By Karen Sloan for Law.com
Editor’s note: This is the first installment in a series called “The True Test: How COVID Is Changing the Course of Legal Education,” which examines how the pandemic is reshaping the way lawyers are trained and licensed. Join us on Twitter Dec. 8 at 11 a.m. EST, where a panel of experts will be sharing their thoughts on what’s next for law schools, at #FutureofLegalEd.
Joan Howarth began researching attorney licensing and efficacy of the bar exam back in the 1990s, but for decades it was lonely work. A handful of fellow legal academics joined her in critiquing the long-standing and little-changed lawyer licensing test and offering ideas for reform over the years, but their proposals fell flat with the bar examiners and state supreme courts that control the attorney admission process.
Flash-forward to 2020, and suddenly Howarth, a professor at the William S. Boyd School of Law at the University of Nevada, Las Vegas, has plenty of company in scrutinizing the bar exam. The COVID-19 pandemic’s upheaval of the bar exam status quo has placed an unprecedented spotlight on the exam, prompting questions about both how it is given and what it actually tests. Moreover, the November completion of several multiyear reviews of the bar exam—one of which was conducted by the national organization that designs the test—offer road maps for a revised bar exam that better gauges the many skills new lawyers need to succeed in practice. Additionally, various influential states, most notably California and New York, recently formed committees to take a fresh look at their own bar exams. Some of these long-in-the-works bar exam studies were prompted by a dramatic decline in passage between 2011 and 2016, when the national first-time pass rate on the July test fell from 83% to 75%.
“I think this is a unique moment for change, different than any I’ve seen since I started thinking and working and writing about bar exam issues in the 1990s,” Howarth said. “We have new research and the new initiatives from the NCBE [National Conference of Bar Examiners]. We have new and important research from the IAALS [Institute for the Advancement of the American Legal System] team led by Deborah Jones Merritt. We have a lot of interesting work being done in California about bar exams. And then we have the shake-up of the pandemic. Each of those is creating opportunities for new thinking about bar exams.”
COVID-19 also forced bar admission authorities to look closely at how they license new attorneys, rather than simply defaulting to the same system that has been in place for decades, Howarth noted. (Thirty jurisdictions offered online bar exams this summer and fall, while a handful enacted diploma privileges that enabled some law graduates to become licensed without passing the test.) That on-the-fly rethinking of bar exam fundamentals could prove pivotal in a larger reimagining of the paths to attorney licensure, she said.
For instance, the pandemic has prompted many to question whether the bar exam is even necessary, though supporters maintain the test helps ensure unqualified new attorneys aren’t unleashed on the public. Hundreds of recent law graduates banded together over the summer to push states to adopt emergency diploma privileges that would allow them to be admitted to practice without passing the bar, arguing not only that in-person testing is unsafe amid a pandemic but that three years of law school adequately prepared them for practice. Their efforts yielded modest results: Louisiana, Oregon, Utah, Washington state and the District of Columbia adopted some form of diploma privilege in the short term.
But organizers of the diploma privilege movement say they will continue to push for changes to attorney licensure even after the pandemic subsides. United for Diploma Privilege, which has served as an umbrella organization for groups pursing diploma privilege at the state level, is in the process of becoming a nonprofit under a new name, according to co-founder Pilar Escontrías. The group is expanding its focus beyond the bar exam to include diversity and pipeline issues and changes to the law school curriculum. The legal profession’s shortcomings on the diversity and equity fronts can’t be solved by changing the attorney licensing process alone, she said.
“We will be formed as a nonprofit by the end of this year,” Escontrías said. “We have a board lined up. And we have a mission statement. We are absolutely invested in continuing this fight. There are many questions for February bar takers. There are still opportunities for advocacy. People are tired. They’re burned out, but the heart is there.”
‘It Will Be Different’
Most insiders say that the bar exam is unlikely to disappear anytime soon. But nearly all agree that 2025’s bar exam will look different than today’s. The extent of those changes remains to be seen, but the NCBE is poised to unveil a series of recommended reforms in January, the culmination of three years of research by its Testing Task Force, which purportedly conducted a top-to-bottom review of the exam. (Officials with the NCBE said that the review doesn’t mean anything is wrong with the existing test. Rather, a periodic assessment is the responsible way to ensure the exam is both a valid and reliable measure of lawyer competence.)
The task force sought feedback from hundreds of stakeholders on the current exam, and gathered data on what new lawyers actually do in a bid to assess whether the existing exam gauges those skills and knowledge. Many stakeholders said they want more emphasis on lawyering skills such as research and writing, and less emphasis on the memorization of doctrinal law. The task force’s practice analysis, which zeroed in on the tasks that new lawyers perform right out of law school, found that legal analysis, legal research, and communication were key, while in-depth knowledge of the law is less important.
Currently, the bar exam is given twice a year over two days, in July and February. (Many states delayed this year’s exam to September and October due to the pandemic.) Every jurisdiction except Louisiana uses the Multistate Bar Exam, which consists of 200 multiple-choice questions completed during one testing day. And most others use a combination of the Multistate Essay Examination, which is six 30-minute, closed-book essays, and the Multistate Performance Test, which requires examinees to draft a legal complaint or client memo using a provided set of reference materials. Some jurisdictions substitute state-specific essay questions in place of those created by the NCBE. Scores from all components are combined, and anyone who fails must retake the entire two-day exam.
Among the reforms the task force considered is administering different portions of the bar exam at different times, such as giving the MBE after the first year of law school, then the remaining sections after graduation. They also contemplated separating scores earned on each section so that if a taker fails one portion, they don’t have to retake the entire exam. Additionally, they considered an exam format that hews more closely to the existing test, in that it would be given all at once, after law school.
The preliminary recommendations are still under wraps, but the Testing Task Force has strongly signaled that the bar exam of the future will be computer-based, rather than given on paper, as it largely is now, said Kellie Early, chief strategy officer for the NCBE.
That doesn’t mean the exam will be given remotely, however. The task force still envisions administering the exam in-person at testing centers or centralized locations, but on computers or laptops. The task force has also made it known that it aims to expand the testing of legal skills, though it’s unclear what form that would take.
“It will be different in what it tests and how it tests,” Early said. “I don’t think it’s going to be something that is unrecognizable to our stakeholders. But I think it will be more changed than potentially some people are expecting. We’ve heard that some people think [the Testing Task Force] is not going to lead to any change. To them I’d say: ‘I hope we haven’t spent the past three years and all the work and effort that has gone into this for naught.’”
‘What Lawyers Actually Do’
Among the people pushing for more radical bar exam reform is Merritt, an Ohio State University law professor who has been studying the bar exam since 2000 and who spearheaded the IAALS’ Building a Better Bar Exam project. That analysis relied on surveys and lawyer focus groups convened over several years to identify 12 minimum competencies—or “building blocks” as the report calls them—that new lawyers need, and highlight where the existing exam falls short in assessing those competencies. The resulting report, released in late October, echoes earlier research concluding that legal skills are given short shrift on the bar exam.
“The thing that surprised me is how little supervision there is in the workplace and how much responsibility new lawyers get in their first year of practice,” Merritt said. “I went into it thinking that was an issue for solo practitioner: How do they survive the first year? It was extraordinary to see how many lawyers, on the day they are licensed, are given cases of their own and are told to go off and take care of it.”
Merritt’s research showed that new lawyers gain legal knowledge quickly on the job, but skills such as negotiation, time management, and client communication are harder to pick up in practice. Supervising attorneys often don’t know how to teach newer lawyers those skills, and the need to keep up with billable hours means new lawyers have little time to observe more experienced lawyers in action. Any exam that doesn’t adequately assess a law graduate’s ability to effectively interact with clients, to communicate, and see the “big picture” of client matters, isn’t a valid exam, Merritt said.
The Building a Better Bar Exam report offers 10 recommendations for the bar exam, which include eliminating or greatly paring back the number of multiple-choice questions on the test; eliminating essay questions and substituting in more performance tests; making any remaining multiple choice or essay questions open book; and giving examinees more time to complete the written portions of the exam.
The practice of law isn’t undertaken in the extreme time constraints mandated by the bar exam, the study concluded, nor are practicing attorneys expected to rely on memorized law when representing clients. To ensure new lawyers have the skills that are difficult to assess on a standardized test, law schools should require students to complete courses in negotiation and clinics in which they can interact with actual clients, the report concluded.
Mike Sims, president of bar exam prep provider BarBri, agrees that the bar exam could use an update. He likes the idea of administering a form of the MBE after the first year of law school, when knowledge of the law is freshest in examinees’ minds, then giving an open-book essay and performance test after graduation.
“There is no question that a test that requires extremely large-scale memorization does not compare to what lawyers actually do,” Sims said. “But a test that requires potential lawyers to analyze a fact pattern and apply the law, that would be a good test in my opinion.”
Early said that the NCBE is unlikely to ditch multiple-choice questions altogether, in part because that format allows for a uniform assessment. It’s more difficult to ensure reliability with individually graded skill components, said Mark Raymond, the NCBE’s director of assessment of design and delivery. Skills testing also introduces a greater risk of bias in the grading of the exam, he added.
Despite those challenges, the bar exam of the future will likely include more skill components. That’s partly why the NCBE expects that it will take anywhere from two to five years to develop and test out any revamped exam. Law students would also need plenty of advanced warning about changes to the test. Plus, jurisdictions will need to sign on to any overhauled bar exam since they control lawyer licensing.
“There will be change,” Early said. “Change is hard. And it’s especially hard when we’ve got a model of a state-by-state licensing and testing process. It may not be possible to make everyone happy. But we have definitely listened and considered everything we’ve heard.”