About Larry Cunningham

Law professor, associate dean, and director of the Center for Trial and Appellate Advocacy at St. John's Law School. Former prosecutor and defense attorney.

Preparing for the NextGen Bar Exam: Questions to Consider

Last week, the National Conference of Bar Examiners released a preliminary set of content outlines for the NextGen Bar Exam. If the timeline holds, this new bar exam will be implemented in 2026. While this seems a long time from now, it is not. Part-time students who enroll this Fall will take this new bar exam. As a result, many schools are now considering how best to adapt to the new bar exam, which has a (1) reduced number of doctrinal subjects tested and (2) increased testing of skills.

Here, I list the questions that I am thinking about both generally and for my school:

  1. What are our school’s goals besides bar passage? It may seem bizarre to start with this question on a blog post about the bar exam, but I think it is the most important. Law schools are not three-year bar preparation programs. We have other goals for our students and ourselves. Career placement, for instance, is an important outcome for law schools. Preparing students for careers in the legal profession requires a curriculum that may not necessarily align with one specifically designed just for the bar exam. Entry-level employers may wish to see students who have knowledge, skills, and values that are not tested on a bar exam. In addition, lifelong success in the legal profession—however we define it—may warrant preparation that does not align with the bar exam. Knowing at the outset what our other goals are will help us to balance what may be competing priorities for a limited number of credits in the curriculum. 
  2. What are our ethical and consumer protection obligations to students? Professor Melissa Shultz (Mitchell Hamline) makes a compelling case in a forthcoming piece in the Journal of Legal Education that law schools have an obligation to prepare students for the new bar exam by taking action now so that students who will sit for that exam will be fully prepared for it. “These monumental changes to the bar exam,” Professor Shulz writes, “do not allow for the legal academy to take a tempered ‘wait-and-see’ approach before taking action.” Her article shares a number of helpful strategies for doing so.
  3. To what extent does our existing curriculum align with the Uniform Bar Exam? If a school has high bar passage and its curriculum is not particularly aligned with the UBE subjects, it may be that only minor modifications are needed. These are likely schools that enroll students who are excellent test takers and will do well with any format of exam. For most schools, however, their curricula may be somewhat or significantly aligned with the UBE content outline. As a result, they will need to do a more significant re-alignment of their curricula to meet the new bar exam while still achieving other curricular goals (#1).  
  4. To what extent does the NextGen bar exam differ from the UBE in doctrinal subjects that are tested? We know that some subjects, such as Secured Transactions, are being dropped from the exam. What are they? Within each subject that remains, how is coverage changing? What is added or dropped? We must all become experts on what the new bar exam is and is not so that we can speak and act thoughtfully on the subject.
  5. Where is the doctrine tested on the NextGen bar exam taught in our existing curriculum? Are there any curricular gaps? Since the NextGen exam largely removes doctrine rather than adding it, I imagine the answer at most schools is that there are no gaps in substantive and procedural law.
  6. Which skills will be tested (or tested more heavily) on the NextGen bar exam? This is the most signficant change to the bar exam in my view—the heavy testing of lawyering skills. 
  7. Where are those skills taught in the existing curriculum, if at all? Are they taught in-depth or just in a cursory fashion? How are they assessed? A curriculum map may be helpful, as are focus groups and surveys of faculty who teach what may currently be specialized electives with low enrollments. 
  8. How do we adapt the curriculum? This is the most significant question and requires consideration of: a school’s other goals (#1), its ethical and consumer protection obligations (#2), the extent to which a school focuses on bar passage (#3), the gaps in doctrine (#5) and skills (#7) between the current curriculum and the new bar exam, and whether there are gaps in faculty expertise that may require a new approach to hiring (#10).
  9. How should our teaching methods and assessments adapt to prepare students for the format of the new bar exam? The new bar exam will employ different assessment tools than we are used to. To what extent should we expose students to them while they are in law school?
  10. How does the NextGen bar exam impact faculty hiring? The number, type, and subject matter expertise of new faculty may need to be reconsidered if a major curriculum realignment is expected.  
  11. What timeline should we follow? Has the NCBE stayed on track with its timeline, suggesting that a 2026 implementation is likely? If so, what steps do we have to take and when to meet our ethical and consumer obligations (#2) and prepare our students for this new exam while still ensuring that current students taking the UBE in interim are well-prepared for that exam?
  12. What should be the process of educating faculty, administrators, and students about the new bar exam and getting buy-in from constituent groups about the new exam? Education and buy-in are two separate considerations.
  13. Will our state’s supreme court adopt the NextGen bar exam? Just as law schools are thinking about the bar exam, state supreme courts are looking at whether they will sign on to the NCBE’s new test or go in a different direction. If most of a law school’s graduates will sit for the bar exam in a state that does not adopt the NextGen bar exam, much of these considerations are mooted. However, the school will still need to think about how it is preparing students for the bar exam that they will take. In addition, a school will need to consider the students who will sit for a NextGen bar exam out-of-state.
  14. Is the NextGen bar exam such a significant shift that it warrants rethinking our admissions criteria? ABA Standard 501(b) requires that law schools only admit students who appear capable of being admitted to the bar exam. If the bar exam changes, it may be that predictors of success change. Should we put different weight on the LSAT/GRE, UGPA, work experience, and references than we do now? Unfortunately, we will not have data on whether our existing admissions framework remains predictive until after the first few cohorts sit for the new bar exam.

It may be there are other questions a school should consider, and I will keep adding to the list as I think of them.

Cross Cultural Competency as a Learning Outcome

The ABA received a flood of comments regarding a series of proposed changes to the Standards. The proposals involve professional identity formation and, broadly, the topic of diversity and inclusion. I added my two cents, suggesting areas where the proposals got it right and where they could use improvement. I noted an overarching concern about the ABA Standards being used to advance particular views of legal education that are better left to individual schools to decide whether to adopt. As I wrote, I can understand fully why schools may wish to specialize or distinguish themselves through professional identity formation or, for that matter, law and economics, public interest law, or international perspectives. That does not mean those views should be imposed on the other roughly 200 schools governed by the Standards as a matter of accreditation.

In any event, one of the proposals would required all law schools to provide training in “bias, cross-cultural competency, and racism” at two points in a student’s education: once at the beginning and at least once before graduation. If a student takes a clinic or externship, the second training must take place before (or concurrent with) enrollment in the clinic or externship. This proposal would add the requirement to Standard 303, the same Standard that requires (a) three broad course sequences (professional responsibility, legal writing, and the 6-credit experiential requirement) and (b) substantial opportunities for clinics, externships, and pro bono work.

In my comment, I suggested a different approach: adding cross-cultural competency to Standard 302.

Continue reading

The New Bar Exam: Out With the Old or More of the Same? Perhaps Some of Both.

The National Conference of Bar Examiners’ “Testing Task Force” is out with its preliminary recommendations for the “Next Generation of the Bar Exam.” 

What’s the Same: Timing and Number of Exams

A lot of it is “same old, same old.” For example, the Task Force recommends sticking with a single bar exam rather than multiple assessments at various points in a student’s law school career. The latter approach is one familiar to medical education, where MD and DO students take exams at three separate points in their careers: after their second year, after their fourth year, and a year into their residencies. In addition, the bar exam will continue to be given only twice a year. I disagree with both decisions. I would rather see students tested on foundational subjects after the first year and given time to retake the exam if needed while still in school. Offering the test more often would also benefit students who do not do well on standardized tests and need to retake one or more portions. The report counters, “The Task Force concluded that the use of an integrated exam with an increased emphasis on assessing skills and more limited depth and breadth of coverage of doctrine addresses the underlying reasons some stakeholders favored multi-event testing.” This is not an unreasonable position, but it remains to be seen how integrated the new exam actually is. If there is still a clear demarcation between doctrine and skills, I would prefer two exams that can be taken independently. The report does not explain why the Task Force is recommending keeping only two administrations, but I imagine it is because of the practicality in developing and scoring multiple sets of questions on an ongoing basis.

Format Changes

Changes that are proposed in the report are significant. For example, the bar exam of the future will be administered all on computer, but the report notes the importance of accounting for takers who require accommodations. In addition, the report proposes a mix of more styles of questions beyond multiple choice, essays, and performance tests (e.g., selected-response, short-answer, and extended constructed-response items). A question “set” might propose one set of facts that are then tested with various kinds of questions that follow. This will make the authorship of questions easier.

Integrating Doctrine and Skills: A Significant Change in Attorney Licensing

In terms of content, the biggest change is that the exam would more clearly test both “doctrine” and “skills.” Doctrine—what the Task Force calls the “Foundational Concepts and Principles”—would be divided between eight subjects (Civil Procedure, Contracts, Evidence, Torts, Business Associations, Constitutional Law, Criminal Law/Procedure, and Property), a significant reduction from the current number tested on the Uniform Bar Exam. If other subjects are tested (e.g., Family Law), the law would be provided to students.

The second set of outcomes are skills based (“Foundational Skills”). This is where it gets interesting. The Testing Task Force proposes to test on legal research, legal writing, issue spotting/analysis, investigation, client counseling, negotiation, and client relationship/management. “Foundational Skills” could be tested in the context of the “Foundational Concepts and Principles,” and vice versa. Moreover, the “new” bar exam might assess skills in “uniform text- or video-based scenarios that require candidates to construct a written response or select the correct response.”

The Task Force envisions that it would take 4-5 years to develop this new type of exam and provide appropriate notice to law schools and law students. The NCBE may be underestimating the notice and lead-up time that is required. For a small number of schools, the bar exam is an afterthought. For most schools, however, the bar exam drives at least some curricular decisions. It is a fact of life (and law school administration) that if the bar exam changes, law schools and their students will respond in some fashion. As an example, look at what happened when states have moved to the UBE. Law schools stopped offering as many as state-specific courses, and fewer students took the electives that remained. In some states, the change to the UBE was done abruptly (unfairly, in my view) without sufficient time for 2Ls and 3Ls to take different courses. I am glad the Task Force recognizes the importance of notice. However, the change in focus from doctrine to a blend of doctrine and skills may require a substantial reconfiguration of the modern, American law school, so that all of the skills tested on the bar exam can be taught and assessed properly. Among other things, the change may force schools to re-think their faculty hiring practices, bringing in fewer JD/PhD’s who focus on theory and scholarship and more practicing attorneys who teach hands-on skills. Courses and degree sequences might need to be designed so that all of the skills tested on the bar exam are taught and assessed. Some schools may have the luxury of ignoring the bar exam, but I suspect many schools will wish to track (at least partly) what is tested on the bar exam. Whether that means courses in investigation and client management will become required for all students remains to be seen. I suspect, though, that schools will not want to risk causing licensure problems for their students and will try to adapt as much as possible to the new outcomes.

The bar exam is as much a test of competency as a signaling device about what is and is not important. As someone who comes from the “skills side” of legal education (I was a legal writing professor and clinician before joining Charleston School of Law), I am thrilled to see more of a focus on the exam on the actual skills that law graduates will need in practice. But rest assured that a shift in focus on the bar exam will have significant ripple effects as law schools adapt their required, core, and elective curricula to prepare students for this important, high stakes exam, especially given how ABA Standard 316 looms over our heads.

But Will the State Supreme Courts Follow NCBE’s Lead?

All of this is very interesting, but it remains to be seen whether the NCBE will actually implement these recommendations and, more importantly, whether the state supreme courts will utilize such a “new” bar exam. At this point, this is a preliminary set of recommendations, although the fact that the Task Force is made up of such influential members in the bar exam world, including Judith Gundersen, makes it likely that the full NCBE will proceed with adopting the report in some fashion. But I would not underestimate the resistance that the NCBE is likely to encounter from state supreme courts, which have the ultimate licensing authority over attorneys. Some courts are likely to prefer a focus on doctrine, since it is the type of exam that they were licensed under as attorneys. On the other hand, there have been complaints from the bench for years that new lawyers did not have the requisite skills to practice law, so perhaps courts may be more receptive to this kind of blended exam.

The other sector that will need to change if this “new” bar exam goes through is the bar preparation field. Teaching doctrine to hundreds or thousands of law graduates at a time is relatively easy since materials (especially video lectures) can be scaled up. Teaching, assessing, and giving feedback on interpersonal lawyering skills (e.g., negotiation) is another matter.

On balance, I am excited for the conversation to be kicked off with this report. I look forward to seeing what various constituent groups—especially the courts—do with the recommendations and proposals.

Personal News

The blog has been on the quiet side — something I hope to change in 2021 — in large part because I moved schools in June. After 12 years as a faculty member at St. John’s, I am now the Dean of Charleston School of Law. Becoming a dean in the middle of a pandemic has been a fun whirlwind but has left little time for writing. Nevertheless, having survived my first semester, I plan to post here more often.

As a bit of background, Charleston School of Law is a young, independent law school. One of only two law schools in South Carolina, CSOL was founded in 2004 with a mission of “pro bono populi” – for the good of the people. Since then, our graduates have donated over a half million hours of pro bono work to the local legal community. While the Law School has had challenges (some common to legal education, some unique), we are very much on the upswing. For instance, our Fall 2020 class was our highest credentialed class since 2012. I hope to be able to share lots of good news about the Law School in 2021. In the meantime, I’ll have a few assessment-related posts coming up.

Moving to Best Practices in Online Learning

By now, all law schools are—or are about to be—fully online. My sense is that professors are mostly teaching synchronously (i.e., live) using Zoom, Webex, or some other platform. In this respect, faculty are trying their best to replicate the in-person law school classroom; in some cases, this even includes cold calling on students to recite cases. Class sizes remain as they were before the Coronavirus hit. That means that some professors are teaching in this format to 60, 80, or even 100 students.

To experts in online teaching and learning, this is not how any of this is supposed to work. Successfully moving a class online requires time, effort, and training, as this helpful post describes. And it typically requires smaller class sizes than we’re used to in legal education, although experts acknowledge that there is no magic number.

A common misconception about moving a course to an online format is that the professor merely delivers the same content in the same manner as a face-to-face classroom … just on video. Not so. Professors new to online learning often view their teaching—incorrectly—through the “lens” of face-to-face classes. In actuality, adapting a course for online learning requires starting anew with course objectives and working backwards from those outcomes the professor hopes to achieve. The steps in students’ development become “modules” (typically on a week-by-week schedule) that are thoughtfully designed to achieve smaller outcomes, like guideposts on a trail. The course is not a march through a casebook.

Each module uses the learning tools that are best designed to achieve that module’s objectives. A module may include a synchronous discussion (as Nina Kohn [Syracuse] persuasively notes) or not. Asynchronous activities, which can be completed at different times in the week, can also be effective. Pre-recorded video lectures (of no more than 15-20 minutes, since attention spans wander after that point), quizzes, discussion posts, writing assignments, readings, and creative projects (such as students creating their own narrated PowerPoints on a subject) are all arrows in the professor’s asynchronous instructional quiver. Many of these activities are themselves formative or summative assessments of students’ progress in meeting the outcomes of the modules and, in turn, the course. The key is to make deliberate, strategic choices about what mode of delivery (synchronous or asynchronous) works best for a particular module or course. Often the best courses will include a blend of the two approaches.

So let us be clear: what we’re doing right now is not best practices; it is a bandaid. It is an emergency response so that we can continue instruction in some format so that our students do not lose the entire semester.

We have a responsibility, however, to do better in the weeks ahead, not just to satisfy accreditors but as part of our obligation to provide a quality education to our students. We should embrace a culture of continuous, quality improvement. This is especially true if the current crisis continues for a significant period and requires law schools to continue to be fully online into the Fall semester.

The Short-Term

There are things we can do in the short-term to improve teaching and learning this semester. In the coming weeks, we should:

  • For faculty who are just recording and posting lectures, encourage them to move to synchronous delivery so students can ask real-time questions and otherwise participate actively.
  • For faculty who are already meeting synchronously, encourage them to adopt active, rather than passive, learning activities in their virtual classrooms. Most videoconference platforms have polling or quizzing features, for instance, which can engage all students, not just those who are participating in a discussion. Get out of the mindset of looking at your online course through a face-to-face lens.
  • Encourage small group interactions to spur discussion and participation. Both Zoom and Webex have “breakout” rooms, and I have become a big fan of using them to spur think-pair-share type exercises. Like polling, breakout sessions turn students into active, rather than passive, participants.
  • Encourage faculty to supplement synchronous sessions with optional, asynchronous exercises, such as quizzes or discussion boards. I recommend making them optional at this point, since we do not want to overwhelm students, who are also adapting to new circumstances.
  • Make ourselves reasonably available for “office hours” through drop-in Zoom or Webex sessions. Not only does this foster personal connection during this time of isolation, but these meetings also give students an opportunity to clarify points they may have missed during pre-recorded lectures or live videoconferences.
  • Help each other.  Having all been thrown in the deep end of the pool at the same time, we are all learning how to swim, some better than others. We can help each other to say afloat. Faculty should share tips, suggestions, and solutions to common problems with one another. As an example, several of us at my school have been struggling with playing videos through our videoconference platform because of “lag” issues. Last night, an adjunct professor came up with an ingenious, but low tech, solution. He turned his laptop (with webcam) to a secondary monitor, brought it close to the monitor, and played the video from the secondary monitor at full volume. It wasn’t perfect, but it allowed students to hear and see the video with decent quality.

I recognize that this is a difficult time for many law professors. Some may be dealing themselves with illness or adjusting to disruption at home. Some of my colleagues now have two full-time jobs: teaching and taking care of school-aged children. Therefore, in my list above, I use words like “try,” “encourage,” and “reasonably available,” not “must” or “require.” The suggestions I have offered must be adapted to each individual faculty member’s circumstances and capabilities at this time.

Nevertheless, to the extent we are able, we should all be thinking about how to get better at teaching in this new format over the next few weeks. For some, that may mean exploring many of the asynchronous tools at our disposal. For others, it may mean trying to facilitate greater discussions versus lecture. Some of the suggestions I list above require no additional time on the professor’s part. If we take these steps, the teaching taking place at the end of April should be better than that being offered now.

The Long-Term

While we are all hopeful that things will get back to normal soon, law schools must nevertheless plan for the possibility that online teaching will need continue into the Fall (depending, of course, on how the virus progresses). What we can get away doing now on an emergency basis is not going to be acceptable to either our accreditors or our students by the Fall. They can, and should, expect a higher quality of instruction from us given the lead-up time we have until the next semester. Here, time is on our side but only if we begin planning now. The key, in my view, is training. Faculty need training on the best practices for online teaching and learning. The Quality Matters Standards and Rubric are a great place to start in terms of where we need to be (eventually) with online learning. The summer months provide an opportunity for such training and development.

Of course, someone has to develop and deliver such training. Law schools that are attached to universities can take advantage of courses that likely already exist through their universities’ centers for teaching and learning. Typically those courses include ones on Blackboard/Canvas, online pedagogy, and similar topics. Standalone law schools should consider working together to develop such courses. Training law faculty, full-time and adjuncts, on a massive scale to deliver “best practices” online teaching will require a great deal of planning to prioritize and rollout development opportunities in a smart way. That planning should take place now, even if the medium-term future is uncertain. I’ll likely have more to say, in a future blog post, on what that planning should look like.

Something else to consider for the Fall is section size. In the academic research, 20 is the recommended, outer limit for class size, and that number is for undergraduate courses. Graduate courses typically have smaller enrollments. Class size affects the ability of a professor to provide interactive experiences and feedback.  The more interactive a course is, the better the learning experience for students. Faculty can successfully provide an interactive experience only to a limited number of students. This may require reconsideration of the class schedule, such as cutting back on electives so that additional, smaller sections of required or core courses can be offered while a law school is in a fully online mode.

Needless to say, these are difficult times for everyone. Adopting a continuous improvement mindset, though, will benefit all of our stakeholders, especially our students.

Assessment in a Time of Coronavirus and Closed Campuses

Today is March 15, 2020, and, by now, most law schools have either announced a transition to fully online teaching or set a date when they will begin doing so.  Although many schools have said that this situation is temporary and will last for no more than a few weeks, my personal prediction is that most schools will not resume face-to-face teaching this semester.  This post invites faculty and administrators to think now about the consequences for assessment during this challenging time.

Although my usual interest is in programmatic assessment, here I am writing specifically about course-based assessment.  On the one hand, the next six seeks or so may be an opportunity for faculty to provide more formative assessments to students, such as low-stakes quizzes, essays, and discussion posts. Such activities are a way to keep students engaged with material.

However, there is a looming assessment issue that will require some attention sooner rather than later: how to engage in the typical end-of-semester summative assessments, such as final exams and, for skills classes, final activities. The questions that a law school must answer are several and complex: Continue reading

Student Services and Assessment

The word “assessment” is often used in the context of students’ academics: measuring the learning that takes place in formal coursework. Let me suggest, though, that there are ways that administrators and staff who work in student services can both engage in their own assessment and help the academic side of the house with learning outcomes assessment.

(I’m speaking on this topic at the AALS Annual Meeting on Saturday, January 4, at 2:45 pm in Washington 1, if you’re interested in hearing more. My slides are attached here.)

Law schools employ many type of professionals besides faculty. A given law school may also have Continue reading

Exam Wrappers and Self-Assessment

As reported today on TaxProf, Professor Sarah Schendel (Suffolk) has a new article on SSRN, “What You Don’t Know (Can Hurt You): Using Exam Wrappers to Foster Self-Assessment Skills in Law Students.”  She describes exam wrappers as a “one page post-exam exercise” that has students self-assess their “exam preparation and exam taking skills, and prompt them to consider changes to their techniques.”  Exam wrappers have been used in a number of disciplines, including physics, chemistry, and second language acquisition; however, they are not widespread in legal education.