Commonwealth Legal Education AssociationConference, Melbourne 2017

Beyond the ‘adhocracy’: building a platform for legal education research and evidence-based policy-making

in the Common Law world

Paul Maharg Julian Webb

Context: ubiquity of reform• USA

– Carnegie Foundation report (2007)– ABA Task Force on the Future of Legal Education

(Report 2014)– [ABA Commission on the Future of Legal Services

(2016)]• UK

– ACLEC ‘First Report’ (1996)– Law Society Training Framework Review (2001-08)– Legal Education and Training Review Report (2013)

• Implementation by SRA/BSB/CILEx 2013-18

• Hong Kong– Redmond-Roper Report (1992)– SCLET Review 2015-17 – changes by 2021?

• Australia– [Productivity Commission Report (2014), Access

to Justice Arrangements]– Law Admissions Consultative Committee?

• Canada– Federation of LS Canada Task Force Report (2009) – [CBA Legal Futures Initiative Report (2014)]

Context: drivers for reform• Costs of education and training (and impact on practice/A2J)• ‘Over-supply’ of graduates• Concerns about practice readiness [innovation + cost of training]• Diversity and access to the profession• Quality + scope and focus of regulation (moves towards more

competence- and/or risk-based regulation)• Impact of new technologies (on practice and learning)• Impact of new practice models: who will be tomorrow’s lawyers and who

will train them?• Neoliberal policy agendas: competition, privatisation, flexibilisation of HE

Our experience: LETR• Driver: Legal Services Act 2007

(E&W) –– ‘independent’ regulation and

market liberalisation agenda• Push from oversight regulator

(LSB)• Greater focus on regulation• Lack of clear problem definition • Multiple stakeholders/interest

groups• Relatively fragmented system of

LET

Address the following issues:

1.What are the skills/knowledge/experience currently required by the legal services sector?2.What skills/knowledge/experience will be required by the legal services sector in 2020?3.What kind of legal education and training (LET) system(s) will deliver the regulatory objectives of the Legal Services Act 2007?4.What kind of LET system(s) will promote flexibility, social mobility and diversity?5.What will be required to ensure the responsiveness of the LET system to emerging needs?6.What scope is there to move towards sector-wide outcomes/activity-based regulation?7.What need is there (if any) for extension of regulation to currently non-regulated groups?

1 LETR Remit

Pathways to qualification (E&W)

LETR’s ‘new paradigm’: understanding (and responding to) social complexity

Lessons from complexity science, work on bounded rationality, ‘wicked problems’, systems theory, and modern regulatory theory….

(eg, Majone, 1989; Reed & Harvey, 1992; Sanderson, 2002, 2006; Baldwin & Black, 2008; Head, 2008, Wegner, 2009)

• Law of unintended consequences and the limits of regulatory/organisational steering

• Dampening effects on policy reform of internal system design/complexity

• Impact of complexity of social and governance networks• One size does not fit all: possibly greater effectiveness of

small-scale, participatory and localised interventions• Beyond rational choice: values, ethical-moral choices,

and desired ends matter

LET as a ‘socially complex’ problemCharacteristics of socially complex problems

Corresponding features of LSET (eg)

There is no definitive definition of the problem Some agreement over a need for reform, but widespread disagreement over the extent, priorities and nature of the changes required

They tend to be intractable General lack of effect from a number of recent education and training reviewsSpecific intractable problems: •Achieving consistency of standards•Reducing costs of training•Managing increasing numbers

The information needed to make sense of the problem is often ill-defined, changing and may be difficult to put into use

Currently operating in rapidly changing work and educational environments

Relative lack of robust, especially longitudinal, data

Costs of deriving meaningful information are relatively high

From Webb et al (2013), Table 1.1Characteristics of socially complex problems

Corresponding features of LSET (eg)

They emerge in fields where there are multiple stakeholders; limited consensus as to who the legitimate stakeholders and/or problem-solvers are, and stakeholders are likely to have different criteria of success

Large number of stakeholders, with different understandings of the problem(s), and different levels of engagement with the process

Legitimacy questions exist, eg, over the extent of professional and regulatory interest in the Bachelor of Laws (LLB)

Evidence of different stakeholders having different ‘objectives’ for the review

Every attempt at a solution matters significantly Reform tends to be a ‘one-shot’ operation so relatively high risk

Exacerbated by uncertainties about the new regulatory environment, and the tendency of LSET system to operate as a relatively low trust environment

Complexity: how not to regulate…1. Debate around ABA Standard 306 (now 311), restricting distance learning

vis-à-vis classroom time:Standard 311. DISTANCE LEARNING

Distance education is an educational process in which more than one-third of the instruction of the course is characterized by: (1) the separation in time or place, or both, between instructor and student; and (2) the use of technology to deliver instruction.

2. But now see:1. ABA Task Force Report2. Mitchell Law School Variation, under Standard 802 (http://bit.ly/1kvD6uR):

Interpretation 802-1(b) states the Council may grant a variance for an experimental program if that program is well-designed and the benefits of the experimental program outweigh its risks.

Colin Scott’s approach:

‘a more fruitful approach would be to seek to understand where the capacities lie within the existing regimes, and perhaps to strengthen those which appear to pull in the right direction and seek to inhibit those that pull in the wrong way’

‘meta-review’: ‘all social and economic spheres in which governments or others might have an interest in controlling already have within mechanisms of steering – whether through hierarchy, competition, community, design or some combination thereof’ (2008, 27).

Regulation and ‘modalities of control’

Norms Feedback Behaviour-al modifi-cation

Example Variant

Hierarchical Legal Rules Monitoring Powers/Duties

Legal Sanctions Classic Agency Model Contractual Rule-making & Enforcement

Competition Price / Quality Ratio

Outcomes of Competition

Striving to Perform Better

Markets Promotion Systems

Community Social Norms Social Observation Social Sanctions, eg Ostrac-ization

Villages, Clubs Professional Ordering

Design Fixed with Architecture

Lack of Response Physical Inhibition Parking Bollards Software Code

Modalities of control (Murray & Scott 2002)

Regulatory alternatives?Shared spaces concept in traffic zones:• Redistributes risk among road users• Treats road users as responsible, imaginative, human• Holds that environment is a stronger influence on behaviour than formal rules &

legislation.‘All those signs are saying to cars,

“this is your space, and we haveorganized your behavior so thatas long as you behave this way,

nothing can happen to you”. That is the wrong story’.

Hans Monderman, http://bit.ly/1p8fC3u

The Art & Science of Shared Streets, http://bit.ly/1p8fr8r See also Hamilton-Baillie (2008).

redesignrelations

Participative regulation

• Portrait of the regulator as:– Not QA but QE – Quality Enhancer, to focus on culture shifts towards

innovation, imagination, change for a democratic society– A hub of creativity, shared research, shared practices & guardian of

debate around that hub– Initiating cycles of funding, research, feedback, feedforward– Archive of ed tech memory in the discipline– Founder of interdisciplinary, inter-professional trading zones

• Regulator as democratic designer

redesignrelations2

We’re not the only ones….“A resounding sentiment was that there needs to be a permanent conversation about the future of the profession – from broad and creative imaginings of future business structures, to assessing emerging technologies, to pedagogical reform in the training of lawyers – to encourage lawyers to innovate within their own practice setting. More often than not, participants affirmed that there should be space within the CBA for these issues to be discussed on a more regular basis.” (Canadian Bar Association, 2014:24)

Legal Services Forum 2016 15

The need for structure(s)... the fundamental tension between education of lawyers as delivering public value and education of lawyers

as delivering private value is structural. The tension may manifest itself in different ways under different conditions, but it will always be with us and must always be managed. Other matters likely to continually give rise to stresses, challenges, and the need for managing change are: the economics of law schools; the rapid evolution in the market for legal services; the function and value of accreditation standards; the financing of legal education; the role of parties other than law schools in legal education; and the role of media in understanding legal education and communicating with the public. Since these forces and factors will always be with us, it is prudent for the system of legal education to institutionalize the process of dealing with them. All parties involved in legal education should support a framework for the continual assessment of strengths and weaknesses and of conditions affecting legal education, and for fostering continual improvement. The process should ensure that not only law schools, but also practicing lawyers, judges, and other interested actors have a voice and an opportunity for meaningful contribution.”

ABA Legal Education Task Force, 2014:29

Legal Services Forum 2016 16

Structures, systems and culture matter

“Innovation is an ongoing process that requires sustained effort and resources as well as a culture that is open to change. To sustain and cultivate future innovation, the ABA should establish a Center for Innovation.”

ABA Commission on the Future of Legal Services – Final Report, 2016: 48

Legal Services Forum 2016 17

Future research needs?1. Map the field & create

taxonomies for research data

3 improveresearch

2. Organise systematic data collection on law school stats across entry/exit points, across jurisdictions (eg using Big Data Project methods)

Future research needs?3. Focus on learning,

not NSS league tables – see USLSSSE… and include longitudinalresearch data, not justsnapshots of place & time

3 improveresearch

4. Provide meta-reviews and systematic summaries of research, where appropriate; literature guides

Themis.space• Collaborative (and cross-disciplinary) research (Melbourne/ANU/Flinders)

– Original research and meta-analysis: the changing profession/market for legal services– Original research and meta-analysis: developments and best practices in legal education and

training– Synthesis: consequences for LET of regulatory/organisational/technological change in legal

services

• Databanking• Resources and training

– Working Paper Series– Systematic Reviews Series– Methods Series

20

Funding needs1. Targeted funding for research initiatives, systematic reviews, eg Cochrane

Collaboration type of initiative2. Funding & admin support to start-up and analyze innovation – eg PBL, public

education in law,legal informatics,data visualization, etc

3. Financial & other support to enableround table meetings with regulatorsand comparative work with otherjurisdictions – globally

4. Creation and maintenanceof a digital hub.

3 improveresearch

If we build it will they (you) come?

22