LegalScope

Results

Exam performance, case reasoning and agreement with human evaluation.

Read the full results summary

0.817

Exam–case association

Model-level Pearson correlation for automatic scores; Spearman correlation is 0.708. Rankings and variant gains do not transfer uniformly.

66.9 / 72.6

Citation / argument

Mean automatic scores on a 0–100 scale. Lawyers show the same direction descriptively; the pooled lawyer interval crosses zero.

0.910 / 0.312

Exam / case judge agreement

Answer-level Pearson correlations between automatic and human scores. Case scores use the equal-weight mean of two lawyers.

Model performance

28 model groups · Reported benchmark results

Download CSV
Best in columnSecond best in columnAll 28 model groups · ties share rank

Loading published results…

LegalScope model scores. Best and second-best displayed values are marked separately within each column over the complete roster.
Automatic evaluationHuman evaluation

All scores are on a 0–100 scale. Automatic scores cover the full tracks. Human scores cover 80 exam items and 10 case prompts; case scores pool two lawyers. Best and second-best marks use the displayed one decimal values across all 28 groups; equal values share a mark, and filtering does not change their rank. The initial order follows the paper. CSV/Parquet additionally include the two descriptive overall means. Highlights are descriptive, not significance tests. Scores do not equate difficulty across tracks.

Explore the score distributions and transfer plots
Per-model score distributions for 861 exam prompts and 276 case prompts across all 28 groups.
Answer-quality distributions, not repeated-scoring stability.
Automatic exam-to-case scatterplot with Pearson correlation 0.817 and Spearman correlation 0.708.
Automatic exam–case association over 28 model groups.
Human exam-to-case scatterplot with Pearson correlation 0.696 and Spearman correlation 0.690.
Human exam scores and pooled lawyer case scores. This is not an automatic–human agreement plot.