What is LMArena and how does it rank models?
How LMArena, formerly Chatbot Arena, turns anonymous head-to-head votes into model scores, how to read the gaps and ties, and what the rankings cannot tell you.
Last updated Oct 1, 2026
When people say an AI model is "number one", they often mean it tops LMArena. It is one of the most watched AI leaderboards, and its scores appear on every model card in Modeldle. Here is how it works, and how to read it without being misled.
From Chatbot Arena to LMArena
The project began in 2023 as Chatbot Arena, run by researchers connected to UC Berkeley under the LMSYS name. The idea was simple: instead of grading models only on fixed tests, let large numbers of real people compare them on their own questions. It grew into one of the most cited AI rankings, was renamed LMArena in 2025, and now operates as its own company. The team describes the method in a 2024 research paper.
How a battle works
- A visitor types a prompt: a coding question, a riddle, a request for a poem, anything at all.
- Two models, picked behind the scenes, each answer. The replies appear side by side, labelled only as model A and model B.
- The visitor votes for the better answer, calls it a tie, or says both are bad.
- Only after the vote are the model names revealed.
Hiding the names is the key design choice. People cannot favor a brand they like, so the vote reflects the answer itself. If a reply gives away which model wrote it, for example by stating its own name, the vote is generally left out of the rankings.
From votes to a score
Millions of these votes are turned into one number per model using a Bradley-Terry model, a statistical method for ranking things from pairwise comparisons. It is a close relative of the Elo system used for chess players, and the scores sit on a similar scale. That is why you see numbers like 1450 or 1500 rather than percentages.
The useful thing about this scale is that the gap between two scores predicts how often the higher model should win a head-to-head vote:
| Score gap | Expected win rate for the higher model |
|---|---|
| 0 points | 50% |
| 10 points | About 51% |
| 50 points | About 57% |
| 100 points | About 64% |
| 200 points | About 76% |
So a 10 point lead is close to a coin flip. Scores are also relative: they describe how a model compares with everything else on the board, not how good it is on some absolute scale. That is why an old flagship can look weak today. GPT-4 sits at 1206 in the catalog, about 300 points below the leaders, because it is being measured against much stronger models.
Confidence intervals and ties
Every score is an estimate, and LMArena shows how sure it is with a confidence interval, such as plus or minus 5 points. A model with few votes has a wide interval. A model with many votes has a narrow one, and brand new models often move noticeably in their first weeks as votes pile up.
When two models' intervals overlap, the leaderboard cannot really tell them apart. LMArena's rank column reflects this: a model's rank depends on how many models are confidently ahead of it, so several models can share a rank. If one model is listed a couple of points above another, treat it as a tie unless the intervals clearly separate.
Category leaderboards
Besides the overall text ranking, LMArena sorts prompts into categories and computes a separate score from just those votes. Modeldle tracks three of them: coding, math and creative writing. The leaders do not always match the overall leader:
| Leaderboard | Top model in the catalog | Runner-up |
|---|---|---|
| Overall text | Claude Opus 5.5, 1512 | Claude Opus 4.6, 1504 |
| Coding | Claude Opus 4.6, 1535 | Claude Opus 5.5, 1534 |
| Math | Gemini 3.8 Flash, 1522 | Claude Opus 4.6, 1517 |
| Creative writing | Claude Opus 5.5, 1530 | Claude Opus 4.6, 1506 |
Two details stand out. In coding, the newer Opus 5.5 trails Opus 4.6 by a single point, which is a statistical tie. In math, Gemini 3.8 Flash, a model from Google's faster and cheaper Flash line, tops every model in the catalog. Just behind the leaders overall are Muse Spark 1.3 at 1490 and Qwen3.8 Max at 1481. LMArena also runs separate arenas for things like web development and image generation, each with its own leaderboard.
What the score tells you, and what it does not
An arena score is a strong signal of one thing: how much people prefer a model's answers to the kinds of questions they tend to ask. That captures helpfulness, clarity and tone in a way that fixed tests often miss.
It does not tell you:
- Whether answers are correct. Voters judge what looks better, and a confident, well formatted wrong answer can win. Because voters also tend to favor longer replies with tidy formatting, LMArena uses style control, a statistical adjustment that tries to separate what an answer says from how long and how decorated it is.
- Price or speed. A top score says nothing about cost. See the guide to token pricing.
- How a model handles your task. Most arena prompts are short chats. Working through long documents, acting as an agent with tools, or answering in a specialist field are different tests.
- The whole story behind a ranking. The authors of The Leaderboard Illusion argued that labs able to test many private versions before a release can gain an edge. LMArena disputed parts of that analysis, but it is a good reminder to treat any single leaderboard as one input, not the final word.
In the quiz
Modeldle asks "Which ranks higher on LMArena?" and category questions like "Which is better at coding, by LMArena votes?", using the scores on each card along with the date they were taken. The quiz avoids near ties when it picks models, but small gaps can still flip from one week to the next, and the category leaders are not always the overall leader. Play today's quiz to see how well you know the board.