Local model evaluation
Which of your models is actually best?
You have eight models pulled in Ollama. Public benchmarks tested none of your prompts on none of your hardware. Turing Chat asks all of them the same question, times every response, and hides the names until you have picked a winner.
How it works
Ask once
One prompt goes to every model you selected. They run one at a time, so they never compete for the same GPU while being measured.
Judge blind
Answers appear under shuffled labels. You cannot tell which came from the 14B model, so you grade the writing instead of the parameter count.
Keep the score
Each pick becomes a pairwise vote. An Elo table builds up across every comparison you run and survives a reload.
What gets measured
Timings you can trust
Everything is measured client-side so the numbers stay comparable between providers, and the interface throttles its own redraws so rendering never leaks into the results.
| Metric | What it tells you |
|---|---|
| TTFT | Time to first token — how long the model sat there before saying anything. Dominated by prompt evaluation and cold model loads. |
| tok/s | Decode throughput, measured after the first token arrives so a slow start does not disguise itself as slow generation. |
| Total | Wall-clock time for the complete response. |
| Elo | Rating from your own pairwise votes, replayed in chronological order so the standings never depend on load order. |
| Median | Performance is reported as a median, not a mean — one cold load would otherwise poison the average for a model. |
In your own app
Two components, no backend
Everything runs in the browser against your local server. Nothing is uploaded, and there is no service to sign up for.
// npm install @turing-chat/react @turing-chat/core
import { ModelArena } from "@turing-chat/react";
import "@turing-chat/react/themes/instrument.css";
export default function Page() {
return <ModelArena baseUrl="http://localhost:11434" />;
}No models installed?
The demo runs on nothing at all
Both demos ship with a simulated provider — three models with different speeds and answering styles — so you can try the whole flow without downloading a single weight.