The 10 most used LLM eval and tracing tools in 2026

Picking a tracing and eval stack usually starts with a vendor comparison page, and the package registry is a better place to start because its numbers are public. Measured by monthly PyPI downloads on 3 November 2026, LangSmith leads LLM eval and tracing tools at 77.2 million, followed by Langfuse, MLflow, Logfire and Braintrust, with Inspect, DeepEval, Opik, Phoenix and Ragas behind them. First place carries a large caveat, covered in its own section. The ranking counts the Python tracing or eval client of each tool, the package your application imports to send a trace or run an evaluation. Below are the ten entries, the figure behind each, and what download counts leave out when you pick one.

How we built the ranking

Every figure here is the trailing 30-day download count from pypistats.org, fetched on 3 November 2026. The site builds its numbers from the Python Software Foundation’s public download data on Google BigQuery, and per its methodology page, “All aggregate download stats ignore known PyPI mirrors (such as bandersnatch) unless noted otherwise.” Nothing in that methodology removes CI pipeline downloads.

We count one package per tool, the client you install to instrument an app or run an eval. Where a vendor’s full package pulls its own tracing client in as a dependency, we count the client, because its number then covers both kinds of user. Tools that ship only on npm are out, since a JavaScript count and a Python count describe different populations. General LLM observability sits beside pure evaluation libraries, because in production the trace is the raw material of the eval. GitHub stars would be a useful second signal, but GitHub refused our requests on the day, so this ranking rests on downloads alone.

The last column is our proxy for momentum, not a growth rate: a package downloading at a steady pace would land near 23.3% (7 days of 30), so a higher share means the last week ran above the month’s average.

1. LangSmith comes with LangChain

langsmith recorded 77,233,437 downloads in the last 30 days, more than the next three entries combined. Most of that is inherited. The langchain-core package page lists langsmith among its dependencies, and langchain-core itself recorded 144.8 million downloads in the same window. Every team that installs LangChain gets the LangSmith SDK, whether or not it ever sends a trace.

Read first place as reach. The SDK already sits inside a very large share of Python LLM projects, so a LangChain team can turn tracing on without adding a package. The platform is the part you pay for, and hosting it yourself is an upgrade: LangSmith’s self-hosting page says “Self-hosted LangSmith is an add-on to the Enterprise plan designed for LangChain’s largest, most security-conscious customers.”

We are LangChain Ambassadors, so weigh our view of this entry accordingly. Two of our published builds, the Open Loyalty AI SDR and the Shen.ai patient education assistant, run on LangSmith traces.

2. Langfuse leads the tools LangChain does not install

langfuse recorded 23,563,200 downloads. It is not among the dependencies of langchain-core, the package behind LangSmith’s count, so we read its number as much closer to teams choosing it. That makes it, on our reading, the most installed tool on this list that a team picked on purpose.

The bigger news is ownership. Langfuse announced that ClickHouse has acquired it, and the same post says the product stays open source and self-hostable with no planned licensing changes. For a buyer that is the line that matters, because the self-hosted path is why teams with data residency rules shortlist it in the first place. Our Freedman International localization pipeline traced every model call in Langfuse with its inputs, outputs and cost, the same control we get from LangSmith on other builds. Whichever backend records it, the trace is the control that matters.

3. MLflow counts in three packages

MLflow’s tracing client, mlflow-tracing, recorded 14,843,852 downloads. The full mlflow package drew 19,887,934 and mlflow-skinny 25,001,641, but those three overlap: the full package pins both of the smaller ones, so every mlflow install is counted in all three. We rank the tracing client, the same rule we apply to Phoenix below. Ranked on mlflow-skinny instead, MLflow would sit second.

Its GenAI documentation now opens with the claim that “MLflow is the largest open source AI engineering platform for agents and LLMs,” and it is a Linux Foundation project, which matters to teams that want a vendor-neutral home for their traces. MLflow has tracked classic machine learning experiments for years, so we expect a share of even the tracing client’s downloads to come from model teams rather than LLM apps. If your data team already runs MLflow, its tracing and judges are the cheapest place to start, because there is nothing new to operate.

4 and 5: Logfire and Braintrust

logfire recorded 13,028,487 downloads, which will surprise anyone who still files it under general app monitoring. The Logfire docs describe it as Pydantic’s AI engineering platform and say it is built on OpenTelemetry, so the spans you already emit for your API and database sit in the same trace as your model calls. Its week share, 34.9%, is the highest in the top five.

braintrust recorded 6,574,708 downloads, the largest count for a commercial platform that is not tied to a framework. Read its deployment model before you sign. Braintrust’s self-hosting guide describes an option that keeps data storage in your environment, and adds that “Braintrust continues to host the control plane.” Your traces can stay in your cloud while the application that reads them stays in Braintrust’s.

6 to 10: the evaluation specialists

Below Braintrust the list changes character. These are tools you run to score outputs, more than tools you leave running in production.

Inspect is “a framework for frontier AI evaluations developed by the UK AI Security Institute and Meridian Labs”, and we read its count as safety and model teams more than app builders. DeepEval calls itself local-first, with evaluations running in your own environment, which is why it sits comfortably inside a CI job as an evaluation harness. Opik, Comet’s platform, has the highest week share on the whole list at 37.8%.

For Phoenix we count arize-phoenix-otel, the tracing client, because the full arize-phoenix server (626,210) lists it as a dependency. The Phoenix self-hosting page says “Phoenix is free to self-host with no feature limitations”. The server package itself is published under the Elastic License 2.0, per its PyPI metadata, a source-available license rather than an OSI one, which your legal team will want to know. Ragas closes the top ten on retrieval scoring; our post on enterprise RAG on LangChain covers why retrieval is where those scores usually fail.

Just outside: Evidently (823,148), Weave from Weights & Biases (817,064), OpenLIT (277,906), AgentOps (94,654) and TruLens (20,121).

What is the meaning of false positive rate?

In an eval, the false positive rate is the share of outputs that were actually fine which your evaluator flags as failures. Whichever tool runs it, an LLM-as-a-judge metric has a false positive rate whether or not the dashboard shows one. A judge that flags 10% of good answers as hallucinations fires on about a hundred of every thousand healthy conversations, and a team that wires that to an alert stops reading the alert within a week. Our glossary entry on the false positive covers the general definition and how it differs from precision.

Measuring it takes an afternoon. Hand label 50 to 100 real outputs into a small evaluation dataset, run the judge over them, and divide the good outputs it failed by all the good outputs. That number belongs next to every metric you report, and no tool can compute it for you without those labels. We wrote up the calibration loop for scoring agent output with an LLM judge in LangSmith, and it transfers to any tool here.

What the counts hide, and how to choose

Before you quote any of these numbers, discount them for the ways a download overstates use. Dependency inheritance is the largest: LangSmith’s count rides on LangChain, and Phoenix’s and MLflow’s clients ride on their own full packages, so none of them tells you how many teams open the traces. CI is the next: an eval library runs in a pipeline, and every uncached run downloads it again, so DeepEval and Ragas probably lean higher per user than a dashboard SDK. And a download says nothing about whether the tool stayed. A trace-based evaluation loop needs a tool you leave running, and the registry cannot separate the team that ships with it from the one that tried it for an afternoon.

For choosing, start with where the traces must live:

Then look at what you already run. LangChain teams have the LangSmith SDK installed, and a stack already on OpenTelemetry fits Logfire or Phoenix without new tracing code. Last, decide between online and offline evaluation. Scoring live traffic needs a platform; scoring a release candidate against a dataset needs only a library like DeepEval or Ragas in CI, and most production teams end up with one of each. The same registry method applied to agent frameworks, where LangChain also leads, is in the most used Python AI agent frameworks.

Our rule: pick the tracing backend first and the eval library second. A trace you can replay is what turns a vague complaint into an agent eval case, and no metric library can score a conversation nobody recorded.

The takeaway

LangSmith’s 77.2 million downloads say LangChain is everywhere; Langfuse’s 23.6 million are the strongest sign of deliberate adoption; Logfire and Opik had the strongest last week. None of the counts measures whether a judge is right, so whichever tool you pick, measure its false positive rate on your own labeled outputs before you trust its dashboard. Our LangSmith services page covers how we set up tracing and evals on client builds.

Sources

• pypistats.org methodology, Python Software Foundation download statistics, package figures fetched 3 November 2026
• langchain-core downloads and dependencies, pypistats.org
• Self-hosted LangSmith, LangChain documentation
• Langfuse joins ClickHouse, Langfuse blog
• MLflow for GenAI, MLflow documentation
• Logfire, get started, Pydantic documentation
• Braintrust self-hosting, Braintrust documentation
• Inspect, UK AI Security Institute
• DeepEval introduction, DeepEval documentation
• Opik documentation, Comet
• arize-phoenix downloads and dependencies, pypistats.org
• Phoenix self-hosting, Arize documentation