Trendingby GlyphLLMOps and evals
Peak · emerged March 2023 · high July 2026 · -12% on the month before · 19 mentions on Trending in 14 days
Measuring, tracing and monitoring model-backed software: eval suites, LLM observability and prompt management. Named by OpenAI: OpenAI Evals: a framework for evaluating LLMs (Mar 14, 2023). Also called LLMOps, LLM evals, Evals, LLM observability, LLM evaluation, LLM-as-a-judge.
Index 0 to 100, monthly
Signals
Hacker News stories, last 12 months244 stories, +63% on the 12 months before
arXiv papers, last 12 months1,409 papers, +64% on the 12 months before
Product releases naming it79 since Feb 19, 2026GitHub repositories4,838Conferences about it1| Signal | Reads as | |
|---|---|---|
| Hacker News stories, last 12 months244 stories, +63% on the 12 months before | 244 stories, +63% on the 12 months before | Adoption |
| arXiv papers, last 12 months1,409 papers, +64% on the 12 months before | 1,409 papers, +64% on the 12 months before | Adoption |
| Product releases naming it79 since Feb 19, 2026 | 79 since Feb 19, 2026 | Adoption |
| GitHub repositories4,838 | 4,838 | Adoption |
| Conferences about it1 | 1 | Adoption |
Companies
| Name | Mentions |
|---|---|
| 7 | |
| 3 | |
| 2 | |
| 1 | |
| 1 | |
| 1 | |
| 1 | |
| 1 |
Releases
79 releases since Feb 19, 2026| Name | |
|---|---|
| Oct 26, 20262 weeks ago | |
| Oct 20, 20263 weeks ago | |
| Oct 15, 20263 weeks ago | |
| Oct 11, 20264 weeks ago | |
| Oct 9, 20264 weeks ago | |
| Oct 9, 20264 weeks ago | |
| Oct 6, 20265 weeks ago | |
| Oct 2, 20265 weeks ago |
GitHub projects
4,838 repositoriesconfident-ai/deepevalThe LLM Evaluation Frameworkopen-compass/opencompassOpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, acroshuggingface/evaluation-guidebookSharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and tjunlp-lab/Awesome-LLMs-Evaluation-PapersThe papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey.allenai/olmesReproducible, flexible LLM evaluationsGPT-Fathom/GPT-FathomGPT-Fathom is an open-source and reproducible LLM evaluation suite, benchmarking 10+ leading open-source and closed-source LLMs as well as O
| Name | Stars | |
|---|---|---|
| confident-ai/deepevalThe LLM Evaluation Framework · Aug 10, 2023 3 years ago | Aug 10, 20233 years ago | 19,216 |
| open-compass/opencompassOpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, acros · Jun 15, 2023 3 years ago | Jun 15, 20233 years ago | 7,560 |
| huggingface/evaluation-guidebookSharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and · Oct 9, 2024 2 years ago | Oct 9, 20242 years ago | 2,163 |
| tjunlp-lab/Awesome-LLMs-Evaluation-PapersThe papers are organized according to our survey: Evaluating Large Language Models: A Comprehensive Survey. · Oct 29, 2023 3 years ago | Oct 29, 20233 years ago | 811 |
| allenai/olmesReproducible, flexible LLM evaluations · Nov 2, 2024 2 years ago | Nov 2, 20242 years ago | 403 |
| GPT-Fathom/GPT-FathomGPT-Fathom is an open-source and reproducible LLM evaluation suite, benchmarking 10+ leading open-source and closed-source LLMs as well as O · Sep 28, 2023 3 years ago | Sep 28, 20233 years ago | 345 |
Partnerships
| Name | |
|---|---|
| Apr 9, 20251 year ago |
Conferences
1 conference| Name | |
|---|---|
| AI Engineering Summit Denver 2026Denver, CO · Oct 28, 2026 13 days ago | Oct 28, 202613 days ago |
Related movements
AI agentsModels that plan and act in steps, calling tools and other services until a task is done.Synthetic dataTraining and evaluation data generated by models or simulators instead of collected from people.Context engineeringDeciding what goes into a model's context window, and in what order: instructions, retrieved documents, tools and memory.
| Movement | Stage |
|---|---|
| AI agentsModels that plan and act in steps, calling tools and other services until a task is done. | Rising |
| Synthetic dataTraining and evaluation data generated by models or simulators instead of collected from people. | Plateau |
| Context engineeringDeciding what goes into a model's context window, and in what order: instructions, retrieved documents, tools and memory. | Plateau |
History
Tracking started: emerged March 2023, peak, 6 other names recorded
| Date | What changed |
|---|---|
| Sep 24, 2026 | Tracking started: emerged March 2023, peak, 6 other names recorded |
Sources: Curve: Hacker News story titles, Wikipedia pageviews and arXiv papers by month, refreshed Nov 1, 2026. Related entities from the other Glyph Ibis sources' public APIs, matched by name. How stages work.