People / Topics / Evals and observability
Topic map · 2026-10-18
Who to follow in Evals and observability
LLM evals, tracing, observability, guardrails and benchmarks. 11 people and 28 labs placed here by the topic tags on their own repos (or a Jev read of their repos when they use no tags), ranked by stars across their repos, shipping and influence.
People1128 labs and companies
Stars, 7 days+24.9k+72.6k in 30 days
Links inside0follows, mentions, pull requests
Top Influence94rUv
Rising this week+10.5kMatt Pocock
Is Evals and observability heating up? All topics on Pulse
Attention: new stars on 243 repos in this topic. Building: new repos created with llm-evaluation, observability, llmops. Each window is compared with the window before it; "vs field" compares it with all 23 topics together. Measured through 2026-10-18.
Skills rising here
Skills from the TutorHero catalog whose repos carry this topic's tags, by measured momentum.
DeepEvalMeasure hallucination rate, answer relevancy, G-Eval score, and context recall with deterministic pass/fail thresholds in unit tests.+171/wkLangfuseInspect detailed step-by-step traces of multi-agent runs, latency bottlenecks, and token spend.+296/wkArize PhoenixIntegrate Arize Phoenix into agentic loops for verified deterministic execution and token efficiency.+93/wkPhoenixPinpoint RAG retrieval failures and hallucinated agent tool parameters with OpenTelemetry-standard traces.+93/wkLiteLLMPrevent downtime by automatically failing over from Anthropic Claude to Gemini 2.5 Pro or DeepSeek if rate limits or 500 errors hit.+458/wkLiteLLM ProxyUsed by modern agentic teams to scale openai-proxy infrastructure with verified provenance.+458/wkPromptfooAutomated assertion testing of agent responses across adversarial prompts in CI/CD.+192/wkFlyteUsed by modern agentic teams to scale large-scale-orchestration infrastructure with verified provenance.+43/wkNew repos to watch
Created in the last 90 days, by stars in the last 30, with their trust check.
usamahz/cpu-performance-engineeringA reading path for CPU performance engineering, from one instruction to production inference. Primary sources only, with a runnable benchmark for every section.66 · New, no references yet+1.2kjwuthri-cloud/Tracely-aiTrace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.59 · New, no references yet+1.0kv-modal/awesome-jev-toolsA curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions.54 · New, no references yet+832lennney/stop-that-shitStop That Shit（别再造史了）｜面向 Codex/GPT 场景的多平台 Hook + Skill Guard：拦截 AI coding agent 无需求的哈希、校验和与任务范围膨胀。 A multi-platform Hook + Skill Guard for AI coding agents in Codex/GPT workflows: stop unrequested has43 · New, no references yet+672superwesleyhys-ux/factcircuitFactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.68 · New, no references yet+591xinxuxin/keystone-benchA clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence. Built on HealthBench.42 · New, no references yet+583Who pays attention to whom in Evals and observability See it on the full map
Bridges to other topics
Accounts here with the most links into other topics.
Matt Pocock25 links to Learning and lists, AI agents, Web and mobile apps, Models and research, Coding agents, Video and animation, Developer tools, Self-hosted apps, RAG and memory, Voice and audio
Berri AI7 links to AI agents, Models and research, MCP servers, Local LLMs and inference, Developer tools, Security
Pydantic7 links to Browser and automation, Local LLMs and inference, Developer tools, AI agents, Voice and audio
rUv9 links to AI agents, Developer tools, Coding agents, Learning and lists
Alexander F. Rødseth4 links to Self-hosted apps, Developer tools, Web and mobile apps, RAG and memory
Langfuse4 links to Local LLMs and inference, AI agents, RAG and memory
Arize AI5 links to Browser and automation, AI agents
calmrocks3 links to Learning and lists, Prompt libraries
AgentOps2 links to Learning and lists, RAG and memory
Mason2 links to Finance and crypto, AI agents
OpenStatus2 links to Web and mobile apps
RediSearch2 links to Self-hosted appsRepos driving it this week
Measured repos of these accounts tagged with this topic, by stars in the last 7 days.
ruvnet/RuViewπ RuView turns commodity WiFi signals into real-time spatial intelligence, vital sign monitoring, and presence detectio…+1.4kBerriAI/litellmThe fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tr…+587langfuse/langfuse🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.+367
promptfoo/promptfooTest your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT,…+234
confident-ai/deepevalThe LLM Evaluation Framework+198
whitecircle/riseRISE: Red-teaming via Iterative Strategy Evolution (NeurIPS 2026)+131Arize-ai/phoenixAI Observability & Evaluation+116
SigNoz/signozSigNoz is an open-source, OpenTelemetry-native observability platform for your team and their AI agents. Get logs, metr…+64
Portkey-AI/gatewayA blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly AP…+55
flyteorg/flyteType-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with…+51Other topics
AI agents186 people · 201 labs
Coding agents131 people · 73 labs
MCP servers52 people · 29 labs
Models and research98 people · 95 labs
Local LLMs and inference69 people · 67 labs
RAG and memory43 people · 51 labs
Image generation42 people · 23 labs
Video and animation25 people · 11 labs
Voice and audio41 people · 29 labs
Browser and automation41 people · 36 labs
Vision and robotics32 people · 49 labs
Developer tools238 people · 150 labs
Self-hosted apps114 people · 58 labs
Learning and lists58 people · 21 labs
Prompt libraries13 people · 7 labs
Agent skills and rules36 people · 18 labs
UI and UX design14 people · 15 labs
Security16 people · 6 labs
Data and databases30 people · 51 labs
Cloud and DevOps18 people · 30 labs
Web and mobile apps86 people · 77 labs
Finance and crypto14 people · 6 labs
Coding agents131 people · 73 labs
MCP servers52 people · 29 labs
Models and research98 people · 95 labs
Local LLMs and inference69 people · 67 labs
RAG and memory43 people · 51 labs
Image generation42 people · 23 labs
Video and animation25 people · 11 labs
Voice and audio41 people · 29 labs
Browser and automation41 people · 36 labs
Vision and robotics32 people · 49 labs
Developer tools238 people · 150 labs
Self-hosted apps114 people · 58 labs
Learning and lists58 people · 21 labs
Prompt libraries13 people · 7 labs
Agent skills and rules36 people · 18 labs
UI and UX design14 people · 15 labs
Security16 people · 6 labs
Data and databases30 people · 51 labs
Cloud and DevOps18 people · 30 labs
Web and mobile apps86 people · 77 labs
Finance and crypto14 people · 6 labs