Previous: WebArena Rank #91 of 300 Skills Next: FastHTML

SWE Arena Rank #91

AutoResearch

karpathy/autoresearch · Author: @karpathy

View on GitHub

Arena ELO

1284

±21

Total Stars

97.4k

+155% w/w

Monthly Traffic

1.9M/mo

2.2x vs median

Search Demand

72,000/mo

+145% YoY

Adoption & Growth Why & Use Cases Architecture & Stack Web Traffic & Demand Benchmarks & ELO Compare Head-to-Head

12-Month Adoption & Star Velocity +155% w/w

Empirical star trajectory for AutoResearch vs Category Median benchmark. Hover along points to inspect exact monthly stats.

AutoResearch Category Median

GROWTH VELOCITY

+155%

2.5x vs category median

ARENA ELO SCORE

1284

+92 vs category median

WEB VISITS MOMENTUM

1.9M/mo

2.2x category median

LATENCY EFFICIENCY

48ms

0.8x faster execution

Ecosystem Adoption Thesis

Across verified open-source agentic tools, AutoResearch holds a position in the top percentile for developer retention and production velocity. Its weekly surge rate of +155% signals sustained real-world adoption rather than speculative hype.

Why Teams & Autonomous Agents Choose AutoResearch

Andrej Karpathy's autonomous agent loop that proposes, tests, verifies, and iterates on research hypotheses and codebases.

Verified Real-World Production Workflow

Primary Implementation:

Automate overnight hyperparameter sweeps, test-driven refactorings, and benchmark verification loops without human intervention.

Engine Stack & Dependencies:

Python, Minimalist LLM API harness, Git diff parser.

Target Persona & Role Fit

Staff AI Engineers

Engineered and benchmarked specifically for Staff AI Engineers demanding deterministic execution, low token overhead, and production reliability in agentic loops.

Production Blueprint & Installation

git clone https://github.com/karpathy/autoresearch

Technical Specification (ASD-STE100)

Andrej Karpathy's autonomous agent loop that proposes, tests, verifies, and iterates on research hypotheses and codebases.

Architecture: Python, Minimalist LLM API harness, Git diff parser.

Domain Tags & Keywords

#karpathy#autonomous-coding#research-loop#minimalist

Compute Efficiency Profile

P95 EXECUTION LATENCY

48ms

0.8x faster than median

TOKEN EFFICIENCY SAVINGS

-95%

Measured via context pruning

HEAD-TO-HEAD WIN RATE

85%

Arena paired matches

Monthly Documentation & Site Visits

1.9M/mo

Measured via Traffic Research bypass engine (2.2x category median)

Google Search Keyword Demand

72,000/mo

+145% YoY expansion

6-Month Web Traffic Velocity

Traffic momentum vs Category Median (850k visits/mo benchmark).

AutoResearch Median

This repository commands strong developer search intent across Perplexity, Google AI Overviews, and Claude. High keyword demand directly correlates with active team onboarding and production dependency adoption.

Arena ELO Rating Stability

Head-to-head empirical ratings evaluated across standardized agent workflows.

1284

±21 CI

WIN RATE

85%

Head-to-head

WEEKLY SURGE

+155%

Adoption velocity

P95 LATENCY

48ms

Execution speed

TOKEN OVERHEAD

-95%

Context saved

Head-to-Head Comparison — AutoResearch vs 300 Skills

Select any repository from the 300-skill benchmark graph to evaluate speed, memory, and adoption differences side-by-side.

Compare with:

Popular Direct Comparisons in Category:

Previous: WebArena Return to Arena Leaderboard Next: FastHTML