Everything / Browser and automation / Web scraping
Web scraping Growing
On TutorHero's Everything Index, Web scraping is growing on GitHub: 1.64 of every 10,000 new public repos over the last three months (1.5k a month), +283% vs a year ago. The 12-month forecast says rising (+53%). By the end of 2032, lines in the same state ended between 0.01x and 2.5x of where they started (median 0.54x).
Since 2019, and the range to 2032
New repos per 10,000 (3-month average, log scale)12-month forecastRange to 2032: middle half and 90% of past similar lines
The range is not a forecast of this line: it is what GitHub lines in the same state (more than doubled its share of new repos in a year) did 1 to 5 years later, 2019 to 2026, applied to today's level. Past 5 years it is stretched, not measured. Checked out of time (learned before 2022, scored after), 71% of outcomes landed inside the 90% range, so read it as wide. Open it in the Everything chart.
Week by week
Wikipedia views a day, last 8 weeks
New public repos with the web-scraping tag each week (Monday to Sunday, UTC). The outlined bars, last week included, are still filling in, as tags are often added days after a repo is created, so their height will grow a little. "Against the field" divides the change by what the typical tag did, so a wave that lifts every tag does not count. Hover for the numbers.
Built vs talked about
Each line is its own share (of all new GitHub repos, all Hacker News stories, all English Wikipedia views, all new stars on the repos we track), shown as a share of its own peak, so the timing and shape compare even though the units differ. Hover for the counts. Tap a name to hide or show a line.
What it means
- Worth learning now: Web scraping is growing, +283% share of new GitHub repos vs a year ago.
- Start by reading firecrawl/firecrawl (+13.2k stars in 3 months) and D4Vinci/Scrapling (+7.4k stars in 3 months).
- People to follow: ScrapeGraphAI, Apify, Evil0ctal.
- Builders and attention both rising: new repos +283%, attention +25% in a year.
- 10.2 new stars per new repo over 3 months, vs 8.50 for the typical trend: about average demand per project.
- The 12-month forecast for its share: +53% (Rising).
- Talked about more than built: Wikipedia 4x vs new repos +283% in a year, more talk per builder than the typical trend, so demand for builders may run ahead of supply.
- History: of past GitHub lines that more than doubled its share of new repos in a year, 61% had a smaller share 3 to 5 years later and 8% at least doubled.
- Attention: 3.15% of new stars on the repos we track, +25% in a year.
- Hacker News: 2 stories a month, -43% share in a year; Wikipedia 51.6k views a month, 4x.
- Not investment advice: this measures what people build on GitHub, not any company.
- Most-starred tools here lately: firecrawl/firecrawl (vouched), D4Vinci/Scrapling (vouched).
- The field is growing, so expect new tools and fast changes; pin versions.
Repos carrying it now
People building it
ScrapeGraphAIAI-powered web scraping tools and workflowsInfluence 52
ApifyPut the web to work.Influence 67
Evil0ctalEat more vegetable 🥬Influence 63
MaxunTurning websites into structured APIs for extraction and automation.Influence 38
lorienlorienInfluence 38
Mark CliTurning websites into compact CLIs for AI agents.Influence 29
GermeySoftware Engineer, majored in Web Scraping, Web Development, Machine Learning.Influence 72
h4ckf0r0dayBuilds headless browsers and AI web scraping tools.Influence 38More in Browser and automation
Questions
Is Web scraping growing on GitHub?
Yes. Its share of all new public GitHub repos is +283% vs a year ago and -2% over the last three months (Sep 2026). Phase: Growing.
How many new Web scraping repos are created each month?
About 1.5k new public repos a month carry the web-scraping tag (three-month average to Sep 2026), or 1.64 of every 10,000 new repos.
What will Web scraping look like in 2032?
Nobody knows, and we say how much. Of past GitHub lines that more than doubled its share of new repos in a year, the middle half had between 0.14x and 1.27x of their share 3 to 5 years later, and 61% ended smaller.
Is Web scraping more talked about than built?
Talked about more than built. Over the year its share of Wikipedia views moved 4x while its share of new GitHub repos moved +283%. New repos grew faster than talk for most AI trends this year, so the label compares it with the typical trend: 3.8 times the talk per builder of the median trend.
Not investment advice. Measurements of what people build and star on GitHub, a statistical forecast, base rates and a model's reading of them. Nothing here is about any company's revenue, valuation or stock.