scrapinghub

Scrapinghub Joined Mar 2011 View on GitHub

Stars: 26,598; across 96 public repos
Rank: #1,363; of 94,827 organizations, top 2%

Repositories with 50+ stars

scrapinghub's repositories with their overall rank
Rank	Repository	Language	Stars
6,115	portia Visual scraping for Scrapy	Python	9,499
15,716	splash Lightweight, scriptable browser as a service with an HTTP API	Python	4,190
23,651	dateparser python parser for human readable dates	Python	2,864
51,099	frontera A scalable frontier for web crawlers	Python	1,337
53,282	slackbot A chat bot for Slack (https://slack.com).	Python	1,278
67,336	extruct Extract embedded metadata from HTML markup	Python	981
74,044	scrapyrt HTTP API for Scrapy spiders	Python	886
84,009	python-crfsuite A python binding for crfsuite	Python	775
112,680	spidermon Scrapy Extension for monitoring spiders execution.	Python	563
160,868	article-extraction-benchmark Article extraction benchmark: dataset and evaluation scripts	Python	377
171,824	price-parser Extract price amount and currency symbol from a raw text string	Python	350
222,849	webstruct NER toolkit for HTML data	HTML	259
271,442	python-scrapinghub A client interface for Scrapinghub's API	Python	204
275,963	adblockparser Python parser for Adblock Plus filters	Python	200
293,109	js2xml Convert Javascript code to an XML document	Python	186
295,733	testspiders Useful test spiders for Scrapy	Python	184
312,780	scrapy-training Scrapy Training companion code	Python	172
368,433	skinfer Skinfer is a tool for inferring and merging JSON schemas	Python	141
375,046	sample-projects Sample projects showcasing Scrapinghub tech	Python	138
391,044	number-parser Parse numbers written in natural language	Python	131
395,913	shub Scrapinghub Command Line Client	Python	129
395,913	scrapy-poet Page Object pattern for Scrapy	Python	129
400,896	python-simhash An efficient simhash implementation for python	C	127
449,446	mdr A python library detect and extract listing data from HTML page.	C	110
459,223	web-poet Web scraping Page Objects core library	Python	107
550,027	aile Automatic Item List Extraction	HTML	85
566,039	wappalyzer-python UNMAINTAINED Python wrapper for Wappalyzer (utility that uncovers the technologies used on websites)	Python	82
679,495	scrapinghub-stack-scrapy Software stack with latest Scrapy and updated deps	Dockerfile	65
742,413	scrapy-autounit Automatic unit test generation for Scrapy.	Python	58
742,413	scrapy-autoextract Zyte Automatic Extraction integration for Scrapy	Python	58
785,713	aduana Frontera backend to guide a crawl using PageRank, HITS or other ranking algorithms based on the link structure of the web graph, even when making big crawls (one billion pages).	C	54
809,287	learn.scrapinghub.com Scrapinghub Learning Center. Report issues in Jira: Report issues in Jira: https://scrapinghub.atlassian.net/projects/WEB	CSS	52
834,115	portia2code	Python	50

Among 94,827 organizations

25k–40k stars: 673 organizations, including scrapinghub

top 50%top 10%top 1%top 0.1%

Stars along the bottom, and organizations in each band on a log scale

Where the stars are

Languages

Python 25.1k stars, 94% · 26 repos
HTML 344 stars, 1% · 2 repos
C 291 stars, 1% · 3 repos
Dockerfile 65 stars, <1% · 1 repo
CSS 52 stars, <1% · 1 repo

When they were made

33 repos with 50+ stars, made between 2011 and 2020

101001k10k dateparsersplashportia

Quick stats

Next rank: +70 stars to reach #1,362
Next milestone: +8,396 stars to reach the top 1%
Public repos: 96, 33 with 50+ stars
Popular repos: 25 over 100 stars, 5 over 1k stars
Top repo: portia, #6,115 of all repos, 36% of stars
Smaller repos: 777 stars from repos under 50