Colly
Colly is a library of web browsing and data extraction with a clean interface written in the Go language. It offers speeds that can exceed a thousand requests per second in a single core; it supports simultaneous, synchronizing, or asynchronous scanning operations. It features built-in features such as request delay management per domain name, automated cookie processing, caching, and robots.txt support.
Colly
100gocolly · Web Scraping & Automation
Common commercial products in this category: Bright Data, Apify, ScrapingBee
- Extract structured data from websites for data mining, data processing and archiving
- Manages maximum concurrentness and request delays per domain name and conducts scanning operations
- With automatic cookie and session management, you configure the crawler by encoding non-unicode responses.
- ↑ PositiveHacker News
Colly – Scraping Framework for Golang
“Please breakup your main `colly.go` file into separate parts. If possible you shouldn't have a 30 line imports definition covering everything from cookies and regex to html and sync access. Make sure to use DNS caching on the box else add it in Go. Colly only supports a single…”
- • NeutralHacker News
Colly – Scraping Framework for Golang
“Good performance matters if you have decent networking infrastructure or your server has limited resources. Bandwidth and IP limits are the most common bottle necks, but these can be solved using multiple proxies and ssh tunnels. Colly has built in support for switching proxies…”
Verified against README · Oct 26, 2026 · source