Plurrrr

a tumblelog
Sun 14 Jul 2019

Avoiding Webscraping Throttling Using Python and Tor as a Proxy

I do not condone the use of this information for creating illegal web crawlers. This was more an informational exercise and I wanted to share it with others. Another thing to note is that some sites are able to automatically block IP’s that are Tor exit nodes, so this may not work for some sites that go to these measures.

Source: Avoiding Webscraping Throttling Using Python and Tor as a Proxy.

I've used Tor as a proxy several times in the past for web scraping projects so I read this article with interest.