I'm trying to scrape some pages that are on a website but to view the pages, I need to be connected to a VPN. My setup is as follows:
I want to run a script on pythonanywhere that connects through the VPN so that the traffic appears to be coming from Australia.
The closest answer I have found so far is:
I see that https://www.privateinternetaccess.com/ has option to use SOCKS5 proxy. If you are using requests module for scraping you may use SOCKS5 like that:
pip install -U requests[socks]
and in the script:
import requests
proxies = {'http': 'socks5://user:pass@host:port',
'https': 'socks5://user:pass@host:port'}
resp = requests.get('http://example.com', proxies=proxies )