How to VPN/Proxy connect in Python?

Liam Flynn picture Liam Flynn · Feb 2, 2016 · Viewed 51.2k times · Source

I'm trying to scrape some pages that are on a website but to view the pages, I need to be connected to a VPN. My setup is as follows:

I want to run a script on pythonanywhere that connects through the VPN so that the traffic appears to be coming from Australia.

The closest answer I have found so far is:

JSoup over VPN/proxy

Answer

Alexey Smirnov picture Alexey Smirnov · Nov 24, 2016

I see that https://www.privateinternetaccess.com/ has option to use SOCKS5 proxy. If you are using requests module for scraping you may use SOCKS5 like that:

pip install -U requests[socks]

and in the script:

import requests
proxies = {'http': 'socks5://user:pass@host:port',
           'https': 'socks5://user:pass@host:port'}

resp = requests.get('http://example.com', proxies=proxies )