So I used Proxy Broker to scrape some proxies. Sometimes the proxies are dead when scraped so I wanted to check them before I used them. So I wrote a program using Python Requests to check them. Here it is:
import time
import random
import requests
lines = open('not_checked.txt').read().splitlines()
check =random.choice(lines)
yaya = {
check
}
for x in range(0 , 10):
requests.get('https://reg.ebay.com/reg/PartialReg?ru=https%3A%2F%2Fwww.ebay.com%2F': proxies=yaya)
r.status_code
print(status_code)
if status_code == 200:
f=open("checked_proxies.txt", "a+")
f.write(proxies)
else:
time.sleep(.001)
However this throws back the "set object has no attribute get". I looked the error up online and the it said it is because I used a comma instead of a colon. So then I tried:
requests.get('https://reg.ebay.com/reg/PartialReg?ru=https%3A%2F%2Fwww.ebay.com%2F': proxies=yaya)
to get a syntax error. What is going on?
proxies
needs to be a dict. It's right there in the docs:
proxies = {
'http': 'http://10.10.1.10:3128',
'https': 'http://10.10.1.10:1080',
}
Your yaya
is a set
not dict
.