我正在尝试使用selenium phantomjs设置webscraping的代理.我正在使用
python.
我在很多地方都看到phantomjs中存在一个bug,因为proxy-auth不起作用.
from selenium.webdriver.common.proxy import *
from selenium import webdriver
from selenium.webdriver.common.by import By
service_args = [
'--proxy=http://fr.proxymesh.com:31280','--proxy-auth=USER:PWD','--proxy-type=http',]
driver = webdriver.PhantomJS(service_args=service_args)
driver.get("https://www.google.com")
print driver.page_source
代理网格建议使用以下代码：
page.customHeaders={‘Proxy-Authorization’: ‘Basic ‘+btoa(‘USERNAME:PASSWORD’)};
但我不知道如何将其转换为python.
这就是我目前拥有的：
from selenium import webdriver
import base64
from selenium.webdriver.common.proxy import *
from selenium import webdriver
from selenium.webdriver.common.by import By
service_args = [
'--proxy=http://fr.proxymesh.com:31280',]
headers = { 'Proxy-Authorization': 'Basic ' + base64.b64encode('USERNAME:PASSWORD')}
for key,value in enumerate(headers):
webdriver.DesiredCapabilities.PHANTOMJS['phantomjs.page.customHeaders.{}'.format(key)] = value
driver = webdriver.PhantomJS(service_args=service_args)
driver.get("https://www.google.com")
print driver.page_source
但它不起作用.
有关如何使其工作的任何建议？
解决方法@H_404_24@
我正在编译答案：
How to correctly pass basic auth (every click) using Selenium and phantomjs webdriver
以及：
base64.b64encode error
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import base64
service_args = [
'--proxy=http://fr.proxymesh.com:31280',]
authentication_token = "Basic " + base64.b64encode(b'username:password')
capa = DesiredCapabilities.PHANTOMJS
capa['phantomjs.page.customHeaders.Proxy-Authorization'] = authentication_token
driver = webdriver.PhantomJS(desired_capabilities=capa,service_args=service_args)
driver.get("http://...")
How to correctly pass basic auth (every click) using Selenium and phantomjs webdriver
以及：
base64.b64encode error
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import base64
service_args = [
'--proxy=http://fr.proxymesh.com:31280',]
authentication_token = "Basic " + base64.b64encode(b'username:password')
capa = DesiredCapabilities.PHANTOMJS
capa['phantomjs.page.customHeaders.Proxy-Authorization'] = authentication_token
driver = webdriver.PhantomJS(desired_capabilities=capa,service_args=service_args)
driver.get("http://...")