当图书馆不为您处理请求时如何从请求中获取特定的 cookie
how to get specific cookies from requests when the library doesn't handle them for you
所以我有这个页面，我需要经常登录，但如果我不在 header 中使用一些 cookie，我什至无法登录该页面。
问题是如果我使用邮递员给我的 cookie，它们最终会过期，我必须在代码中再次替换它们。
#####this makes the login
url = "https://www.APAGE.com"#login url
payload = "user="+dude["user"]+"&password="+dude["password"]+"&action=login"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #for some reason i need this or the page returns error
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cookie': "ASP.NET_SessionId=<an usually expirable cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<another expirable cookie>",#i need THESE!
'Cache-Control': "no-cache",
}
login = session.post(url, data=payload, headers=headers)# makes the login
print "Login open"
cookie = session.cookies.get_dict() #get the recursive cookie
#this here is me trying to grab the request-cookies just after the login so i can repass them so they don't expire
print '================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print login.headers
print cookie2
print login.cookies.get_dict()
#makes a get request to change to the initial page
url = "www.APAGE-after-login.com"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Referer': "www.APAGE-after-login.com",
'Cookie': "ASP.NET_SessionId=<the cookie again>.; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<the other cookie>; .ASPXAUTH="+str(cookie['.ASPXAUTH']), #here i need to repost the .ASPAUTH cookie every time after a request or the session expires
'Upgrade-Insecure-Requests': "1",
'Cache-Control': "no-cache",
}
moving = session.get(url,headers=headers)
cookie = session.cookies.get_dict()
我在这里需要帮助来获取这些 cookie，这样当它们发生变化时，我就不必一次又一次地更改整个代码部分。
有谁知道我如何拦截那些 request-cookies 以便我可以使用它们？
谢谢！
编辑：我已经在代码中声明了 session = requests.session() 并且我已经尝试了几种解决方案来解决这个问题...如果我手动放置代码就可以工作headers 上的 cookie 但 cookie 将 几天后过期...请求库出于某种原因未自动处理 cookie...
如果我使用这个 header:
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Cookie': .ASPXAUTH="+str(cookie['.ASPXAUTH']),
'Cache-Control': "no-cache",
}
或任何其他变体如
moving = session.get(url,headers=headers,cookies=cookie) #cookie that i tried to get before
登录根本不起作用。它 returns 一个错误页面。
谢谢
编辑2：
for customer in customers:
session = requests.session()
##create a folder
if not os.path.exists("C:\Users\Desktop\customers\" + customer["dir"] + "/page"):
os.makedirs("C:\Users\Desktop\customers\" + customer["dir"] + "/page", 0755)
search_date= datetime.datetime.now().strftime("%d-%m-%Y-%H-%M-%S")
search_date_end= (datetime.datetime.now() - timedelta(days = 30)).strftime("%d/%m/%Y")
search_date_begining= (datetime.datetime.now() - timedelta(days = 30)).strftime("%d/%m/%Y")
search_date_closing= (datetime.datetime.now() - timedelta(days = 45)).strftime("%d/%m/%Y")
search_date_closing= urllib.quote_plus(data_busca_fechamento)
search_date_begining= urllib.quote_plus(data_busca_inicio)
search_date_end= urllib.quote_plus(data_busca_fim)
print str(search_date_end)
######makes the login
url = "www.ASITE.com/aunthenticate/APAGELogin" #login
payload = "user="+customer["user"]+"&password="+customer["pass"]+"&action=login"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #for some reason i need this or the login breaks
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cookie': "ASP.NET_SessionId=<some cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<part1cookie>-<part2cookie>-<part3cookie>", #i need to get these cookies to login, for some reason i cant get them by any means
'Cache-Control': "no-cache",
}
login = session.post(url, data=payload, headers=headers)#open the login session on the page
print "Login session open"
cookie = session.cookies.get_dict() #when i get this cookie i only get the recursive cookie '.ASPXAUTH' that i need to get again every request or the session expires
print login.text
#The response has only one line with some site data confirming the login
#if the login fails it returns an HTML with the error message
#here i try to get the request cookies and not the response ones, but the headers dont return any cookies at all
print '================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print login.headers
print cookie2
print login.cookies.get_dict() #this cookie is returned, but just the '.ASPXAUTH' one, the one i already know how to get
#makes the get request to the index page
url = "www.ASITE/index/home"
headers = {
'Accept': "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #again i need to pass the user-agent
'Accept-Language': "pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3",
'Cookie': "ASP.NET_SessionId=<a cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<other long cookie>; .ASPXAUTH="+str(cookie['.ASPXAUTH']), #here i need to start passing the recursive cookie again and again every request on the site
'Upgrade-Insecure-Requests': "1",
'Cache-Control': "no-cache",
}
moving = session.get(url,headers=headers)
cookie = session.cookies.get_dict() #get the '.ASPXAUTH' again
这里的问题是，如果我手动设置丢失的 cookie，代码 将 工作几天，但是当它们过期时或者如果另一台机器使用我必须的代码再次手动设置它们。
通过这种方式，我尝试了几种方法来在请求之前获取另外两个 cookie，none 确实有效，并且由于某种原因，'requests' 库 不是 自动处理它们......老实说，我不知道该怎么做了。
代码开始运行。好消息是现在的代码以正确的方式获取 cookie；坏消息是我完全不知道那是怎么发生的。
我唯一添加的是这段代码：（问题是我昨天添加了它，然后它不起作用......现在它起作用了）。
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cache-Control': "no-cache",
}
url = "www.asite.com/login" #login page
login = session.get(url, headers=headers) #login get
print 'login.request.headers================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print 'login headers ============================='
print login.headers
headers = login.headers
print '\n\n\n'
print 'login.headers.get(''Set-Cookie'') ================================'
print cookie2
print '\n\n\n'
print "login.cookies.get_dict() ========================="
test = login.cookies.get_dict()
print test
print '\n\n\n'
昨天 login.cookies.get_dict() 刚刚返回空字典或 none 或者，如果放置在登录之后，则只返回递归 cookie...现在...它正在工作。
所以我有这个页面，我需要经常登录，但如果我不在 header 中使用一些 cookie，我什至无法登录该页面。
问题是如果我使用邮递员给我的 cookie，它们最终会过期，我必须在代码中再次替换它们。
#####this makes the login
url = "https://www.APAGE.com"#login url
payload = "user="+dude["user"]+"&password="+dude["password"]+"&action=login"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #for some reason i need this or the page returns error
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cookie': "ASP.NET_SessionId=<an usually expirable cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<another expirable cookie>",#i need THESE!
'Cache-Control': "no-cache",
}
login = session.post(url, data=payload, headers=headers)# makes the login
print "Login open"
cookie = session.cookies.get_dict() #get the recursive cookie
#this here is me trying to grab the request-cookies just after the login so i can repass them so they don't expire
print '================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print login.headers
print cookie2
print login.cookies.get_dict()
#makes a get request to change to the initial page
url = "www.APAGE-after-login.com"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Referer': "www.APAGE-after-login.com",
'Cookie': "ASP.NET_SessionId=<the cookie again>.; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<the other cookie>; .ASPXAUTH="+str(cookie['.ASPXAUTH']), #here i need to repost the .ASPAUTH cookie every time after a request or the session expires
'Upgrade-Insecure-Requests': "1",
'Cache-Control': "no-cache",
}
moving = session.get(url,headers=headers)
cookie = session.cookies.get_dict()
我在这里需要帮助来获取这些 cookie，这样当它们发生变化时，我就不必一次又一次地更改整个代码部分。 有谁知道我如何拦截那些 request-cookies 以便我可以使用它们？ 谢谢！
编辑：我已经在代码中声明了 session = requests.session() 并且我已经尝试了几种解决方案来解决这个问题...如果我手动放置代码就可以工作headers 上的 cookie 但 cookie 将 几天后过期...请求库出于某种原因未自动处理 cookie...
如果我使用这个 header:
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Cookie': .ASPXAUTH="+str(cookie['.ASPXAUTH']),
'Cache-Control': "no-cache",
}
或任何其他变体如
moving = session.get(url,headers=headers,cookies=cookie) #cookie that i tried to get before
登录根本不起作用。它 returns 一个错误页面。
谢谢
编辑2：
for customer in customers:
session = requests.session()
##create a folder
if not os.path.exists("C:\Users\Desktop\customers\" + customer["dir"] + "/page"):
os.makedirs("C:\Users\Desktop\customers\" + customer["dir"] + "/page", 0755)
search_date= datetime.datetime.now().strftime("%d-%m-%Y-%H-%M-%S")
search_date_end= (datetime.datetime.now() - timedelta(days = 30)).strftime("%d/%m/%Y")
search_date_begining= (datetime.datetime.now() - timedelta(days = 30)).strftime("%d/%m/%Y")
search_date_closing= (datetime.datetime.now() - timedelta(days = 45)).strftime("%d/%m/%Y")
search_date_closing= urllib.quote_plus(data_busca_fechamento)
search_date_begining= urllib.quote_plus(data_busca_inicio)
search_date_end= urllib.quote_plus(data_busca_fim)
print str(search_date_end)
######makes the login
url = "www.ASITE.com/aunthenticate/APAGELogin" #login
payload = "user="+customer["user"]+"&password="+customer["pass"]+"&action=login"
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #for some reason i need this or the login breaks
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cookie': "ASP.NET_SessionId=<some cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<part1cookie>-<part2cookie>-<part3cookie>", #i need to get these cookies to login, for some reason i cant get them by any means
'Cache-Control': "no-cache",
}
login = session.post(url, data=payload, headers=headers)#open the login session on the page
print "Login session open"
cookie = session.cookies.get_dict() #when i get this cookie i only get the recursive cookie '.ASPXAUTH' that i need to get again every request or the session expires
print login.text
#The response has only one line with some site data confirming the login
#if the login fails it returns an HTML with the error message
#here i try to get the request cookies and not the response ones, but the headers dont return any cookies at all
print '================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print login.headers
print cookie2
print login.cookies.get_dict() #this cookie is returned, but just the '.ASPXAUTH' one, the one i already know how to get
#makes the get request to the index page
url = "www.ASITE/index/home"
headers = {
'Accept': "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0", #again i need to pass the user-agent
'Accept-Language': "pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3",
'Cookie': "ASP.NET_SessionId=<a cookie>; __RequestVerificationToken_L0NvbnN1bHRhV2Vi0=<other long cookie>; .ASPXAUTH="+str(cookie['.ASPXAUTH']), #here i need to start passing the recursive cookie again and again every request on the site
'Upgrade-Insecure-Requests': "1",
'Cache-Control': "no-cache",
}
moving = session.get(url,headers=headers)
cookie = session.cookies.get_dict() #get the '.ASPXAUTH' again
这里的问题是，如果我手动设置丢失的 cookie，代码 将 工作几天，但是当它们过期时或者如果另一台机器使用我必须的代码再次手动设置它们。 通过这种方式，我尝试了几种方法来在请求之前获取另外两个 cookie，none 确实有效，并且由于某种原因，'requests' 库 不是 自动处理它们......老实说，我不知道该怎么做了。
代码开始运行。好消息是现在的代码以正确的方式获取 cookie；坏消息是我完全不知道那是怎么发生的。
我唯一添加的是这段代码：（问题是我昨天添加了它，然后它不起作用......现在它起作用了）。
headers = {
'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0",
'Content-Type': "application/x-www-form-urlencoded",
'X-Requested-With': "XMLHttpRequest",
'Cache-Control': "no-cache",
}
url = "www.asite.com/login" #login page
login = session.get(url, headers=headers) #login get
print 'login.request.headers================'
print login.request.headers
print '================'
print '\n\n\n'
cookie2 = login.headers.get('Set-Cookie')
print 'login headers ============================='
print login.headers
headers = login.headers
print '\n\n\n'
print 'login.headers.get(''Set-Cookie'') ================================'
print cookie2
print '\n\n\n'
print "login.cookies.get_dict() ========================="
test = login.cookies.get_dict()
print test
print '\n\n\n'
昨天 login.cookies.get_dict() 刚刚返回空字典或 none 或者，如果放置在登录之后，则只返回递归 cookie...现在...它正在工作。