Python 监控网站的变化

Question

我想登录网站，获取数据，将其保存到文件中，一段时间后获取新数据并将其与旧（已保存）数据进行比较，如果有变化则打印出来。我怎么做？登录有效，但比较无效。为什么？

提前致谢！

我的代码：

# -*- coding: utf-8 -*-
import urllib
import urllib2
import cookielib
import time


def login():
    username = "username"
    password = "password"

    cj = cookielib.CookieJar()
    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
    login_data = urllib.urlencode({'login_username' : username, 'login_password' : password})
    opener.open('lol.com/login', login_data)
    resp = opener.open('lol.com/login')
    data = resp.read()
    print data    
    write_data(data)

def write_data(data):
    file = open("htmlString", "w")
    file.write(data)
    file.close()
    monitor(data)



def monitor(data):
    string1 = open("htmlString", "r").read()
    string2 = data
    while True:
        time.sleep(5)
        login()
        if string1 == string2:
            print "Nothing has changed"
        else:
            print "Something has changed"




login()

Answer 1

我发现您的代码很难理解。这是一个未经测试的替代方案，应该接近您要实现的目标。

def fetch_html():
    # fetch logic
    return html  # string

def write_html(html):  # string
    # write logic

def read_html():
    with open('page.html','r') as f:
        return f.read()

def monitor():
    write_html(fetch_html())
    while True:
        time.sleep(5)
        new_html = fetch_html()
        if new_html == read_html():
            print('Nothing has changed')
        else:
            print('Something has changed')
            write_html(new_html)

monitor()

Answer 2

问题是当您调用 login() 时 string2 没有更新。您应该制作 login() return data 并将其分配给每个循环 string2。

Python 监控网站的变化

Python Monitor Website for changes

html

python

screen-scraping

web