如何使用 gevent 完成任务 "read - search - write"？

Question

我正在尝试处理如下任务：

读取本地文件中的一行
将行发送到网络服务进行查询
将结果写入本地文件

`

def read(fin):
    for query in fin:
        return query

def search(query):
    # Send the query to something like Google Search
    result = google(query)
    return result

def write(fout, result):
    fout.write(result)

`

我参考了Gevent教程，但是没弄明白如何把这三个函数放到gevent中。

Answer 1

我不知道你的示例与 gevent 相关的要点。不管怎样，还是可以通过下面的示例代码来处理的。

import gevent

def read(fin):
    for query in fin:
        return query

def search(query):
    # Send the query to something like Google Search
    result = google(query)
    return result

def write(fout, result):
    fout.write(result)

# read from different files
input_files = ["input1.json", "input2.json" ... ] 

# write result into output file
fout = open("output.json", 'wb')

# get queries from input files
queries = []
for input_file in input_files:
    with open(input_file, 'rb') as fin:
        queries.append(gevent.joinall([gevent.spawn(read, fin)]))

# store results of queries
results = []
for item in [query for query in queries]:
    results.append(gevent.joinall([gevent.spawn(search, item[0].value)]))

# save results into output file
for item in [result for result in results]:
    gevent.joinall([gevent.spawn(write, fout, item[0].value)])

读取到搜索和搜索到写入都有依赖关系，这是无法使用异步方式实现的。因此在例子中，它只是可以处理每个功能并保存return。

如何使用 gevent 完成任务 "read - search - write"？

How to use gevent for a task "read - search - write"?

python

gevent

coroutine