使用@ray装饰器后无法向字典中添加数据

Question

我写了一个函数，它从 API 和 return 中获取一些数据到字典。当我执行它工作正常。问题是当我尝试使用 ray 库在 "the same time" 中执行此函数两次时。数据从 api 正确获取，但未添加到字典中。

Companies = dict()

def call_company_api(company_id, dictionary):
    data = requests.get(API_CALL_COMPANY_URL.format(company_id)).json()
    # name = data['data']['krs_podmioty.nazwa'] FULL NAME
    name = data['data']['krs_podmioty.nazwa_skrocona']
    city = data['data']['krs_podmioty.adres_poczta']
    nip = data['data']['krs_podmioty.nip']
    community_id = data['data']['krs_podmioty.gmina_id']
    county_id = data['data']['krs_podmioty.powiat_id']
    voivodeship_id = data['data']['krs_podmioty.wojewodztwo_id']

    try:
        community = gminy_list[community_id]
        county = powiaty_list[county_id]
        voivodeship = wojewodztwa_list[voivodeship_id]

    except KeyError:
        community = community_id
        county = county_id
        voivodeship = voivodeship_id

    dictionary[name] = [city, county, community , voivodeship, nip]

当我执行下面这段代码时工作正常


def call_company():
    for k in comapanies_list:
        call_company_api(k, Companies)

call_company()


print(Companies) --> {'BELKA19': ['Warszawa', 'Warszawa', 'Warszawa', 'Mazowieckie', '5252786971'], 'GSW CONSTRUCTION': ['Kraków', 'Kraków', 'Kraków', 'Małopolskie', '6762564804']}

在这种情况下数据没有添加到字典中，您知道如何解决吗？我尝试为所有 call_items 函数单独添加字典，但它也没有像我预期的那样正常工作。


ray.init()



@ray.remote
def call_l1_items():
    for k in l1:
        call_company_api(k, Companies)



@ray.remote
def call_l2_items():
    for k in l2:
        call_company_api(k, Companies)



ret_id1 = call_l1_items.remote()
ret_id2 = call_l2_items.remote()
ret1, ret2 = ray.get([ret_id1, ret_id2])

print(Companies) --> {}

Answer 1

问题是 Ray 任务在单独的进程中执行（与线程相反），因此当您定义使用 Companies 字典的 call_l1_items 函数时，会创建一个副本Companies 实际执行任务的工作进程字典。所以字典的远程副本会发生变异，但主脚本中的原始副本不会。

您可以通过从函数返回项目然后在主脚本中更新原始字典来解决此问题。

使用@ray装饰器后无法向字典中添加数据

After using @ray decorator can't add data to dictionary

python

ray