如何使用等待迭代并附加到数据框

Question

我正在尝试了解如何将股票列表附加到单个数据框中。

有人说我需要使用 wait 语句（如果我想使用 append 语句进行迭代）。我想我已经设置好了，但我什至不能做一个简单的迭代

from concurrent.futures import wait, ALL_COMPLETED

import concurrent.futures
import datetime
from datetime import timedelta
import yfinance as yf

pool = concurrent.futures.ThreadPoolExecutor(8)

end=datetime.date.today()
start=end - timedelta(weeks=104)

symbols = ['GOOG','CSCO']

def dl(stock):
    #sleep(randint(1, 5))
    #print(stock)
    return yf.download(stock, start=start, end=end).iloc[: , :5].dropna(axis=0, how='any')

futures = [pool.submit(dl, args) for args in symbols]
wait(futures, timeout=10, return_when=ALL_COMPLETED)

#print(futures[1])
futures[0].result()

stocks=[]

for x in range(len(symbols)):
    print(x)
    stocks.append(futures[x].result())
    futures[x].result()
    
print(stocks)

所以...如果我执行以下操作

stocks = []
# CHANGE IN THE BELOW LINE
for x in range(len(futures)):
    #print(x)
    stocks.append(futures[x].result())
    #futures[x].result()

print(stocks)

它会打印，但是它是两个块，每个块有 502 行......我想要一个数据帧（即 1004 行）。我之前能够在不使用等待的情况下完成同样的行为...

Answer 1

from concurrent.futures import wait, ALL_COMPLETED

import concurrent.futures
import datetime
from datetime import timedelta
import yfinance as yf

pool = concurrent.futures.ThreadPoolExecutor(8)

end = datetime.date.today()
start = end - timedelta(weeks=104)

stocks = ['GOOG', 'CSCO']


def dl(stock):
    # sleep(randint(1, 5))
    # print(stock)
    return yf.download(stock, start=start, end=end).iloc[:, :5].dropna(axis=0, how='any')


futures = [pool.submit(dl, args) for args in stocks]
wait(futures, timeout=10, return_when=ALL_COMPLETED)


# CHANGE IN THE BELOW LINE
stocks_data = pd.DataFrame()
for x in range(0,len(stocks)):
    stocks_data = pd.concat([stocks_data,pd.DataFrame(futures[x].result())])
print(stocks_data.shape)
(1004, 5)

Answer 2

功劳归功于 Rafael Valero

但我想我会 post 最终代码...我仍然经常收到“keyerror”，但偶尔会填充整个数据框

from concurrent.futures import wait, ALL_COMPLETED

import concurrent.futures
import datetime
from datetime import timedelta
import yfinance as yf
import pandas as pd

pool = concurrent.futures.ThreadPoolExecutor(8)

end = datetime.date.today()
start = end - timedelta(weeks=104)

stocks = ['GOOG', 'CSCO']


def dl(stock):
    return yf.download(stock, start=start, end=end).iloc[:, :5].dropna(axis=0, how='any')


futures = [pool.submit(dl, args) for args in stocks]
wait(futures, return_when=ALL_COMPLETED)

stocks_data = pd.DataFrame()
for x in range(0,len(stocks)):
    prices = pd.DataFrame(futures[x].result())
    prices['Symbol'] = stocks[x]
    stocks_data = pd.concat([stocks_data,prices])

print(stocks_data)

如何使用等待迭代并附加到数据框

How to use wait to iterate and append to a dataframe

python

parallel-processing

concurrent.futures