在 Python 中模拟 10,000 次抛硬币非常慢

Question

我正在编写一个模拟，创建 10,000 个周期，每组 25 组，每组包含 48 次抛硬币。此代码中的某些内容使其运行非常缓慢。它已经运行ning 至少 20 分钟了，它仍在工作。 R 运行s 中的类似模拟不到 10 秒。

这是我正在使用的 python 代码：

import pandas as pd
from random import choices

threshold=17
all_periods = pd.DataFrame()

for i in range(10000):
    simulated_period = pd.DataFrame()
    for j in range(25):
        #Data frame with 48 weeks as rows. Each run through loop adds one more year as column until there are 25
        simulated_period = pd.concat([simulated_period, pd.DataFrame(choices([1, -1], k=48))],\
                                      ignore_index=True, axis=1)
        positives = simulated_period[simulated_period==1].count(axis=1)
        negatives = simulated_period[simulated_period==-1].count(axis=1)
        #Combine positives and negatives that are more than the threshold into single dataframe
        sig = pd.DataFrame([[sum(positives>=threshold), sum(negatives>=threshold)]], columns=['positive', 'negative'])
        sig['total'] = sig['positive'] + sig['negative']
    #Add summary of individual simulation to the others
    all_periods = pd.concat([all_periods, sig])

如果有帮助，这里是正在快速运行的 R 脚本：

flip <- function(threshold=17){
  #threshold is min number of persistent results we want to see. For example, 17/25 positive or 17/25 negative

  outcomes <- c(1, -1)
  trial <- do.call(cbind, lapply(1:25, function (i) sample(outcomes, 48, replace=T)))
  trial <- as.data.frame(t(trial)) #48 weeks in columns, 25 years in rows.

  summary <- sapply(trial, function(x) c(pos=length(x[x==1]), neg=length(x[x==-1])))
  summary <- as.data.frame(t(summary)) #use data frame so $pos/$neg can be used instead of [1,]/[2,]

  sig.pos <- length(summary$pos[summary$pos>=threshold])
  sig.neg <- length(summary$neg[summary$neg>=threshold])

  significant <- c(pos=sig.pos, neg=sig.neg, total=sig.pos+sig.neg) 

  return(significant)
}

  results <- do.call(rbind, lapply(1:10000, function(i) flip(threshold)))
  results <- as.data.frame(results)

任何人都可以告诉我我在 python 中运行是什么减慢了进程？谢谢。

Answer 1

你为什么不生成整个大集合

idx = pd.MultiIndex.from_product((range(10000), range(25)),
                                 names=('period', 'set'))
df = pd.DataFrame(data=np.random.choice([1,-1], (10000*25, 48)), index=idx)

在我的电脑上花费了大约 120 毫秒。然后是其他操作：

positives = df.eq(1).sum(level=0).gt(17).sum(axis=1).to_frame(name='positives')
negatives = df.eq(-1).sum(level=0).gt(17).sum(axis=1).to_frame(name='negatives')

all_periods = pd.concat( (positives, negatives), axis=1 )

all_periods['total'] = all_periods.sum(1)

额外需要大约 600 毫秒。

在 Python 中模拟 10,000 次抛硬币非常慢

Simulating 10,000 Coinflips in Python Very Slow

python-3.x

pandas

dataframe

simulation

coin-flipping