如何在Python3中同时运行两个或多个函数？

Question

我想运行同时运行

我还想知道如何在进程结束时获取 return 值。

这是我的代码↓

[函数1：训练模型]

def train_model(model, EPOCHS, BATCH_SIZE, train_input, train_output, tb_logdir):
    check_path(tb_logdir)
    tb_callback = tf.keras.callbacks.TensorBoard(log_dir=tb_logdir + datetime.now().strftime('%Y%m%d-%H%M%S'), profile_batch=50)#,
                                                 #histogram_freq=1,
                                                 #profile_batch=5)

    print('- EPOCHS => ', EPOCHS)
    print('- BATCH_SIZE => ', BATCH_SIZE)
    s_time = timeit.default_timer()

    history = model.fit(train_input, train_output,
                        batch_size=BATCH_SIZE, epochs=EPOCHS,
                        # valication_data = (x_val, x_val),
                        callbacks=[tb_callback])

    loss, acc = model.evaluate(train_input, train_output)
    print('- Train Loss : {}'.format(loss))
    print('- Train Accuracy : {}'.format(acc))

    duration, _ = time_taken(s_time)
    print('-- Training time [', duration, ' (sec)]')

    return model, history

[功能2：查看系统使用情况]

def check_sys_usage():
    usage = 100
    gpus = GPUtil.getGPUs()
    while usage > 10 :
        usage = psutil.cpu_percent()
        memory_usage = psutil.virtual_memory()
        print('** cpu =>', usage, '%, mem => ', memory_usage, '%')
        for gpu in gpus:
            print('** ', gpu.id, ' => ', gpu.load*100,'%')
        time.sleep(10)

[主要]

from multiprocessing import Process

if __name__ == '__main__':
    p1 = Process(target=modules.train_model, args=(model, 100, 50, train_input, train_output, tb_logdir))
    p2 = Process(target=modules.check_sys_usage)


    p1.run()
    p2.run()

    model = p1.model  ***# error***
    history = p1.history  ***# error***

我使用了“Process”，但它仍然运行连续。

我想从 "p1"

中获取 return 值

但它 returns "AttributeError: 'Process' 对象没有属性 'model'"

[控制台结果]

>     - EPOCHS =>  100
>     - BATCH_SIZE =>  50
>     Epoch 1/100
>     12/12 [==============================] - 0s 24ms/step - loss: 2.3979 - accuracy: 0.0932
>     Epoch 2/100
>     12/12 [==============================] - 0s 26ms/step - loss: 2.3726 - accuracy: 0.1000
>     .
>     .
>     .
>     Epoch 100/100
>     12/12 [==============================] - 0s 20ms/step - loss: 2.0228 - accuracy: 0.2288
>     19/19 [==============================] - 0s 7ms/step - loss: 2.0046 - accuracy: 0.2525
>     - Train Loss : 2.0046160221099854
>     - Train Accuracy : 0.2525423765182495
>     -- Training time [ 29.3114186  (sec)]
>     ** cpu => 75.1 %, mem =>  svmem(total=16943194112, available=6091567104, percent=64.0, used=10851627008, free=6091567104)
> %
>     ** cpu => 6.1 %, mem =>  svmem(total=16943194112, available=6090833920, percent=64.1, used=10852360192, free=6090833920)
> %
>     Traceback (most recent call last):
>       File "D:/projects/python/classification/LSTM_basic_.py", line 105, in <module>
>         model = p1.model
>     AttributeError: 'Process' object has no attribute 'model'

Answer 1

我使用 'Thread' 而不是多处理解决了这个问题。

由于 GIL（全局解释器锁）多处理不起作用。

下面是我的固定代码。

from threading import Thread

th1 = Thread(target=modules.train_model, args=(model, 10, 50, train_input, train_output, tb_logdir))
th2 = Thread(target=modules.check_sys_usage)

th1.start()
th2.start()

th1.join()
th2.join()

如何在Python3中同时运行两个或多个函数？

How to run two or more functions at the same time in Python3?

python-3.x

python-multithreading

multiprocess