Python 运行 Macbook pro m1 max 错误(运行 在 Tensorflow 上)
Python Running Error on Macbook pro m1 max (Running on Tensorflow)
我正尝试在我的新 macbook pro max M1 芯片上 运行 来自 github binary-bot 的代码:
金属设备设置为:
Apple M1 Max
systemMemory: 32.00 GB
maxCacheSize: 10.67 GB
我收到以下错误。有什么建议吗?
2021-12-19 17:26:25.248041: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-12-19 17:26:25.248181: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
1 Physical GPUs, 1 Logical GPUs
1 Physical GPUs, 1 Logical GPUs
Trying to connect to IqOption
Successfully Connected!
/Users/abdallahmohamed/Downloads/binary-bot-master/training.py:35: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
df = df.drop("future", 1)
/Users/abdallahmohamed/Downloads/binary-bot-master/training.py:35: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
df = df.drop("future", 1)
train data: 836 validation: 68
sells: 418, buys: 418
VALIDATION sells: 34, buys : 34
0.001-5-SEQ-2-40-16-PRED-1639927591
1 Physical GPUs, 1 Logical GPUs
2021-12-19 17:26:32.262259: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/40
2021-12-19 17:26:33.223096: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.592036: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.646351: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.684523: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.786763: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.866171: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.932667: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
53/53 [==============================] - ETA: 0s - loss: 0.8687 - accuracy: 0.52272021-12-19 17:26:35.527128: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.652789: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.681668: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.710022: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 4s 34ms/step - loss: 0.8687 - accuracy: 0.5227 - val_loss: 0.6928 - val_accuracy: 0.5000
Epoch 2/40
53/53 [==============================] - ETA: 0s - loss: 0.7844 - accuracy: 0.5335WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7844 - accuracy: 0.5335 - val_loss: 0.6890 - val_accuracy: 0.5000
Epoch 3/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7257 - accuracy: 0.5588WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7293 - accuracy: 0.5562 - val_loss: 0.6836 - val_accuracy: 0.5735
Epoch 4/40
52/53 [============================>.] - ETA: 0s - loss: 0.7421 - accuracy: 0.5649WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7411 - accuracy: 0.5658 - val_loss: 0.7035 - val_accuracy: 0.4412
Epoch 5/40
52/53 [============================>.] - ETA: 0s - loss: 0.7205 - accuracy: 0.5565WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7193 - accuracy: 0.5586 - val_loss: 0.7327 - val_accuracy: 0.4412
Epoch 6/40
52/53 [============================>.] - ETA: 0s - loss: 0.7233 - accuracy: 0.5637WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7228 - accuracy: 0.5634 - val_loss: 0.7023 - val_accuracy: 0.5441
Epoch 7/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7192 - accuracy: 0.5588WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7187 - accuracy: 0.5586 - val_loss: 0.8523 - val_accuracy: 0.4559
Epoch 8/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7111 - accuracy: 0.5613WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7105 - accuracy: 0.5634 - val_loss: 0.7727 - val_accuracy: 0.4559
Epoch 9/40
53/53 [==============================] - ETA: 0s - loss: 0.7151 - accuracy: 0.5514WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7151 - accuracy: 0.5514 - val_loss: 0.7105 - val_accuracy: 0.5147
Epoch 10/40
53/53 [==============================] - ETA: 0s - loss: 0.7046 - accuracy: 0.5371WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7046 - accuracy: 0.5371 - val_loss: 0.6940 - val_accuracy: 0.5588
Epoch 11/40
53/53 [==============================] - ETA: 0s - loss: 0.7064 - accuracy: 0.5455WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7064 - accuracy: 0.5455 - val_loss: 0.7433 - val_accuracy: 0.3971
Epoch 12/40
51/53 [===========================>..] - ETA: 0s - loss: 0.6991 - accuracy: 0.5784WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.6988 - accuracy: 0.5778 - val_loss: 0.6902 - val_accuracy: 0.5147
Epoch 13/40
52/53 [============================>.] - ETA: 0s - loss: 0.6812 - accuracy: 0.5757WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6818 - accuracy: 0.5754 - val_loss: 0.8100 - val_accuracy: 0.4118
Epoch 14/40
52/53 [============================>.] - ETA: 0s - loss: 0.6876 - accuracy: 0.5673WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6888 - accuracy: 0.5658 - val_loss: 0.7208 - val_accuracy: 0.5294
Epoch 15/40
52/53 [============================>.] - ETA: 0s - loss: 0.6815 - accuracy: 0.5505WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 23ms/step - loss: 0.6809 - accuracy: 0.5502 - val_loss: 0.6965 - val_accuracy: 0.5441
Epoch 16/40
51/53 [===========================>..] - ETA: 0s - loss: 0.6886 - accuracy: 0.5711WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6900 - accuracy: 0.5670 - val_loss: 0.6529 - val_accuracy: 0.6029
Epoch 17/40
53/53 [==============================] - ETA: 0s - loss: 0.6959 - accuracy: 0.5598WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6959 - accuracy: 0.5598 - val_loss: 0.7832 - val_accuracy: 0.4118
Epoch 18/40
52/53 [============================>.] - ETA: 0s - loss: 0.7002 - accuracy: 0.5325WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6999 - accuracy: 0.5335 - val_loss: 0.7270 - val_accuracy: 0.3676
WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
WARNING:tensorflow:Layer lstm_4 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
WARNING:tensorflow:Layer lstm_5 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
Trying to connect to IqOption
Successfully Connected!
2021-12-19 17:27:01.637577: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:27:01.792362: F tensorflow/core/framework/tensor.cc:681] Check failed: IsAligned() ptr = 0x2d0efcee0
zsh: abort /Users/abdallahmohamed/miniforge3/envs/env_tensorflow/bin/python
我删除了存储的模型并保存了一个新模型后,它起作用了
我的感觉是这里有两个明显的问题:
- 使用 Metal 引擎在 M1 机器上安装 Tensorflow 的步骤。
- 要使用的包的
requirements.txt
文件。
背景:
在新的 M1 处理器上工作时,由于某些命名约定和版本控制假设往往包含在包 requirement.txt
文件中,因此在安装 M1 之前的 python 包时往往会遇到问题.
另外需要注意的是,M1 处理器仅支持 Python 3.8 及更高版本。 las,Python 3.7 和 python 的所有先前版本都是在 M1 处理器发布之前开发的,并且(显然)没有向后移植关键补丁的计划。
Tensorflow-macos 和 Tensorflow-metal 安装
安装Tensorflow-macos
和TensorFlow-metal
的步骤很详细here,使用mini-forge可以总结如下:
conda create -n tf python=3.8 -y
conda activate tf
conda install -c apple tensorflow-deps -y
# Navigate the issue with conda environments
# built against pre-macOS 11 SDK use (Just in case)
# this solves the Intel / M1 install issues:
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-macos
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-metal
所以我们可以测试它是否有效:
#!/usr/bin/env bash
PYCMD=$(cat <<EOF
import tensorflow as tf
tf.config.list_physical_devices()
with tf.device('/GPU'):
a = tf.random.normal(shape=(2,), dtype=tf.float32)
b = tf.nn.relu(a)
print(a)
print(b)
EOF
)
python3 -c "$PYCMD" 2>/dev/null
我的测试机器的预期输出:
>./val.sh
Metal device set to: Apple M1
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
tf.Tensor([-0.4044231 0.08157467], shape=(2,), dtype=float32)
tf.Tensor([0. 0.08157467], shape=(2,), dtype=float32)
好的,一切(假设)都在工作...
二进制 Bot 安装问题说明:
现在我们来看看 Binary Bot...
首先,我们需要通过 git hub
安装
git clone https://github.com/ItamarRocha/binary-bot
好的,现在在安装之前让我们看一下 requirements.txt 文件...
iqoptionapi @ git+git://github.com/Lu-Yi-Hsun/iqoptionapi.git@e96ba2c5b905a139a4765167b08c5df48cf57773
numpy==1.18.0
pandas==1.1.1
tensorflow==2.3.1
scikit-learn==0.23.2
grpcio==1.24.3
重要的是要注意两个关键问题 ==
和 tensorflow
。需求文件中的 ==
表示与包版本完全匹配。同时我们还没有安装tensorflow
,我们安装了tensorflow-macos
和tensorflow-metal
。我们可以检查一下:
>pip show numpy pandas tensorflow scikit-learn grpcio | egrep 'Name:|Version:'
WARNING: Package(s) not found: pandas, scikit-learn, tensorflow
Name: numpy
Version: 1.19.5
Name: grpcio
Version: 1.43.0
因此如果我们 运行:
python3 pip install -r requirements.txt
我们收到大量错误。为什么?好吧,因为 requirements.txt
文件需要 exact ==
包匹配而不是兼容的 ~=
包匹配或下载大于或等于规定版本的任何版本>=
。
为了解决这个问题,我们可以更改为 >=
,这(希望)应该允许我们安装与 Apple M1 芯片兼容的软件包,但如果开发人员依赖于旧的假设,我们可能会破坏兼容性包。
此外,requirements.txt
将尝试安装 tensorflow
包,而不是 tensorflow-macosx
或 tensorflow-metal
包,正如我们所看到的那样,当我们 运行 requirements.txt
文件不变。
那么如何导航?
好吧,一种选择是简单地将 ==
替换为 >=
,并将 tensorflow
替换或删除为 tensorflow-macosx
。
sed -i.bak 's/==/>=/g;s/tensorflow/tensorflow-macos/g' requirements.txt
导致:
cat requirements.txt
iqoptionapi @ git+git://github.com/Lu-Yi-Hsun/iqoptionapi.git@e96ba2c5b905a139a4765167b08c5df48cf57773
numpy>=1.18.0
pandas>=1.1.1
tensorflow-macos>=2.3.1
scikit-learn>=0.23.2
grpcio>=1.24.3
现在如果我们 运行:
pip install -r requirements.txt
我们安装了最新版本的软件包 >=
导致:
pip show numpy pandas tensorflow-macos tensorflow-metal scikit-learn grpcio | egrep 'Name:|Version:'
Name: numpy
Version: 1.22.0
Name: pandas
Version: 1.3.5
Name: tensorflow-macos
Version: 2.7.0
Name: tensorflow-metal
Version: 0.3.0
Name: scikit-learn
Version: 1.0.2
Name: grpcio
Version: 1.43.0
所以,我们现在应该能够测试一切是否正常...
python3 testing.py
Metal device set to: Apple M1
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
2022-01-08 20:40:40.591430: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-01-08 20:40:40.591754: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
1 Physical GPUs, 1 Logical GPUs
1 Physical GPUs, 1 Logical GPUs
Trying to connect to IqOption
Error when trying to connect
<iqoptionapi.stable_api.IQ_Option object at 0x1031074c0>
Retrying
所以包现在安装,所有需要的是将您的名称和密码添加到 iq.py 文件,并检查结果是否符合预期。
19 if iq == None:
20 print("Trying to connect to IqOption")
21 iq=IQ_Option('USERNAME','PASSWORD') # YOU HAVE TO ADD YOUR USERNAME AND PASSWORD
22 iq.connect()
参考 'deleting the model' 上的观点,这是公认的答案。我推测这是指 binary-bot
目录中 models
目录中的 LSTM-best.model
。
tree binary-bot binary-bot
├── README.md
├── __pycache__
│ ├── iq.cpython-38.pyc
│ └── training.cpython-38.pyc
├── imgs
│ ├── mac_forex.jpg
│ └── result_binary.png
├── iq.py
├── models
│ └── LSTM-best.model
├── requirements.txt
├── requirements.txt.bak
├── result_binary.png
├── testing.py
└── training.py
唉,我目前不在允许我注册服务 https://iqoption.net/
的国家/地区,以进行全面测试,但我的感觉是,事情应该可以达到可以根据以下条件验证结果的程度以上。
希望这能为人们指明正确的方向,让他们思考让 python 软件包与使用 TensorFlow 金属插件的 Apple-M1 / Apple-Intel 笔记本电脑一起工作所涉及的障碍。
注意安全。
我正尝试在我的新 macbook pro max M1 芯片上 运行 来自 github binary-bot 的代码:
金属设备设置为:
Apple M1 Max
systemMemory: 32.00 GB
maxCacheSize: 10.67 GB
我收到以下错误。有什么建议吗?
2021-12-19 17:26:25.248041: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2021-12-19 17:26:25.248181: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
1 Physical GPUs, 1 Logical GPUs
1 Physical GPUs, 1 Logical GPUs
Trying to connect to IqOption
Successfully Connected!
/Users/abdallahmohamed/Downloads/binary-bot-master/training.py:35: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
df = df.drop("future", 1)
/Users/abdallahmohamed/Downloads/binary-bot-master/training.py:35: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only
df = df.drop("future", 1)
train data: 836 validation: 68
sells: 418, buys: 418
VALIDATION sells: 34, buys : 34
0.001-5-SEQ-2-40-16-PRED-1639927591
1 Physical GPUs, 1 Logical GPUs
2021-12-19 17:26:32.262259: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Epoch 1/40
2021-12-19 17:26:33.223096: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.592036: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.646351: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.684523: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.786763: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.866171: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:33.932667: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
53/53 [==============================] - ETA: 0s - loss: 0.8687 - accuracy: 0.52272021-12-19 17:26:35.527128: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.652789: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.681668: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:26:35.710022: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 4s 34ms/step - loss: 0.8687 - accuracy: 0.5227 - val_loss: 0.6928 - val_accuracy: 0.5000
Epoch 2/40
53/53 [==============================] - ETA: 0s - loss: 0.7844 - accuracy: 0.5335WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7844 - accuracy: 0.5335 - val_loss: 0.6890 - val_accuracy: 0.5000
Epoch 3/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7257 - accuracy: 0.5588WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7293 - accuracy: 0.5562 - val_loss: 0.6836 - val_accuracy: 0.5735
Epoch 4/40
52/53 [============================>.] - ETA: 0s - loss: 0.7421 - accuracy: 0.5649WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7411 - accuracy: 0.5658 - val_loss: 0.7035 - val_accuracy: 0.4412
Epoch 5/40
52/53 [============================>.] - ETA: 0s - loss: 0.7205 - accuracy: 0.5565WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7193 - accuracy: 0.5586 - val_loss: 0.7327 - val_accuracy: 0.4412
Epoch 6/40
52/53 [============================>.] - ETA: 0s - loss: 0.7233 - accuracy: 0.5637WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7228 - accuracy: 0.5634 - val_loss: 0.7023 - val_accuracy: 0.5441
Epoch 7/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7192 - accuracy: 0.5588WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7187 - accuracy: 0.5586 - val_loss: 0.8523 - val_accuracy: 0.4559
Epoch 8/40
51/53 [===========================>..] - ETA: 0s - loss: 0.7111 - accuracy: 0.5613WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.7105 - accuracy: 0.5634 - val_loss: 0.7727 - val_accuracy: 0.4559
Epoch 9/40
53/53 [==============================] - ETA: 0s - loss: 0.7151 - accuracy: 0.5514WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7151 - accuracy: 0.5514 - val_loss: 0.7105 - val_accuracy: 0.5147
Epoch 10/40
53/53 [==============================] - ETA: 0s - loss: 0.7046 - accuracy: 0.5371WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7046 - accuracy: 0.5371 - val_loss: 0.6940 - val_accuracy: 0.5588
Epoch 11/40
53/53 [==============================] - ETA: 0s - loss: 0.7064 - accuracy: 0.5455WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.7064 - accuracy: 0.5455 - val_loss: 0.7433 - val_accuracy: 0.3971
Epoch 12/40
51/53 [===========================>..] - ETA: 0s - loss: 0.6991 - accuracy: 0.5784WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 25ms/step - loss: 0.6988 - accuracy: 0.5778 - val_loss: 0.6902 - val_accuracy: 0.5147
Epoch 13/40
52/53 [============================>.] - ETA: 0s - loss: 0.6812 - accuracy: 0.5757WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6818 - accuracy: 0.5754 - val_loss: 0.8100 - val_accuracy: 0.4118
Epoch 14/40
52/53 [============================>.] - ETA: 0s - loss: 0.6876 - accuracy: 0.5673WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6888 - accuracy: 0.5658 - val_loss: 0.7208 - val_accuracy: 0.5294
Epoch 15/40
52/53 [============================>.] - ETA: 0s - loss: 0.6815 - accuracy: 0.5505WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 23ms/step - loss: 0.6809 - accuracy: 0.5502 - val_loss: 0.6965 - val_accuracy: 0.5441
Epoch 16/40
51/53 [===========================>..] - ETA: 0s - loss: 0.6886 - accuracy: 0.5711WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6900 - accuracy: 0.5670 - val_loss: 0.6529 - val_accuracy: 0.6029
Epoch 17/40
53/53 [==============================] - ETA: 0s - loss: 0.6959 - accuracy: 0.5598WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6959 - accuracy: 0.5598 - val_loss: 0.7832 - val_accuracy: 0.4118
Epoch 18/40
52/53 [============================>.] - ETA: 0s - loss: 0.7002 - accuracy: 0.5325WARNING:tensorflow:Can save best model only with val_acc available, skipping.
53/53 [==============================] - 1s 24ms/step - loss: 0.6999 - accuracy: 0.5335 - val_loss: 0.7270 - val_accuracy: 0.3676
WARNING:tensorflow:Layer lstm_3 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
WARNING:tensorflow:Layer lstm_4 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
WARNING:tensorflow:Layer lstm_5 will not use cuDNN kernels since it doesn't meet the criteria. It will use a generic GPU kernel as fallback when running on GPU.
Trying to connect to IqOption
Successfully Connected!
2021-12-19 17:27:01.637577: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
2021-12-19 17:27:01.792362: F tensorflow/core/framework/tensor.cc:681] Check failed: IsAligned() ptr = 0x2d0efcee0
zsh: abort /Users/abdallahmohamed/miniforge3/envs/env_tensorflow/bin/python
我删除了存储的模型并保存了一个新模型后,它起作用了
我的感觉是这里有两个明显的问题:
- 使用 Metal 引擎在 M1 机器上安装 Tensorflow 的步骤。
- 要使用的包的
requirements.txt
文件。
背景:
在新的 M1 处理器上工作时,由于某些命名约定和版本控制假设往往包含在包 requirement.txt
文件中,因此在安装 M1 之前的 python 包时往往会遇到问题.
另外需要注意的是,M1 处理器仅支持 Python 3.8 及更高版本。 las,Python 3.7 和 python 的所有先前版本都是在 M1 处理器发布之前开发的,并且(显然)没有向后移植关键补丁的计划。
Tensorflow-macos 和 Tensorflow-metal 安装
安装Tensorflow-macos
和TensorFlow-metal
的步骤很详细here,使用mini-forge可以总结如下:
conda create -n tf python=3.8 -y
conda activate tf
conda install -c apple tensorflow-deps -y
# Navigate the issue with conda environments
# built against pre-macOS 11 SDK use (Just in case)
# this solves the Intel / M1 install issues:
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-macos
SYSTEM_VERSION_COMPAT=0 pip install tensorflow-metal
所以我们可以测试它是否有效:
#!/usr/bin/env bash
PYCMD=$(cat <<EOF
import tensorflow as tf
tf.config.list_physical_devices()
with tf.device('/GPU'):
a = tf.random.normal(shape=(2,), dtype=tf.float32)
b = tf.nn.relu(a)
print(a)
print(b)
EOF
)
python3 -c "$PYCMD" 2>/dev/null
我的测试机器的预期输出:
>./val.sh
Metal device set to: Apple M1
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
tf.Tensor([-0.4044231 0.08157467], shape=(2,), dtype=float32)
tf.Tensor([0. 0.08157467], shape=(2,), dtype=float32)
好的,一切(假设)都在工作...
二进制 Bot 安装问题说明:
现在我们来看看 Binary Bot...
首先,我们需要通过 git hub
安装git clone https://github.com/ItamarRocha/binary-bot
好的,现在在安装之前让我们看一下 requirements.txt 文件...
iqoptionapi @ git+git://github.com/Lu-Yi-Hsun/iqoptionapi.git@e96ba2c5b905a139a4765167b08c5df48cf57773
numpy==1.18.0
pandas==1.1.1
tensorflow==2.3.1
scikit-learn==0.23.2
grpcio==1.24.3
重要的是要注意两个关键问题 ==
和 tensorflow
。需求文件中的 ==
表示与包版本完全匹配。同时我们还没有安装tensorflow
,我们安装了tensorflow-macos
和tensorflow-metal
。我们可以检查一下:
>pip show numpy pandas tensorflow scikit-learn grpcio | egrep 'Name:|Version:'
WARNING: Package(s) not found: pandas, scikit-learn, tensorflow
Name: numpy
Version: 1.19.5
Name: grpcio
Version: 1.43.0
因此如果我们 运行:
python3 pip install -r requirements.txt
我们收到大量错误。为什么?好吧,因为 requirements.txt
文件需要 exact ==
包匹配而不是兼容的 ~=
包匹配或下载大于或等于规定版本的任何版本>=
。
为了解决这个问题,我们可以更改为 >=
,这(希望)应该允许我们安装与 Apple M1 芯片兼容的软件包,但如果开发人员依赖于旧的假设,我们可能会破坏兼容性包。
此外,requirements.txt
将尝试安装 tensorflow
包,而不是 tensorflow-macosx
或 tensorflow-metal
包,正如我们所看到的那样,当我们 运行 requirements.txt
文件不变。
那么如何导航?
好吧,一种选择是简单地将 ==
替换为 >=
,并将 tensorflow
替换或删除为 tensorflow-macosx
。
sed -i.bak 's/==/>=/g;s/tensorflow/tensorflow-macos/g' requirements.txt
导致:
cat requirements.txt
iqoptionapi @ git+git://github.com/Lu-Yi-Hsun/iqoptionapi.git@e96ba2c5b905a139a4765167b08c5df48cf57773
numpy>=1.18.0
pandas>=1.1.1
tensorflow-macos>=2.3.1
scikit-learn>=0.23.2
grpcio>=1.24.3
现在如果我们 运行:
pip install -r requirements.txt
我们安装了最新版本的软件包 >=
导致:
pip show numpy pandas tensorflow-macos tensorflow-metal scikit-learn grpcio | egrep 'Name:|Version:'
Name: numpy
Version: 1.22.0
Name: pandas
Version: 1.3.5
Name: tensorflow-macos
Version: 2.7.0
Name: tensorflow-metal
Version: 0.3.0
Name: scikit-learn
Version: 1.0.2
Name: grpcio
Version: 1.43.0
所以,我们现在应该能够测试一切是否正常...
python3 testing.py
Metal device set to: Apple M1
systemMemory: 16.00 GB
maxCacheSize: 5.33 GB
2022-01-08 20:40:40.591430: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.
2022-01-08 20:40:40.591754: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)
1 Physical GPUs, 1 Logical GPUs
1 Physical GPUs, 1 Logical GPUs
Trying to connect to IqOption
Error when trying to connect
<iqoptionapi.stable_api.IQ_Option object at 0x1031074c0>
Retrying
所以包现在安装,所有需要的是将您的名称和密码添加到 iq.py 文件,并检查结果是否符合预期。
19 if iq == None:
20 print("Trying to connect to IqOption")
21 iq=IQ_Option('USERNAME','PASSWORD') # YOU HAVE TO ADD YOUR USERNAME AND PASSWORD
22 iq.connect()
参考 'deleting the model' 上的观点,这是公认的答案。我推测这是指 binary-bot
目录中 models
目录中的 LSTM-best.model
。
tree binary-bot binary-bot
├── README.md
├── __pycache__
│ ├── iq.cpython-38.pyc
│ └── training.cpython-38.pyc
├── imgs
│ ├── mac_forex.jpg
│ └── result_binary.png
├── iq.py
├── models
│ └── LSTM-best.model
├── requirements.txt
├── requirements.txt.bak
├── result_binary.png
├── testing.py
└── training.py
唉,我目前不在允许我注册服务 https://iqoption.net/
的国家/地区,以进行全面测试,但我的感觉是,事情应该可以达到可以根据以下条件验证结果的程度以上。
希望这能为人们指明正确的方向,让他们思考让 python 软件包与使用 TensorFlow 金属插件的 Apple-M1 / Apple-Intel 笔记本电脑一起工作所涉及的障碍。
注意安全。