张量板:配置文件选项未显示
tensorboard: profile option not showing
我正在尝试设置 tensorboard 以显示配置文件信息,以便我可以调试慢速模型。
我已按照 https://www.tensorflow.org/guide/profiler
上的步骤操作
运行 pip freeze | grep tensorboard
显示:
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-profile==2.8.0
tensorboard-plugin-wit==1.8.1
运行 /sbin/ldconfig -N -v $(sed 's/:/ /g' <<< $LD_LIBRARY_PATH) | grep libcupti
显示:
/sbin/ldconfig.real: Path `/usr/local/cuda-11.6/targets/x86_64-linux/lib' given more than once
/sbin/ldconfig.real: Path `/usr/local/cuda-11.4/targets/x86_64-linux/lib' given more than once
/sbin/ldconfig.real: Can't stat /usr/local/lib/x86_64-linux-gnu: No such file or directory
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib' given more than once
libcupti.so.11.6 -> libcupti.so.2022.1.1
libcupti.so.11.4 -> libcupti.so.2021.2.0
/sbin/ldconfig.real: /usr/lib/wsl/lib/libcuda.so.1 is not a symbolic link
libcupti.so.10.1 -> libcupti.so.10.1.208
/sbin/ldconfig.real: /lib/x86_64-linux-gnu/ld-2.31.so is the dynamic linker, ignoring
说明页面不清楚应该显示什么输出,但我假设对 libcupti.so 的引用表示成功。
Tensorboard 以 tensorboard --logdir logs/
启动,输出:
2022-04-08 17:42:03.872466: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-04-08 17:42:03.947825: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-04-08 17:42:03.948334: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
NOTE: Using experimental fast data loading logic. To disable, pass
"--load_fast=false" and report issues on GitHub. More details:
https://github.com/tensorflow/tensorboard/issues/4784
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
但是 tensorboard GUI 不显示配置文件选项卡:
我错过了什么?
我缺少添加张量板回调的代码:
tensorboard_callback = tf.keras.callbacks.TensorBoard(
log_dir=os.path.join(self.run_folder.value, 'logs'),
histogram_freq=1,
profile_batch='100,120'
我正在尝试设置 tensorboard 以显示配置文件信息,以便我可以调试慢速模型。
我已按照 https://www.tensorflow.org/guide/profiler
上的步骤操作运行 pip freeze | grep tensorboard
显示:
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-profile==2.8.0
tensorboard-plugin-wit==1.8.1
运行 /sbin/ldconfig -N -v $(sed 's/:/ /g' <<< $LD_LIBRARY_PATH) | grep libcupti
显示:
/sbin/ldconfig.real: Path `/usr/local/cuda-11.6/targets/x86_64-linux/lib' given more than once
/sbin/ldconfig.real: Path `/usr/local/cuda-11.4/targets/x86_64-linux/lib' given more than once
/sbin/ldconfig.real: Can't stat /usr/local/lib/x86_64-linux-gnu: No such file or directory
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib' given more than once
libcupti.so.11.6 -> libcupti.so.2022.1.1
libcupti.so.11.4 -> libcupti.so.2021.2.0
/sbin/ldconfig.real: /usr/lib/wsl/lib/libcuda.so.1 is not a symbolic link
libcupti.so.10.1 -> libcupti.so.10.1.208
/sbin/ldconfig.real: /lib/x86_64-linux-gnu/ld-2.31.so is the dynamic linker, ignoring
说明页面不清楚应该显示什么输出,但我假设对 libcupti.so 的引用表示成功。
Tensorboard 以 tensorboard --logdir logs/
启动,输出:
2022-04-08 17:42:03.872466: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-04-08 17:42:03.947825: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-04-08 17:42:03.948334: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:922] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
NOTE: Using experimental fast data loading logic. To disable, pass
"--load_fast=false" and report issues on GitHub. More details:
https://github.com/tensorflow/tensorboard/issues/4784
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
但是 tensorboard GUI 不显示配置文件选项卡:
我错过了什么?
我缺少添加张量板回调的代码:
tensorboard_callback = tf.keras.callbacks.TensorBoard(
log_dir=os.path.join(self.run_folder.value, 'logs'),
histogram_freq=1,
profile_batch='100,120'