如何以编程方式读取 Tensorboard 文件？

Question

如何在不启动 GUI tensorboard --logdir=... 的情况下编写 python 脚本来读取 Tensorboard 日志文件，提取损失和准确度以及其他数值数据？

Answer 1

可以使用TensorBoard的Python类或者脚本来提取数据：

If you'd like to export data to visualize elsewhere (e.g. iPython Notebook), that's possible too. You can directly depend on the underlying classes that TensorBoard uses for loading data: python/summary/event_accumulator.py (for loading data from a single run) or python/summary/event_multiplexer.py (for loading data from multiple runs, and keeping it organized). These classes load groups of event files, discard data that was "orphaned" by TensorFlow crashes, and organize the data by tag.

As another option, there is a script (tensorboard/scripts/serialize_tensorboard.py) which will load a logdir just like TensorBoard does, but write all of the data out to disk as json instead of starting a server. This script is setup to make "fake TensorBoard backends" for testing, so it is a bit rough around the edges.

使用EventAccumulator：

# In [1]: from tensorflow.python.summary import event_accumulator  # deprecated
In [1]: from tensorboard.backend.event_processing import event_accumulator

In [2]: ea = event_accumulator.EventAccumulator('events.out.tfevents.x.ip-x-x-x-x',
   ...:  size_guidance={ # see below regarding this argument
   ...:      event_accumulator.COMPRESSED_HISTOGRAMS: 500,
   ...:      event_accumulator.IMAGES: 4,
   ...:      event_accumulator.AUDIO: 4,
   ...:      event_accumulator.SCALARS: 0,
   ...:      event_accumulator.HISTOGRAMS: 1,
   ...:  })

In [3]: ea.Reload() # loads events from file
Out[3]: <tensorflow.python.summary.event_accumulator.EventAccumulator at 0x7fdbe5ff59e8>

In [4]: ea.Tags()
Out[4]: 
{'audio': [],
 'compressedHistograms': [],
 'graph': True,
 'histograms': [],
 'images': [],
 'run_metadata': [],
 'scalars': ['Loss', 'Epsilon', 'Learning_rate']}

In [5]: ea.Scalars('Loss')
Out[5]: 
[ScalarEvent(wall_time=1481232633.080754, step=1, value=1.6365480422973633),
 ScalarEvent(wall_time=1481232633.2001867, step=2, value=1.2162202596664429),
 ScalarEvent(wall_time=1481232633.3877788, step=3, value=1.4660096168518066),
 ScalarEvent(wall_time=1481232633.5749283, step=4, value=1.2405034303665161),
 ScalarEvent(wall_time=1481232633.7419815, step=5, value=0.897326648235321),
 ...]

size_guidance:

size_guidance: Information on how much data the EventAccumulator should
  store in memory. The DEFAULT_SIZE_GUIDANCE tries not to store too much
  so as to avoid OOMing the client. The size_guidance should be a map
  from a `tagType` string to an integer representing the number of
  items to keep per tag for items of that `tagType`. If the size is 0,
  all events are stored.

Answer 2

要完成 user1501961 的回答，您可以使用 pandas pd.DataFrame(ea.Scalars('Loss)).to_csv('Loss.csv')

轻松地将标量列表导出到 csv 文件

Answer 3

对于任何感兴趣的人，我已经将 user1501961 的答案改编成一个函数，用于将张量板标量解析为 pandas 数据帧的字典：

from tensorboard.backend.event_processing import event_accumulator
import pandas as pd


def parse_tensorboard(path, scalars):
    """returns a dictionary of pandas dataframes for each requested scalar"""
    ea = event_accumulator.EventAccumulator(
        path,
        size_guidance={event_accumulator.SCALARS: 0},
    )
    _absorb_print = ea.Reload()
    # make sure the scalars are in the event accumulator tags
    assert all(
        s in ea.Tags()["scalars"] for s in scalars
    ), "some scalars were not found in the event accumulator"
    return {k: pd.DataFrame(ea.Scalars(k)) for k in scalars}

如何以编程方式读取 Tensorboard 文件？

How do you read Tensorboard files programmatically?

python

machine-learning

tensorflow

tensorboard