MLflow：如何从现有运行中读取指标或参数？

Question

我尝试以这种方式读取指标：

 data, info = mlflow.get_run(run_id)
 print(data[1].metrics)
 # example of output: {'loss': 0.01}

但它只得到最后一个值。是否可以手动读取特定指标的所有步骤？

Answer 1

我粗暴地解决了：我用特定的[run_id].

读取了特定的[metric_name]的原始文件

path = f'./mlruns/0/[run_id]/metrics/[metric_name]'
with open(path) as f:
    content = f.readlines()
metrics_for_step = [float(x.split(' ')[1]) for x in content]

Answer 2

是的，您可以使用 MlffowClient APIs 获取实验和运行信息。它将 returns MLflow 数据结构作为字典，您可以对其进行迭代以提取 listcomp 中所需的内容。这是一个例子：


def print_experiment_details(experiment_id, run_id):
    """
    Method to print experiment run info and a specific run details
    :param experiment_id: MLflow experiment ID
    :param run_id: MLflow run ID within an experiment
    :return: none
    """
    print("Finished MLflow Run with run_id {} and experiment_id {}".format(run_id, experiment_id))

    # Use MlflowClient API to list experiments and run info
    client = MlflowClient()
    print("=" * 80)
    # Get a list of all experiments
    print("List of all Experiments")
    print("=" * 80)
    [print(pprint.pprint(dict(exp), indent=4))
     for exp in client.list_experiments()]
    print("=" * 80)
    print(f"List Run info for run_id={run_id}")
    print(pprint.pprint(dict(mlflow.get_run(run_id))))

这输出：

Running local model registry=sqlite:///mlruns.db
Finished MLflow Run with run_id 3f3b827dd6814649a2f84ebae09b26c6 and experiment_id 0
================================================================================
List of all Experiments
================================================================================
{   'artifact_location': './mlruns/0',
    'experiment_id': '0',
    'lifecycle_stage': 'active',
    'name': 'ODSC_TUTORIALS',
    'tags': {   'mlflow.note.content': 'This is experiment for getting started '
                                       'with MLflow ...'}}
None
================================================================================
List Run info for run_id=3f3b827dd6814649a2f84ebae09b26c6
{'data': <RunData: metrics={'metric_1': 0.9236238251076615,
 'metric_2': 1.6732389715754346,
 'metric_3': 2.249979396736294}, params={'n_estimators': '3', 'random_state': '42'}, tags={'mlflow.log-model.history': '[{"run_id": "3f3b827dd6814649a2f84ebae09b26c6", '
                             '"artifact_path": "sklearn-model", '
                             '"utc_time_created": "2020-03-18 '
                             '22:25:33.083332", "flavors": {"python_function": '
                             '{"loader_module": "mlflow.sklearn", '
                             '"python_version": "3.7.5", "data": "model.pkl", '
                             '"env": "conda.yaml"}, "sklearn": '
                             '{"pickled_model": "model.pkl", '
                             '"sklearn_version": "0.22.2.post1", '
                             '"serialization_format": "cloudpickle"}}}]',
 'mlflow.note.content': 'This Run is for getting started with MLflow ...',
 'mlflow.runName': 'LOCAL_REGISTRY',
 'mlflow.source.git.commit': '0a3c6a3739deab77631318eca7fb9690b6dbad66',
 'mlflow.source.name': '/Users/julesdamji/gits/tutorials/mlflow/labs/00_get_started.py',
 'mlflow.source.type': 'LOCAL',
 'mlflow.user': 'julesdamji'}>,
 'info': <RunInfo: artifact_uri='./mlruns/0/3f3b827dd6814649a2f84ebae09b26c6/artifacts', end_time=1584570333841, experiment_id='0', lifecycle_stage='active', run_id='3f3b827dd6814649a2f84ebae09b26c6', run_uuid='3f3b827dd6814649a2f84ebae09b26c6', start_time=1584570332914, status='FINISHED', user_id='julesdamji'>}

您可以获得完整代码here

希望对您有所帮助。

Answer 3

我运行遇到了同样的问题，并且能够通过使用 mlflow.tracking.MlflowClient().get_metric_history 获取指标的所有值。这将 return 您使用 mlflow.log_metric(key, value).

记录的每个值

快速示例（未经测试）

import mlflow
trackingDir = 'file:///....'
registryDir = 'file:///...'
runID = 'my run id'
metricKey = 'loss'

client = mlflow.tracking.MlflowClient(
            tracking_uri=trackingDir,
            registry_uri=registryDir,
        )

metrics = client.get_metric_history(runID, metricKey)

From the docs

get_metric_history(run_id, key)[source] Return a list of metric objects corresponding to all values logged for a given metric.

Parameters run_id – Unique identifier for run

key – Metric name within the run

Returns A list of mlflow.entities.Metric entities if logged, else empty list

from mlflow.tracking import MlflowClient

def print_metric_info(history):
    for m in history:
        print("name: {}".format(m.key))
        print("value: {}".format(m.value))
        print("step: {}".format(m.step))
        print("timestamp: {}".format(m.timestamp))
        print("--")

# Create a run under the default experiment (whose id is "0"). Since this is low-level
# CRUD operation, the method will create a run. To end the run, you'll have
# to explicitly end it. 
client = MlflowClient() 
experiment_id = "0" 
run = client.create_run(experiment_id) 
print("run_id:{}".format(run.info.run_id))
print("--")

# Log couple of metrics, update their initial value, and fetch each
# logged metrics' history. 
for k, v in [("m1", 1.5), ("m2", 2.5)]:
    client.log_metric(run.info.run_id, k, v, step=0)
    client.log_metric(run.info.run_id, k, v + 1, step=1)
    print_metric_info(client.get_metric_history(run.info.run_id, k))
client.set_terminated(run.info.run_id)

Answer 4

使用 MLflow 客户端 (MlflowClient)，您可以使用 get_run(id).data:

轻松获取所有或选定的参数和指标

# create an instance of the MLflowClient,
# connected to the tracking_server_url
mlflow_client = mlflow.tracking.MlflowClient(
    tracking_uri=tracking_server_url)

# list all experiment at this Tracking server
# mlflow_client.list_experiments()

# extract params/metrics data for run `test_run_id` in a single dict 
run_data_dict = mlflow_client.get_run(test_run_id).data.to_dictionary()

# list all params and metrics for this run (test_run_id)
# pprint(run_data_dict)

print(run_data_dict['params']['algo'])
print(run_data_dict['metrics']['RMSE'])

MLflow：如何从现有 运行 中读取指标或参数？

MLflow: how to read metrics or params from an existing run?

python

metrics

mlflow

MLflow：如何从现有运行中读取指标或参数？