Metrics Handler

class codestral_ros2_gen.utils.metrics_handler.MetricsHandler(metrics_file)

Bases: object

Handler for collecting, recording, and analyzing code generation metrics.

This class manages per-attempt metrics and overall aggregated metrics for the code generation process. It provides functionalities to record metrics, save and load metrics from a JSONL file, generate summary statistics, analyze error patterns, and produce plots.

Parameters:

metrics_file (str | Path)

__init__(metrics_file)

Initialize the MetricsHandler.

Parameters:

metrics_file (str | Path) – The path to the JSONL file where metrics will be stored.

record_attempt(iteration_number, attempt_number, attempt_metrics)

Record the metrics for a single generation attempt.

Parameters:

• iteration_number (int) – The sequential number of the iteration.

• attempt_number (int) – The sequential number of the attempt.

• attempt_metrics (AttemptMetrics) – An AttemptMetrics instance containing metrics for the attempt.

Return type:

None

_save_metrics()

Save the current metrics DataFrame to the JSONL file.

Raises:

RuntimeError – If saving fails.

Return type:

None

load_metrics()

Load metrics from an existing JSONL file into the internal DataFrame.

Raises:

RuntimeError – If the file is not found or an error occurs during loading.

Return type:

None

get_report()

Generate a report based on the collected metrics.

Returns:

The generated report.

Return type:

str