HyperparameterTuner

hyperparameter_tuner.HyperparameterTuner

Performs rolling-window hyperparameter sweeps for a task-model pair.

The tuner evaluates hyperparameter combinations across rolling validation windows, selects optimal parameters based on tuning loss, generates visualizations, and aggregates cross-window evaluation metrics.

__init__(self, job_config: JobConfig, job_id: str | None = None, results_callback: Callable[..., None] | None = None)

Initialize tuner with job configuration.

Parameter	Type	Default	Description
job_config	JobConfig	-	Fully validated JobConfig produced by ConfigManager.generate_run_configs. Provides benchmark settings, task metadata, and model hyperparameter grid.
job_id	str \| None	None	Optional run identifier for result sinks (passed to results_callback).
results_callback	Callable[..., None] \| None	None	Optional callback(job_id, model, task, window_idx, metrics, forecast_data?) for per-window metrics/forecasts (optional persistence layer).

optimize_hyperparameters(self, context_steps: int, train_steps: int, validate_steps: int) -> Tuple[dict, dict]

Evaluate every hyperparameter combination on rolling windows and select the best.

The tuner iterates over the dataset using context/train/validate windows, executes the model for each hyperparameter configuration, logs metrics, generates visualizations, and aggregates cross-window statistics. The best hyperparameters are selected based on the tuning loss metric averaged across validation windows.

Parameter	Type	Default	Description
context_steps	int	-	Number of historical context points supplied to the model.
train_steps	int	-	Number of points used for the training/fit segment within each window.
validate_steps	int	-	Number of points reserved for evaluation within each window.

Returns: Tuple[dict, dict] A tuple containing: 1. Nested dictionary keyed by model name then dataset path, containing averaged evaluation metrics across windows. 2. Nested dictionary keyed by model name then dataset path, containing ordered lists of best hyperparameter assignments for each window.