convert_pydantic_errors

configs.convert_pydantic_errors

Convert Pydantic validation errors to a readable string format.

This function takes a Pydantic ValidationError and formats it into a human-readable string that shows the field path and error message for each validation failure.

Parameter	Type	Default	Description
validation_error	PydanticValidationError	-	The Pydantic ValidationError to convert.

Returns: str (Formatted error string with field paths and error messages separated by semicolons.)

EvaluationConfig

configs.EvaluationConfig

Evaluation configuration model.

This class defines the configuration parameters for model evaluation including tuning loss selection, window generation, and metric computation settings.

__init__(self, task_path=None, task_paths=None, tuning_loss='mae', max_windows=5, max_num_variates=None, num_samples=100, num_quantiles=10, point_forecast_statistic='mean')

Parameter	Type	Default	Description
task_path	Optional[str]	None	Single task path pattern
task_paths	Optional[List[str]]	None	List of task paths; when set, overrides task_path for multiple tasks
tuning_loss	Optional[Literal["mae", "mase", "mape", "rmse"]]	"mae"	Tuning loss for trainable models such as ARIMA, LSTM, DeepAR, SVR. Only deterministic (point) metrics are allowed: mae, mase, mape, rmse.
max_windows	int	5	Maximum number of rolling windows to generate for evaluation (must be between 1 and 5, inclusive)
max_num_variates	Optional[int]	None	Maximum number of variates to extract from dataset for evaluation (use None for all variates)
num_samples	int	100	Number of samples to generate for stochastic metrics
num_quantiles	int	10	Number of quantiles to compute for quantile-based metrics
point_forecast_statistic	Literal["mean", "median"]	"mean"	Statistic to use for converting stochastic predictions to point forecasts

ModelConfig

configs.ModelConfig

Model configuration model.

This class defines the configuration for a single model including its name and hyperparameter search space. The hyperparameter values are specified as lists to enable grid search over all combinations.

__init__(self, model_name: str, **kwargs)

Parameter	Type	Default	Description
model_name	str	-	Name of the model (must match folder name in models directory).
**kwargs		-	(undocumented)

DatasetConfig

configs.DatasetConfig

Dataset configuration model for individual task folders.

This class defines dataset-specific configuration including file name, missing value handling strategy, and normalization settings.

__init__(self, file_name, handle_missing='interpolate', normalize=True)

Parameter	Type	Default	Description
handle_missing	Literal["interpolate", "mean", "median", "drop", "forward_fill", "backward_fill"]	"interpolate"	Strategy for handling missing values
file_name	str	-	Dataset file name
normalize	bool	True	Whether to normalize the data

TaskConfig

configs.TaskConfig

Task configuration model for individual task folders.

This class defines task-specific configuration including task name, paths, forecast horizon, context window, and dataset settings.

__init__(self, task_name, task_path, forecast_horizon, context_window, dataset)

Parameter	Type	Default	Description
task_name	str	-	Task name (must match folder name)
task_path	str	-	Task path
forecast_horizon	int	-	Number of steps to forecast ahead (max 128)
context_window	int	-	Number of context steps for training
dataset	DatasetConfig	-	Dataset configuration for this task

EvaluationSetting

configs.EvaluationSetting

System-wide evaluation settings configuration.

This class defines global settings for logging, TensorBoard, and conda environment management across all models and tasks.

__init__(self, file_logging, console_logging, tensorboard_logging, conda_env_prefix, reinstall_conda, file_log_level='DEBUG', console_log_level='INFO', verbose=False)

Parameter	Type	Default	Description
file_logging	bool	-	Enable file logging
file_log_level	Literal["DEBUG", "INFO", "WARNING", "ERROR"]	"DEBUG"	File logging level (DEBUG, INFO, WARNING, ERROR)
console_logging	bool	-	Enable console logging
console_log_level	Literal["DEBUG", "INFO", "WARNING", "ERROR"]	"INFO"	Console logging level (DEBUG, INFO, WARNING, ERROR)
tensorboard_logging	bool	-	Enable TensorBoard logging
conda_env_prefix	str	-	Prefix for conda environment names
reinstall_conda	bool	-	Whether to reinstall the conda environments for each model
verbose	bool	False	Whether to print verbose output

JobConfig

configs.JobConfig

Unified configuration model for a single benchmarking job.

This class aggregates all configuration components for a single job execution, combining evaluation settings, model configuration, task configuration, and runtime settings into a single object.

__init__(self, evaluation_config: EvaluationConfig, evaluation_setting: EvaluationSetting, model_config: ModelConfig, model_setting: Dict[str, Any], task_config: TaskConfig, run_path: str, task_datasets_dir: Optional[str] = None)

Initialize job configuration with all components.

Parameter	Type	Default	Description
evaluation_config	EvaluationConfig	-	Evaluation-specific configuration.
evaluation_setting	EvaluationSetting	-	System-wide evaluation settings.
model_config	ModelConfig	-	Model-specific configuration and hyperparameters.
model_setting	Dict[str, Any]	-	Model execution settings.
task_config	TaskConfig	-	Task-specific configuration.
run_path	str	-	Path to run directory for outputs.
task_datasets_dir	Optional[str]	None	Directory for pickled task datasets; set by BenchmarkRunner.

to_dict(self) -> Dict[str, Any]

Convert this JobConfig object to a dictionary for JSON serialization.

This method serializes all components of this JobConfig into a dictionary format that can be safely converted to JSON. Pydantic models are converted using model_dump(), and ModelConfig attributes are extracted from its __dict__.

Returns: Dict[str, Any] (Dictionary representation of the JobConfig suitable for JSON serialization.)