Preprocessor

preprocessor.Preprocessor

Cleans raw CSV payloads and applies normalization strategies.

The Preprocessor handles parsing raw target data from CSV files, handling missing values using various strategies, normalizing data if configured, and ensuring data is in the correct format for model consumption.

__init__(self, task_config, evaluation_config)

Initialize the preprocessor for a concrete task configuration.

Parameter	Type	Default	Description
task_config	TaskConfig	-	Task configuration object that includes dataset metadata and preprocessing directives for the active task.
evaluation_config	EvaluationConfig	-	Evaluation configuration object that includes benchmark settings for evaluation.

clean(self, time_start, freq, target_raw, normalize, handle_missing)

Clean raw target data by parsing, handling missing values, and normalizing.

This method performs the complete preprocessing pipeline: parsing raw target data, capping variates if configured, validating timestamps and frequency, handling missing values, and optionally normalizing the data.

Parameter	Type	Default	Description
time_start	str	-	Start time as string (will be converted to pandas Timestamp).
freq	str	-	Frequency as string (pandas-compatible frequency).
target_raw	str	-	Raw target data as string (JSON-like array format).
normalize	bool	-	Whether to normalize the data using StandardScaler.
handle_missing	str	-	Strategy for handling missing values. Options: 'drop', 'mean', 'median', 'interpolate', 'forward_fill', 'backward_fill'.

Returns: Tuple[np.ndarray, str, str, np.ndarray, Optional[StandardScaler]]

Key	Type	Description
timestamps	np.ndarray	Cleaned timestamps array of shape (num_steps,).
time_start	str	Sanitized start timestamp string.
freq	str	Validated frequency string.
target	np.ndarray	Processed target array of shape (num_steps, num_targets).
scaler	Optional[StandardScaler]	Scaler instance if normalization was applied, None otherwise.

Raises: * ValueError: If target array is empty, has incorrect dimensions, or frequency is invalid or missing.