run_lsf_eval
Evaluates a TOTO model on LSF datasets, supporting multiple configurations and summarizing results.
EvalTask
run_lsf_eval.EvalTask
A dataclass representing an evaluation task for a TOTO model on LSF datasets.
__init__(self, dataset, checkpoint_path, data_split, prediction_length, context_length, eval_stride, batch_size, num_samples, samples_per_batch, seed, use_kv_cache)
(Automatically generated by @dataclass)
| Parameter | Type | Default | Description |
|---|---|---|---|
dataset |
LSFDatasetName |
- | (undocumented) |
checkpoint_path |
str |
- | (undocumented) |
data_split |
str |
- | (undocumented) |
prediction_length |
int |
- | (undocumented) |
context_length |
int |
- | (undocumented) |
eval_stride |
int |
- | (undocumented) |
batch_size |
int |
- | (undocumented) |
num_samples |
int |
- | (undocumented) |
samples_per_batch |
int |
- | (undocumented) |
seed |
int |
- | (undocumented) |
use_kv_cache |
bool |
- | (undocumented) |
evaluate_checkpoint
run_lsf_eval.evaluate_checkpoint(task)
Evaluate a TOTO model on LSF datasets.
Fetches the model from Hugging Face Hub, evaluates it on LSF datasets, and returns the evaluation results as a DataFrame.
| Parameter | Type | Default | Description |
|---|---|---|---|
task |
EvalTask |
- | (undocumented) |
Returns: pd.DataFrame (The evaluation results.)