input_data and input_token mismatch for empty timewindow in validation

[shmh40]

What happened?

Input data and input tokens have different lengths in validation if we are processing an empty sample.

File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/run_train.py", line 130, in train_continue_from_args
trainer.run(cf, devices, args.from_run_id, args.mini_epoch)
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/train/trainer.py", line 367, in run
self.validate(mini_epoch, self.validation_cfg, self.batch_size_validation_per_gpu)
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/train/trainer.py", line 531, in validate
for bidx, batch in enumerate(dataset_val_iter):
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 708, in next
data = self._next_data()
^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1480, in _next_data
return self._process_data(data)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/utils/data/dataloader.py", line 1505, in _process_data
data.reraise()
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/_utils.py", line 733, in reraise
raise exception
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop
data = fetcher.fetch(index) # type: ignore[possibly-undefined]
^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/.venv/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 42, in fetch
data = next(self.dataset_iter)
^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/datasets/multi_stream_data_sampler.py", line 759, in iter
batch = self._get_batch(idx, num_forecast_steps)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/datasets/multi_stream_data_sampler.py", line 682, in _get_batch
sdata = self._build_stream_data(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/datasets/multi_stream_data_sampler.py", line 503, in _build_stream_data
stream_data = self._build_stream_data_input(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/users/shickman/work/wg_base/test-kerem-pr/WeatherGenerator/src/weathergen/datasets/multi_stream_data_sampler.py", line 393, in _build_stream_data_input
token_data = input_tokens[-(step + 1)]
~~~~~~~~~~~~^^^^^^^^^^^^^
IndexError: list index out of range

What are the steps to reproduce the bug?

validation when training/finetuning on a dataset with empty time windows.

Hedgedoc link to logs and more information. This ticket is public, do not attach files directly.

No response