Conversation
This property was unused and had a type inconsistency (returning torch.tensor([]) instead of list[torch.Tensor] for the empty case). Co-authored-by: Cursor <cursoragent@cursor.com>
Add a `static_inputs: dict[str, str] | None` field to TrainerConfig that maps variable names to file paths. The build method now constructs StaticInputs from this config rather than inferring topography from the fine training data path. Co-authored-by: Cursor <cursoragent@cursor.com>
…nputs - Rename `Topography` class to `StaticInput` and export it publicly - Update `GriddedData` and `PairedGriddedData` to store `static_inputs: StaticInputs` instead of `topography: Topography` - Update `_get_input_from_coarse` in DiffusionModel to accept StaticInputs and loop over all fields rather than a single topography - Rename `build_topography` to `build_static_inputs` in DataLoaderConfig - Update `PairedDataLoaderConfig.build` to pass StaticInputs - Remove unused `get_topography` from CheckpointModelConfig Co-authored-by: Cursor <cursoragent@cursor.com>
Rename topography -> static_inputs throughout the codebase: - train.py: training and validation loop variables - predict.py: downscaler and event downscaler - evaluator.py: evaluator and event evaluator - inference/: downscaler, output config, work items - _deterministic_models.py: model methods - predictors/cascade.py: cascade predictor and config - predictors/composite.py: patch predictor Co-authored-by: Cursor <cursoragent@cursor.com>
Update all test files to use StaticInput/StaticInputs instead of Topography: - test_topography.py: rename Topography -> StaticInput in test cases - test_config.py: pass StaticInputs to DataLoaderConfig.build - test_patching.py: use StaticInputs wrapper for patching tests - test_models.py: update model train/generate tests - test_predict.py: update predictor integration tests - test_train.py: add static_inputs config to test setup - test_train_config.yaml: add static_inputs field - test_inference.py: update inference/downscaler tests - test_output.py: pass StaticInputs to output config build - test_cascade.py: update cascade predictor tests - test_composite.py: update composite predictor tests Co-authored-by: Cursor <cursoragent@cursor.com>
| @dataclasses.dataclass | ||
| class DiffusionModelConfig: | ||
| """ | ||
| f""" |
| coarse_extent[0] * downscale_factor, coarse_extent[1] * downscale_factor | ||
| ), | ||
| paired_batch_data.fine.latlon_coordinates[0], | ||
| static_inputs = StaticInputs( |
There was a problem hiding this comment.
I would consider a helper function for creating static inputs in tests where there are multiple instances.
|
|
||
|
|
||
| def test_StaticInputs_generate_from_patches(): | ||
| def testStaticInputs_generate_from_patches(): |
There was a problem hiding this comment.
test_S
| def testStaticInputs_generate_from_patches(): | |
| def test_StaticInputs_generate_from_patches(): |
|
|
||
|
|
||
| def test_StaticInputs_serialize(): | ||
| def testStaticInputs_serialize(): |
There was a problem hiding this comment.
| def testStaticInputs_serialize(): | |
| def test_StaticInputs_serialize(): |
| else: | ||
| # Join the normalized topography to the input (see dataset for details) | ||
| topo = topography.data.unsqueeze(self._channel_axis) | ||
| topo = static_inputs.fields[0].data.unsqueeze(self._channel_axis) |
There was a problem hiding this comment.
We are going to remove _deterministic_models, right? Only asking because the use of topography only as item[0] maybe is a bit confusing here. Ignore if this is all going to disappear.
| predict_residual: bool | ||
| use_fine_topography: bool = False | ||
| use_amp_bf16: bool = False | ||
| static_inputs: dict[str, str] | None = None |
There was a problem hiding this comment.
Does this field get used? There's also a static inputs on the TrainConfig that gets used to build and is fed into the build of this class.
| raise ValueError( | ||
| "Topography shape must be evenly divisible by data shape. " | ||
| f"Got topography {self.topography.shape} and data {self.shape}" | ||
| f"Got topography {self.static_inputs.shape} and data {self.shape}" |
There was a problem hiding this comment.
Error still references Topography. Does this need to be updated? Could be internal to the StaticInputs so we don't have to reference attribute specifics outside of their source.
|
|
||
|
|
||
| @pytest.mark.parametrize("static_inputs_on_model", [True, False]) | ||
| @pytest.mark.parametrize("static_inputs_on_model", [True]) |
There was a problem hiding this comment.
Assuming the False is removed because we will always expect static inputs on the model? No need for parametrization.
This refactors the downscaling code to use the full
StaticInputsclass ingeneratecalls.DiffusionModel._get_input_from_coarseconcats allStaticInputs.fields.dataas additional input channelsbreaking change:
TrainConfighas a new fieldstatic_inputswhich should be a mapping of{var_name: path_to_dataset}. This should be used to provide the static inputs fields- they will no longer be loaded automatically from the fine dataset. In the future,use_fine_topographyin the config should be deprecated since thestatic_inputsfield is the source of this information.GriddedData, PairedGriddedDataare updated to have aStaticInputsattribute instead of the singleTopographyreferences to variables named
topographyare updated to the more generalstatic_inputtrain, predict, inference, evaluatorentrypoint code and their tests are updated to pass theStaticInputsto generate callslast three commits are minor updates to use the correct number of input channels given the number of static inputs, add the static inputs to the
from_statemethod, and update tests for these changes(Exact diff of v1 branch but used cursor to organize the commits to be more reviewable)