Skip to content

Custom data loaders in Base Runner class #77

@gespyrop

Description

@gespyrop

Hello and thank you for this framework. I have a suggestion that might improve the reusability of the Runner.

After looking at train_nhp_with_features.py, I was wondering whether the TPPRunner (or another Runner subclass) could be used in such custom scenarios.

Using a custom model is already possible since TorchBaseModel.generate_model_from_config searches for TorchBaseModel subclasses named after the given model_id.

However, using custom data loaders is not possible since the Runner directly instantiates a TPPDataLoader which is in turn tightly coupled to the TPPDataset and EventTokenizer classes.

I believe providing a way to inject a custom TPPDataLoader subclass in the Runner would enable it to be used with custom datasets and tokenizers (e.g. TPPDatasetV2 and EventTokenizerV2 from train_nhp_with_features.py).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions