Robustar future roadmap beyond v0.3

[PaulCCCCCCH]

Version 0.3 will be a good one to demo to users and gather their feedback, especially on the usefulness of features. We expect to pivot on the feature side. However, there are some things we need to rethink, no matter how we pivot, and we will need a detailed planning for them. Here is a (maybe incomplete) list of items

• Robustar has a core set of functionalities: model training & evaluation & management, model-dataset insights (influcence, visualization...), dataset exploration & augmentation. We may want to extract these into a robustar-core python library with a clearly-defined and generic set of APIs, and decouple the rest of Robustar (front-end , launcher, ...). This will have the following benefits:
  • Others can build their own front end based on our robustar-core library. This is a good open-source pattern to follow.
  • We can release Robustar and robustar-core separately.
  • We can build other things on top of robustar-core. The generic APIs for models and dataset will be a powerful SDK to bootstrap any interactive model training applications. This saves us cost when we pivot.
• We will eventually host Robustar on the cloud, and provide a demo. We need to design or search for an architecture that can handle Robustar workload. I believe this system is generic to all model training workload. The minimum set of functionalities is:
  • Access control (e.g., use aws Cognito)
  • Runtime management (e.g., create a k8s pod (w/ or w/o GPU, in different sizes, depending on user tier) for a connecting user)
  • Data management (e.g., save user dataset / insight artifacts / model ckpt in S3)
  • User management (e.g., hosted DB service)
• We need to identify how we deliver Robustar to different users of different stages, e.g.,
  • For users need a demo, direct all of them to a single running instance of Robustar
  • For users with programming capabilities, provide robustar-core library
  • For research scientists who want fast experiments, provide robustar on cloud.

We will use this issue to document any related design & discussion.