Investigate performance of locally trained models

[SteffenRenisch]

We want to understand the performance, convergence etc. of the locally trained models to have a baseline for comparing the swarm trained model.

[oleschwen]

Follows #45 and #23

[SteffenRenisch]

Ideas for evaluation:

• train models on local data for individual sites
• train model on swarm
• train model centralized on challenge data
• train models on challenge data split by sites

Can we evaluate the models on swarm? Can we use the CrossSiteEvaluation workflow for that?

Technically this requires:

• creation of a test split per site
• creation of an evaluation step either as part of the swarm training of as a separate workflow

[SteffenRenisch]

Suggested steps:

• check current log files from local trainings - what information do we get?
• organize brainstorming with TUD/Mevis to define set of metrics and strategy for implementation
• including automatic collection of log files Add mechanism to collect log files from swarm #160

[Ultimate-Storm]

Logs and models from a successful training on 16.12.2025 are stored under
https://drive.google.com/drive/folders/1Y1RKDcvX75BzUHmIegOZJT-a-WS6ZWhP?usp=sharing