Empty data and factors after model finished training

[dmalzl]

Hi,

I am frequently using your tool to get insight into the biology of my data and I feel its great. So first of all thank you for providing it.
However, I recently tried to replicated the microbiome analysis detailed here. For the sake of convenience I use the interface provided by muon. This worked great previously but I am experiencing some difficulties with the setup described in the microbiome analysis tutorial which in my mind is either the number of groups, the sparseness of the data or a combination of both. The dataset at hand contains 32 individuals over 7 timepoints and we measured 794 species in at least one of these individual x timepoints. In brief, I do the following:

1. Filtering the data to retain only species which have at least 8 measurements which retains 441 (I did this to retain only species which are present in at least 8 individuals but I just realized this is not what the actual code does anyway)
2. I then run MEFISTO via the mu.tl.mofa function (as far as I understand the code there is nothing that would filter the data further, I also tested a couple of things and the dataset gets assigned correctly to the model, which is also confirmed by the model displaying all the right messages and even starting and finishing to train without any errors or warnings)

However, mu.tl.mofa throws an error after the model finished training because it fails to load the data and factors back from the saved model. Looking at it manually with mofax the output is the following:

MOFA+ model: species only quarter 10.h5ad
Samples (cells): 0
Features: 441
Groups: 1 (7), 13 (6), 14 (6), 15 (7), 16 (7), 28 (7), 29 (7), 3 (7), 30 (7), 31 (7), 33 (6), 34 (7), 35 (7), 36 (7), 43 (7), 44 (7), 48 (6), 49 (6), 5 (7), 51 (6), 53 (6), 56 (6), 57 (5), 59 (7), 6 (7), 60 (5), 61 (6), 62 (7), 65 (6), 66 (6), 67 (5), 68 (6)
Views: data (441)
Factors: 10
Expectations: Sigma, W, Z

MEFISTO:
Covariates available: time

and when calling trying to get the data or the factors I only get empty arrays.

My question now is why this is happening and if there is a solution to this. This could also well be a muon related problem but for now I think it is a problem of mofapy2 as the model trains and then saves it with empty factors and data.

Thanks in advance for looking into this

[dmalzl]

I got a slight hint from the python tutorial of the microbiome analysis. I am not able to pinpoint exactly where it happens but I'd like to leave my interpretation of the situation here.

I think the culprit here is/was my use of integer group names. In the tutorial they do something like this:

microbiome_groups = [microbiome.loc[microbiome.group == group,["sample", "group", "month"]].drop_duplicates().set_index("sample") for group in ent.data_opts["groups_names"]]
ent.data_opts["samples_metadata"] = microbiome_groups

after initializing the model. When trying to do the same with my data setting the smooth covariate further downstream fails because the "samples_metadata" list is just empty dataframes. Since looking at the code revealed that "groups_names" is a list of unique values of the grouping column converted to string, this raised my suspicion that the mismatch between string and integer values may be the causing problem here.

Indeed simply converting my group names to some string representation the model finishes training as expected and now also contains the data and metadata / factors as expected

MOFA+ model: species only quarter 10.h5ad
Samples (cells): 206
Features: 441
Groups: mouse_1 (7), mouse_13 (6), mouse_14 (6), mouse_15 (7), mouse_16 (7), mouse_28 (7), mouse_29 (7), mouse_3 (7), mouse_30 (7), mouse_31 (7), mouse_33 (6), mouse_34 (7), mouse_35 (7), mouse_36 (7), mouse_43 (7), mouse_44 (7), mouse_48 (6), mouse_49 (6), mouse_5 (7), mouse_51 (6), mouse_53 (6), mouse_56 (6), mouse_57 (5), mouse_59 (7), mouse_6 (7), mouse_60 (5), mouse_61 (6), mouse_62 (7), mouse_65 (6), mouse_66 (6), mouse_67 (5), mouse_68 (6)
Views: data (441)
Factors: 10
Expectations: Sigma, W, Z

So I guess somewhere in the code there is a conditional for matching values of the grouping column with values of "groups_names" and since they don't match all samples are lost in some way without any complaint.