I keep getting error when running spatialdm_local AttributeError: 'NoneType' object has no attribute 'to_adata'

[smk5g5]

Hi,

I followed the spatial dm example but I keep getting errors when running spatialdm_local.
Below is the code I am running.

start = time.time()
sdm.spatialdm_local(adata, n_perm=1000, method='both', specified_ind=None, nproc=1)     # local spot selection
sdm.sig_spots(adata, method='permutation', fdr=False, threshold=0.1)     # significant local spots
print("%.3f seconds" %(time.time()-start))

And here is the error I am getting.

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[47], line 2
      1 start = time.time()
----> 2 sdm.spatialdm_local(adata, n_perm=1000, method='both', specified_ind=None, nproc=1)     # local spot selection
      3 sdm.sig_spots(adata, method='permutation', fdr=False, threshold=0.1)     # significant local spots
      4 print("%.3f seconds" %(time.time()-start))

File /n/data1/mgh/neuro/petti/lab/Users/khan.saad/Conda_envs/python_3.10/lib/python3.10/site-packages/spatialdm/main.py:275, in spatialdm_local(adata, n_perm, method, specified_ind, nproc, scale_X)
    273 ## different approaches
    274 with threadpool_limits(limits=nproc, user_api='blas'):
--> 275     spot_selection_matrix(adata, ligand, receptor, ind, n_perm, method, scale_X)

File /n/data1/mgh/neuro/petti/lab/Users/khan.saad/Conda_envs/python_3.10/lib/python3.10/site-packages/spatialdm/utils.py:179, in spot_selection_matrix(adata, ligand, receptor, ind, n_perm, method, scale_X)
    176 def spot_selection_matrix(adata, ligand, receptor, ind, n_perm, method, scale_X=True):
    177     # local variables (only live in this function scope)
    178     # normalize raw counts
--> 179     raw_norm = adata.raw.to_adata()
    180     raw_norm.X = csr_matrix([norm_max(X) for X in raw_norm.X.T]).T
    181     import scanpy as sc

AttributeError: 'NoneType' object has no attribute 'to_adata'

[ABU-TO]

After my testing, you might need to store the original expression matrix in .raw.
—> adata.raw = adata.

[smk5g5]

Thank you that fixed it but now I am getting a new error when I am trying to save anndata object using adata.write.

Here is the error.

  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 282, in h5py.h5d.DatasetID.write
  File "h5py/_proxy.pyx", line 147, in h5py._proxy.dset_rw
  File "h5py/_conv.pyx", line 442, in h5py._conv.str2vlen
  File "h5py/_conv.pyx", line 96, in h5py._conv.generic_converter
  File "h5py/_conv.pyx", line 247, in h5py._conv.conv_str2vlen
TypeError: Can't implicitly convert non-string objects to strings
Error raised while writing key 'Ligand1' of <class 'h5py._hl.group.Group'> to /

Can you please comment on why is that and how to circumvent it?

[ABU-TO]

It may be an issue with the current version of Anndata, and I have encountered a similar problem before. I tried to individually save some items stored in the .uns attribute to bypass this bug.
For example:

adata.uns['local_z_p'].to_csv(savedir+'sdm_test_local_z_p.csv')
adata.uns['selected_spots'].to_csv(savedir+'sdm_test_selected_spots.csv')
del adata.uns['local_z_p']
del adata.uns['selected_spots']
# Use sdm for storage
sdm.write_spatialdm_h5ad(adata, filename=savedir+'save_sdm__test_.h5ad')

Additionally, it is also possible that some values stored in the .obs attribute are null, leading to this issue.
adata.obs['item'] = adata.obs['item'] .astype(str)
You might want to give it a try, and in the end, using either write_spatialdm_h5ad or write_h5ad should yield the same result.

[smk5g5]

Do I have to remove all the adata.uns objects. I have not looked at your code in detail but a cursory glance indicates that following uns data might have to be removed before saving the adata object.

adata.uns['global_res'], 
adata.uns['global_I'],
adata.uns['global_stat'],
adata.uns['global_res']
adata.uns['local_stat']
adata.uns['local_z']
adata.uns['local_z_p']
adata.uns['local_perm_p']

Let me know if I am missing something. For many of them to_csv might not work. I tried to pickle.dump(adata_out.uns['local_stat'] some objects but even that is throwing error. I would ideally like to be able to comeback and look at the above things and need to save them to do so. I was wondering if anything else in the uns also needs to be deleted that I am missing?

Also, which version of anndata/scanpy did you test/publish your code (where you did not get any errors while saving anndata object)?

Because I am getting a lot of errors with mine while trying to save the anndata.

My anndata version is : 0.10.3
My scanpy version is : 1.9.6
My spatialdm version : 0.1.0

[ABU-TO]

Have you checked for 'null' values in your 'obs', 'obsm', and 'uns'? It seems that there are some null values in the DataFrame. Personally, I have encountered an issue where 'adata.uns['local_z_p']' and 'adata.uns['selected_spots']' cannot be saved using 'adata.write_h5ad', so I saved them using 'to_csv'.

My anndata version is : 0.8.0
My scanpy version is : 1.9.6
My spatialdm version : 0.1.0

[smk5g5]

I understand why some of unstructured data may not work when writing anndata file. What I don't understand is why I would run into errors when doing pickle dump of that unstructured data. e.g. When i try to do the following :

filehandler = open(b"test_sdm_allunsdata.pkl","wb")
pickle.dump({'global_res':adata.uns['global_res'], 'global_I':adata.uns['global_I'],'global_stat':adata.uns['global_stat'],'global_res':adata.uns['global_res'],'local_stat':adata.uns['local_stat'],
'local_z':adata.uns['local_z'],'local_z_p':adata.uns['local_z_p'],'local_perm_p':adata.uns['local_perm_p']},filehandler)

I still get error saving picke file.