errors when testing "Detect"

[pinghongyeh]

Hi Maxime,

Thank you for proving the tool.
I used the sample files in the tests directory to run "Detect" using web browser.

Here is the command i ran;
./bin/Detect --i ./tests/SourceDataFig3.xlsx --demog ./tests/scores_RISH0_AutoEncoder_demo.csv

Attached is the screenshot showing the error message.
Thank you.
Ping

[chamberm]

Hi Ping,

as far as I recall, those files are the output from Detect. If you want to run a test you have to use the files in the ressource folder (matching by letter for demog and data).

[pinghongyeh]

Hi Maxime,

Thank you for the clarification. Detect runs fine now.

I will test Detect on my own data, which I have used tractometry ouput from TractSeg. Do the streamlines need to align in the template space between subjects ?

Best,
Ping

[pinghongyeh]

Hi Maxime,

I am using my own data files for running DETECT. The demographic files and data tract-profiles were displayed correctly, but it was not display the plots of tract profiles, and with the following errors

FileNotFoundError: [Errno 2] No such file or directory: 'figures/NDI_AF_profile.svg'
Traceback:

File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 554, in run_script
exec(code, module.dict)
File "/home/yehp/bin/Detect-master/bin/../Detect/detect.py", line 163, in
main()
File "/home/yehp/bin/Detect-master/bin/../Detect/detect.py", line 100, in main
fig = explorer.plot_profile(df_data, df_demog, tract_selection, metric, plot_controls, plot_patients, group, show_indiv)
File "/home/yehp/bin/Detect-master/bin/../Detect/utils/explorer.py", line 96, in plot_profile
fig.savefig('figures/'+metric+''+tract_profile+'_profile.svg', dpi=200)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/figure.py", line 3058, in savefig
self.canvas.print_figure(fname, **kwargs)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2325, in print_figure
**kwargs)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 1648, in wrapper
return func(*args, **kwargs)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/_api/deprecation.py", line 389, in wrapper
return func(*inner_args, **inner_kwargs)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/_api/deprecation.py", line 415, in wrapper
return func(*inner_args, **inner_kwargs)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backends/backend_svg.py", line 1321, in print_svg
with cbook.open_file_cm(filename, "w", encoding="utf-8") as fh:
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/cbook/init.py", line 465, in open_file_cm
fh, opened = to_filehandle(path_or_file, mode, True, encoding)
File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/cbook/init.py", line 451, in to_filehandle
fh = open(fname, flag, encoding=encoding)
tractometry_results_test.xlsx
FBA_NBS_demogrphics_detect.csv

Do you any comments in fixing the errors?
Thank you.
Ping

[chamberm]

Hi, you have to add one extra column in the tract profiles file with your subjects ID (same as in the demographics file)  yet the plotting seems to not show the healthy controls - maybe you can check under the hood why that occurs - maybe related to some hardcoded variable for controls name .. Maxime

…

On Wed, Jul 26, 2023 at 11:50 PM pinghongyeh ***@***.***> wrote: Hi Maxime, I am using my own data files for running DETECT. The demographic files and data tract-profiles were displayed correctly, but it was not display the plots of tract profiles, and with the following errors FileNotFoundError: [Errno 2] No such file or directory: 'figures/NDI_AF_profile.svg' Traceback: File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 554, in *run_script exec(code, module.dict) File "/home/yehp/bin/Detect-master/bin/../Detect/detect.py", line 163, in main() File "/home/yehp/bin/Detect-master/bin/../Detect/detect.py", line 100, in main fig = explorer.plot_profile(df_data, df_demog, tract_selection, metric, plot_controls, plot_patients, group, show_indiv) File "/home/yehp/bin/Detect-master/bin/../Detect/utils/explorer.py", line 96, in plot_profile fig.savefig('figures/'+metric+'*'+tract_profile+'_profile.svg', dpi=200) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/figure.py", line 3058, in savefig self.canvas.print_figure(fname, **kwargs) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 2325, in print_figure **kwargs) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backend_bases.py", line 1648, in wrapper return func(*args, **kwargs) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/_api/deprecation.py", line 389, in wrapper return func(*inner_args, **inner_kwargs) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/_api/deprecation.py", line 415, in wrapper return func(*inner_args, **inner_kwargs) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/backends/backend_svg.py", line 1321, in print_svg with cbook.open_file_cm(filename, "w", encoding="utf-8") as fh: File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/cbook/ *init*.py", line 465, in open_file_cm fh, opened = to_filehandle(path_or_file, mode, True, encoding) File "/home/yehp/anaconda3/envs/py37/lib/python3.7/site-packages/matplotlib/cbook/ *init*.py", line 451, in to_filehandle fh = open(fname, flag, encoding=encoding) tractometry_results_test.xlsx <https://github.com/chamberm/Detect/files/12177404/tractometry_results_test.xlsx> FBA_NBS_demogrphics_detect.csv <https://github.com/chamberm/Detect/files/12177405/FBA_NBS_demogrphics_detect.csv> Do you any comments in fixing the errors? Thank you. Ping — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA42N3LAUAL47PICRU3UMNLXSGGMFANCNFSM6AAAAAAZU2LGWE> . You are receiving this because you commented.Message ID: ***@***.***>

[pinghongyeh]

Hi Maxime,

Thanks for looking into this.
Indeed, I was using the file which has the ID column. I have uploaded the updated one.
I have noticed that display was working when DETECT was running using one of our Linux machines, but not the others. I suspect probably due to difference version of Pandas.
Anyhow, I'm still not able to get Analysis running, below is the error message,

Using Z-score and the following tracts:

Computing permutations ... estimated time: 9.47 minutes.
ZeroDivisionError: division by zero
Traceback:

File "/home/yehp/anaconda3/envs/Detect/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 554, in _run_script
exec(code, module.dict)
File "bin/../Detect/detect.py", line 163, in
main()
File "bin/../Detect/detect.py", line 150, in main
AUC, result, fpr, tpr = launcher.run(method, df_data, df_demog, regress, tract_profile, group, hemi, metric, reps)
File "./bin/../Detect/utils/launcher.py", line 86, in run
ratio = numOfdis/numOfhcs

Any further suggestions?
Thank you.
Ping

[chamberm]

Hi, the subjects group have to be 0 for controls, and 1 for patients. I ran the z-score and autoencoder, both now work (make sure to adapt the ODI and NDI features files to correct the ID column). However results aren't great based on a quick 10 iterations run (AUC 0.45 ...). You may need to select which tract features are more important based on your hypothesis. Maxime

…

On Thu, Jul 27, 2023 at 11:03 PM pinghongyeh ***@***.***> wrote: Hi Maxime, Thanks for looking into this. Indeed, I was using the file which has the ID column. I have uploaded the updated one. I have noticed that display was working when DETECT was running using one of our Linux machines, but not the others. I suspect probably due to difference version of Pandas. Anyhow, I'm still not able to get Analysis running, below is the error message, Using Z-score and the following tracts: Computing permutations ... estimated time: 9.47 minutes. ZeroDivisionError: division by zero Traceback: File "/home/yehp/anaconda3/envs/Detect/lib/python3.7/site-packages/streamlit/scriptrunner/script_runner.py", line 554, in _run_script exec(code, module.*dict*) File "bin/../Detect/detect.py", line 163, in main() File "bin/../Detect/detect.py", line 150, in main AUC, result, fpr, tpr = launcher.run(method, df_data, df_demog, regress, tract_profile, group, hemi, metric, reps) File "./bin/../Detect/utils/launcher.py", line 86, in run ratio = numOfdis/numOfhcs Any further suggestions? Thank you. Ping — Reply to this email directly, view it on GitHub <#10 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA42N3MNYQLGUCSLZ3KAYWLXSLJSLANCNFSM6AAAAAAZU2LGWE> . You are receiving this because you commented.Message ID: ***@***.***>

[pinghongyeh]

Hi Maxime,

I am back to work on DETECT. How can I change the tracts to match my file for the display? Currently the DETECT only show the default tracts. I have more tracts in the file though.
Are the AUCROC curves based on all the subjects?
Thank you.
Ping

[chamberm]

Hi Ping,

if you want to add more tracts, you have to specify them in the file detect.py (or inspect.py)

tract_list = ['AF', 'ATR', 'CA', 'CC_1', 'CC_2', 'CC_3', 'CC_4', 'CC_5', 'CC_6', 'CC_7', 'CG', 'CST', 'FX', 'IFO', 'ILF', 'OR', 'SLF_I', 'SLF_II', 'SLF_III', 'UF', 'All']

in the app you can select which ones to turn on and off.

Re: AUCROC, the final reported one (after all iterations) is based on the average of all runs - and yes looped on all subjects.

[ning0430]

Hi Maxime,

I encountered an error while running the z-scores. Is there a way to resolve this issue? I would greatly appreciate your help.

Thank you,

[Ning]

[chamberm]

Hi Ning,

are the other methods working ? (AE or PCA?) is it only z-score who failed?
Could it be that your tract list is not what the tool expects (I hardcoded those coming from tractseg if I recall)

[ning0430]

Hi Maxime,
Thank you for your message. The other methods, AE and PCA, are running smoothly without any issues. It’s only the z-score method that is causing the error. I’ve tried modifying the code several times, but unfortunately, I still haven’t been able to resolve the problem.
The attached image shows the naming of my fiber tracts. They work fine with the AE and PCA methods, but the z-score method fails. I really hope you can help me with this. Could you please provide some guidance or suggestions on how to fix it?
Thank you very much for your help.
Best regards,
Ning