pnet.Brain_Template.file_FS_surf not a gzipped file

[chiuhoward]

Summary of what happened:

I am trying to run the non-DL version of pNet on my longitudinal resting state data, and provided CIFTIs that are not from HCP. As such, I selected the FS_surf template, but the Data_Input seems to be looking for a gzip file. I’ve also tried unzipping it manually or loading src/pnet/Brain_Template/FreeSurfer_fsaverage5/Brain_Template.json.zip as a json file but neither approach worked.

Command used (and if a helper script was used, a link to the helper script or the command generated):

import pnet
import glob
import os

print(dir(pnet.Brain_Template))

template_path = pnet.Brain_Template.file_FS_surf
print(f"Template exists: {os.path.exists(template_path)}”)
print(f"Template is file: {os.path.isfile(template_path)}”)
if os.path.isfile(template_path):
    print(f"File size: {os.path.getsize(template_path)} bytes”)
    with open(template_path, 'rb') as f:
        print(f"First few bytes: {f.read(10)}”)
        
template_dir = os.path.dirname(template_path)
print(f"Contents of template directory:”)
for item in os.listdir(template_dir):
    print(f"  {item}”)

dir_pnet_result = '/output/pNet_rs_FN17_noxcpd’

# Find all resting state files
bids_dir = '/fmriprep'  # This matches the binding in the shell script
rs_files = glob.glob(os.path.join(bids_dir, 'sub-*/ses-*/func/*task-rest*dtseries.nii'), recursive=True)

print(f"Found {len(rs_files)} resting state files”)

# Write to file list
file_scan = '/data/bids_rs_files.txt'  # This matches the /tmp binding in the shell script
with open(file_scan, 'w') as f:
    for file_path in sorted(rs_files):
        # Adjust the path to match what it will be inside the container
        container_path = file_path.replace('/path/to/fmriprep, '/fmriprep’)
        f.write(container_path + '\n’)

# Parameters
dataType = ‘Surface’
dataFormat = ‘FreeSurfer’
file_Brain_Template = pnet.Brain_Template.file_FS_surf
K = 17
sampleSize = 100  # recommended >= 100
nBS = 50  # recommended >= 10
# nTPoints not specified = use all time points

pnet.workflow_simple(
    dir_pnet_result=dir_pnet_result,
    dataType=dataType,
    dataFormat=dataFormat,
    file_scan=file_scan,
    file_Brain_Template=file_Brain_Template,
    K=K,
    sampleSize=sampleSize,
    nBS=nBS
    # nTPoints omitted to use all available time points
)

Version:

docker://mldataanalytics/fmripnet:latest

Environment (Docker, Singularity / Apptainer, custom installation):

#!/bin/bash

# Paths
FMRIPREP_DIR=“/path/to/fmriprep"
OUTPUT_DIR="/path/to/output"
WORK_DIR=“/path/to/work"
SCRIPT_DIR="/home/users/chiuh/scripts"

# Create directories
mkdir -p ${OUTPUT_DIR}
mkdir -p ${WORK_DIR}

echo "Input directory: ${FMRIPREP_DIR}"
echo "Output directory: ${OUTPUT_DIR}"

# Run pnet workflow
singularity exec --containall --writable-tmpfs --cleanenv \
-B ${FMRIPREP_DIR}:/fmriprep:ro \
-B ${OUTPUT_DIR}:/output:rw \
-B ${WORK_DIR}:/work:rw \
-B ${SCRIPT_DIR}:/scripts:ro \
-B /tmp:/data:rw \
/path/to/pnet_nodl.sif \
bash -c "source /opt/conda/etc/profile.d/conda.sh && conda activate fmripnet && python /scripts/pnet_workflow.py”

Relevant log outputs (up to 20 lines):

WARNING: Not virtualizing pid namespace by configuration
Template path: /opt/conda/envs/fmripnet/lib/python3.8/site-packages/pnet/Brain_Template/FreeSurfer_fsaverage5/Brain_Template.json.zip
Template exists: True
Template is file: True
First few bytes: b'\x1f\xef\xbf\xbd\x08\x08\xef\xbf\xbd\xef’
Found 54 resting state files
Traceback (most recent call last):
File "/scripts/pnet_workflow.py", line 39, in <module>
pnet.workflow_simple(
File "/opt/conda/envs/fmripnet/lib/python3.8/site-packages/pnet/Workflow/Workflow_Func.py", line 266, in workflow_simple
setup_brain_template(dir_pnet_dataInput, file_Brain_Template)
File "/opt/conda/envs/fmripnet/lib/python3.8/site-packages/pnet/Module/Data_Input.py", line 1066, in setup_brain_template
Brain_Template = load_brain_template(file_Brain_Template, logFile=logFile)
File "/opt/conda/envs/fmripnet/lib/python3.8/site-packages/pnet/Module/Data_Input.py", line 917, in load_brain_template
Brain_Template = load_json_setting(file_Brain_Template)
File "/opt/conda/envs/fmripnet/lib/python3.8/site-packages/pnet/Module/Data_Input.py", line 201, in load_json_setting
json_string = file.read()
File "/opt/conda/envs/fmripnet/lib/python3.8/gzip.py", line 292, in read
return self._buffer.read(size)
File "/opt/conda/envs/fmripnet/lib/python3.8/gzip.py", line 479, in read
if not self._read_gzip_header():
File "/opt/conda/envs/fmripnet/lib/python3.8/gzip.py", line 427, in _read_gzip_header
raise BadGzipFile('Not a gzipped file (%r)' % magic)
gzip.BadGzipFile: Not a gzipped file (b'\x1f\xef’)

[MLDataAnalytics]

The template file might be corrupted. Could you share how you prepared your fMRI surface data so we can make a new one?

[chiuhoward]

It was generated from a longitudinal dataset where each individual subject had up to 5 sessions of fMRI data.

fmriprep-23.1.3 was run (older version used so as to be consistent with some other analyses done with the same dataset) and using these flags (including the optional --cifti-output so I could get the space-fsLR_den-91k_bold.dtseries.nii files for each subject and session.

singularity run --cleanenv /path/to/fmriprep-23.1.3.simg \
/path/to/bids/input \
/path/to/output/derivatives participant --participant-label sub-$SLURM_ARRAY_TASK_ID \
-w /path/to/working/directory \
--output-spaces anat fsnative fsaverage MNI152NLin2009cAsym \
--dummy-scans 4 \
--ignore sbref \
--skip-bids-validation \
--skull-strip-t1w skip \
--output-layout legacy \
--bids-filter-file /path/to/filter_T1w.json \
--fs-subjects-dir /file/to/bids/derivatives/freesurfer \
--cifti-output

and this is the content of the filter json file

{
 "t1w": {"acquisition": “regmask”}
}

[MLDataAnalytics]

An updated template file is now available at:
https://github.com/MLDataAnalytics/pNet/tree/main/src/pnet/Brain_Template/FreeSurfer_fsaverage5

A new script for generating template files will be released soon.