ImageSource.save Optics Block

[j-c-c]

Save ImageSource's with Relion >= 3.1 convention for starfiles, with separate data_optics and data_particle blocks.

Some other additions in this PR are:

• Always store amplitudes in doubles
• Rename _rlnAmplitude metadata field to _aspireAmplitude, since _rlnAmplitude is not a valid Relion field name
• Save ImageSource mrcs files with pixel/voxel size in header. Some Relion command line tools extract pixel size (angpix) from here.
• Save with optics block required fields _rlnImageSize and _rlnImageDimensionality
• Testing for added features
• Loading new file format in Relion was verified using several Relion CLI tools with Relion version 5.0.1

[garrettwrong]

Nice, thanks. Couple minor things noted.

Can you add a test that covers slicing the source? I don't think arbitrary slicing will mess up your logic, but it might in the future. Better to add test now while it works :D.

If you haven't let's make sure you can convert some real STAR files that we use. (Read them in and write them out in a re-usable way, ideally with the user doing nothing...). If we can't, that is going to be a problem and I'm not sure we could take the code yet. This may be fine already, so don't interpret it as a code criticism 😇 .

Can we document in a tutorial an example that shows generating a realistic Simulation, saving with this new format, and loading with Relion? (This can be a non-running, display only, example). I see your testing that we can load one back ourselves, but, we're probably not the target application...

Thanks!

[codecov]

Codecov Report

❌ Patch coverage is 98.30508% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.72%. Comparing base (12d94e8) to head (41adb21).
⚠️ Report is 31 commits behind head on develop.

Files with missing lines	Patch %	Lines
src/aspire/source/image.py	98.14%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1337      +/-   ##
===========================================
+ Coverage    90.69%   90.72%   +0.02%     
===========================================
  Files          135      135              
  Lines        14503    14558      +55     
===========================================
+ Hits         13154    13208      +54     
- Misses        1349     1350       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[j-c-c]

Here are some updates for the recent changes to this PR:

1. Always Build Optics Block: as mentioned in the meeting we must always have an optics block if we want to load a starfile in Relion. This is relevant for clean simulations, class average sources, or oriented sources that are all absent of ctf metadata. This was accomplished by using dummy variables (0's) for missing optics block fields. Additionally, when we add dummy variables, an _aspireNoCTF field is appended to the optics block so we can detect the presence of ASPIRE-generated dummy variables when re-loading one of these datasets. A test was added to check the expected behavior.
2. Gallery Example: I added a gallery example in experiments (which does not run) that generates a realistic Simulation, saves it off in a star file, and prints out a Relion command to reconstruct a volume from the data. I've confirmed that this runs and the printed command produces a reasonable volume reconstruction
3. Added a unit test for saving sliced sources.
4. Confirmed that we can save in ASPIRE and load/reconstruct in Relion with the following data:
   • Clean Simulation's
   • Simulation's with CTF filters and noise
   • OrientedSource's: I used the oriented source from the pipeline demo.
   • Experimental Data
     • 10073: load in ASPIRE, save in ASPIRE, load and reconstruct in Relion
     • 10028: load in ASPIRE, save in ASPIRE, load only in Relion (no angles for reconstruction)

[garrettwrong]

Example looks good, thanks for adding. Two tiny changes and two questions.

[j-c-c]

Investigating these failures that popped up after updating the branch. Appears to be related to cvxpy.

[j-c-c]

Investigating these failures that popped up after updating the branch. Appears to be related to cvxpy.

Ok, looks like cvxpy added the use of the numpy.exceptions module in version 1.7.4 which was released yesterday. The exceptions module was added in NumPy 1.25, so is not available in the CI env with NumPy 1.23.5. Working on a patch.

[janden]

Looks great! Just one comment.

[janden]

Maybe document why float64.

[j-c-c]

Sure. Added a comment. Thanks!