Skip to content

Nisar feature branch v11: extended csxtools to AXIS detector data #98

Merged
ambarb merged 48 commits intoNSLS-II-CSX:masterfrom
nisarnk:nisar-feature-branch_v11
Jul 2, 2025
Merged

Nisar feature branch v11: extended csxtools to AXIS detector data #98
ambarb merged 48 commits intoNSLS-II-CSX:masterfrom
nisarnk:nisar-feature-branch_v11

Conversation

@nisarnk
Copy link
Contributor

@nisarnk nisarnk commented Jun 18, 2025

Summary

This PR extends the csxtools package to support data collected using the AXIS detector and renames/adds functions for improved clarity and performance.

Changes

  • Added new functions for AXIS detector support:

    • get_axis_images
    • get_axis_flatfield
    • get_axis_timestamps
  • Refactored part of the AXIS image correction pipeline:

    • Some NumPy-based operations (e.g., rotate90) are now handled by the C extension module.
    • This makes the package run approximately twice as fast for AXIS data.
  • Updated __init__.py to reflect the new function names.

  • Added backward-compatible aliases to avoid breaking existing code.

Testing

  • All existing tests pass locally.

nisarnk and others added 30 commits February 18, 2025 19:33
…to include roation by 90 degree for AXIS part
Copy link
Member

@ambarb ambarb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nisarnk I've finished reviewing.

there are a few things worth discussing, but most of the changes I am requisition is the uniform use of axis not axis1 (or AXIS not AXIS1)

I'll make some additional issues to so that I can address things that are leftover from helpers

Copy link
Member

@ambarb ambarb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this imports setup tools.Extension 2 times now (lines 4 and 7). what happened to distutils.core? Can you give me a bit more context so I understand the change, @nisarnk ?

@nisarnk
Copy link
Contributor Author

nisarnk commented Jun 26, 2025

this imports setup tools.Extension 2 times now (lines 4 and 7). what happened to distutils.core? Can you give me a bit more context so I understand the change, @nisarnk ?

Got it! Here's a concise PR comment explaining why the three setuptools-related imports are used:


Clarification on the imports in setup.py:

  • from setuptools import Extension: used to define C extension modules (fastccd, image, etc.). This replaces the deprecated distutils.Extension and ensures compatibility with Python 3.12+.
  • import setuptools: needed to call setuptools.setup(...) for building and installing the package.
  • from setuptools.command.build_ext import build_ext: imported to subclass build_ext in CustomBuildExt, which delays NumPy import and strips platform-specific suffixes from compiled .so files.

Each import serves a distinct role in customizing and safely building the C-accelerated Python package.

@nisarnk
Copy link
Contributor Author

nisarnk commented Jun 26, 2025

@nisarnk I've finished reviewing.

there are a few things worth discussing, but most of the changes I am requisition is the uniform use of axis not axis1 (or AXIS not AXIS1)

I'll make some additional issues to so that I can address things that are leftover from helpers

Regarding the change from 'axis1' to 'axis', this is a straightforward but sensitive change. It needs to be made consistently across several files in the package, and if not done carefully, it can lead to unexpected results or bugs due to conflicting names in the namespace. That’s why I considered it a lower-priority issue. The public APIs are already correctly named (e.g., from csxtools import get_axis_images, get_axis_flatfield), and the use of 'axis1' is hidden from the user.

@ambarb
Copy link
Member

ambarb commented Jun 26, 2025

We have made separate GitHub issues to address the items that need to be fixed in subsequent PRs. This one is getting too complex. All code works, either raising errors or completing as expected, just some affordance issues need to be addressed.

  1. CI/CD errors self-assigned to @ambarb fix helper widget to page through images #97
  2. request to make everything axis - needs to be done carefully clean up use of AXIS1 and AXIS #99
  3. too many default tag possibilities for axis functions retrieving data address complex needs of different axis detectors and allow for some automation with tag=None #100

Additionally, everything in helpers/fastccd.py and helpers/overscan.py are code style changes so these can be ignored.

There is 1 unresolved code comment about the roi functionality. Please contribute.

@maffettone @thomashopkins32 ready for one of you to review.

Copy link
Contributor

@maffettone maffettone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with some agita.

  1. In general, it is best practice to separate MAINT PRs from FEAT PRs. This way, a reviewer is only looking at changes that make a difference. Looking through the 36 files changed, many of them are just formatting fixes, which can be largely accepted at face value. This means MAINT can go quickly through, while the followup FEAT PR can get a more detailed look. Not a blocker, just a note for the future :).
  2. I'm not here to yuck anyone's yums, but am a little concerned about the injection of C code to perform optimized array optimizations (when this may already by accomplished by Dask or other libraries). It makes maintenance a bit more challenging. Again, I won't consider that a blocker, just an opinion.
  3. I'm admittedly not an expert with Python C bindings, so I will accept the extra components at face value. Had a look over the array math and that LGTM.

nisarnk added 2 commits July 1, 2025 10:51
Remove Travis CI badge (inactive) and replace with GitHub Actions test badge
@ambarb ambarb merged commit 7d55564 into NSLS-II-CSX:master Jul 2, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments