fix: Zarr v3 push API improvements and 6D mode fixes #27

hongquanli · 2026-01-25T23:28:26Z

Summary

Follow-up fixes for #25 discovered during Squid integration testing.

Bug Fixes

6D array path: Use array_path="" instead of "0" for 6D mode (array is at root, not /0 subpath)
Store readiness check: Add zarr.json existence check for 6D regions mode to prevent errors during live acquisition
Metadata merging: For 6D mode, merge OME-NGFF attributes into existing array zarr.json instead of overwriting
OME-NGFF dataset path: Use "." for 6D mode (array at root) vs "0" for per_fov mode
TensorStore caching: Add recheck_cached_metadata: True to see fresh writes from writer process during live acquisition

API Changes

start_zarr_acquisition() now requires fov_paths as first parameter (breaking change - Squid updated)
Added input validation: empty fov_paths raises ValueError

Refactoring

Remove single-store mode: Used incorrect axis order. Replaced by:
- per_fov with --n-fov 1 for single FOV cases
- 6d for multi-FOV 6D cases
Simulation script default changed from single to per_fov

Improvements

Improved error logging with error type: {type(e).__name__}: {e}
Updated docstrings for _load_zarr_plane() documenting both modes

Test plan

All 215 tests pass
python simulate_zarr_acquisition.py --structure per_fov --n-fov 4
python simulate_zarr_acquisition.py --structure 6d --n-regions 2 --fovs-per-region 3 3
python simulate_zarr_acquisition.py --structure 6d --n-regions 3 --fovs-per-region 2 4 3
python simulate_zarr_acquisition.py --structure hcs --wells A1 A2 B1 B2 --fov-per-well 2
Verify no black images during live acquisition

🤖 Generated with Claude Code

- 6D mode: array is directly at acquisition.zarr, not at /0 subpath - Add store readiness check for 6D regions mode (retry if zarr.json doesn't exist yet during early acquisition) Ported from Squid repo testing. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR makes follow‑up fixes to the zarr v3 push API, specifically for 6D regions mode, to align behavior with Squid integration testing and improve robustness during live acquisition.

Changes:

Adjust 6D regions mode to open the TensorStore array at the zarr store root (array_path="") instead of under the "0" subdirectory.
Add a readiness check for 6D regions mode that verifies the presence of zarr.json for the region store before building the dask array, mirroring the existing per‑FOV readiness logic.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-25T23:32:32Z

ndviewer_light/core.py

+                    # 6D mode: array is directly at acquisition.zarr, not at /0
+                    ts_arr = open_zarr_tensorstore(region_path, array_path="")
                    if ts_arr is None:
                        logger.debug(


Changing 6D regions mode to use array_path="" means open_zarr_tensorstore will now open the TensorStore at the region root (e.g. .../acquisition.zarr), but simulate_zarr_acquisition.py currently creates the 6D arrays under a "0" subdirectory (see simulate_zarr_acquisition.py:314-332). As a result, the 6D simulation path (--structure 6d) will no longer line up with the viewer’s read path and the viewer won’t see the data written by the simulator. If the goal is to support Squid’s "array at root" layout, consider either (a) updating the simulator to create the 6D array at the store root instead of .../0, or (b) teaching the reader to detect whether the array lives at the root vs "0" and handle both layouts transparently.

Suggested change

# 6D mode: array is directly at acquisition.zarr, not at /0

ts_arr = open_zarr_tensorstore(region_path, array_path="")

if ts_arr is None:

logger.debug(

# 6D mode: prefer array at acquisition.zarr root, but fall back to /0 for

# older/simulated layouts that store the array under a "0" subdirectory.

ts_arr = open_zarr_tensorstore(region_path, array_path="")

if ts_arr is None:

logger.debug(

f"Root-level zarr array not found for region {region_idx}, "

f"trying '0' subdirectory layout"

)

ts_arr = open_zarr_tensorstore(region_path, array_path="0")

if ts_arr is None:

logger.debug(

[Claude Code] Fixed in commit 9a346b0 - updated simulator to create 6D array at zarr root instead of /0 subdirectory.

Copilot · 2026-01-25T23:32:32Z

ndviewer_light/core.py

+        # Check if store is ready for 6D regions mode
+        if self._zarr_6d_regions_mode and self._zarr_region_paths:
+            region_idx, _ = self._global_to_region_fov(fov_idx)
+            if region_idx < len(self._zarr_region_paths):
+                zarr_json = self._zarr_region_paths[region_idx] / "zarr.json"
+                if not zarr_json.exists():
+                    # Store not ready yet, retry later
+                    if self._zarr_acquisition_active:
+                        QTimer.singleShot(500, self._load_current_zarr_fov)
+                    return


The PR description’s test plan suggests running simulate_zarr_acquisition.py --structure 6d_regions, but the simulator’s CLI only accepts --structure values "single", "6d", "per_fov", and "hcs" (simulate_zarr_acquisition.py:591-595). This discrepancy will make it hard to follow the documented test plan; either the CLI needs a 6d_regions option or the PR description/test instructions should be updated to refer to --structure 6d with the appropriate multi-region flags.

[Claude Code] Fixed - updated PR description to use --structure 6d instead of --structure 6d_regions.

…ions The single-store mode used incorrect axis order (T, FOV, C, Z, Y, X) instead of the OME-NGFF standard (FOV, T, C, Z, Y, X). Rather than fix it, remove the mode entirely since it can be replaced by: - per_fov with --n-fov 1 for single FOV cases - 6d_regions for multi-FOV cases Changes: - Remove _zarr_acquisition_path and _zarr_acquisition_store variables - Make fov_paths required in start_zarr_acquisition() (was optional) - Remove single-store handling in _load_zarr_plane() - Update simulation script to remove "single" structure option - Default structure is now "per_fov" instead of "single" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Validates that fov_paths is not empty in start_zarr_acquisition(), matching the validation pattern in start_zarr_acquisition_6d(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The viewer reads 6D arrays from the zarr root (array_path=""), so the simulator must also create arrays there to match. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-25T23:57:52Z

ndviewer_light/core.py

+                    # 6D mode: array is directly at acquisition.zarr, not at /0
+                    ts_arr = open_zarr_tensorstore(region_path, array_path="")


Switching 6D regions mode to open the array at the store root (array_path="") fixes the mismatch with Squid-written datasets, but the simulator currently creates 6D zarrs with the array under a "0" subdirectory, and the static 6D loader _load_zarr_v3_6d() still opens array_path="0". This inconsistency means simulated 6D acquisitions are unlikely to display correctly in live mode, and real 6D datasets with root-level arrays will still fail when opened via the file loader. To avoid hard-to-debug behavior, align all 6D code paths on a single layout (either move simulator writes and _load_zarr_v3_6d to use root-level arrays, or keep using the "0" subpath everywhere and revert this call).

Suggested change

# 6D mode: array is directly at acquisition.zarr, not at /0

ts_arr = open_zarr_tensorstore(region_path, array_path="")

# 6D mode: array is stored under the '0' subdirectory

ts_arr = open_zarr_tensorstore(region_path, array_path="0")

Copilot · 2026-01-25T23:57:52Z

simulate_zarr_acquisition.py

    ap.add_argument(
        "--structure",
-        choices=["single", "6d", "per_fov", "hcs"],
-        default="single",
-        help="Zarr structure type (default: single). "
+        choices=["6d", "per_fov", "hcs"],
+        default="per_fov",


The CLI no longer allows the "single" structure and the PR description says single-store mode is removed, but ZarrAcquisitionSimulator._setup_fov_structure() still has an else: # single branch that configures fov_paths for that mode. Since this branch is now unreachable and _create_zarr_stores() has dropped support for structure == "single", it effectively defines a broken, dead mode and can confuse future readers; consider deleting the leftover branch (and its comments) to keep the simulator aligned with the supported structures.

When the 6D array is created at the zarr root (not /0 subdirectory), tensorstore creates an array zarr.json. Previously, _write_zarr_metadata would overwrite this with a group zarr.json, destroying the array info. Now for 6D mode, we merge the OME-NGFF attributes into the existing array zarr.json instead of overwriting it. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The OME-NGFF metadata `datasets[0].path` should be: - "." when array is at root (6D mode) - "0" when array is at /0 subdirectory (per_fov/hcs mode) Previously it was always "0", which was incorrect for 6D mode. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…rocesses During live acquisition, the writer process and viewer run in separate processes. Without these options, TensorStore may return cached data instead of seeing fresh writes from the writer process. Adding recheck_cached_metadata and recheck_cached_data ensures TensorStore validates data freshness on each read, which is much cheaper than reopening stores for every read. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

recheck_cached_data defaults to True in TensorStore, so explicitly setting it has no effect. Only recheck_cached_metadata needs to be set since its default is "open" (recheck only at open time). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Update _load_zarr_plane() docstring to document both 6D regions and per-FOV modes, and clarify that zeros are returned when no zarr mode is configured - Add unit tests for _write_zarr_metadata() merge_into_existing parameter: - test_merge_into_existing_preserves_array_structure: verifies 6D mode preserves array zarr.json and uses path "." - test_non_merge_creates_group_structure: verifies per_fov mode creates group zarr.json and uses path "0" Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Remove reference to _zarr_acquisition_path which was removed from the implementation. Add test case for fov_paths mode detection. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Add error type to warning log messages (e.g., "OSError: ...") to help distinguish between different failure modes - Add TestZarrApiValidation class with tests for: - Empty fov_paths validation raises ValueError - Mismatched fov_paths/fov_labels truncation behavior Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

hongquanli requested a review from Copilot January 25, 2026 23:29

Copilot started reviewing on behalf of hongquanli January 25, 2026 23:29 View session

Copilot AI reviewed Jan 25, 2026

View reviewed changes

hongquanli changed the title ~~fix: 6D mode array path and store readiness check~~ fix: 6D mode fixes and remove single-store mode Jan 25, 2026

fix: Add input validation for empty fov_paths

d7a9934

Validates that fov_paths is not empty in start_zarr_acquisition(), matching the validation pattern in start_zarr_acquisition_6d(). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

hongquanli requested a review from Copilot January 25, 2026 23:52

Copilot started reviewing on behalf of hongquanli January 25, 2026 23:53 View session

fix: Update 6D simulator to create array at root, not /0 subdirectory

9a346b0

The viewer reads 6D arrays from the zarr root (array_path=""), so the simulator must also create arrays there to match. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot AI reviewed Jan 25, 2026

View reviewed changes

hongquanli and others added 4 commits January 25, 2026 15:58

hongquanli requested a review from Copilot January 26, 2026 02:22

Copilot started reviewing on behalf of hongquanli January 26, 2026 02:22 View session

hongquanli and others added 3 commits January 25, 2026 18:25

fix: Update test to match is_zarr_push_mode_active implementation

fad892a

Remove reference to _zarr_acquisition_path which was removed from the implementation. Add test case for fov_paths mode detection. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

hongquanli changed the title ~~fix: 6D mode fixes and remove single-store mode~~ fix: Zarr v3 push API improvements and 6D mode fixes Jan 26, 2026

hongquanli merged commit fbff5e4 into main Jan 26, 2026
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Zarr v3 push API improvements and 6D mode fixes #27

fix: Zarr v3 push API improvements and 6D mode fixes #27

Uh oh!

hongquanli commented Jan 25, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 25, 2026

Uh oh!

hongquanli Jan 25, 2026

Uh oh!

Copilot AI Jan 25, 2026

Uh oh!

hongquanli Jan 25, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 25, 2026

Uh oh!

Copilot AI Jan 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		# 6D mode: array is directly at acquisition.zarr, not at /0
		ts_arr = open_zarr_tensorstore(region_path, array_path="")

fix: Zarr v3 push API improvements and 6D mode fixes #27

fix: Zarr v3 push API improvements and 6D mode fixes #27

Uh oh!

Conversation

hongquanli commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Bug Fixes

API Changes

Refactoring

Improvements

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

hongquanli Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

hongquanli Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hongquanli commented Jan 25, 2026 •

edited

Loading