Conversation
…dataR into keller-mark/zarr
…milar to HDF5AnnData
Simplify how obs and var names handled in ZarrAnnData
More zarr-related changes
Update comments
rcannood
left a comment
rcannood
left a comment
There was a problem hiding this comment.
Fantastic work @keller-mark and @Artur-man !
I went through the PR for a first time and left some minor comments. I will review the code by running it a couple of times next :)
R/read_zarr_helpers.R
Outdated
| attrs <- g$get_attrs()$to_list() | ||
|
|
||
| if (!all(c("encoding-type", "encoding-version") %in% names(attrs))) { | ||
| path <- name |
There was a problem hiding this comment.
where are a lot of linting issues in this file -- could you run lintr::lint_package() and fix any issues that pop up?
There was a problem hiding this comment.
done ... did a full lint_package check and corrected some R check issues too!
inst/extdata/example2.zarr/.zgroup
Outdated
There was a problem hiding this comment.
This file should probably be removed
| expect_equal(array_h5ad, array_zarr) | ||
| }) | ||
|
|
||
| test_that("reading mappings is same for h5ad and zarr", { |
There was a problem hiding this comment.
There is a difference between rhdf5, Rarr and h5py on how rec arrays and structured data types are read.
To me, h5py and Rarr reads it correctly, but rhdf5 does not:
Huber-group-EMBL/rhdf5#192
There was a problem hiding this comment.
Maybe we need an issue for this here as well? From our point of view it doesn't matter so much what is returned by {rhdf5} as long as what we return from the AnnData matches Python.
Going the other way (writing a recarray) is more difficult because we don't know when to use that format. I don't think we test this. Also, I don't think the AnnData disk spec really covers alternative arrays like recarray.
|
We are really close, tests are passing on my local but we have to wait until changes in Thanks @Bisaloo again for answering all issues of Rarr! |
|
As discussed in the bioconductor zulip, you're not likely to get up-to-date macOS x86 binaries for Rarr. So you'll need to either force installing from source, or add the bioc r-universe as an additional repository for this test to pass. Alternatively, setting 1.11.24 as minimum required version for Rarr might do the trick to force installing from source when a recent enough binary isn't available. |
|
Bencher Report| Branch | keller-mark/zarr |
| Testbed | ubuntu-latest |
🐰 View full continuous benchmarking report in Bencher
⚠️ WARNING: Truncated view!The full continuous benchmarking report exceeds the maximum length allowed on this platform.
Could you please explain what the issue with this is? Is it something on the Bioconductor side so do we need to look into it? I'm guessing the current release check is also expected to fail because we need a later version of {Rarr}? |
BioC macOS x86 builder is no longer working. They're planning on pulling the binaries from r-universe, as they are doing with windows binaries, but it's still a work in progress. But you can still work around the issue if you need to. By requiring installation from source (this should happen automatically if you request a version more recent than what the binaries provide). Or by getting the binaries from r-universe yourself.
Yes, exactly |
5 participants
Fixes #91
These changes are from both me and @Artur-man
The main public-facing changes here are:
ZarrAnnDataclassread_zarrandwrite_zarrtop-level functionsfrom_Seurat(output_class="ZarrAnnData")from_SingleCellExperiment(output_class="ZarrAnnData")Internally:
read_zarr_helpers.Ris the zarr analog ofread_h5ad_helpers.Rwrite_zarr_helpers.Ris the zarr analog ofwrite_h5ad_helpers.Rinst/extdata/example.zarr(this makes the diff noisy, apologies)test-Zarr-read.R(35 new tests)test-Zarr-write.R(70)test-ZarrAnnData.R(26)test-h5ad-zarr.R(17)A number of these functions generate warnings in the R console that are intended to be followed up on to improve the code (and should probably be resolved as end users may not appreciate them), but the tests still pass despite these warnings.
Known things that are not implemented here:
recarraysmode = c("r", "r+", "a", "w", "w-", "x")parameter value