Start on CUDA extension

[kshyatt]

This splits off basic support for CUDA tensor maps from #320, and doesn't cover Diagonals or factorizations at all. I made some modifications to things in src to make them a little more generic also. Some things aren't working yet:

• converting from an AdjointTensorMap of a CuTensorMap to a CuTensorMap of a different scalar type
• converting to a CuArray
• the hasfusiontensor(I) tests
• diag/diagm
• index flipping

[github-actions]

Your PR requires formatting changes to meet the project's style guidelines.
Please consider running Runic (git runic main) to apply these changes.

Click here to view the suggested changes.

diff --git a/src/tensors/tensor.jl b/src/tensors/tensor.jl
index e60d4d9..6946796 100644
--- a/src/tensors/tensor.jl
+++ b/src/tensors/tensor.jl
@@ -316,15 +316,15 @@ end
 
 for randf in (:rand, :randn, :randexp, :randisometry)
     _docstr = """
-        $randf([rng=default_rng()], [TorA=Float64], codomain::ProductSpace{S,N₁},
-                 domain::ProductSpace{S,N₂}) where {S,N₁,N₂,T} -> t
-        $randf([rng=default_rng()], [TorA=Float64], codomain ← domain) -> t
+            $randf([rng=default_rng()], [TorA=Float64], codomain::ProductSpace{S,N₁},
+                     domain::ProductSpace{S,N₂}) where {S,N₁,N₂,T} -> t
+            $randf([rng=default_rng()], [TorA=Float64], codomain ← domain) -> t
         
     Generate a tensor `t` with entries generated by `$randf`.
-    The type `TorA` can be used to control the element type and
-    data type generated. For example, if `TorA` is a `CuVector{ComplexF32}`
-    or `ROCVector{Float64}`, then the final output `TensorMap` will have
-    that as its storage type.
+        The type `TorA` can be used to control the element type and
+        data type generated. For example, if `TorA` is a `CuVector{ComplexF32}`
+        or `ROCVector{Float64}`, then the final output `TensorMap` will have
+        that as its storage type.
 
     See also [`Random.$(randf)!`](@ref).
     """

[lkdvos]

I kind of failed to realize that fusiontensor(a, b, c) always returns arrays so these convert(Array, t) based methods are a bit more painful than expected. It might be reasonable to consider testing the GPU methods by comparing to the CPU implementations rather than comparing to the Array implementations?

[kshyatt]

I think I have them mostly working now, actually... let me push in a moment

[codecov]

Codecov Report

❌ Patch coverage is 71.15385% with 30 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
ext/TensorKitCUDAExt/cutensormap.jl	62.50%	27 Missing ⚠️
src/tensors/tensor.jl	86.36%	3 Missing ⚠️

Files with missing lines	Coverage Δ
ext/TensorKitCUDAExt/TensorKitCUDAExt.jl	100.00% <100.00%> (ø)
src/tensors/linalg.jl	82.33% <100.00%> (-0.37%)	⬇️
src/tensors/tensor.jl	85.31% <86.36%> (+0.01%)	⬆️
ext/TensorKitCUDAExt/cutensormap.jl	62.50% <62.50%> (ø)

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jutho]

Haven't looked at the tests in detail yet, but certainly looks very good already.

[kshyatt]

Re the tests, they are mostly copied from the CPU tensors tests. I have added a few to test CUDA specific code paths and to test conversion to/from CPU based TensorMap. Some tests do not work yet (the basic linear algebra ones, sylvester) because some of the functionality isn't yet supported on CUDA.

[kshyatt]

OK I think I've responded to everything 🧐

[kshyatt]

I can't get the format issue to appear locally...

[kshyatt]

OK, is there anything else this needs besides #333 and somehow fixing the format check?

[lkdvos]

Except for my remaining question about the convert, and the comment about the tagged MatrixAlgebraKit version, I'm happy to merge this.

As always, thanks for the enormous amount of work (and patience), much appreciated!

[lkdvos]

This looks like there might be some slight indenting issues?

[kshyatt]

Was me fighting with the formatter