Working EIG + CUDA AD

[kshyatt]

Added the Hermitian wrapper to avoid cholesky yelling that V' * V isn't Hermitian.

[github-actions]

Your PR no longer requires formatting changes. Thank you for your contribution!

[kshyatt]

Sigh... this was working locally for me... back to the CUDA mines we go!

[kshyatt]

What the helly, it's passing locally for me still...

[kshyatt]

OK the 1.10 failure is RNG related

[lkdvos]

Was it just accidentally using a pathological case, or is this actually not very stable to begin with?

[kshyatt]

I think the former, considering it worked on 1.12...

[lkdvos]

If tests pass, happy to merge this!

[kshyatt]

Was it just accidentally using a pathological case, or is this actually not very stable to begin with?

Or maybe not. Something really weird on 1.10 here

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
...MatrixAlgebraKitCUDAExt/MatrixAlgebraKitCUDAExt.jl	78.12% <100.00%> (+0.70%)	⬆️
src/MatrixAlgebraKit.jl	100.00% <ø> (ø)
src/pullbacks/eig.jl	92.72% <100.00%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jutho]

There does seem to be something going on, potentially with degenerate eigenvalues. There is a "gauge sensitive" warning.

Output generated during execution of 'mooncake/eig':
┌ 7-element Vector{Tuple{Float64, Float64}}:
│  (-0.701143421209963, -1.2117169941375323)
│  (-13314.483677792308, -1.2117169941375323)
│  (-0.6760447251535879, -1.2117169941375323)
│  (-1.5543476938692038, -1.2117169941375323)
│  (3.792817048932193, -1.2117169941375323)
│  (-92.0990632729372, -1.2117169941375323)
│  (0.09235575050115585, -1.2117169941375323)
│ 
│ ┌ Warning: `eig` cotangents sensitive to gauge choice: (|Δgauge| = 1.3540607507349773)
└ └ @ MatrixAlgebraKit ~/work/MatrixAlgebraKit.jl/MatrixAlgebraKit.jl/src/pullbacks/eig.jl:36

I would also have to look into it more closely to see what is going on. Not sure what this vector of tuples exactly means.

[kshyatt]

@Jutho it's annoyingly hard to interpret, basically it's a comparison of the summed (y, x) differences from AD (right column) and finite differences (left column) for various epsilon in FD.

[Jutho]

Ok so that is why the right column is constant, there is no epsilon in AD. But the left column then shows no convergence or consistency for different epsilon, so I assume there is some degeneracy messing things up. I have been wondering if it would be useful to have the "gauge sensitivity" warning be more informative, to explain what exactly is contributing to this.