Rework fixed-point differentiation

[leburgel]

Attempt at reorganizing the fixed-point differentiation approach. The main motivation was to remove the iterscheme keyword that was previously used to select a specific flavor of fixed-point differentiation, which has now become trivial since we only support one value.

As a result it now made more sense to switch the logic around: instead of using a solver algorithm struct as a representation of a gradient algorithm where the type parameter specifies how the gradient is actually computed, I now represent the gradient algorithm as a specific type and store the solver algorithm inside that. This involved coming up with a new name, which ended up being FixedPointGradient, and wiring this into the algorithm selection. It looks a bit silly now since there's only one option, but hopefully this is a good setup for adding different styles of implicit differentiation.

Very open to suggestions and changes, let me know what you think.

[lkdvos]

Left some very small comments, but this looks pretty good to me actually

[github-actions]

After the build completes, the updated documentation will be available here

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...rithms/optimization/fixed_point_differentiation.jl	93.02%	3 Missing ⚠️

Files with missing lines	Coverage Δ
src/Defaults.jl	85.71% <ø> (ø)
src/algorithms/optimization/peps_optimization.jl	94.66% <100.00%> (ø)
src/algorithms/select_algorithm.jl	76.92% <ø> (-1.65%)	⬇️
...rithms/optimization/fixed_point_differentiation.jl	88.79% <93.02%> (-0.56%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[pbrehmer]

This is really great, thanks! I find that it definitely cleans up and improves the entire (fixed-point) gradient framework. I only left a few minor comments.

[pbrehmer]

With all the example changes we have to make sure to rerun all of them such that the rendered version reflect the changes.

[pbrehmer]

Although this is not really necessary, what do we think about renaming this to GradientMode{A}? I think the abbreviation is not really useful here and GradientMode sounds better in my opinion. (And we're already renaming and breaking a lot anyway.)

[leburgel]

I'm happy to rename it. What do you think about GradientAlgorithm as an alternative option? I was never too sure about Mode to be honest.

[pbrehmer]

I like GradientAlgorithm even better since it is more consistent with the rest of our algorithm struct names.

[leburgel]

Then I would go for that.