recommend robust method for iteratively calculated functions

[tpapp]

I have a computation that solves an implicit equation using iterative numerical methods (bisection, Brent, etc). I would like to use FiniteDifferences.jl for unit testing partials I write for ChainRulesCore.

The problem is that since the calculated function is not technically continuous, using eg central_fdm(5, 1) gives me bogus results (occasionally off by a factor of 5--10). I had some success with a high number of grid points, eg central_fdm(20, 1), but too much adaptation (eg 2 steps) makes things go bad again.

I would appreciate recommendations on how to deal with this in a robust way for unit testing. Since it is for testing, it does not have to be super-fast, just robust.

[wesselb]

Hey @tpapp,

How do the results look like it you set factor to a high number, e.g. central_fdm(5, 1; factor=1e4)?

[tpapp]

Improved, compared to central_fdm(5, 1), but central_fdm(20, 1) is still better (2-3 digits more accuracy).

factor = 1e6 works fine though. Is that what I should do in this case, instead of more gridpoints?

[wesselb]

If factor=1e6 works fine, then that would be the more robust solution, I think. factor controls how much error derives from the function evaluations and is relative to the machine epsilon. You can even try something like central_fdm(10, 1; factor=1e6).

[oxinabox]

I would like to use FiniteDifferences.jl for unit testing partials I write for ChainRulesCore.

Aside: If you are not already I recomment using ChainRulesTestUtils, which wraps FiniteDifferences.jl.

The default fdm in ChainRulesTestUtils is central_fdm(5, 1; max_range=1e-2)
That might not work for you (you can pass your own fdm into test_rrule to overwrite

[wesselb]

Hey @tpapp! Would you be able to confirm whether you've found a satisfactory way to compute finite differences? If so, we'll add a recommendation to the README. :)

[tpapp]

@wesselb: I am in the process of finishing up the unit tests for the relevant package, it seems to be working for now but I will get back here once I am done.

[wesselb]

Thanks for responding quickly! No need to rush at all, of course. :)

[tpapp]

@wesselb: I am done, and I can confirm that central_fdm(5, 1; factor = 1e6) works reasonably for my purposes. I can even combine it with max_range to bound away from singularities.

You may close this issue, or keep it as a reminder for the readme change if you prefer.

Thanks for all the help everyone. I appreciate it, and also this great package.

[wesselb]

That's fantastic to hear! I've opened a PR to add a small section to the README (#174).