What’s so great about derivative-free solvers for SDEs?

Question

I am trying to familiarise myself with SDEs and have been reading a few review papers on the topic. They leave the impression that a great deal of work has been put into solvers that are derivative-free. To my understanding this means that for a DDE like $$\newcommand\diff{\mathop{}\!\mathrm{d}} \diff X = f(X)\diff t + g(X) \diff W, $$ the derivatives of $f$ and $g$ are not required for the method (correct me if I am wrong).

I can understand that this property is useful in some applications where the derivative is difficult or computationally infeasible to obtain or does not exist. However, I would not expect such problems to be very relevant in application.

This suggests to me that at least one of the following applies:

• There is some further relevant advantage to derivative-free solvers that I am missing.

• Problems where derivative-free solvers are required (due to the above reason) are more relevant than I think they are.

• The demand for derivative-free solvers is lower than the “supply”, i.e., the attention given to them by those who develop solvers.

Which is it?

Answer 1

I can understand that this property is useful in some applications where the derivative is difficult or computationally infeasible to obtain or does not exist. However, I would not expect such problems to be very relevant in application.

If an analytical solution for the derivative is not known, it's very costly and error prone. Calculating the Jacobian is $n^2$ entries, but numerical differentiation techniques will need to do multiple function calls per entry. Then, to get this right, numerical differentiation techniques have to divide by a small number when computing the derivative, which causes a lot of numerical issues.

With autodifferentiation tools this cost is reduced, but it can still be significant. So when analytical Jacobians are not prescribed, it's usually good to stay away from methods which require derivatives.

However, I would not expect such problems to be very relevant in application.

For most things like nonlinear SPDEs or large systems of SDEs (1000's) coming from biology, getting the Jacobian written out can be nearly impossible and error prone. I would say that it's the other way around: expecting an analytical Jacobian to be provided is not a good idea.

There are some further advantages as well. Runge-Kutta methods are derivative-free methods, and they can do a lot of coefficient optimization.

The demand for derivative-free solvers is lower than the “supply”, i.e., the attention given to them by those who develop solvers.

That is not the case. In DifferentialEquations.jl derivative-free methods were implemented before the KPS Stochastic Taylor Series methods because, for most users, it will lead to ease-of-use and increased performance. That said, in the field of differential equations, you can always find a counter example where that's not the case, so I do plan on implementing some methods which explicitly use derivatives. However, I am sure most users will probably just default to the derivative-free methods because the cognitive load on their end is much lower.

Answer 2

I am not an expert in specifically stochastic differential equations, but I would assume that my answer will still be of some value.

1. Computation of the derivative can be challenging, as you mentioned in your question. However, this would be even more pronounced in a multidimensional case, as one would have to calculate Jacobian matrices ($n^2$ entries). So, non-derivative-free solvers will suffer from the curse of dimensionality. The situation becomes even worse when higher-order derivatives are required for a scheme.
2. Computation of a derivative by itself generally amplifies numerical noise. So, for example, if the underlying function ($f$ or $g$) is not analytical, the error in the derivative might completely distort the solution.