Optimal Derivatives of Noisy Simulations

Computational noise in deterministic simulations is as ill-defined a concept as can be found in scientific computing. Roundoff errors, discretizations, numerical solutions to systems of equations, and adaptive techniques can destroy the smoothness of the processes underlying a simulation. Such noise complicates optimization, sensitivity analysis, and other applications that depend on the simulation output.

We present a new method for estimating the computational noise that arises in virtually all numerical HPC simulations. We use an estimate of the computational noise to address a longstanding problem in derivative estimation: How should finite-difference parameters be determined when working with a noisy function? Our near-optimal parameters are easy to compute and come with provable approximation bounds.