Exploring Automatic Differentiation in Racket
The code in this repository depends on a working Racket installation. To get started with Racket, see https://racket-lang.org
To install the package, run (in this directory):
raco pkg install
You can also run
raco pkg install http://github.com/ots22/racket-ad-demos
to fetch the package and install it directly from GitHub.
This repository contains implementations of a few automatic differentiation algorithms.
This is a straightfoward implementation of AD with dual-numbers.
#lang racket
(require ad/dual-numbers)
(define (f x y)
(+ (* x x) (* x y)))
(f 10.0 2.0) ; => 120.0
((D/f f) 10.0 2.0) ; => #(22.0 10.0)The implementation uses a single dual part, and so suffers from perturbation confusion in certain circumstances.
#lang ad/trace
(require ad/trace/diff)
;; ...Program tracing has a large overhead. Each primitive operation does quite a lot of extra work to manage the trace.
Uses the same algorithm (and code) as for finding a derivative via a program trace, but at expansion time, and only on "straight line" code.
This is currently a single example, illustrating how reverse-mode AD can be implemented using dual numbers and continuations (with the shift/reset higher-order control operators). It is based on Wang et al (2018).
The slides for the presentation at Lambda Days 2020 (video) are available as a pdf or as a Racket slideshow program.
We follow the lead of the main Racket license: This code is distributed under the MIT license and the Apache version 2.0 license, at your option.