# Automatic differentiation

## Idea

Given a function defined by a section of computer code, automatic differentiation produces a piece of computer code which, except for certain extreme cases, computes its derivative function. This page accumulates some notes of it.

## Details

### Forward rules

Extended number system based rules for forward automatic differentiation:

(1)$\langle u,u'\rangle +\langle v,v'\rangle = \langle u+v, u'+v' \rangle$
(2)$\langle u,u'\rangle -\langle v,v'\rangle = \langle u-v, u'-v' \rangle$
(3)$\langle u,u'\rangle *\langle v,v'\rangle = \langle u v, u' v+u v' \rangle$
(4)$\langle u,u'\rangle /\langle v,v'\rangle = \left\langle \frac{u}{v}, \frac{u' v-u v'}{v^2} \right\rangle \quad ( v\ne 0)$
(5)$\sin\langle u,u'\rangle = \langle \sin(u) , u' \cos(u) \rangle$
(6)$\cos\langle u,u'\rangle = \langle \cos(u) , -u' \sin(u) \rangle$
(7)$\exp\langle u,u'\rangle = \langle \exp u , u' \exp u \rangle$
(8)$\log\langle u,u'\rangle = \langle \log(u) , u'/u \rangle \quad (u \gt 0)$
(9)$\langle u,u'\rangle^k = \langle u^k , k u^{k-1} u' \rangle \quad (u \ne 0)$
(10)$\left| \langle u,u'\rangle \right| = \langle \left| u \right| , u' \sign u \rangle \quad (u \ne 0)$