Automatic differentiation

Idea

Given a function defined by a section of computer code, automatic differentiation produces a piece of computer code which, except for certain extreme cases, computes its derivative function. This page accumulates some notes of it.

Details

Forward rules

Extended number system based rules for forward automatic differentiation:

(1)$\langle u,u'\rangle +\langle v,v'\rangle = \langle u+v, u'+v' \rangle$
(2)$\langle u,u'\rangle -\langle v,v'\rangle = \langle u-v, u'-v' \rangle$
(3)$\langle u,u'\rangle *\langle v,v'\rangle = \langle u v, u' v+u v' \rangle$
(4)$\langle u,u'\rangle /\langle v,v'\rangle = \left\langle \frac{u}{v}, \frac{u' v-u v'}{v^2} \right\rangle \quad ( v\ne 0)$
(5)$\sin\langle u,u'\rangle = \langle \sin(u) , u' \cos(u) \rangle$
(6)$\cos\langle u,u'\rangle = \langle \cos(u) , -u' \sin(u) \rangle$
(7)$\exp\langle u,u'\rangle = \langle \exp u , u' \exp u \rangle$
(8)$\log\langle u,u'\rangle = \langle \log(u) , u'/u \rangle \quad (u \gt 0)$
(9)$\langle u,u'\rangle^k = \langle u^k , k u^{k-1} u' \rangle \quad (u \ne 0)$
(10)$\left| \langle u,u'\rangle \right| = \langle \left| u \right| , u' \sign u \rangle \quad (u \ne 0)$

Will chrome allow me to post this? No, the browser is deleting the user’s content: cool feature!