Generative Programming for Automatic Differentiation in C++0x
M. Nehmeier. 483. University of Würzburg, (December 2011)
Abstract
The computation of the first or higher derivatives of a function is a common
problem in scientific computing.
The most obvious approach to compute the derivative values is the symbolic
differentiation which applies the well known rules of differentiation onto
the expression to compute a formal expression of the derivative function.
Another method called numerical differentiation is the approximation of the
derivative with difference quotients.
Superior to these approaches is the automatic differentiation which is a
technique to compute a value of the expression and the derivative together
by applying the well known rules of differentiation. The difference to
symbolic differentiation is that it propagates numerical values instead
of formal expressions.
Commonly used implementations of automatic differentiation may be divided
into two categories, the implementations using operator overloading to
compute the values of the function and the derivative together in contrast
to special tools applying the technique of source transformation to mix in
the expressions to compute the derivatives. Both of these techniques are
somehow inflexible in the manner that the operators are not overloaded for
high order differentiation or additional tools are required for the source
transformation.
In this paper we use expression templates and template meta programming
to mix in the code for the partial differentiation. The main concept is
to apply the symbolic differentiation at compile time only onto the single
operations of the expression tree to create the run time code for the
automatic differentiation. This has two advantages, we can, in theory,
compute derivatives of any order and secondly the differentiation of the
operations is done with types. This means that the symbolic differentiation
of an operation with a specified order is performed only once by the compiler.
Our generic approach provides a domain specific language for partial
automatic differentiation of an arbitrary order which is easily extendable by
new types. Several C++ and especially C++0x
template programming concepts and techniques like variadic templates, type
lists or variadic tuples are used to realize the specification of the automatic
differentiation at compile time. Common template meta programming techniques
are refined.
We tested the functionality, correctness and performance of our implementation
in different case studies for floating point as well as interval data types and
compared it against other implementations.
%0 Report
%1 nehmeierTR483
%A Nehmeier, Marco
%D 2011
%K info2 techReportPub
%N 483
%T Generative Programming for Automatic Differentiation in C++0x
%U http://wwwi2.informatik.uni-wuerzburg.de/publications/TRnehmeier11MetaDiff.pdf
%X The computation of the first or higher derivatives of a function is a common
problem in scientific computing.
The most obvious approach to compute the derivative values is the symbolic
differentiation which applies the well known rules of differentiation onto
the expression to compute a formal expression of the derivative function.
Another method called numerical differentiation is the approximation of the
derivative with difference quotients.
Superior to these approaches is the automatic differentiation which is a
technique to compute a value of the expression and the derivative together
by applying the well known rules of differentiation. The difference to
symbolic differentiation is that it propagates numerical values instead
of formal expressions.
Commonly used implementations of automatic differentiation may be divided
into two categories, the implementations using operator overloading to
compute the values of the function and the derivative together in contrast
to special tools applying the technique of source transformation to mix in
the expressions to compute the derivatives. Both of these techniques are
somehow inflexible in the manner that the operators are not overloaded for
high order differentiation or additional tools are required for the source
transformation.
In this paper we use expression templates and template meta programming
to mix in the code for the partial differentiation. The main concept is
to apply the symbolic differentiation at compile time only onto the single
operations of the expression tree to create the run time code for the
automatic differentiation. This has two advantages, we can, in theory,
compute derivatives of any order and secondly the differentiation of the
operations is done with types. This means that the symbolic differentiation
of an operation with a specified order is performed only once by the compiler.
Our generic approach provides a domain specific language for partial
automatic differentiation of an arbitrary order which is easily extendable by
new types. Several C++ and especially C++0x
template programming concepts and techniques like variadic templates, type
lists or variadic tuples are used to realize the specification of the automatic
differentiation at compile time. Common template meta programming techniques
are refined.
We tested the functionality, correctness and performance of our implementation
in different case studies for floating point as well as interval data types and
compared it against other implementations.
@techreport{nehmeierTR483,
abstract = {The computation of the first or higher derivatives of a function is a common
problem in scientific computing.
The most obvious approach to compute the derivative values is the symbolic
differentiation which applies the well known rules of differentiation onto
the expression to compute a formal expression of the derivative function.
Another method called numerical differentiation is the approximation of the
derivative with difference quotients.
Superior to these approaches is the automatic differentiation which is a
technique to compute a value of the expression and the derivative together
by applying the well known rules of differentiation. The difference to
symbolic differentiation is that it propagates numerical values instead
of formal expressions.
Commonly used implementations of automatic differentiation may be divided
into two categories, the implementations using operator overloading to
compute the values of the function and the derivative together in contrast
to special tools applying the technique of source transformation to mix in
the expressions to compute the derivatives. Both of these techniques are
somehow inflexible in the manner that the operators are not overloaded for
high order differentiation or additional tools are required for the source
transformation.
In this paper we use expression templates and template meta programming
to mix in the code for the partial differentiation. The main concept is
to apply the symbolic differentiation at compile time only onto the single
operations of the expression tree to create the run time code for the
automatic differentiation. This has two advantages, we can, in theory,
compute derivatives of any order and secondly the differentiation of the
operations is done with types. This means that the symbolic differentiation
of an operation with a specified order is performed only once by the compiler.
Our generic approach provides a domain specific language for partial
automatic differentiation of an arbitrary order which is easily extendable by
new types. Several C++ and especially C++0x
template programming concepts and techniques like variadic templates, type
lists or variadic tuples are used to realize the specification of the automatic
differentiation at compile time. Common template meta programming techniques
are refined.
We tested the functionality, correctness and performance of our implementation
in different case studies for floating point as well as interval data types and
compared it against other implementations.},
added-at = {2011-12-19T13:32:22.000+0100},
author = {Nehmeier, Marco},
biburl = {https://www.bibsonomy.org/bibtex/211968e1b42249ab14ef9ee370b9a59d3/nehmeier},
institution = {University of W\"urzburg},
interhash = {545885e1ada1b95b58a9ecf058cfdc7f},
intrahash = {11968e1b42249ab14ef9ee370b9a59d3},
keywords = {info2 techReportPub},
month = dec,
number = 483,
timestamp = {2014-03-24T14:45:02.000+0100},
title = {Generative Programming for Automatic Differentiation in C++0x },
url = {http://wwwi2.informatik.uni-wuerzburg.de/publications/TRnehmeier11MetaDiff.pdf},
year = 2011
}