autodiff and ast2ast

[Konrad1991]

I tried to combine ast2ast (https://github.com/Konrad1991/ast2ast) and autodiff and could implement a first working example.

What is your approach? Do you want to use autodiff for R functions?

[fangzhou-xie]

Thanks for reaching out! I just noticed your dfdr repo yesterday and your ast2ast project also seems to be very interesting!

My original idea for this repo (autodiffr) was taking the header-only c++ project (autodiff) so that it can be used in writing Rcpp. But it relies on operator-overloading and hence would require users to completely rewrite the function to be differentiated. I wonder if using ast2ast could avoid the rewriting-process for users to differentiate any custom R function?

I was recently considering a different approach, similar to Zygote.jl in Julia. This would be done by translating R code into SSA form, then generate gradient function (either in R or in C++). By doing so, the users will not need to rewrite the function and ideally all R-native functions could be differentiated automatically. Specifically, I was thinking to implement the single-pass SSA algorithm by Brandis and Mössenböck (1994). This algorithm seems to be simple with the drawback of not supporting goto control flow. But this shouldn't be an issue for R.

I wonder what is your take on this and I appreciate any comments.

Reference:
Brandis, M. M., & Mössenböck, H. (1994). Single-pass generation of static single-assignment form for structured languages. ACM Transactions on Programming Languages and Systems, 16(6), 1684–1698. https://doi.org/10.1145/197320.197331

[Konrad1991]

In theory, it is possible to use autodiff and ast2ast together. The central buffer class in ast2ast is a generic class (https://github.com/Konrad1991/ast2ast/blob/master/inst/include/etr_bits/vec.hpp). Beyond that, the operators are overloaded for this class. However, it is necessary to add some changes to ast2ast e.g. adding some methods for specific classes (https://github.com/Konrad1991/ast2ast/blob/master/inst/include/etr_bits/add.hpp). The approach works but it is not trivial. The package dfdr fulfills my needs by calculating jacobian-functions, especially required for paropt (https://github.com/Konrad1991/paropt). Therefore, I will not keep working on the autodiff approach.

Do you have a use case in mind?

I had a very quick view of the paper you added. This looks really interesting. One could also think about extending dfdr using the SSA algorithm... Currently, I have no time to do this :(

[fangzhou-xie]

I don't have a particular use case in mind at the moment. All I was thinking was to provide a generic way to automatically differentiate functions in R. This might eventually enable differentiable programming for R.

[fangzhou-xie]

BTW, this autodiffr package is not published on CRAN, but only on Github right now. If you need to do something with it, please let me know. I can maybe update the headers and publish them on CRAN.