zigasajovic / dcpp Goto Github PK

Could you please add comments inside the code to explain how to construct the differentiable virtual memory?

Try "return​ x / ​foo​(x / ​2​, y) + y * ​foo​(x, y / 4);"
It's cpp and binary, you know..

Infinite differentials + free integration at signature, just test it with 1/x and see if you find 2/x. It's easy.

Could I revise the value in var so that I can evaluate equation at different points?

Considering the following snippet of code:

  #include <iostream>
  #include <dCpp.h> 
  #include <dEigen.h>
  #include <Eigen/Dense>

  #define BATCH_SIZE 3
  #define NUM_FEATURES 2
  #define NUM_CLASSES 4

  template<typename A, typename B>
  void print_derivative(A &input, B &w) {
      auto nRows = input.rows();
      auto nCols = input.cols();
      Eigen::MatrixXd toprint(nRows, nCols);
      for (size_t i = 0; i < nRows; ++i){
          for (size_t j = 0; j < nCols; ++j) {
              toprint(i, j) = input(i, j).d(&w);
          }
      }
      std::cout << toprint << std::endl;
  }

  int main() {
      dCpp::initSpace(1);

      // x.W + b
      Eigen::Matrix<dCpp::var, BATCH_SIZE, NUM_FEATURES> x;
      x = Eigen::Matrix<dCpp::var, BATCH_SIZE, NUM_FEATURES>::Ones(BATCH_SIZE, 
  NUM_FEATURES);
      dCpp::init(x);

      Eigen::Matrix<dCpp::var, NUM_FEATURES, NUM_CLASSES> w;
      w = Eigen::Matrix<dCpp::var, NUM_FEATURES, NUM_CLASSES>::Zero(NUM_FEATURES, NUM_CLASSES);
      dCpp::init(w);

      Eigen::Matrix<dCpp::var, BATCH_SIZE, NUM_CLASSES> b;
      b = Eigen::Matrix<dCpp::var, BATCH_SIZE, NUM_CLASSES>::Zero(BATCH_SIZE, NUM_CLASSES);
      dCpp::init(b);


      Eigen::Matrix<dCpp::var, BATCH_SIZE, NUM_CLASSES> y = x*w + b;
      print_derivative(y, w);

      return 0;
  }

The previous snippet of code does not compile because of the following line:

  toprint(i, j) = input(i, j).d(&w);

My question is how, while leveraging dCpp, can I obtain a matrix representing the first derivate of y in respect to w?

Hi,

what would be the implications if one runs dCpp in a multithreaded environment?

baricenterGD -> barycenterGD

Hi,

I'm playing a bit with your nice AD tool and I'm having a bug with C++ operator and expressions.

Please have a look to https://gist.github.com/dcoeurjo/ce2b7f5e16edd348b7e4ca061ae6ceb5

For short: I have four points in the plane and I compute a kind of energy (sum of squared euclidean distance to a anchor point) and I want to differentiate this energy w.r.t. the anchor point.

If I expand the for loop when computing the energy (line 28), everything looks fine. When I use a for loop (line 17), then the differentiation fails (returning 0.0 for de/dx for instance). In terms of C++ operators, both energy expressions look similar.

Would you have any idea ?