RcppEigen::fastLm about fixest HOT 4 CLOSED

No, that's not worth it:

library(fixest)
library(RcppEigen)

n = 1e6
base = data.frame(x1 = rnorm(n), x2 = rnorm(n), x3 = rnorm(n))
base$y = with(base, x1 + x2 + x3 + rnorm(n, sd = 20))

system.time(a <- fastLm(y ~ x1 + x2 + x3, base))
#>    user  system elapsed 
#>    0.42    0.03    0.45 
system.time(b <- feols(y ~ x.[1:3], base))
#>    user  system elapsed 
#>    0.03    0.00    0.03 

rbind(coef(a), coef(b))
#>      (Intercept)       x1       x2        x3
#> [1,]  0.02777233 1.026369 1.009945 0.9609391
#> [2,]  0.02777233 1.026369 1.009945 0.9609391

Please Wayne, do insert a use case next time!

For the record, I have tested the idea of using Eigen for linear algebra a long time ago.

The problem is that Eigen is single threaded, such that while it is clearly an improvement for single threaded mode, its benefit wanes quickly when compared to my (rather simple) multi-threaded code.
Hence implementing it wasn't an unconditional win (and it would add a huge dependency) so I didn't use it in the end.

from fixest.

Hi @lrberge (maybe I am misunderstanding) - RcppEigen is in fact multithreaded, e.g. matrix multiplication can easily be run via multiple threads, with sometimes very nice performance gains - a while back I have collected some benchmarks here. Though I have no idea if this would help speed up ´fixest´. For fwildclusterboot, where I have to right-multiply with a potentially very large bootstrap weights matrix (G x B; dim of clusters x bootstrap iterations), it definitely made a difference. :)

from fixest.

Eigen has been able to use multiple threads via OpenMP for many years. https://eigen.tuxfamily.org/dox/TopicMultiThreading.html

The function fastLm does not use multiple threads but can when the OpenMP compiler option is enabled.
RcppCore/RcppEigen#129

from fixest.

Not exactly what I wrote in RcppCore/RcppEigen#129. In any event, if someone wants a change, it would seem that starting with some well-chosen benchmark examples would be a good start, as opposed to a two-line issue without support or reasoning. My bet is that a linear model is so quick already that going the multiple threads is unlikely to be much of a game changer. But as always, actual empirics could change my mind.

from fixest.