Porting this package to use text2vec instead of topicmodels about ldatuning HOT 4 CLOSED

Now even better - implements WarpLDA. Sampler doesn't depend on number of topics, so I expect that it will be more than 10x faster.
Function to calculate perplexity is exported as well.

from ldatuning.

++@dselivanov - the author of text2vec package

from ldatuning.

Bit of a necro-bump, but am I right in assuming that none of the currently provided metrics apply to WarpLDA? I hacked on this and tried to get at least Griffiths's metric working with text2vec's WarpLDA, but to no avail.

In essence, I used text2vec as a drop-in replacement for topicmodels, and provide the result of fitting a model to the function for calculating Griffiths. In there, I access the model's loglikelihood attribute, but these seem to be different likelihoods than those provided by a topicmodels::LDA_Gibbs result...?

Some (simplified and hence untested!) code of my endeavours:

Griffiths2004 <- function(model) {
    logLiks <- lapply(models, function(model) {

        # the result of fitting text2vec's LDA also has an attribute "likelihood"
        # containing a column "loglikelihood"
        ll <- do.call(c, attr(model, "likelihood")["loglikelihood"])

        utils::tail(ll, n = length(ll))
    })

    # harmonic means for every model
    metrics <- sapply(logLiks, function(x) {
        # code is a little tricky, see explanation in [Ponweiser2012 p. 36]
        # ToDo: add variant without "Rmpfr"
        llMed <- stats::median(x)
        metric <- as.double(
          llMed - log( Rmpfr::mean( exp( -Rmpfr::mpfr(x, prec=2000L) + llMed )))
        )
        return(metric)
    })
return(metrics)
}

model <- text2vec::LDA$new(n_topics = x)
model$fit_transform(
    x = dtm,
    n_iter = 1000,
    convergence_tol = 0.001,
    n_check_convergence = 1,
    progressbar = FALSE
)

result <- Griffiths2004(model)
FindTopicsNumber_plot(result)

This gives a plot similar to this:

If these seems naive: it is 😃 I'm only just getting into R and topic modelling, but hyperparameter tuning is currently an important task for me. Running ldatuning with the Gibbs sampling algorithm takes an unfeasible amount of time on my dataset, so I'm researching for a more speedy but equally reliable solution. Any insight into this is thus much appreciated!

from ldatuning.

It looks like text2vec no longer returns log-likelihood calculations as of v0.5.0 per dselivanov/text2vec#212. As such, I'm closing this issue. Feel free to open a new issue if there's another way to accomplish the port.

from ldatuning.