how to control level of sparsity?

Consider to remove alpha_max because it is not implemented in the original tipping code neither in the pronto implementation (only used in James implementation)

which kernel is the best for unbalanced one hot data for classifier?
for one hot data: only 0s and 1s in data

will it work for unbalanced data for classifier?
for one hot data: only 0s and 1s in data
is it possible to set class weights like
https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html
class_weight dict or ‘balanced’, default=None
Weights associated with classes in the form {class_label: weight}. If not given, all classes are supposed to have weight one

Using the EMRVR method, some models after training have only the bias on phi.
This make not possible to perform predictions of unseen data.

This problem might be due to the inversion of the matrix hessian.

Adding both to same class and removing the fast version of RVR to have everything working on version v0.0.3

Extend documentation for each of the functions.
Do a list of the needed improvements to documentation.

Sorry, I'm far from machine learning, just helping a friend to figure out problems with the code left by someone else.

After some debugging, I found that:

    def _prune(self):
        """Remove basis functions based on alpha values."""
        keep_alpha = self.alpha_ < self.threshold_alpha

        if not np.any(keep_alpha):
            keep_alpha[0] = True

        if self.bias_used:
            if not keep_alpha[0]:
                self.bias_used = False
            if self.kernel != "precomputed":
                self.relevance_vectors_ = self.relevance_vectors_[
                    keep_alpha[1:]]
            # Breakpoint added by me
            if not self.relevance_vectors_.shape[0]:
                import pdb
                pdb.set_trace()
            self.relevance_ = self.relevance_[keep_alpha[1:]]
        else:
            if self.kernel != "precomputed":
                self.relevance_vectors_ = self.relevance_vectors_[keep_alpha]
            self.relevance_ = self.relevance_[keep_alpha]

        self.alpha_ = self.alpha_[keep_alpha]
        self._alpha_old = self._alpha_old[keep_alpha]
        self.gamma_ = self.gamma_[keep_alpha]
        self.Phi_ = self.Phi_[:, keep_alpha]
        self.Sigma_ = self.Sigma_[np.ix_(keep_alpha, keep_alpha)]
        self.mu_ = self.mu_[keep_alpha]

taken from em_rvm.py is responsible for "nonsense" values in self.relevance_vectors_. This happens when keep_alpha is [True, False, ..., False]. Unfortunately, variables like "alpha" or other Greek letters don't help me understand why the values are the way they are, so I have to leave this report in the state that it is. The bottom line: the particular value of self.alpha_, when it's causing creation of zero-size self.relevance_vectors_ should be treated exceptionally. It's either an error (Why?) or, if not an error, then it should be made to do something sensible.

RVC does not compute the kernel when running prediction. Need to be added.

Similar to RVR, add cholesky decomposition to RVC to get a more efficient inverse of the hessian.

will it work on windows, or only for linux?

Create unit test appropriate for this version

It would be good to add verbose for both interpretability and debugging.

can it be used for categorical data classier for binary target?

since there is example for continues data ?

or categorical data should be transformed to one hot data?

Hi @Warvito,

I am trying to use Sklearn_rvm on a dataset and I have this error when computing the dot product between the mu_ and relevant_vectors_.

Can you please provide insight on how you want to extract the feature importance (coefficient) from relevant vectors and the mean "mu_" ?

Need to implement predict_proba in order to run predict for the RVC.

RVR does not permit yet to have beta fixed through out training. Consider implementing it.

Similar to https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html, it would be good to allow callable kernel to be more similar to SKLEARN

Class RVR has 3 gamma variables that can be confusing. Consider changing the names to something more suitable.

The choice of a threshold_alpha would change based on the dimensionality of the data. To overcome this issue, the Phi should be scaled when the kernel is calculated.