marco299 / deeprob-kit Goto Github PK

DeeProb-kit is a general-purpose Python library that implements several deep probabilistic models, putting an effort into unifying and standardizing the experiments across most of them. The implementation in a single library of different models permits to easily combine them, as a common practice in deep learning research. The library consists of a collection of deep probabilistic models such as various kinds of Sum-Product Networks, Normalizing Flows and their possible combinations for density estimation. Some models are implemented using PyTorch for fast training and inference on GPUs.

• Inference algorithms for SPNs. ^{1 4}
• Learning algorithms for SPNs structure. ^{1 2 3 4 5}
• Chow-Liu Trees (CLT) as SPN leaves. ^{12 13}
• Batch Expectation-Maximization (EM) for SPNs with arbitrarily leaves. ^{14 15}
• Structural marginalization and pruning algorithms for SPNs.
• High-order moments computation for SPNs.
• JSON I/O operations for SPNs and CLTs. ⁴
• Plotting operations based on NetworkX for SPNs and CLTs. ⁴
• Randomized And Tensorized SPNs (RAT-SPNs) using PyTorch. ⁶
• Masked Autoregressive Flows (MAFs) using PyTorch. ⁷
• Real Non-Volume-Preserving (RealNVP) and Non-linear Independent Component Estimation (NICE) flows. ^{8 9}
• Deep Generalized Convolutional SPNs (DGC-SPNs) using PyTorch. ¹¹

The collection of implemented models is summarized in the following table. The supported data dimensionality for each model is showed in the Input Dimensionality column. Moreover, the Supervised column tells which model is suitable for a supervised learning task, other than density estimation task.

The library can be installed either from PIP repository or by source code.

# Install from PIP repository
pip install deeprob-kit

# Install from `main` git branch
pip install -e git+https://github.com/deeprob-org/deeprob-kit.git@main#egg=deeprob-kit

The documentation is generated automatically by Sphinx using sources stored in the docs directory.

A collection of code examples and experiments can be found in the examples and experiments directories respectively. Moreover, benchmark code can be found in the benchmark directory.

• SPFlow
• RAT-SPN
• Random-PC
• LibSPN-Keras
• MAF
• RealNVP

1. Peharz et al. On Theoretical Properties of Sum-Product Networks. AISTATS (2015).

2. Poon and Domingos. Sum-Product Networks: A New Deep Architecture. UAI (2011).

3. Molina, Vergari et al. Mixed Sum-Product Networks: A Deep Architecture for Hybrid Domains. AAAI (2018).

4. Molina, Vergari et al. SPFLOW : An easy and extensible library for deep probabilistic learning using Sum-Product Networks. CoRR (2019).

5. Di Mauro et al. Sum-Product Network structure learning by efficient product nodes discovery. AIxIA (2018).

6. Peharz et al. Probabilistic Deep Learning using Random Sum-Product Networks. UAI (2020).

7. Papamakarios et al. Masked Autoregressive Flow for Density Estimation. NeurIPS (2017).

8. Dinh et al. Density Estimation using RealNVP. ICLR (2017).

9. Dinh et al. NICE: Non-linear Independent Components Estimation. ICLR (2015).

10. Papamakarios, Nalisnick et al. Normalizing Flows for Probabilistic Modeling and Inference. JMLR (2021).

11. Van de Wolfshaar and Pronobis. Deep Generalized Convolutional Sum-Product Networks for Probabilistic Image Representations. PGM (2020).

12. Rahman et al. Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees. ECML-PKDD (2014).

13. Di Mauro, Gala et al. Random Probabilistic Circuits. UAI (2021).

14. Desana and Schnörr. Learning Arbitrary Sum-Product Network Leaves with Expectation-Maximization. CoRR (2016).

15. Peharz et al. Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuits. ICML (2020).