lucmos / relreps Goto Github PK

Relative representations can be leveraged to enable solving tasks regarding "latent communication": from zero-shot model stitching to latent space comparison between diverse settings.

Home Page: https://openreview.net/forum?id=SrC-nwieGJ

License: MIT License

Python 15.39% Shell 0.53% Jupyter Notebook 84.07%

invariance representation-learning stitching zero-shot latent-communication relative-representation

relreps's Introduction

Relative representations enable zero-shot latent space communication

Slides | OpenReview | arXiv | BibTeX

Luca Moschella*, Valentino Maiorca*, Marco Fumero, Antonio Norelli, Francesco Locatello, Emanuele Rodolà

* equal contribution

Installation

pip install git+ssh://[email protected]/lucmos/relreps.git

Quickstart

Development installation

Setup the development environment:

git clone [email protected]:lucmos/relreps.git
cd relreps
conda env create -f env.yaml
conda activate relreps
pre-commit install
dvc pull

Refer to the template documentation for an high level overview of the code structure.

Update the dependencies

Re-install the project in edit mode:

pip install -e .[dev]

BibTeX

@inproceedings{
    moschella2023relative,
    title={Relative representations enable zero-shot latent space communication},
    author={Luca Moschella and Valentino Maiorca and Marco Fumero and Antonio Norelli and Francesco Locatello and Emanuele Rodol{\`a}},
    booktitle={The Eleventh International Conference on Learning Representations },
    year={2023},
    url={https://openreview.net/forum?id=SrC-nwieGJ}
}

relreps's People

Contributors

Stargazers

Watchers

Forkers

zhoubay matekrk zhizhangxian zerkclown imalshad themirachathumina

relreps's Issues

AE latent space training code

I am trying to reproduce figure 1 from the paper:

I've found the code in fig:latent-rotation/visualize.ipynb and am attempting to get it to work. IIUC - it appears to assume that pre-baked model checkpoints are downloaded with the down.sh helper script.

If I've got this right - could you provide a pointer to the AE training of these MNIST checkpoints themselves? My interest is in recreating your results including model training so that I can do follow up experiments which vary the upstream parameters on the latent spaces.

Error when running abs-rel performance comparison exp

Hi,
Great work! I am getting the error below when running your experiment. Can you help?
Best,
Sebastian

The command: bash experiments/tab:abs-rel-performance-comparison/classification/train.sh

Error executing job with overrides: ['core.tags=[classification,relative,tab1]', 'nn/data/datasets=vision/fmnist', 'nn/module=classifier', 'nn/module/model=vision/relresnet', 'nn.module.model.resnet_size=18', 'nn.module.model.input_size=224', 'nn.module.model.use_pretrained=True', 'nn.module.model.finetune=False', 'nn.module.model.hidden_features=512', 'nn.module.model.relative_attention.output_normalization_mode=layernorm', 'train.seed_index=0', 'train.trainer.max_epochs=10']
Traceback (most recent call last):
File "src/rae/run.py", line 123, in main
run(cfg)
File "src/rae/run.py", line 80, in run
datamodule: pl.LightningDataModule = hydra.utils.instantiate(cfg.nn.data, recursive=False)
File "/home/sebek/.conda/envs/relrep2/lib/python3.8/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 220, in instantiate
OmegaConf.resolve(config)
omegaconf.errors.InterpolationKeyError: Interpolation key 'nn.data.datasets.val_fixed_sample_idxs' not found

Request for code related to word embedding

Hi Luca,

I'm curious about your work about comparing communicating different word embeddings latent space, but I find it hard to figure out which part of your repo is about this.

Could you be so kind to point them out for me?

Thank you!
Xibin

Question about model stitching

Hi, thanks for providing such a wonderful work.

I have a question about the mechanism of stitching models with relative represenation.
When performing model stitching, if similarity vectors are created, are they normalized (with l2 norm) and communicated, or are they just connected?

Thanks

lucmos / relreps Goto Github PK

relreps's Introduction

Relative representations enable zero-shot latent space communication

Installation

Quickstart

Development installation

Update the dependencies

BibTeX

relreps's People

Contributors

Stargazers

Watchers

Forkers

relreps's Issues

AE latent space training code

Error when running abs-rel performance comparison exp

Request for code related to word embedding

Question about model stitching

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent