This is the authors' official PyTorch implementation for CIDA. This repo contains code for experiments in the ICML 2020 paper 'Continuously Indexed Domain Adaptation'.

• Beyond Domain Adaptation: Brief Introduction for CIDA
• Sample Results
• Method Overview
• IPython Notebooks and Environment
• Quantitative Results
• Theory
• Reference

Essentially CIDA asks the question of whether and how to go beyond current (categorical) domain adaptation regime and proposes the first approach to adapt across continuously indexed domains. For example, instead of adapting from domain A to domain B, we would like to simultaneously adapt across infintely many domains in a manifold. This allows us to go beyond domain adaption and perform both domain interpolation and domain extrapolation. See the following toy example.

For a more visual introduction, feel free to take a look at this video.

If we use domains [1, 6] as source domains and the rest as target domains, below are some sample results from previous domain adaptation methods and CIDA, where CIDA successfully learns how the decision boundary evolves with the domain index.

We provide a simple yet effective learning framework with theoretical guarantees (see the Theory section at the end of this README). Below is a quick comparison between previous domain adaptation methods and CIDA (differences marked in red).

• Previous domain adaptation methods use a discriminator is classifify different domains (as categorical values), while CIDA's discriminator directly regresses the domain indices (as continuous values).
• Previous domain adaptation methods' encoders ignore domain IDs, while CIDA takes the domain index as input.

Below are some IPython Notebooks for the experiments. We strongly recommend starting from the simplest case, i.e., Experiments for Toy Datasets (Quarter Circle) to get familar with the data and settings.

• Experiments for Toy Datasets (Quarter Circle)
• Experiments for Toy Datasets (Half Circle)
• Experiments for Toy Datasets (Sine)
• Experiments for Rotating MNIST

Besides using IPython notebooks, you can also directly run the following command for the Rotating MNIST experiments inside the folder 'rotatingMNIST':

bash run_all_exp.sh

• Python 3.6
• PyTorch 1.5.1
• Numpy 1.19

In the intra-dataset setting, we consider both domain extrapolation and domain interpolation (see the figure below).

Denoting the domain index as u and the encoding as z, we have (check the paper for full theorems):

• Theorem 1 (Informal). CIDA converges, if and only if, the expectation of domain index 𝔼[𝑢 | z] is identical for any embedding z.
• Theorem 2 (Informal). PCIDA converges, if and only if, the expectation and the variance of domain index, 𝔼[𝑢 | z] and 𝕍[𝑢 | z] are identical for any z.
• Theorem 3 (Informal). The global optimum of the two-player game between 𝐸 and 𝐷 matches the global optimum of the three-play game between 𝐸, 𝐹, and 𝐷.

Continuously Indexed Domain Adaptation

@inproceedings{DBLP:conf/icml/WangHK20,
  author    = {Hao Wang and
               Hao He and
               Dina Katabi},
  title     = {Continuously Indexed Domain Adaptation},
  booktitle = {ICML},
  year      = {2020}
}