This repository contains the code for the paper "Bridging Graph Position Encodings for Transformers with Weighted Graph-Walking Automata" (Soga and Chiang, 2023).

There are 2 main directories in this repository. graphs contains code for running the graph dataset experiments, and mt contains code for running the machine translation experiments.

To set up the environment for graphs, we recommend creating a new virtual environment with Python 3.10. Run

python -m venv ~/.virtualenvs/gape-graphs

To install dependencies for graphs, run the following:

1. cd graphs
2. source ~/.virtualenvs/gape-graphs/bin/activate
3. pip3 install -r requirements.txt

Dependencies for mt require an environment with Python 3.7 instead, which is easy to install separate from your main installation with Conda. Run the following after deactivating gape-graphs:

1. cd mt
2. conda env create -f environment.yml
3. conda activate gape-mt

Run the shell scripts in graphs/data/ to download the graph datasets. Run any preparation scripts in each dataset's respective folder. The only exception is OGB-PCQM4Mv2 which is downloaded automatically upon running graphs/scripts/run_OGB_graph_regression.sh. The English-Vietnamese sentence pairs are already in mt/nmt/data.

Bash scripts to test each PE per dataset are located in graphs/scripts. By default, the scripts assume only one GPU and run experiments for each PE in series over 4 random seeds. The evaluation metrics and training outputs are logged in DEBUG.log.

To verify the BLEU scores in the paper, run python bleu.py OUT-[model]-[expected BLEU] nmt/data/en2vi/test.vi.tok substituting the model name and the expected BLEU score. To train a model, run train.sh replacing the PE variable in TASK=${SL}2${TL}_${PE} with the desired PE scheme. See nmt/configurations.py for the list of PE schemes and their hyperparameters.

Below is a table of all references and sources of code that is not ours.