-Introduction

-Getting Starting

-Citation

-Contact

This repository hosts an implementation of the Audio Spectrogram Transformer (AST) based on the official implementation provided by the authors of the paper "Parameter-Efficient Transfer Learning of Audio Spectrogram Transformers" -- PETL_AST. The paper introduces Parameter-Efficient Transfer Learning (PETL) methods tailored to the Audio Spectrogram Transformer architecture. In this repository, particular emphasis is placed on the LoRA (Low Rank) adapter, which has demonstrated the best results.

Low Rank Adaption is the most widely used PETL methods. Provides the approach to fine-tuning a model for downstream tasks and the capability of transfer learning with fewer resources.

Step 1. Clone or download this repository and set it as the working directory, create a virtual environment and install the dependencies.

cd ast-lora/ 
python3 -m venv venvast
source venvast/bin/activate
pip install -r requirements.txt 

Step 2: Running Experiment

We just need set some parameters in train.yaml:

• ``` lr_LoRA ```
• ``` weight_decay ```
• ``` final_output ```
• ``` patch_size ```
• ``` hidden_size ```

In main.sh just need set data path ( train, validation and test ) and some parameters.

bash  main.sh

Citing the original paper:

@misc{cappellazzo2024efficient,
      title={Efficient Fine-tuning of Audio Spectrogram Transformers via Soft Mixture of Adapters}, 
      author={Umberto Cappellazzo and Daniele Falavigna and Alessio Brutti},
      year={2024},
      eprint={2402.00828},
      archivePrefix={arXiv},
      primaryClass={eess.AS}
}

If you have a question, or just want to share how you have use this, send me an email at [email protected]