cwitkowitz / guitar-transcription-continuous Goto Github PK

Code for the paper "FretNet: Continuous-Valued Pitch Contour Streaming for Polyphonic Guitar Tablature Transcription". This repository contains scripts which do the following (and more):

• Implement the proposed continuous-valued (relative) pitch estimation output layer
• Implement the FretNet model and obtain continuous-valued multi-pitch estimates as described in the paper
• Compute note and continuous-valued multi-pitch metrics at the string-level using mir_eval
• Perform cluster-based note and pitch contour grouping to cope with noisy ground-truth pitch annotations
• Perform six-fold cross-validation experiments on GuitarSet
• Exemplify how to use FretNet for inference and visualize note-contour grouping

The repository utilizes amt-tools, a more general music transcription repository, and guitar-transcription-with-inhibition, a repository built for previous research on reformulating and applying inhibition to the tablature output layer of TabCNN.

Clone the repository, install the requirements, then install the package:

git clone https://github.com/cwitkowitz/guitar-transcription-continuous
pip install -r guitar-transcription-continuous/requirements.txt
pip install -e guitar-transcription-continuous/

TODO

The scripts experiment.py and evaluation.py under six_fold_cv_scripts are available as a more complete example of how to train and evaluate the proposed model under the six-fold cross-validation schema using amt-tools.

Execution of six_fold_cv_scripts/experiment.py will generate the following under <root_dir> (defined at the top of the script):

• n/ - folder (beginning at n = 1)¹ containing sacred experiment files:
  • config.json - parameter values used for the experiment
  • cout.txt - contains any text printed to console
  • metrics.json - evaluation results for final model checkpoints
  • run.json - system and experiment information
  • six-fold-*.json - final six-fold cross-validation results for various criteria (generated by six_fold_cv_scripts/evaluation.py)
• models/ - folder containing saved model and optimizer state at each checkpoint, as well as an events file (for each execution) readable by tensorboard
• results/ - folder containing separate evaluation results at final model checkpoints for each track within the test set
• _sources/ - folder containing copies of scripts at the time(s) execution

Additionally, ground-truth and features will be saved under the path specified by gset_cache, unless save_data=False.

¹An additional folder (n += 1) containing similar files is created for each execution with the same experiment name <EX_NAME>.

During training, losses and various validation metrics can be analyzed in real-time by running:

tensorboard --logdir=<root_dir>/models --port=<port>

Here we assume the current working directory contains <root_dir>, and <port> is an integer corresponding to an available port (port = 6006 if unspecified).

After running the above command, navigate to http://localhost:<port> with an internet browser to view any reported training or validation observations within the tensorboard interface.

@article{cwitkowitz2023fretnet,
  title     = {FretNet: Continuous-Valued Pitch Contour Streaming for Polyphonic Guitar Tablature Transcription},
  author    = {Cwitkowitz, Frank and Hirvonen, Toni and Klapuri, Anssi},
  year      = 2023,
  booktitle = {Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)}
}