Code Monkey home page Code Monkey logo

acoss's Introduction

acoss: Audio Cover Song Suite

PyPI pyversions Build Status PyPI version shields.io License: AGPL v3

acoss: Audio Cover Song Suite is a feature extraction and benchmarking frameworks for the cover song identification (CSI) tasks. This tool has been developed along with the new DA-TACOS dataset.

acoss includes a standard feature extraction framework with state-of-art audio features for CSI task and open source implementations of seven state-of-the-art CSI algorithms to facilitate the future work in this line of research. Using this framework, researchers can easily compare existing algorithms on different datasets, and we encourage all CSI researchers to incorporate their algorithms into this framework which can easily done following the usage examples.

Please site our paper if you use this tool in your resarch.

Furkan Yesiler, Chris Tralie, Albin Correya, Diego F. Silva, Philip Tovstogan, Emilia Gómez, and Xavier Serra. Da-TACOS: A Dataset for Cover Song Identification and Understanding. In 20th International Society for Music Information Retrieval Conference (ISMIR 2019), Delft, The Netherlands, 2019.

Benchmarking results on Da-Tacos dataset can be found in the paper.

Setup & Installation

We recommend you to install the package inside a python virtualenv.

Install using pip

pip install acoss

OR

Install from source (recommended)

  • Clone or download the repo.
  • Install acoss package by using the following command inside the directory.
python3 setup.py install

Additional dependencies

acoss requires a local installation of madmom library for computing some audio features and essentia library for similarity algorithms.

For linux-based distro users,

pip install "acoss[extra-deps]"

or if you are a Mac OSX user,

you can install madmom from pip

pip install madmom

you can install essentia from homebrew

brew tap MTG/essentia
brew install essentia --HEAD

Documentation and examples

acoss mainly provides the following python sub-modules-

  • acoss.algorithms: Sub-module with various cover identification algorithms, utilities for similarity comparison and an template for adding new algorithms.

  • acoss.coverid: Interface to benchmark a specific cover identification algorithms.

  • acoss.features: Sub-module with implementation of various audio features.

  • acoss.extractors : Interface to do efficient batch audio feature extraction for an audio dataset.

  • acoss.utils : Utility functions used in acoss package.

Cover Song Identification algorithms in acoss
Serra09 Paper
LateFusionChen Paper
EarlyFusionTraile Paper
SiMPle Paper
FTM2D Paper
MOVE adding soon ...
{
	"chroma_cens": numpy.ndarray,
	"crema": numpy.ndarray,
	"hpcp": numpy.ndarray,
	"key_extractor": {
		"key": numpy.str_,
		"scale": numpy.str_,_
		"strength": numpy.float64
	},
	"madmom_features": {
		"novfn": numpy.ndarray, 
		"onsets": numpy.ndarray,
		"snovfn": numpy.ndarray,
		"tempos": numpy.ndarray
	}
	"mfcc_htk": numpy.ndarray,
	"label": numpy.str_,
	"track_id": numpy.str_
}

Dataset structure required for acoss

Audio data

audio_dir
    /work_id
        /track_id.mp3

Feature file

feature_dir
    /work_id
        /track_id.h5 
import deepdish as dd

feature = dd.load("feature_file.h5")

An example feature file will be in the following structure.

{
   'feature_1': [],
   'feature_2': [],
   'feature_3': {'type_1': [], 'type_2': [], ...},
   ......  
}

CSV annotation file for a dataset

work_id track_id
W_163930 P_546633
... ...

acoss benchmark methods expect the dataset annotation csv file in the above given format.

There are also some utility functions in acoss which generates the csv annotation file automatically for da-tacos from it's subset metadata file and for covers80 dataset from it's audio data directory.

from acoss.utils import da_tacos_metadata_to_acoss_csv
da_tacos_metadata_to_acoss_csv("da-tacos_benchmark_subset_metadata.json", 
                            "da-tacos_benchmark_subset.csv")


from acoss.utils import generate_covers80_acoss_csv
generate_covers80_acoss_csv("coversongs/covers32k/", 
                            "covers80_annotations.csv")

For quick prototyping, let's use the tiny covers80, dataset.

Audio feature extraction

from acoss.utils import COVERS_80_CSV
from acoss.extractors import batch_feature_extractor
from acoss.extractors import PROFILE

print(PROFILE)

{ 'sample_rate': 44100, 'input_audio_format': '.mp3', 'downsample_audio': False, 'downsample_factor': 2, 'endtime': None, 'features': ['hpcp', 'key_extractor', 'madmom_features', 'mfcc_htk'] }

Compute

# Let's define a custom acoss extractor profile
extractor_profile = {
           'sample_rate': 32000,
           'input_audio_format': '.mp3',
           'downsample_audio': True,
           'downsample_factor': 2,
           'endtime': None,
           'features': ['hpcp']
}

# path to audio data
audio_dir = "../coversongs/covers32k/"
# path where you wanna store your data
feature_dir = "features/"

# Run the batch feature extractor with 4 parallel workers
batch_feature_extractor(dataset_csv=COVERS_80_CSV, 
                        audio_dir=audio_dir, 
                        feature_dir=feature_dir,
                        n_workers=4,
                        mode="parallel", 
                        params=extractor_profile)

Benchmarking

from acoss.coverid import benchmark, algorithm_names
from acoss.utils import COVERS_80_CSV

# path to where your audio feature h5 files are located
feature_dir = "features/"

# list all the available algorithms in acoss 
print(algorithm_names)

# we can easily benchmark any of the given cover id algorithm
# on the given dataset using the following function
benchmark(dataset_csv=COVERS_80_CSV, 
        feature_dir=feature_dir,
        algorithm="Serra09", 
        parallel=False)
# result of the evaluation will be stored in a csv file 
# in the current working directory.

How to add my algorithm in acoss?

Defining my algorithm

from acoss.algorithms.algorithm_template import CoverSimilarity

class MyCoverIDAlgorithm(CoverSimilarity):
    def __init__(self, 
                dataset_csv, 
                datapath, 
                name="MyAlgorithm", 
                shortname="mca"):
        CoverAlgorithm.__init__(self, 
                                dataset_csv=dataset_csv, 
                                name=name, 
                                datapath=datapath, 
                                shortname=shortname)

    def load_features(self, i):
        """Define how to want to load the features"""
        feats = CoverAlgorithm.load_features(self, i)
        # add your modification to feature arrays
        return

    def similarity(self, idxs):
        """Define how you want to compute the cover song similarity"""
        return

Running benchmark on my algorithm

# create an instance your algorithm
my_awesome_algorithm = MyCoverIDAlgorithm(dataset_csv, datapath)

# run pairwise comparison
my_awesome_algorithm.all_pairwise()

# Compute standard evaluation metrics
for similarity_type in my_awesome_algorithm.Ds.keys():
        print(similarity_type)
        my_awesome_algorithm.getEvalStatistics(similarity_type)

my_awesome_algorithm.cleanup_memmap()

How to contribute?

  • Fork the repo!
  • Create your feature branch: git checkout -b my-new-feature
  • Add your new audio feature or cover identification algorithm to acoss.
  • Commit your changes: git commit -am 'Add some feature'
  • Push to the branch: git push origin my-new-feature
  • Submit a pull request

Acknowledgments

This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 765068 (MIP-Frontiers).

This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 770376 (Trompa).

acoss's People

Contributors

albincorreya avatar ctralie avatar diegofurts avatar furkanyesiler avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

acoss's Issues

Comparing between two audio

Hi,

according to the README, everything seems to be end-to-end when running benchmark across the whole covers80 dataset.

Is there a way to simply compare two audio using any of the algorithms, and determine whether they are indeed cover versions of each other?

NameError: name 'n_threads' is not defined

Hi, I git cloned this on 9th April 2020.

I was running

batch_feature_extractor(dataset_csv=COVERS_80_CSV, 
                        audio_dir=audio_dir, 
                        feature_dir=feature_dir,
                        n_workers=4,
                        mode="parallel", 
                        params=extractor_profile)

And the following error was returned

Traceback (most recent call last):
  File "./debug.py", line 56, in <module>
    batch_feature_extractor(dataset_csv=COVERS_80_CSV,
  File "/data3/lootiangkuan/AudioHashing/QbH/acoss/acoss/extractors.py", line 150, in batch_feature_extractor
    Parallel(n_jobs=n_threads, verbose=1)(delayed(compute_features_from_list_file)\
NameError: name 'n_threads' is not defined

UnboundLocalError: local variable 'progressbar' referenced before assignment

Hi, i am trying to run the benchmark.

benchmark(dataset_csv=COVERS_80_CSV, 
    feature_dir=feature_dir,
    algorithm="Serra09", 
    parallel=True,
    n_workers=2)

This error is given.

Traceback (most recent call last):
  File "/data3/lootiangkuan/AudioHashing/QbH/acoss/acoss/coverid.py", line 65, in benchmark
    serra09.all_pairwise(parallel, n_cores=n_workers, symmetric=True)
  File "/data3/lootiangkuan/AudioHashing/QbH/acoss/acoss/algorithms/algorithm_template.py", line 191, in all_pairwise
    progressbar.next()
UnboundLocalError: local variable 'progressbar' referenced before assignment

data filepath format problem

Hi! In acoss/util.py, line 101, if the work_id or track_id is all composed of integers, then Pandas will infer that the data type is numpy.int64, and there will be an error when joining int and string together. Maybe a datatype convertion is needed here.

Code refactoring TODOs before beta release

The following 'code-refactoring-todos' were identified along with @furkanyesiler

The changes are made in the packaging dev branch of the repo.

Preprocess -

Extractors

  • Proper docstrings for all the functions
  • Logging error audio files in the batch extraction process
  • Specify a list of features as cmd args.
  • Change exception in compute features from list file
  • Unify string formatting using {}.format
  • Add cmd-line option for the feature extraction of other datasets (eg. from a given path)
  • Refactor cmd-line args for running non-parallel jobs (eg. cluster, cpu thing).
  • Make feature extractor unison for any file paths.

Features

  • Wrapper to call feature methods on a specific python version. If not available throw an exception.
  • Clean the file

Local config

  • Single local config file for the whole repo.

Requirements

  • Separate requirements.txt file for various python versions

README

  • Proper description and toy examples.

Benchmarking and CoverStats (@ctralie can you give me a hand on this part? )

  • Adhere code to pep-8 standards
  • Make getEvalStatistics clean
  • Follow conventions for writing docstrings
  • Standardize argparse of cover similarity algorithms
  • Clean all functions in statistics/ with pep8, etc
  • Make a single command line interface to call all benchmarking algorithms with a consistent set of parameters

Main repo and packaging

  • Add CI tools and unit tests.
  • Package acoss as a python library and build pip wheels
  • Logo for the repo and dataset
  • Scripts for building automated docs.
  • Proper README with setup instructions and example files
  • Check file storage formats for distributing datasets.

Cross-referencing with MusicBrainz

  • get the small batch of metadata
  • write initial script for cross-referencing
  • make sure the intersection is large enough on full metadata (+ subsets)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.