Code Monkey home page Code Monkey logo

spotify-rehydrator's Introduction

Spotify Rehydrator

https://img.shields.io/badge/Python-3.9-blue

Recreate a full dataset of audio features of songs downloaded through Spotify's download my data facility.

This requires the files named StreamingHistory{n}.json where {n} represents the file number that starts at 0, and goes up to however many files were retrieved.

Quick start

Extended documentation is available on ReadTheDocs. First, install the package using pip. An example of using the package to rehydrate a folder of json files is then:

# main.py
from spotifyrehydrator import Rehydrator
import os
import pathlib

if __name__ == "__main__":
    Rehydrator(
        os.path.join(pathlib.Path(__file__).parent.absolute(), "input"),
        os.path.join(pathlib.Path(__file__).parent.absolute(), "output"),
        client_id=os.getenv("SPOTIFY_CLIENT_ID"),
        client_secret=os.getenv("SPOTIFY_CLIENT_SECRET"),
    ).run(return_all=True)

Run takes boolean arguments for audio_features and artist info, or for return_all which then returns both. These will determine how much information is retrieved to make up the full dataset that is saved into the output folder.

How it works

  1. The files for each person are read from the specified input folder.
  2. The name and artist provided are searched with the Spotify API. The first result is taken to be the track, and the track ID is recorded.
  3. Additional information is searched on other endpoints if audio_features, artist info or return_all were set to True.
  4. The matched track ID and audio features are saved as one tab delimited .tsv file per person into the specified output folder.

Good to know

  • Not all tracks can be retreived from the API. In our experience about 5% of tracks cannot be found on the API. These will have a value of NONE in the output files.
  • There is not a guaranteed match between the first returned item in a search and the track you want. Comparing msPlayed with the track length is a good way to test this since msPlayed should not exceed the track length.

P.S. Thanks to Pixel perfect for the title icon. ๐Ÿ™‚

spotify-rehydrator's People

Contributors

altanner avatar ninadicara avatar

Watchers

 avatar  avatar  avatar

spotify-rehydrator's Issues

Functions need separating.

Both get_track_id and get_track_features are now very long and would benefit from being separated into different functions.

  1. Retrieving data to be processed
  2. Processing the data
  3. Saving it out

Maybe a good time to consider a Data Class?

Convert to package for easier reuse

This will require re-organising the repo a little, and changing the default arguments.

Other things that could be achieved through this option include:

  • Choosing whether to get TrackIDs, Features, or something else? (Lyrics)

  • Re-organise into package structure

  • setup.py

  • Register with PyPi

  • Change input/output file options to arguments.

Complete test set

Check the joins match as expected etc and any data manipulation is tested.
Will need to mock the API calls.

  • get_ids
  • read
  • unmatched_tracks
  • get_URI
  • add_URI
  • add_feature_cols
  • run_pipeline
  • rehydrate

Output file values for empty tracks

Instead of having 'None' as the value in the output csv it would be better to have a null placeholder in each column. Otherwise reading the file back in can cause problems.

Understand why some tracks aren't returned by the API

About 5% of tracks aren't returned by the API. Quite a few of them seem to have special characters so that could be the reason. It would be good to understand if there's anything we can change about the search queries to increase chances of them returning successfully.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.