Code Monkey home page Code Monkey logo

alphaxmas's Introduction

alphaXmas

This project, AlphaXmas, is a multimedia installation aimed to bring the audience into an experience of watching a computer generated tree growing, with a computer generated music melody in the background. The growing plant is generated using a 3D model built from an L-system and the music melody is generated using a LSTM neural networks. The generated music melodies are short monophonic Christmas carol compositions obtained from a LSTM-RNN Model. We train the model using the Hymns and Carols of Christmas dataset with around 1k historical compositions.

This is a Hackathon project for a.y. 2020-2021 Creative Programming and Computing Course of MSc. in Music and Acoustic Engineering @ Politecnico di Milano. The hackathon date is on Dec. 18th 2020, while the final presentation is on Sep. 8th 2021.

Links

Resources

  1. Keras, Tensorflow: Machine Learning in Python
  2. p5.js: JavaScript port of Processing
  3. Music21: Computer-aided Musicology Tool

Bibliography

  1. Noam Chomsky. "Three models for the description of language". In: IRE Transactions on information theory 2.3 (1956), pp. 113โ€“124.
  2. James Hanan. Parametric L-systems and their application to the modelling and visualization of plants. Citeseer, 1992.
  3. Yan-Bin Jia. "Quaternions and rotations". In: Com S 477.577 (2008), p. 15.
  4. Qi Lyu et al. "Modelling high-dimensional sequences with lstm-rtrbm: Application to polyphonic music generation". In: Twenty-Fourth International Joint Conference on Artificial Intelligence. 2015.
  5. Jon McCormack et al. "Interactive evolution of L-system grammars for computer graphics modelling". In: Complex Systems: from biology to computation (1993), pp. 118โ€“130.
  6. Przemyslaw Prusinkiewicz, Mark S Hammel, and Eric Mjolsness. "Animation of plant development". In: Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 1993, pp. 351โ€“360.
  7. Przemyslaw Prusinkiewicz and Aristid Lindenmayer. The algorithmic beauty of plants. Springer Science & Business Media, 2012.
  8. Falak Shah, Twisha Naik, and Nisarg Vyas. "LSTM based music generation". In: 2019 International Conference on Machine Learning and Data Engineering (iCMLDE). IEEE. 2019, pp. 48โ€“53.
  9. Massimiliano Zanoni. Creative Programming and Computing: Course material of MSc in Music and Acousting Engineering. 2020.

alphaxmas's People

Contributors

mttbernardini avatar yilin10 avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar

Forkers

jf-xia 5l1v3r1

alphaxmas's Issues

Make postcards

The idea is to use "lo-fi" quality: black&white dithered textures and 8-bit synthesis of the melodies.

Basic use case:

  1. User generates a postcard
  2. Back-end generates melody and "painting rules", front-end displays empty postcard
  3. If user likes the postcard they can write their message and save it, otherwise repeat from (1)
  4. Back-end saves melody, "painting rules" and message in DB and returns permalink
  5. User can share permalink with others (contents are non-editable)

Make meaningful music

The idea is to have a trained model on the back-end and generate short looped carol melodies which "make sense" when requested (and later store them in the db).

Basically, all the generation part that is now hard-coded in the front-end should be moved to the back-end and the model should be evaluated and, if necessary, trained again.

Make continuous growth

Apply some improved systems to make the growth of the tree slower and more continuous, rather than step-wise.

Interactions

  • strengthen the interaction between the generated melody and the graphical part
  • add one more feature: if the generated melody is "good", put an star on top of the tree

Fluent graphics

Right now we have an issue with the grammar model: when the "sentence" gets too long, the painting process is slowed down dramatically.

We need to understand what's wrong / simplify the model / simplify the instructions.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.