envelopes / looping about soundfont-player HOT 12 OPEN

Here's a JS soundfont parser that I came across through an email to the web audio api mailing list:

https://github.com/notator/WebMIDISynthHost/blob/master/synths/standard/sf2Synth1/soundFontParser.js

It includes loop points and all the other goodies. I haven't tried it out with my own code, but there are live examples that seem to work well. Most impressively, it's directly parsing the soundfont file without pre-processed, intermediary .js representation, although I assume this greatly increases the amount of time to initially load a file...

from soundfont-player.

@charlieroberts Thanks for starring my repository, and mentioning it here, but I've reorganised it recently and the soundFont parser you mention has moved to
https://github.com/notator/WebMIDISynthHost/blob/master/sf2Synth1/soundFontParser.js
The main repository is at
https://github.com/notator/WebMIDISynthHost

Load times are not as bad as one might think. The time spent processing sf2 files is negligible compared to the time spent loading them.
And apropos load times: There are free sf2 editors out there that can be used to create sf2 files containing only the presets one really needs in a particular case. I think an sf2 file containing just a grand piano probably loads as fast as a folder containing an mp3 for every note. The sf2 only contains samples for a few notes, and interpolates the rest. The rest of the sf2 file contains "all the other goodies". :-)

Looping has to be done very carefully at a very low level, otherwise clicks happen at the joins. Maybe one could use an mp3 editor to find out where the loop points should be, but why should one first throw away all that info (by creating a flat set of mp3s from an sf2 file) only to have to do a lot of work for each preset to put it back again later.
One could write a utility for extracting the loop points from an sf2, but then one might as well write the whole sf2 parser. Maybe I'm wrong, but I think that building on MIDI.js effectively means expanding the binary sf2 into text plus mp3s. And that leads, in the end, to larger files and longer load times.

So my own feeling is that the answer is to create a proper, bug-free sf2 parser, and to embed it in a web soundFont synthesizer that can be used in large numbers of web applications. That's a tough nut to crack, but its a well-defined target that would only have to be done once. When such software exists, web sites will be able to use sf2 files, or subsets thereof, as they stand. There is a large off-web community working with sf2 files, and a lot of work has been put into optimising them. We shouldn't have to do all that work again.

My (gree's) sf2 parser is a good start, but it has its shortcomings/bugs. I don't pretend to understand it all yet, and would welcome some help. Bug reports and other issues are kept at:
https://github.com/notator/SimpleSoundFontSynthHost/issues.

All the best,
James

from soundfont-player.

I really like this idea for one; wondering, do you have a link to the
soundfont spec describing this feature?
On Nov 23, 2015 9:53 AM, "charlie roberts" [email protected] wrote:

Hi,

This looks like a great library; congrats! Any plans to support looping
(for sustain)? I know the SoundFont spec supports including loop points in
instruments, although the files included with MIDI.js don't have this info
unfortunately.

I think it would make a huge difference in the quality of current
browser-based SoundFonts if this could be supported at some point, but I
imagine it would be a great deal of work... anyways, if you're looking for
some future work, having "endless" sustain would be a great feature,
especially for instruments like organ. - Charlie

—
Reply to this email directly or view it on GitHub
#7.

from soundfont-player.

Section 9.1.2 of this document:

http://freepats.zenvoid.org/sf2/sfspec24.pdf

from soundfont-player.

er code formatting didn't work for this comment (sent via email) ... see my later comment.

from soundfont-player.

Very interesting, thanks for sharing that document! This would be very
useful for me - I would like to see it implemented.

I await danigb's thoughts, but here are some of my own:

1. soundfont-player should have override capabilities related to looping
   behavior (I.e. no loop, loop during release, or loop until release).
2. Actual loop point implementation should be contributed directly to the
   MIDI.js project - it's too messy and sound-dependent to leave up to the
   player, I think. Also, MIDI.js soundfonts should include a default looping
   behavior (which could be overridden). I'm not sure if MP3s or OGGs can
   encode this type of information, but I'm sure it could be included in
   .mps.js and .ogg.js files. In fact, here's an example file:
   https://raw.githubusercontent.com/gleitz/midi-js-soundfonts/master/FluidR3_GM/contrabass-ogg.js.
   I can imagine the loop points and behavior being implemented one of two
   (non-breaking) ways: as members of the Soundfont.instrumentname object, or
   perhaps better, as objects accessible via instrumentname keys in a separate
   Soundfont.loopInfo object.

E.g.

Soundfont.contrabass.loopInfo = {
  start: 0.12475,
  end: 0.5860,
  behavior: 'loopDuringRelease'
};

Or

Soundfont.loopInfo['contrabass'] = {
  start: 0.12475,
  end: 0.5860,
  behavior: 'loopDuringRelease'
};

Those are just my ideas.

from soundfont-player.

It's sounds like an interesting feature. The only thing is that, initially, I think this module basically as a sound-font loader with simple play support. Maybe it's a good idea to split it into two projects: soundfont-loader and soundfont-player

About the looping itself it's seems easy to implement, but I would prefer something like:

var piano = soundfont.intstrument('piano')
piano.setLoop(0.12375, 0.5660, 'loopDuringRelease')
piano.play('C4', 0, 2)

Currently there's no envelope support so I don't know if loop during release and loop until release has any sense. Maybe add envelop support is the first step.

Some other questions arise:

• harder: is there a dataset of loop points or it's just a user decision?
• simpler: what happens if the duration is not specified?

from soundfont-player.

To the "hard" question (is there a dataset of loop points): I can't imagine a practical application of the user just deciding, and doing that sounds difficult (if we're trying to make a soundfont sound like it's sustaining). I really think it should be a part of soundfont production (i.e. in the MIDI.js soundfont-generator). Including them in the soundfont-player seems like a temporary hack, not a long term solution. I guess the question is, would there be a situation where the user would need to override loop point presets?

from soundfont-player.

Aja... understood. Anyone feel adventurous to figure out how to extract that information from the sources? For me it's a too big task right now...

Here's https://github.com/mudcube/MIDI.js/blob/master/generator/ruby/soundfont_builder.rb

from soundfont-player.

Harder: I agree with @benwiley4000, this should be included in the soundfont and not be user definable. A hack would be to find the loops points and manually add them into the appropriate .js files rather than figuring out how to parse them in the soundfont_builder.rb file, but the best scenario would be for someone to figure out how to parse the .sf2 headers so that we can get -all- the additional information provided (such as how much chorus and reverb to apply etc.) which could then be selectively implemented in various soundfont players.

Simpler: In this case, just exhibit the current behavior... play through the file with no looping.

I just found this thread that I chimed in on over a year ago (and forgot about): mudcube/MIDI.js#73

You can see the loop points for each audiofile (note that they are per-file, and not per-instrument) and all the other metadata that is currently ignored in Polyphone(http://www.polyphone.fr/index.php?lang=en&page=home). Perhaps its source code could be one way to figure out parsing the .sf2 headers...

Anyways, I guess this issue makes more sense for MIDI.js... or rather, it seems like there should be a fork that specifically handles this one task (generating JSON representations of .sf2 data) rather than the variety of tasks that MIDI.js handles currently. Feel free to close this issue and/or comment further.

from soundfont-player.

Actually @charlieroberts if I understand it correctly, @danigb's "simpler" question is about notes that are played without a specified ending duration-- currently, if none is specified, the note must be manually stopped using the stop() method on the audio node returned by the instrument's play() method. In my own use case, I have my own release() function that tells a node to stop() after a specified delay time (no envelope). However, if envelopes/looping are implemented, it would be ideal to allow the sound to complete its final loop then decay before ending. For this to work, the instrument itself would probably need to keep track of nodes for its own sounds in progress (perhaps in an object with note names, e.g. 'A4', as keys) and contain a release(noteName) method (or similar) for ending them.

ETA: Perhaps it should be more like the OscillatorNode plays; no duration, loop once; -1 as duration, loop until interruption.

Having a release method for abstraction purposes wouldn't hurt, anyway.

from soundfont-player.

Of course I think this is kind of theoretical until MIDI.js soundbanks can support this feature 😝 Though, I would support the implementation of internal node tracking in Instrument objects, and a release method, to help abstract away some of the work I've had to do 😃

from soundfont-player.