Comments (3)
Thanks for opening the issue! :)
Right now in IJulia,
using AudioIO, Interact #, Gadfly
s1 = SinOsc(220)
s2 = SinOsc(220)
@manipulate for f1=100:880, f2 = 110:880
s1.renderer.freq = f1
s2.renderer.freq = f2
#plot(t->sin(f1*2pi*t) + sin(f2*2pi*t), 0, 2pi)
end
play(s1)
play(s2)
gives you sliders which control the two frequencies of sine waves being played.
That macro call translates to:
lift(f1->s1.renderer.freq = f1, signal(slider(110:880)))
lift(f2->s2.renderer.freq = f2, signal(slider(110:880)))
where signal(slider(110:880))
is a signal of slider values.
This sort of API requiring side effects on s1
, s2
doesn't feel ideal though. But it's still cool!
what makes the Interact API nice for plotting is IJulia's display
mechanism. Interact is able to override display(::Signal{T})
to display an updating view of a value of T, in the richest mime type T supports.
if play(::Signal{AudioNode{T}})
could be similarly handled (by playing via the current renderer in the signal's current AudioNode value at any given time) then we are realizing more power.
In such a set up,
f1 = slider(1:800)
f2 = slider(1:800)
sound = @lift AudioMixer(SinOsc(f1), SinOsc(f2))
play(sound)
would produce the same result.
this would translate to a simpler @manipulate
macro call:
sound = @manipulate for f1=1:800, f2=1:800
AudioMixer(SinOsc(f1), SinOsc(f2))
end
play(sound) # or this could be made implicit
Moreover it is pretty damn simple to add sources of signals. E.g. it is trivial to turn an accelerometer reading or mouse position event stream into a signal. Which opens up possibilities of creating sounds from a lot of sources. I don't know much about interfacing musical instruments via MIDI, but once able to receive signals, it should be easy to turn them into reactive's signal.
An ambitious, and also unfeasible reactive programming approach would involve piping actual Float32 values from/into source/sink at 44100fps. It would look something like:
t = timestamp(fps(44100))
y1 = @lift sin(2pi * f1 * t)
y2 = @lift sin(2pi * f2 * t)
y = @lift y1 + y2
lift(sink, y)
Someday.
from audioio.jl.
I'm trying to figure out whether lift
ing is appropriate here. As I understand it lift
takes a transformation function and a signal, and returns a new signal which is a transformed version of the original signal. That doesn't seem to be what I want here, as I'm not really thinking of an AudioNode itself as a Signal, it would just take Signals as control parameters.
The SinOscRenderer is a parametric type parameterized on the frequency control, so you can have SinOscRenderer{Float32}
for a constant oscillator, or SinOscRenderer{AudioNode}
to control the frequency with another audio-rate AudioNode. Right now I'm considering adding SinOscRenderer{Signal}
, so once you have a Signal created you can just instantiate the oscillator:
f1 = slider(1:800)
f2 = slider(1:800)
sound = AudioMixer(SinOsc(signal(f1)), SinOsc(signal(f2)))
play(sound)
So rather than using lift
, SinOsc(signal(f1))
returns a value of type AudioNode{SinOscRenderer{Signal{Float32}}}
(which for convenience is type-aliased to SinOsc{Signal{Float32}}
)
It seems that with your example proposal, it's instantiating a new AudioNode every time the signal changes, but maybe I'm misunderstanding an important concept here.
The other thing I would do is set up writemime
so that any AudioNode
with signal inputs displays widgets for all of them using the Interact.jl machinery.
from audioio.jl.
That's right, I was suggesting that the manipulate expressions instantiate AudioNode
s when the control signals change. :) I imagine AudioNode
s as recipes for generating the sound for a short amount of time. We need a different recipe to generate another sound. I imagined the recipes themselves are inexpensive to create: E.g. SinOsc(440)
would just be just that, an immutable type SinOsc
, and a frequency 440. However I realize this is not how it is implemented. But it's a direction that I think would be profitable to go. There may be clever ways to get this to work.
In my mind, Reactive is best kept orthogonal to the functions it is intended to be used with. This has some benefits:
- users do not need to learn about extensions to AudioIO API with
Signal
s. They only need to know thatplay(::Signal{AudioNode})
is possible, and can bring in their knowledge of the two APIs (Reactive, AudioIO) and start doing nice things. - Composability: other arbitrary pure functions can be used. Also makes it easy to conceive things with other
Reactive
primitives than justlift
.
# Change the AudioNode being played.
switch = togglebuttons(["sine" => SinOsc(440), "noise" => WhiteNoise()])
play(signal(switch))
# Or alternate every second
ticks = every(1)
flip(a, b) = (a[2], a[1])
snd = lift(x -> x[1], foldl(flip, (SinOsc(440), WhiteNoise()), ticks))
play(signal(snd))
I can imagine it will be powerful if signal combinators like merge
, sampleon
, keepif
, keepwhen
, dropwhen
, droprepeats
were allowed to operate on signals of AudioNodes.
I tried to implement play
this way, but didn't have much success, the stop condition remains false once set to false, I will take a look at this tomorrow.
switch(a, b) = begin play(b); stop(a); b end
play(s::Signal) = foldl(switch, NullNode(), s) # NullNode plays nothing
from audioio.jl.
Related Issues (20)
- WARNING: libportaudio: Input overflowed HOT 4
- AudioIO should have a `versioninfo` method
- [PkgEval] AudioIO may have a testing issue on Julia 0.3 (2014-10-17) HOT 4
- Playing a file with a sample rate != 44100 does not work
- Use Wav.jl for opening .wav files HOT 4
- support writemime
- Support auto samplerate conversion
- design contract for buffers of AudioNode HOT 2
- Warning: libportaudio: Input overflowed HOT 7
- The play()-function is broken HOT 2
- SinOsc should take an initial phase HOT 2
- AudioIO uses deprecated Dict syntax
- Close audio stream when nothing is playing HOT 1
- Hook into FileIO `load` `save` mechanism
- Real Time audio recording examples missing HOT 5
- Coordinate various audio and DSP packages HOT 10
- PaStream as Ptr{Void} loses information about the data stream HOT 1
- Make Error in Fedora 23 HOT 1
- not working in Julia version 0.5.0-rc3+0 HOT 2
- Info about upcoming removal of packages in the General registry
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from audioio.jl.