Comments (4)
Thanks for your interest!
That output is bizarre. The only time I've ever seen something like that is when I tried training a WaveGAN using a frequency-domain discriminator. The time-domain generator learned to create little "grains" of sound which had the right spectral properties to fool the discriminator but were not very listenable.
One possibility is that you might be discriminating on "patches" of the output rather than the full output. Are you aggregating the final convolutional output of the discriminator to a single scalar output? If you are operating on patches (certain approaches like pix2pix intentionally do this), the generator might only be trying to make coherent patches but have no context of the entire waveform.
The training pattern I observed mostly started with noisy sounds and became more and more speech like with less noise as training progressed.
Discriminator and generator's weights should be initialized to random noise. I used Tensorflow's default weight initialization, which I believe defaults to "Glorot uniform initialization"
from wavegan.
Thank you for your comment.
I think the bizarre output seems to be due to the shape of sc09 training set. Since most of the sc09 .wav files are shorter than the desired slice length of 16384, which type of padding did you use for the real data input x? I used "edge" padding only on the right side, so just repeating the last sample's value for 16384 - len(signal) times at the end.
Also, for loader.py in your wavegan repository, in the following code segment:
(Randomize starting phase:)
if slice_randomize_offset:
start = tf.random_uniform([], maxval=slice_len, dtype=tf.int32)
audio = audio[start:]
Were you proposing a phase shuffle of random value between [0, 16384] for the real data x?
I noticed that when I trained with drums/ training set, where all the .wav files are exactly 16384 samples, this bizarre output pattern did not occur...
Thank you again!
from wavegan.
Actually, the model started working extremely well (recognizable articulation of digits 0-9 as early as Epoch 30) when I changed the model dim (d) to 32 from 64!
from wavegan.
To pad the SC09 digits to 16384 samples, I used zero padding at the end of the audio files. This is more typical for audio signal processing than the "edge" padding you described (though for this dataset they are approximately the same)
Your interpretation of the code snippet you mentioned is correct. It chooses a random offset into the waveform to produce training examples. However, I do not use this option for the digits dataset nor the drum sound effects, only for the more variable-length datasets like music or bird sounds.
Surprised to hear things improved when you reduced the size of the model. But I'm glad to hear that something worked :)
from wavegan.
Related Issues (20)
- Transfer Learning using pre-trained checkpoints HOT 2
- Invalid argument: You must feed a value for placeholder tensor 'ngl' with dtype int32 HOT 2
- Can we change (increase) amount of training data in between sessions?
- Last layer of the generator in the CNN (size 16384) HOT 1
- Tensorflow2.4 support HOT 5
- Multi-channel audio doesn't work with --data_num_channels 2 in Jupyter Lab HOT 1
- Can you fix collab
- Generate MFCC
- Can't get this to run anymore - Need information on environment
- Single-frequency noisy sound in the result HOT 4
- Reading WaveGAN models using Tensorflow C API
- Code for continue training model from last ckpt? HOT 1
- Training starts but no updates are dumping to checkpoints? HOT 1
- Training higher quality audio for ~5 seconds
- Overtraining HOT 2
- Can't train the model
- Can't run HOT 4
- OSS License compatibility question
- Massive Tensorflow Error Message When Attempting to Train WaveGAN
- shuffle buffer size is 4096 but I wish to use a dataset of 49k+ samples?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wavegan.