zeroone-universe / realtimebwe Goto Github PK

I have read your code, where self.generator = SEANet(min_dim = 8, causality = True). But you haven't implemented streaming SEANet. I guess the generator is a offline SEANet.

google has implemented streaming mode for SEANet:
https://github.com/google-research/google-research/tree/master/kws_streaming

Do you have plan to implement streaming SEANet in pytorch-lightning?

Thanks!

Hello,

when I attempt to reproduce your work, I got these error:

Sanity Checking DataLoader 0: 0%| | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
File "/home/ma-user/work/c00573026/AISR/RealTimeBWE-master/main.py", line 45, in
main(config)
File "/home/ma-user/work/c00573026/AISR/RealTimeBWE-master/main.py", line 39, in main
trainer.fit(rtbwe_train, rtbwe_datamodule)
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 520, in fit
call._call_and_handle_interrupt(
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py", line 42, in _call_and_handle_interrupt
return trainer.strategy.launcher.launch(trainer_fn, *args, trainer=trainer, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/strategies/launchers/subprocess_script.py", line 92, in launch
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 559, in _fit_impl
self._run(model, ckpt_path=ckpt_path)
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 935, in _run
results = self._run_stage()
^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 976, in _run_stage
self._run_sanity_check()
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/trainer.py", line 1005, in _run_sanity_check
val_loop.run()
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/loops/utilities.py", line 174, in _decorator
return loop_run(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 115, in run
self._evaluation_step(batch, batch_idx, dataloader_idx)
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/loops/evaluation_loop.py", line 375, in _evaluation_step
output = call._call_strategy_hook(trainer, hook_name, *step_kwargs.values())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/trainer/call.py", line 288, in _call_strategy_hook
output = fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/strategies/ddp.py", line 337, in validation_step
return self.model(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/torch/nn/parallel/distributed.py", line 1156, in forward
output = self._run_ddp_forward(*inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/torch/nn/parallel/distributed.py", line 1110, in _run_ddp_forward
return module_to_run(*inputs[0], **kwargs[0]) # type: ignore[index]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/pytorch_lightning/overrides/base.py", line 102, in forward
return self._forward_module.validation_step(*inputs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/work/c00573026/AISR/RealTimeBWE-master/train.py", line 93, in validation_step
wav_bwe = self.forward(wav_nb)
^^^^^^^^^^^^^^^^^^^^
File "/home/ma-user/work/c00573026/AISR/RealTimeBWE-master/train.py", line 41, in forward
x = self.resampler(x)
^^^^^^^^^^^^^^
File "/home/ma-user/anaconda3/envs/seanet_clf/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1614, in getattr
raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'RTBWETrain' object has no attribute 'resampler'

Can you offer the resampler method?

Hello,

While I was reproducing the result, I found that the latest model could not generate valid high frequency part with small part of vctk dataset (i.e. one speaker). After I used all the vctk dataset to train the latest model, it converge in 10 epochs.