The vits_web_demo from chocolatedesue

vits_web_demo's Introduction

onnx inference server in docker container

huggingface demo

Copy the demo web from link

Thanks a lot to @CjangCjengh Thanks a lot to wetts

Only used for entertainment. Don't used for bussiness

quick start

To use other model and config
please use -v /path/to/dir:/app/.model to mount your model and config

export name=vits_onnx
docker stop $name
docker rm $name
docker run -d \
--name $name \
-p 7860:7860 \
ccdesue/vits_demo:onnx
# -v /path/to/dir:/app/.model

Helpful info

please read the source code to better understand
refer to the demo config.json to tail to your own model config
refer the dockerfile
to use gpu, please install onnxruntime-gpu

limitation

only test on japanese_cleaners2 and zh_ja_mixture_cleaners in config.json with raw vits

Reference

license

GPLv2

vits_web_demo's People

Contributors

Stargazers

Watchers

vits_web_demo's Issues

Question with dynamic graph when export

Thank you for your sharing. I', export vits model to onnx but with following and got no idea how to deal with it. Do you have some advice ?

    seq = torch.randint(low=0, high=phone_num, size=(1, 10), dtype=torch.long)
    seq_len = torch.IntTensor([seq.size(1)]).long()
    print(f"{seq.size()}")
    print(f"{seq}")
    print("--------------------")
    print(f"{seq_len.size()}")
    print(f"{seq_len}")


    # noise(可用于控制感情等变化程度) lenth(可用于控制整体语速) noisew(控制音素发音长度变化程度)
    # 参考 https://github.com/gbxh/genshinTTS
    scales = torch.FloatTensor([0.667, 1.0, 0.8])
    # make triton dynamic shape happy
    scales = scales.unsqueeze(0)
    sid = torch.IntTensor([0]).long()
    # sid=torch.LongTensor([0])
    print(f"{scales.size()}")
    print(f"{sid.size()}")
    tmp=to_numpy(sid)
    print(f"{type(sid)}, tmp={type(tmp)}")



    dummy_input = (seq, seq_len, scales, sid)
    torch.onnx.export(model=net_g,
                      args=dummy_input,
                      f=args.onnx_model,
                      input_names=['input', 'input_lengths', 'scales', 'sid'],
                      output_names=['output'],
                      dynamic_axes={
                          'input': {
                              0: 'batch',
                              1: 'phonemes'
                          },
                          'input_lengths': {
                              0: 'phonemes_len'
                          },
                          'scales': {
                              0: 'batch'
                          },
                          'sid': {
                              0: 'batch'
                          },
                          'output': {
                              0: 'batch',
                              1: 'audio',
                              2: 'audio_length'
                          }
                      },
                      opset_version=11,
                      verbose=False)
    # Test case
    seq = torch.randint(low=0, high=phone_num, size=(1, 15), dtype=torch.long)
    seq_len = torch.IntTensor([seq.size(1)]).long()

    # Verify onnx precision
    print(f"In Multi speaker {num_speakers}")
    torch_output = net_g(seq, seq_len, scales, sid)
    providers = [args.providers]
    ort_sess = ort.InferenceSession(args.onnx_model, providers=providers)
    ort_inputs = {
            'input': to_numpy(seq),
            'input_lengths': to_numpy(seq_len),
            'scales': to_numpy(scales),
            'sid': to_numpy(sid),
    }

It has the following error

2023-06-05 09:18:47.766977537 [E:onnxruntime:, sequential_executor.cc:339 Execute] Non-zero status code returned while running Expand node. Name:'Expand_2897' Status Message: invalid expand shape
Traceback (most recent call last):
  File "export_onnx.py", line 203, in <module>
    main()
  File "export_onnx.py", line 195, in main
    audio = np.squeeze(ort_sess.run(None, ort_inputs))
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 188, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Expand node. Name:'Expand_2897' Status Message: invalid expand shape

Recommend Projects