Skip to content

Commit

Permalink
Add RTF for TTS models (#690)
Browse files Browse the repository at this point in the history
  • Loading branch information
csukuangfj authored Jan 2, 2025
1 parent ee6ea5e commit 8843fd4
Show file tree
Hide file tree
Showing 4 changed files with 421 additions and 1 deletion.
1 change: 1 addition & 0 deletions docs/source/onnx/tts/pretrained_models/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ This page list pre-trained models for text-to-speech.
.. toctree::
:maxdepth: 5

./rtf
./matcha
./vits
31 changes: 31 additions & 0 deletions docs/source/onnx/tts/pretrained_models/matcha.rst
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,8 @@ Generate speech with Python script
</tr>
</table>

.. _matcha-icefall-zh-baker:

matcha-icefall-zh-baker (Chinese, 1 female speaker)
---------------------------------------------------

Expand Down Expand Up @@ -368,3 +370,32 @@ After running, it will generate a file ``matcha-baker-zh-2.wav`` in the current
</td>
</tr>
</table>

RTF on Raspberry Pi 4 Model B Rev 1.5
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We use the following command to test the RTF of this model on Raspberry Pi 4 Model B Rev 1.5:

.. code-block:: bash
for t in 1 2 3 4; do
build/bin/sherpa-onnx-offline-tts \
--num-threads=$t \
--matcha-acoustic-model=./matcha-icefall-zh-baker/model-steps-3.onnx \
--matcha-vocoder=./hifigan_v2.onnx \
--matcha-lexicon=./matcha-icefall-zh-baker/lexicon.txt \
--matcha-tokens=./matcha-icefall-zh-baker/tokens.txt \
--matcha-dict-dir=./matcha-icefall-zh-baker/dict \
--output-filename=./matcha-baker-0.wav \
"当夜幕降临,星光点点,伴随着微风拂面,我在静谧中感受着时光的流转,思念如涟漪荡漾,梦境如画卷展开,我与自然融为一体,沉静在这片宁静的美丽之中,感受 着生命的奇迹与温柔."
done
The results are given below:

+-------------+-------+-------+-------+-------+
| num_threads | 1 | 2 | 3 | 4 |
+=============+=======+=======+=======+=======+
| RTF | 0.892 | 0.536 | 0.432 | 0.391 |
+-------------+-------+-------+-------+-------+

94 changes: 94 additions & 0 deletions docs/source/onnx/tts/pretrained_models/rtf.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
RTF of pre-trained models
==========================

The following table lists the RTF of pre-trained models on
``Raspberry Pi 4 Model B Rev 1.5``.

.. list-table::

* - Number of threads
- 1
- 2
- 3
- 4
-
* - :ref:`vits-melo-tts-zh_en`
- 6.727
- 3.877
- 2.914
- 2.518
- 163 MB
* - :ref:`vits-piper-en_US-glados`
- 0.812
- 0.480
- 0.391
- 0.349
- 61 MB
* - :ref:`vits-piper-en_US-libritts_r-medium`
- 0.790
- 0.493
- 0.392
- 0.357
- 75 MB
* - :ref:`vits-model-vits-ljspeech`
- 6.057
- 3.517
- 2.535
- 2.206
- 109 MB
* - :ref:`vits-model-vits-vctk`
- 6.079
- 3.483
- 2.537
- 2.226
- 116 MB
* - :ref:`sherpa-onnx-vits-zh-ll`
- 4.275
- 2.494
- 1.840
- 1.593
- 116 MB
* - :ref:`vits-zh-hf-fanchen-C`
- 4.306
- 2.451
- 1.846
- 1.600
- 116 MB
* - :ref:`vits-zh-hf-fanchen-wnj`
- 4.276
- 2.505
- 1.827
- 1.608
- 116 MB
* - :ref:`vits-zh-hf-theresa`
- 6.032
- 3.448
- 2.566
- 2.210
- 117 MB
* - :ref:`vits-zh-hf-eula`
- 6.011
- 3.473
- 2.537
- 2.231
- 117 MB
* - :ref:`vits-model-aishell3`
- 0.365
- 0.220
- 0.171
- 0.156
- 30 MB
* - :ref:`vits-model-en_US-lessac-medium`
- 0.774
- 0.482
- 0.390
- 0.357
- 61 MB
* - :ref:`matcha-icefall-zh-baker`
- 0.892
- 0.536
- 0.432
- 0.391
- 73 MB


Loading

0 comments on commit 8843fd4

Please sign in to comment.