{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "542ef4dd",
"metadata": {},
"outputs": [],
"source": [
"# SPDX-FileCopyrightText: Copyright (c) 2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.\n",
"# SPDX-License-Identifier: MIT"
]
},
{
"cell_type": "markdown",
"id": "c97d9d54",
"metadata": {},
"source": [
"# TTS API tutorial\n",
"\n",
"This tutorial demonstates how to use Python Riva API.\n",
"\n",
"## Server\n",
"\n",
"Before running client part of Riva, please set up a server. The simplest\n",
"way to do this is to follow\n",
"[quick start guide](https://docs.nvidia.com/deeplearning/riva/user-guide/docs/quick-start-guide.html#local-deployment-using-quick-start-scripts).\n",
"\n",
"\n",
"## Authentication\n",
"\n",
"Before using Riva services you will need to establish connection with a server."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bd57f561",
"metadata": {},
"outputs": [],
"source": [
"import riva.client\n",
"\n",
"uri = \"localhost:50051\" # Default value\n",
"\n",
"auth = riva.client.Auth(uri=uri)"
]
},
{
"cell_type": "markdown",
"id": "c5319eae",
"metadata": {},
"source": [
"## Setting up service\n",
"\n",
"To instantiate a service pass `riva.client.Auth` instance to a constructor."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fb75ba38",
"metadata": {},
"outputs": [],
"source": [
"tts_service = riva.client.SpeechSynthesisService(auth)"
]
},
{
"cell_type": "markdown",
"id": "bd4f6a1c",
"metadata": {},
"source": [
"## Offline synthesis\n",
"\n",
"In offline mode a result is returned in one response."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eaadb63f",
"metadata": {},
"outputs": [],
"source": [
"language_code = 'en-US'\n",
"sample_rate_hz = 16000\n",
"nchannels = 1\n",
"sampwidth = 2\n",
"text = (\n",
" \"The United States of America, commonly known as the United States or America, \"\n",
" \"is a country primarily located in North America. It consists of 50 states, \"\n",
" \"a federal district, five major unincorporated territories, 326 Indian reservations, \"\n",
" \"and nine minor outlying islands.\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94e0e2c7",
"metadata": {},
"outputs": [],
"source": [
"resp = tts_service.synthesize(text, language_code=language_code, sample_rate_hz=sample_rate_hz)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59aecdc6",
"metadata": {},
"outputs": [],
"source": [
"audio = resp.audio\n",
"meta = resp.meta"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d8ecf5e7",
"metadata": {},
"outputs": [],
"source": [
"print(len(audio))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5d83b0ce",
"metadata": {},
"outputs": [],
"source": [
"processed_text = meta.processed_text\n",
"predicted_durations = meta.predicted_durations"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4f1e9173",
"metadata": {},
"outputs": [],
"source": [
"print(processed_text)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "81edd1ce",
"metadata": {},
"outputs": [],
"source": [
"print(len(predicted_durations))\n",
"print(predicted_durations[0])"
]
},
{
"cell_type": "markdown",
"id": "31900d07",
"metadata": {},
"source": [
"Now we can write audio to a file."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ded9fb4a",
"metadata": {},
"outputs": [],
"source": [
"import wave\n",
"offline_output_file = \"my_offline_synthesized_speech.wav\"\n",
"with wave.open(offline_output_file, 'wb') as out_f:\n",
" out_f.setnchannels(nchannels)\n",
" out_f.setsampwidth(sampwidth)\n",
" out_f.setframerate(sample_rate_hz)\n",
" out_f.writeframesraw(resp.audio)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "08f864f9",
"metadata": {},
"outputs": [],
"source": [
"import IPython\n",
"IPython.display.Audio(offline_output_file)"
]
},
{
"cell_type": "markdown",
"id": "fcfaa16d",
"metadata": {},
"source": [
"### Asynchronous calls\n",
"\n",
"You can perform speech synthesis in an asynchronous manner by setting parameter `future=True`. In such case, `SpeechRecognitionService.synthesize()` will return a future object. You may get final response from a future object by calling `result()` method."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb6490d8",
"metadata": {},
"outputs": [],
"source": [
"from time import time\n",
"num_repeats = 10"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "12cef7f3",
"metadata": {},
"outputs": [],
"source": [
"start_time = time()\n",
"sync_audio = []\n",
"for _ in range(num_repeats):\n",
" sync_audio.append(\n",
" tts_service.synthesize(text, language_code=language_code, sample_rate_hz=sample_rate_hz).audio\n",
" )\n",
"print(f\"Synchronous calls time: {time() - start_time:.2f}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "596a95b2",
"metadata": {},
"outputs": [],
"source": [
"start_time = time()\n",
"async_audio = []\n",
"futures = []\n",
"for _ in range(num_repeats):\n",
" futures.append(\n",
" tts_service.synthesize(\n",
" text, language_code=language_code, sample_rate_hz=sample_rate_hz, future=True\n",
" )\n",
" )\n",
"for f in futures:\n",
" async_audio.append(f.result().audio)\n",
"print(f\"Async calls time: {time() - start_time:.2f}\")"
]
},
{
"cell_type": "markdown",
"id": "ade75ad9",
"metadata": {},
"source": [
"## Streaming synthesis\n",
"\n",
"In streaming mode an audio is returned in several responses. Responses are returned as soon as audio chunk is ready."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f3723b2e",
"metadata": {},
"outputs": [],
"source": [
"responses = tts_service.synthesize_online(text, language_code=language_code, sample_rate_hz=sample_rate_hz)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e257e237",
"metadata": {},
"outputs": [],
"source": [
"streaming_audio = b''\n",
"for resp in responses:\n",
" streaming_audio += resp.audio"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3ff6c2a7",
"metadata": {},
"outputs": [],
"source": [
"import wave\n",
"streaming_output_file = \"my_streaming_synthesized_speech.wav\"\n",
"with wave.open(streaming_output_file, 'wb') as out_f:\n",
" out_f.setnchannels(nchannels)\n",
" out_f.setsampwidth(sampwidth)\n",
" out_f.setframerate(sample_rate_hz)\n",
" out_f.writeframesraw(streaming_audio)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ce5bd83",
"metadata": {},
"outputs": [],
"source": [
"import IPython\n",
"IPython.display.Audio(streaming_output_file)"
]
},
{
"cell_type": "markdown",
"id": "72b40e15",
"metadata": {},
"source": [
"## Audio output\n",
"\n",
"For using audio input and output you need to install PyAudio.\n",
"\n",
"```bash\n",
"conda install -c anaconda pyaudio\n",
"```\n",
"\n",
"### Playing audio during synthesis\n",
"\n",
"For playing audio during synthesis you will need to pass audio chunks to `riva.client.audio_io.SoundCallBack` as they arrive."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "be28c2e6",
"metadata": {},
"outputs": [],
"source": [
"import riva.client.audio_io\n",
"# show available output devices\n",
"riva.client.audio_io.list_output_devices()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9097d307",
"metadata": {},
"outputs": [],
"source": [
"output_device = None # use default device\n",
"sound_stream = riva.client.audio_io.SoundCallBack(\n",
" output_device, nchannels=nchannels, sampwidth=sampwidth, framerate=sample_rate_hz\n",
")\n",
"for resp in tts_service.synthesize_online(text, language_code=language_code, sample_rate_hz=sample_rate_hz):\n",
" sound_stream(resp.audio)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.13"
}
},
"nbformat": 4,
"nbformat_minor": 5
}