08 - Speech.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["# Speech\n","\n","Increasingly, we expect to be able to communicate with artificial intelligence (AI) systems by talking to them, often with the expectation of a spoken response.\n","\n","![A robot speaking](./images/speech.jpg)\n","\n","*Speech recognition* (an AI system interpreting spoken language) and *speech synthesis* (an AI system generating a spoken response) are the key components of a speech-enabled AI solution.\n","\n","## Create a Cognitive Services resource\n","\n","To build software that can interpret audible speech and respond verbally, you can use the **Speech** cognitive service, which provides a simple way to transcribe spoken language into text and vice-versa.\n","\n","If you don't already have one, use the following steps to create a **Cognitive Services** resource in your Azure subscription:\n","\n","> **Note**: If you already have a Cognitive Services resource, just open its **Quick start** page in the Azure portal and copy its key and endpoint to the cell below. Otherwise, follow the steps below to create one.\n","\n","1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n","2. Click the **&#65291;Create a resource** button, search for *Cognitive Services*, and create a **Cognitive Services** resource with the following settings:\n","    - **Subscription**: *Your Azure subscription*.\n","    - **Resource group**: *Select or create a resource group with a unique name*.\n","    - **Region**: *Choose any available region*:\n","    - **Name**: *Enter a unique name*.\n","    - **Pricing tier**: S0\n","    - **I confirm I have read and understood the notices**: Selected.\n","3. Wait for deployment to complete. Then go to your cognitive services resource, and on the **Overview** page, click the link to manage the keys for the service. You will need the key and location to connect to your cognitive services resource from client applications.\n","\n","### Get the Key and Location for your Cognitive Services resource\n","\n","To use your cognitive services resource, client applications need its authentication key and location:\n","\n","1. In the Azure portal, on the **Keys and Endpoint** page for your cognitive service resource, copy the **Key1** for your resource and paste it in the code below, replacing **YOUR_COG_KEY**.\n","2. Copy the **Location** for your resource and and paste it in the code below, replacing **YOUR_COG_LOCATION**.\n",">**Note**: Stay on the **Keys and Endpoint** page and copy the **Location** from this page (example: _westus_). Please _do not_ add spaces between words for the Location field. \n","3. Run the code below by clicking the **Run cell** (&#9655;) button to the left of the cell."]},{"cell_type":"code","execution_count":null,"metadata":{"gather":{"logged":1599695240794}},"outputs":[],"source":["cog_key = 'YOUR_COG_KEY'\n","cog_location = 'YOUR_COG_LOCATION'\n","\n","print('Ready to use cognitive services in {} using key {}'.format(cog_location, cog_key))"]},{"cell_type":"markdown","metadata":{},"source":["## Speech recognition\n","\n","Suppose you want to build a home automation system that accepts spoken instructions, such as \"turn the light on\" or \"turn the light off\". Your application needs to be able to take the audio-based input (your spoken instruction), and interpret it by transcribing it to text that it can then parse and analyze.\n","\n","Now you're ready to transcribe some speech. The input can be from a **microphone** or an **audio file**. \n"]},{"cell_type":"markdown","metadata":{},"source":["### Speech Recognition with an audio file\n","\n","Run the cell below to see the Speech Recognition service in action with an **audio file**. \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["import os\n","from playsound import playsound\n","from azure.cognitiveservices.speech import SpeechConfig, SpeechRecognizer, AudioConfig\n","\n","# Get spoken command from audio file\n","file_name = 'light-on.wav'\n","audio_file = os.path.join('data', 'speech', file_name)\n","\n","# Configure speech recognizer\n","speech_config = SpeechConfig(cog_key, cog_location)\n","speech_config.speech_synthesis_voice_name = 'en-US-ChristopherNeural'\n","audio_config = AudioConfig(filename=audio_file) # Use file instead of default (microphone)\n","speech_recognizer = SpeechRecognizer(speech_config, audio_config)\n","\n","# Use a one-time, synchronous call to transcribe the speech\n","speech = speech_recognizer.recognize_once()\n","\n","# Play the original audio file\n","playsound(audio_file)\n","\n","# Show transcribed text from audio file\n","print(speech.text)"]},{"cell_type":"markdown","metadata":{},"source":["## Speech synthesis\n","\n","So now you've seen how the Speech service can be used to transcribe speech into text; but what about the opposite? How can you convert text into speech?\n","\n","Well, let's assume your home automation system has interpreted a command to turn the light on. An appropriate response might be to acknowledge the command verbally (as well as actually performing the commanded task!)"]},{"cell_type":"code","execution_count":null,"metadata":{"gather":{"logged":1599695261170}},"outputs":[],"source":["import os\n","import matplotlib.pyplot as plt\n","from PIL import Image\n","from azure.cognitiveservices.speech import SpeechConfig, SpeechSynthesizer, AudioConfig\n","%matplotlib inline\n","\n","# Get text to be spoken\n","response_text = 'Turning the light on.'\n","\n","# Configure speech synthesis\n","speech_config = SpeechConfig(cog_key, cog_location)\n","speech_config.speech_synthesis_voice_name = 'en-US-ChristopherNeural'\n","speech_synthesizer = SpeechSynthesizer(speech_config)\n","\n","# Transcribe text into speech\n","result = speech_synthesizer.speak_text(response_text)\n","\n","# Display an appropriate image \n","file_name = response_text.lower() + \"jpg\"\n","img = Image.open(os.path.join(\"data\", \"speech\", file_name))\n","plt.axis('off')\n","plt. imshow(img)"]},{"cell_type":"markdown","metadata":{},"source":["Try changing the **response_text** variable to *Turning the light off.* (including the period at the end) and run the cell again to hear the result.\n","\n","## Learn more\n","\n","You've seen a very simple example of using the Speech cognitive service in this notebook. You can learn more about [speech-to-text](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-speech-to-text) and [text-to-speech](https://docs.microsoft.com/azure/cognitive-services/speech-service/index-text-to-speech) in the Speech service documentation."]}],"metadata":{"kernel_info":{"name":"python3-azureml"},"kernelspec":{"display_name":"Python 3.8.5 32-bit","metadata":{"interpreter":{"hash":"177429bd1865e7f7a0dbecbac90518c0d9641b1102b2e6c0df4b82dc948b5cb2"}},"name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.0"},"nteract":{"version":"nteract-front-end@1.0.0"}},"nbformat":4,"nbformat_minor":2}