Create Your Own VR Enviornment Using Voice Commands and Controllers Input with IBM Watson Speech-to-Text, Assistant, and Text-To-Speech on Unity
Austin Stanbury, Ines Said, Patrick Molen, Rashaan Siddiqui
10/18/2020
This projects implements three IBM Watson Services: Speech-To-Text, Text-To-Speech and Assistant!
WonderLab places the user in a digital realm accompanied by the assistant robot Rollanda ("Roll", get it?). Rolanda can make anything you desire appear in the space, and has
access to a large library of assets in many colors. She can make rooms, trees, garden fences, chairs, paintings, computers, kitchen items and anything else you can imagine come
out of thin air, just by telling her "I want a [object]". For example, if you wanted a room, you simply tell Rollanda "I want a room", and she will make one appear for you.
She'll also give a verbal affirmation that she did what you asked, including her favorite word "voila!" (pronounced voy - lah).
This means that anybody, at any skill level can jump into the WonderLab and reimagine their reality.
Start by creating your IBM Watson Speech-To-Text, Text-To-Speech and Assistant.
Follow the steps below to find your creditionals:
1- Go to your resource list by clicking the panel at the top left of your screen.
2- Under services, click the service you want to get creditionals for.
3- Navigate to Manage and under Credentials, you can find API key and URL (See Below).
1- Go to your resource list by clicking the panel at the top left of your screen.
2- Under services, click IBM Watson Assistant.
3- Navigate to Manage and under Credentials, you can find API key and URL. You only need to get the URL from this page, as it is different from the assistant URL.
4- Click the big blue "Launch Watson Assistant" button.
5- Click the three dots at the side of your assistant (as shown below).
6- Click on Settings.
7- Navigate to API details on the left of the window. In this page, you can see all the credentials needed.
WARNING !! THE COPY BUTTON DOES NOT WORK ON THE ASSISTANT PAGE. MAKE SURE TO HIGHLIGHT THE CREDENTIALS AND MANUALLY COPY AND PASTE THEM.
Once you have your credentials saved, make sure to add them to Watson Settings in Unity:
1- Search Watson Settings on Unity search bar.
2- Select the red Icon.
3- Paste credentials.
Oculus APK is already included in this repo. There is no need to import it again. Make sure VR is still enabled in your project by following these steps:
1- Navigate to Unity Build Settings, Player Settings, XR settings
2- Make sure Oculus is added to the Virtual Reality SDKs, otherwise click the + button and add it.
3- Make sure Shared Depth Buffer and Dash Support are checked.
There are some lag issues if too many huge objects are placed in the scene.