This is a hacking robot recipe — it takes 30 minutes tops, really just beginner programing/cloud skills and about $50 in ingredients (Raspberry Pi 3) — plus cardboard.
Recently my 11yo son got obsessed with one of the personal assistants available on the market. He made me (thru Santa Clause, Easter Bunny, or Tooth Fairy — yeah, good for him — he still believes in them — just till the middle school, eh!!!) to buy him one of the popular ones out there.
Now I want to get those download the ghost abilities of creators of Scarlet Johanson’s role of Major in exciting Ghost in the Shell
Unfortunately I failed to convince him, that I am going to help him building his own personal assistant from scratch with Raspberry Pi, IBM Watson and TJBot cardboard robot that is connected to the cloud. He knew better than that!
To my luck fortunetly my 8yo daughter felt into my TJBot trap, and I even made her an example of the first Proof of Technology.
Back in 2014 I was talking to VP of Engineering from Jibo, on their challenges with building, marketing, and delivering a Jibo robot over that so successful Indigogo crowdfunding campaign. Have you seen this campaign? That was one of the sexiest socially enabled robot assistants of the time. Today anyone can build it — more less — in matter of days, if not hours — based on the existing cloud resources of AI, various SaaS based systems. I will show you how to do a small robot — an early prototype of Jibo robot just with cardboard, Raspberry Pi and its connection to the cloud based AI. Check the Jibo crowdfunding video — https://www.indiegogo.com/projects/jibo-the-world-s-first-social-robot-for-the-home#/
For Jibo, due to the long cycle from the crowdfunding to delivery (more than 2 years!!!) it lost its cutting edge — and now the Jibo might be wishing to be acquired. Regretfully I know the same pain from my startup of missing the window of opportunity — here are some outdated features that could make it become the hit again: https://youtu.be/U1RASlbIIVc
One would say that this might have been the best animated personal assistant, that you could relate to as an object with personality. It missed its mark, but there is the hope. Because the robot ghost 👻 could be in a cloud ☁️ so it could be easily downloaded into your robot shell.
Let the hack begin
So without further due let’s start. To do your personal assistant you need the following ingredients:
- Raspberry Pi 3
- usb microphone 🎤
- some speaker 🔊 (or earphones)
- IBM Cloud ⛅️ account (lite account would do and it is completely free — yay 😀 !)
- and the cardboard for TJ Bot robot 🤖
setting up raspberry pi
Raspberry Pi 3 is one of the latest IoT type computers out there. It is very simple to setup. After buying it thru the online store — the bare board is about $35 (you need to have a memory card, a USB microphone, and attach it to a small/any speaker). Just follow this link to boot it, and set it up (the Youtube video with the recipe is coming soon…). Do not forget to get your keyboard, and mouse with a monitor (attached via HDMI) to do an initial setup. Later you might just to SSH to the board.
- setting it up: https://www.raspberrypi.org/help/noobs-setup/2/
- adding necessary libraries like
- clonnig git repo for TJBot joke telling bot: https://github.com/ibmtjbot/tjbot/tree/master/recipes/conversation
setting up IBM Cloud
You need IBM Cloud ⛅️ account — the free lite account would do — for Speech to Text, Text to Speech and the chatbot Assistant AI services . All these services are provided by IBM Watson. After logging in go to catalog, and provision — otherwise create — the following services:
- Watson Speech to Text; 🎤
- Watson Text to Speech; 🔊
- Watson Assistant (aka Conversation) — in addition to provisioning the Watson Assistant service you need to import the Watson Assistant workspace workspace-sample.json from TJ Bot joke telling library provided above.
Now you can try the dialog out as it is shown on the next screen:
When you come back to the Watson Assistant dashboard simply click 3 dot mini hamburger to get the details of the workspace as it is shown on the below picture:
connecting the dots . . .
Now you are ready to connect TJBot node.js library with the IBM Watson services. You need to copy the credentials from the above mentioned 3 services, and the workspace id from the Watson Assistant in the config.js file:
$ nano config.js
You will see the following fields to be filled with the above mentioned information:
exports.conversationWorkspaceId = ‘’;
exports.credentials = {};
exports.credentials.conversation = {
password: ‘’,
username: ‘’
};
exports.credentials.speech_to_text = {
password: ‘’,
username: ‘’
};
exports.credentials.text_to_speech = {
password: ‘’,
username: ‘’
};
In the configuration.js file (use again the nano editor, or other) you can configure the “attention” word — for example “computer” (like the other systems are having: Jibo, Hi Google, Alexa, or Hey Siri). And the voice type (male, or female). Then you can run the program and test the conversation with the TJBot with the following command:
$ sudo node conversation.js
Folding cardboard of TJBot robot
And now there is the final step to use the cardboard to fold it into the TJBot robot 🤖 , and insert into it your Raspberry Pi. The details on how to do it you can find it here: https://www.research.ibm.com/tjbot/
et voila!
Where to go from here?
- AI to manage the attention word — you will quickly notice that Watson Speech to Text service constantly listens to all the voices (yes, it means it sends all the sounds to the cloud ☁️ ) until it gets back the attention word — then it sends to the Watson Assistant service. In order to change it you might want to use an AI service that runs on Raspberry Pi that monitors sounds locally (without sending them to the cloud). As soon as it hears the attention word — only then Raspberry Pi starts to listen to the voice through the cloud ☁️ based AI service. One of the example programs that works like this is Snowboy ⛄️ — check the steps on doing so here: http://docs.kitt.ai/snowboy/ But otherwise it would not send a sound to the cloud ☁️.
- new TJBot skills, like “computer, what is the weather like today?” — these skills you can easily add through composing cloud services like Weather Data Service. You can find plenty of DIY recipes on the resources here: https://github.com/ibmtjbot
- other dialogs — check the community bot exchange: https://developer.ibm.com/code/exchanges/bots/
I hope you find this hack entertaining and adventorous. Feel free to follow me on twitter.
