Hijack your robot vacuum to tell you a joke — Back to the Past!
Two years! See how many things changed — and how many people went ahead.
Long time ago (2–3 years ago) when JavaOne Conference wasn’t called Oracle CodeOne I was invited to present at JavaOne on the subject of “Transforming the lightweight IoT with Voice UI, Java, AI, and Cloud”. Easier said than done. Over the summer of 2017 I had to collect couple technologies in order to deliver a compelling presentation for demanding attendees of JavaOne conference.
With Dan Zeitman in 2017 I worked to develop a small robot assistant that searches the Cloudinary.com media resources for your blog posts with verbal requests like “hi bot, find me a JavaOne Duke, please”. As the robot assistant we used TJ-Bot. After delivering the picture to your Slack channel the bot — if requested — would work on the format and transformations of the picture. This proof of technology gave me some deeper insights into IBM Watson Conversation service — the AI service available from IBM Cloud — Bluemix. We presented the solution in Galvanize in San Francisco in the middle of September.
The code for the Cloudinary service that tells you how good your picture is against the “perfect picture” classifier looked like that on Android in Java (the repository: https://github.com/blumareks/cloudinary-watson — from lab2):
VisualRecognitionOptions options2 =
new VisualRecognitionOptions.Builder().images(image).build(); DetectedFaces faces = visualService.detectFaces(options2).execute(); String showAgeMin, showAgeMax, showGender;
//checking if there are any faces on the picture
if (faces!=null && !faces.toString().equalsIgnoreCase(“{}”)) {
System.out.println(“faces obj not null”);
if (!faces.getImages().get(0).getFaces().isEmpty()) {
Face face = faces.getImages().get(0).getFaces().get(0);
Face.Age age =
faces.getImages().get(0).getFaces().get(0).getAge();
showAgeMin = Integer.toString(age.getMin());
showAgeMax = Integer.toString(age.getMax());
showGender = face.getGender().getGender();
ttsResponse = ttsResponse + “. And I see you are taking
picture of a person — probably “ + showGender
+ “ who is about “ + showAgeMin + “ years old.”;
}//invoking Text to Speech service:
streamPlayer = new StreamPlayer(); streamPlayer.playStream(textToSpeechService.synthesize(ttsResponse , Voice.EN_MICHAEL).execute());
Furthermore with JeanCarl Bisson I developed a small iOS and server-side swift aplications to manage the group of TJ-Bots. The presentation was awesome — 8 TJ-Bots blinked the LED eyes, took pictures and spoke when asked to use IBM Watson Text-to-Speech service. JeanCarl was folding the cardboard robots over entire weekend. It is a great adventure for one robot (or supervising kids to do it), but rather mundane task if repeated 8–10 times. But the group of robots gave us perfect introduction to processing more information on board of the Raspberry Pi, the little and not expensive computer heart of TJ-Bot (the computer costs ~$35). The group of TJ-Bots were accessible thru IBM API Connect as the API end point. JeanCarl and I tested the solution remotely the night before the presentation starting at excruciating 9am — mind the South Bay is prone to outstanding AM traffic around the San Jose Convention Center, where we presented at API World end of September.
The code that was used to invoke the code on Kitura the API Code the WAVE command for TJ-Bots is in the following github repository https://github.com/blumareks/2017-api-world).
I reused the Massive Online Open Course on https://developer.ibm.com/courses/all/robots-coming-build-iot-apps-ibm-watson-swift-node-red/ from developerWorks that Lennart Frantzel (AKA Developer Advocate in Silicon Valley) and I created beginning of this year — where a mobile app drives a home robot with iOS Swift, Node-red, and IBM Watson IoT Platform foundation.
During the summer together with John Cutter I presented another voice powered solution — this time it was the VR game leveraging IBM Watson Speech to Text service in Ubisoft’s Start Trek Bridge VR game at Galvanize Cognitive weekend.
So these technologies helped me to build the solution I am able to share with you now. I built it in these 3 simple steps:
- get the conversation going — using the Mobile Kick-start project for Android Java (or iOS Swift) and IBM Watson Conversation service;
- adding the TJ-Bot Voice UI in Javascript— now my Raspberry Pi was able to crack robot jokes — the basic functionality for the social robots; I used the TJBot’s recipe for that;
- adding the Voice UI, Conversation services to the existing solution that enables driving — the hijacked home robot with Watson — from the cloud: docking, taking pictures.
Expanding the capabilities of the robot exposed me to new ideas that I might be able to demonstrate in coming months. Would it help me run the elderly care robot campaign on a crowdfunding site? I will share with you this shortly.
The link to my JavaOne github repository with all the above mentioned resources, solution videos, and additional information is here: bitly.com/CodeCampSV2017
As you can see a lot of things moved over in past 3 years. It is outstanding to see how the things are moving out. Yet this blog post might be useful to some.
Follow me on my twitter: https://twitter.com/blumareks
