How to develop your actions on google? In these slides we covered the main aspects: API.AI, Voice UI, google assistant and more.

1. Developing Conversation
Actions for the Google
Assistant
+GreenIdo
@greenido
Developer Advocate
June 2017

3. Google Home — voice-activated speaker powered
The Google Assistant — A conversation between you and
Google that helps you get more done in your world.
Actions on Google — How developers can extend the
assistant (via Conversation Actions)

6. “Ok Google, talk to personal chef”
Conversation API, Actions SDK
Invoke “personal chef” action
“Sure, here’s personal chef.
Hi, I’m your personal chef, what
are you in the mood for?”
Speech to Text
“What protein would you
like to use?”
“Well, it’s kind of cold outside, so I’d like
something to …”
Text to Speech
“Sure, here’s your personal chef”
Speech to Text, NLP,
Knowledge Graph,
ML Ranking, User
Profile, Text to
Speech

9. Webhook
(optional)
“Ok Google, talk to personal chef”
api.ai
Invoke “personal chef” action
“Well, it’s kind of cold outside, so I’d like
something to …”
“Sure, here’s your personal chef”
Speech to Text, NLP,
Knowledge Graph,
ML Ranking, User
Profile, Text to
Speech
“Sure, here’s personal chef.
Hi, I’m your personal chef, what
are you in the mood for?”
“What protein would you
like to use?”
Speech to Text
Text to Speech

12. Intent
Triggered via a series of “user says” phrases
Share a common set of entities
- Create new Intent otherwise

13. Intent name
Name to use in API.AI console
Only used by the developer, not referenced in your code

14. Action name
String passed to the webhook that indicates what action is being performed
Can share the same action name across different Intents
API.AI specific use of “Action” compared to Google Assistant “Conversation Action”

15. Input and Output Contexts
You can require a context to be available before an Intent is enabled
Intent can set context to enable other Intents
Implement flow control between some Intents

16. Entities
Values that we are trying to capture from the user phrases
Can specify a parameter name and a type of value
Values can be optional
Values can be a list of fixed values

17. Text response
Spoken when the Intent is complete
Embed entity values if desired
- “I found a recipe for $protein and $vegetable!”
Webhook can dynamically adjust this response

20. Prompts
Specify follow up questions if a user doesn’t specify certain values
Read out in random order to make it more natural

21. Webhook with HTTPS POST request

23. Node.js client library
https://developers.google.com/actions/apiai/fulfillment
'use strict';
// Boilerplate setup
let ApiAiAssistant = require('actions-on-google').ApiAiAssistant;
let express = require('express');
let bodyParser = require('body-parser');
let app = express();
app.set('port', (process.env.PORT || 8080));
app.use(bodyParser.json({type: 'application/json'}));
// Create an instance of ApiAiAssistant
app.post('/', function (request, response) {
const assistant = new ApiAiAssistant(
{request: request, response: response});
});
// Create functions to handle requests here
// Start the server
let server = app.listen(app.get('port'), function () {
console.log('App listening on port %s', server.address().port);
console.log('Press Ctrl+C to quit.');
});

24. Node.js client library
https://developers.google.com/actions/apiai/fulfillment
const assistant = new ApiAiAssistant({request: request, response: response});
const WELCOME_INTENT = 'input.welcome'; // the action name from the API.AI intent
const NUMBER_INTENT = 'input.number'; // the action name from the API.AI intent
const NUMBER_ARGUMENT = 'input.mynum'; // the action name from the API.AI intent
function welcomeIntent (assistant) {
assistant.ask('Welcome to action snippets! Say a number.');
}
function numberIntent (assistant) {
let number = assistant.getArgument(NUMBER_ARGUMENT);
assistant.tell('You said ' + number);
}
let actionMap = new Map();
actionMap.set(WELCOME_INTENT, welcomeIntent);
actionMap.set(NUMBER_INTENT, numberIntent);
assistant.handleRequest(actionMap);

25. Facts About Google sample (context and deep links)
https://developers.google.com/actions/apiai/fulfillment

26. Requesting user’s name and location
https://developers.google.com/actions/assistant/helpers
Q: "Recommend me a local bookstore"
A: "To find bookstores near you, I'll just need to get your zip code from Google. Is that
okay?"
NAME (Registered device user’s full name)
DEVICE_PRECISE_LOCATION (Coordinates and street address)
DEVICE_COARSE_LOCATION (Zip code and city)
// assistant is an instance of ApiAiAssistant or ActionsSdkAssistant
let permission = assistant.SupportedPermissions.NAME;
assistant.askForPermission('To address you by name', permission);

27. Account linking with OAuth 2.0
https://developers.google.com/actions/identity/account-linking
"It looks like your <invocation name> account is not linked yet. You can link <invocation
name> to your Google Account from the Google Home app."
Google Home app provides a link to your login page.
OAuth token is presented to your webhook.
Will only connect to a login page on your domain, not a 3rd party login.

28. Speech Synthesis Markup Language (SSML) & Earcons
https://developers.google.com/actions/reference/ssml
<speak> <!-- Must be at the start of the string -->
<say-as interpet-as="characters">SSML</say-as>
<break time="3s"/>.
<audio src="https://example.com/file.mp3"></audio>
<say-as interpret-as="cardinal">10</say-as>.
<say-as interpret-as="ordinal">10</say-as>
<say-as interpret-as="characters">10</say-as>.
<sub alias="World Wide Web">WWW</sub>.
<p><s>This is one.</s><s>This is two.</s></p>
</speak> <!-- Must be at the end of the string -->
“S S M L”
[3 second pause]
[audio file plays]
“Ten”
“Tenth”
“One Oh”
World Wide Web
[two sentences]

29. g.co/actionswebsim

31. Action naming and policies
https://developers.google.com/actions/policies/general-policies
● Policies about what Conversation Actions are allowed
● See the guidelines for more information
● Invocation names must be unique since it is a global name
● Direct Actions not widely available to build and deploy,
register for interest

32. Conversation Design

34. GUI :: VUI What’s the difference?

35. Top 3 Design Tips

36. Create a
persona
1

37. Persona is conveyed
through:
● Tone
● Word and phrase choices
● Functional design
● Style
● Technique
● Voice
...and based on:
1. Your user population and
their needs
2. The imagery & qualities
associated with your
brand

38. Preserve and reinforce your
persona by engaging the
user as a separate entity
from the Google Assistant.
Own it.
Hey! This is ___
Welcome to ___
Ready to play ____
Hi! ___ here.
Hello. I’m ___
Greetings, human.
Welcome back to ___
Hey again. ___
Let’s play ___
Here’s your ___
Brought to you by ___
Hi there, ___
Let’s get started.
Ready for your ___
___, here to…
Live from ___
This is ___
What’s up, ___
and more...

39. Think outside
the box.
Literally.
2

40. Confidential and proprietary
code code code code code
code code code code
code code “ dialog string” code
code code code
code code code code code
code “dialog string” code code
code code code code code code code
code code code
code code
code
code code
code
code
code code code
code “dialog string” code code code
code code
code code code
code code code code code code code code
code code code code code code code code
code
STRUCTURE + CODE

41. Confidential and proprietary
DIALOG + STRUCTURE

42. In conversations,
there are no
“errors”.
3

43. real life
user
conditions{

44. “I don’t understand you”

45. When a so-called “error” occurs in a
conversation, it should be treated simply as
a new turn in the dialog, only with different
conditions.

46. User’s have intent

47. g.co/dev/ActionsDesign

48. Actions on Google Developer
Community
https://g.co/actionsdev
Actions on Google Developers
https://developers.google.com/actions
@ActionsOnGoogle

49. Thank you!
+GreenIdo
@greenido
Developer Advocate
June 2017