The document provides an overview of building conversational experiences using Actions on Google. It discusses the Google Assistant, Google Home/mobile devices, and Actions on Google as the way for developers to extend the Assistant via apps. It then covers various aspects of developing for Actions on Google like intents, entities, contexts, fulfillment, and supported programming languages. Tips are provided around designing conversational experiences, building a persona, supporting multiple surfaces, and user information. Finally, it discusses ways for developers to reach users through implicit and explicit triggering in the Assistant and promoting their Assistant app.

1. Building Conversational
Experiences with
Actions on Google
+GreenIdo
@greenido
Developer Advocate
June 2017

2. The Google Assistant — A conversation between you
and Google that helps you get things done in your
world.
Google Home/Mobile device — The surface to
interact with the Assistant.
Actions on Google — How developers can extend the
Assistant (via Assistant apps)

3. The next session about Android is “Building Data
Driven applications with Android” in room 404 on
Tuesday at 4. Would you like to hear more?
Sure! Here’s Google IO 17.
Ok Google, let’s talk to Google IO 17
Enter Earcon
Exit Earcon
Hi, I’m one of the Googlers running I/O this year! I can
help you explore topics or pick a session to attend.
What would you like to know?
When is the next Android talk?

5. How does it work?
Design Develop Discover

6. Assistant app
{ conversation
api request }
{ conversation
api response }
user input
app
response

7. Ok Google,
let’s talk to
Google IO 17
I want to know
about Machine
Learning
Sure, here’s
Google IO 17
Hi! Welcome
to Google IO!
The next ML
session is...
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile
Handle
“action.intent.MAIN”
Speech to Text
Text to Speech
...
...
...

8. “
Intent Matching — Match and categorize user utterances to
an intent.
Entity Extraction — Identify key words and phrases spoken by
the user.
@

9. Actions on Google API.AI
Intent - type of input Event
actions.intent.TEXT Intent (user utterance)
Action - app entry point Triggering Intent
N/A Action - fulfillment alias
Terminology

10. Ok Google,
let’s talk to
Google IO 17
I want to know
about Machine
Learning
Sure, here’s
Google IO 17
Hi! Welcome
to Google IO!
The next ML
session is...
Speech to Text
NLP
Knowledge Graph
ML Ranking
User Profile Convert
“action.intent.MAIN”
to WELCOME event
Speech to Text
Text to Speech
...
...
...
...
NLP:
Intent Matching
Entity Extraction

12. Intent
Triggered via a series of “user says” phrases or platform based events
Can collect entity values
Matched at every turn of conversation

13. Follow-up Intents

14. Input and Output Contexts
You can require a context to be available before an Intent is enabled
Intent can set context to enable other Intents
Retain data across intents

15. Entities (slot-filling)
Values that we are trying to capture from the user phrases
Can specify a parameter name and a type of value
Values can be optional
Values can be a list of fixed values

16. System Entities
● Time
● Date
● Time Period
● Number
● Cardinal
● Ordinal
● Temperature
● Speed
● Volume
● Weight
● Age
● Currency
● Country
● Location
● Language
● Artist
● Music
● ….

17. Developer Entities

18. Specify follow up questions if a user doesn’t specify certain values
Read out in random order to make it more natural
Prompts

19. Fulfillment

21. Node.js Client library
https://github.com/actions-on-google/actions-on-google-nodejs
const App = require('actions-on-google').ApiAiApp;
exports.yourApp = (request, response) => {
const app = new App({request, response});
console.log('Request headers: ' + JSON.stringify(request.headers));
console.log('Request body: ' + JSON.stringify(request.body));
// Fulfill action business logic
function responseHandler (app) {
// Complete your fulfillment logic and send a response
app.ask('Hello, World!');
}
const actionMap = new Map();
actionMap.set('<API.AI_action_name>', responseHandler);
app.handleRequest(actionMap);
};

22. Facts about Google sample (context and deep links)
https://developers.google.com/actions/develop/apiai/tutorials/google-facts

23. Facts about Google sample (context and deep links)
https://developers.google.com/actions/develop/apiai/tutorials/google-facts

24. Facts about Google sample (context and deep links)
https://developers.google.com/actions/develop/apiai/tutorials/google-facts

25. Facts about Google sample (context and deep links)
https://developers.google.com/actions/develop/apiai/tutorials/google-facts

26. if (app.hasSurfaceCapability(app.SurfaceCapabilities.SCREEN_OUTPUT)) {
app.ask(app.buildRichResponse()
.addSimpleResponse(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`)
.addBasicCard(
app.buildBasicCard('Google is an amazing company.')
.setImage(GOOGLE_LOGO_SRC))
.addSuggestions(['History', 'Headquarters']));
} else {
app.ask(`Here's a fact for you. ${fact} Which one ` +
`do you want to hear about next, Google's history or headquarters?`);
}
Facts about Google Code Snippet
Designing for multiple surfaces

27. From ‘app’ to ‘experience’...

28. Create a persona.

29. Preserve and reinforce your persona by
engaging the user as a separate entity
from the Google Assistant.
Own it.
Hey! This is ___
Welcome to ___
Ready to play ____
Hi! ___ here.
Hello. I’m ___
Greetings, human.
Welcome back to ___
Hey again. ___
Let’s play ___
Here’s your ___
Brought to you by ___
Hi there, ___
Let’s get started.
Ready for your ___
___, here to…
Live from ___
This is ___
What’s up, ___
and more...

30. <speak> <!-- Must be at the start of the string -->
<say-as interpret-as="characters">SSML</say-as>
<break time="3s"/>.
<audio src="https://example.com/file.mp3"></audio>
<say-as interpret-as="cardinal">10</say-as>.
<say-as interpret-as="ordinal">10</say-as>
<say-as interpret-as="characters">10</say-as>.
<sub alias="World Wide Web">WWW</sub>.
<p><s>This is one.</s><s>This is two.</s></p>
</speak> <!-- Must be at the end of the string -->
Reinforce it with SSML
Speech Synthesis Markup language
“S S M L”
[3 second pause]
[audio file plays]
“Ten”
“Tenth”
“One Oh”
World Wide Web
[two sentences]

31. function welcome (app) {
return isPreviousUser(app.getUser().userId).then((userHasVisited) => {
if (userHasVisited) {
app.ask(`Welcome to Number Genie!...`, NO_INPUT_PROMPTS);
} else {
app.ask(`Hey you're back...`, NO_INPUT_PROMPTS);
}
});
}
Reinforce it with persistence
https://developers.google.com/actions/assistant/best-practices

32. Design for multiple surfaces.

33. Support different surface capabilities
https://developers.google.com/actions/assistant/surface-capabilities
AUDIO_OUTPUT SCREEN_OUTPUT

34. function simpleResponse (app) {
app.ask({
speech: 'Howdy! I can tell you fun facts about ' +
'almost any number, like 42. What do you have in mind?',
displayText: 'Howdy! I can tell you fun facts about ' +
'almost any number. What do you have in mind?'
});
}
Support speech and display text
https://developers.google.com/actions/assistant/responses
Chat text should be a subset of audio

35. function suggestionChips (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse('Howdy! I can tell you fun facts ‘ +
‘about almost any number. What number do you have ‘ +
‘in mind?')
.addSuggestions(['0', '42', '100', 'Never mind'])
);
}
Guide the user (suggestion chips)
https://developers.google.com/actions/assistant/responses
Chip text should be under 25 characters

36. function basicCard (app) {
app.ask(app.buildRichResponse()
.addSimpleResponse('Math and prime numbers it is!')
.addBasicCard(
app.buildBasicCard(`42 is an even composite number. It ` +
`is composed of three distinct prime numbers multiplied together. It ` +
`has a total of eight divisors. 42 is an abundant number, because the ` +
`sum of its proper divisors 54 is greater than itself. To count from ` +
`1 to 42 would take you about twenty-one…`)
.setTitle('Math & prime numbers')
.addButton('Read more')
.setImage('https://example.google.com/42.png', 'Image alternate text')
)
);
}
Display basic cards
https://developers.google.com/actions/assistant/responses

37. Lists and carousels for selection
https://developers.google.com/actions/assistant/responses
Used for easy selection
<10 items
Used for comparison
<30 items

38. Know the
user

39. Google Home Mobile Device
NAME Registered device user’s full name Registered device user’s full name
DEVICE_COARSE_LOCATION Zip code and city N/A
DEVICE_PRECISE_LOCATION Coordinates and street address Coordinates
let permission = app.SupportedPermissions.DEVICE_COARSE_LOCATION;
app.askForPermission('To find bookstores near you', permission);
Q: "Recommend me a local bookstore"
A: "To find bookstores near you, I'll just need to get your zip code from Google. Is that okay?"
Ask for information
https://developers.google.com/actions/assistant/helpers#user_information

40. Link an account (OAuth2)
https://developers.google.com/actions/identity/
Seamless account linking with Google Sign-in

41. Transact with the user
https://developers.google.com/actions/identity/
Build orders
Use Google provided payment
instrument
Use your payment processor
(Stripe, Braintree, Vantiv, more
coming)
Update order status

43. Reach users...

44. In Dialogue Discovery:
Explicit Triggering (of Actions)
Ok Google, ask Google I/O 17 about what was announced
Trigger Phrase App Name
Developer Specified
Action
Preposition
Action Phrase
Developer Specified
Partner Examples: Let’s speak to Domino’s
At Akinator
Ask Dr. Doggy if dogs can eat chocolate
Ok Google, let’s talk to Google I/O 17
Trigger Phrase App Name
Developer Specified

45. Hey Google, I want to play a game.Hey Google, tell me a joke.
In Dialogue Discovery:
Implicit Triggering
Hey Google, I want to work out. Hey Google, what’s the surf report

46. Discovery in the Google
Assistant Directory

47. Link to your
Assistant App
From anywhere
Share through social media.
Promote through your own site or apps.
Encourage press to drive traffic
to your Assistant app.

48. Where do I start?

49. Actions Console
console.actions.google.com

50. 0. Put away the code
1. Draft a persona
2. Write some dialog

51. DIALOG + STRUCTURE

52. g.co/dev/ActionsDesign

53. developers.google.com/actions

54. Actions on Google Developer Community
https://g.co/actionsdev
Actions on Google Developers
https://developers.google.com/actions
Actions on Google Developer Challenge
https://g.co/actionschallenge
+GreenIdo
@greenido
Developer Advocate