Posted by Payam Shodjai, Director, Product Management Google Assistant
With 2020 coming to a close, we wanted to reflect on everything we have launched this year to help you, our developers and partners, create powerful voice experiences with Google Assistant.
Today, many top brands and developers turn to Google Assistant to help users get things done on their phones and on Smart Displays. Over the last year, the number of Actions built by third-party developers has more than doubled. Below is a snapshot of some of our partners who’ve integrated with Google Assistant:
Below are a few highlights of what we have launched in 2020:
1. Integrate your Android mobile Apps with Google Assistant
App Actions allow your users to jump right into existing functionality in your Android app with the help of Google Assistant. It makes it easier for users to find what they're looking for in your app in a natural way by using their voice. We take care of all the Natural Language Understanding (NLU) processing, making it easy to develop in only a few days. In 2020, we announced that App Actions are now available for all Android developers to voicify their apps and integrate with Google Assistant.
For common tasks such as opening your apps, opening specific pages in your apps or searching within apps, we introduced Common Intents. For a deeper integration, we’ve expanded our vertical-specific built-in intents (BIIs), to cover more than 60 intents across 10 verticals, adding new categories like Social, Games, Travel & Local, Productivity, Shopping and Communications.
For cases where there isn't a built-in intent for your app functionality, you can instead create custom intents that are unique to your Android app. Like BIIs, custom intents follow the actions.xml schema and act as connection points between Assistant and your defined fulfillments.
Learn more about how to integrate your app with Google Assistant here.
2. Create new experiences for Smart Displays
We also announced new developer tools to help you build high quality, engaging experiences to reach users at home by building for Smart Displays.
Actions Builder is a new web-based IDE that provides a graphical interface to show the entire conversation flow. It allows you to manage Natural Language Understanding (NLU) training data and provides advanced debugging tools. And, it is fully integrated into the Actions Console so you can now build, debug, test, release, and analyze your Actions - all in one place.
Actions SDK, a file based representation of your Action and the ability to use a local IDE. The SDK not only enables local authoring of NLU and conversation schemas, but it also allows bulk import and export of training data to improve conversation quality. The Actions SDK is accompanied by a command line interface, so you can build and manage an Action fully in code using your favorite source control and continuous integration tools.
Interactive Canvas allows you to add visual, immersive experiences to Conversational Actions. We announced the expansion of Interactive Canvas to support Storytelling and Education verticals earlier this year.
Continuous Match Mode allows the Assistant to respond immediately to a user’s speech for more fluid experiences by recognizing defined words and phrases set by you.
We also created a central hub for you to find resources to build games on Smart Displays. This site is filled with a game design playbook, interviews with game creators, code samples, tools access, and everything you need to create awesome games for smart displays.
Actions API provides a new programmatic way to test your critical user journeys more thoroughly and effectively, to help you ensure your Action's conversations run smoothly.
The Dialogflow migration tool inside the Actions Console automates much of the work to move projects to the new and improved Actions Builder tool.
We also worked with partners such as Voiceflow and Jovo, to launch integrations to support voice application development on the Assistant. This effort is part of our commitment to enable you to leverage your favorite development tools, while building for Google Assistant.
We launched several other new features that help you build high quality experiences for the home, such as Media APIs, new and improved voices (available in Actions Console), home storage API.
Get started building for Smart Displays here.
3. Discovery features
Once you build high quality Actions, you are ready for your users to discover them. We have designed new touch points to help your users easily learn about your Actions..
For example, on Android mobile, we’ll be recommending relevant Apps Actions even when the user doesn't mention the app’s name explicitly by showing suggestions. Google Assistant will also be suggesting apps proactively, depending on individual app usage patterns. Android mobile users will also be able to customize their experience, creating their own way to automate their most common tasks with app shortcuts, enabling people to set up quick phrases to enable app functions they frequently use. By simply saying "Hey Google, shortcuts", they can set up and explore suggested shortcuts in the settings screen. We’ll also make proactive suggestions for shortcuts throughout Google Assistants’ mobile experience, tailored to how you use your phone.
Assistant Links deep link to your conversational Action to deliver rich Google Assistant experiences to your websites, so you can send your users directly to your conversational Actions from anywhere on the web.
We also recently opened two new built-in intents (BIIs) for public registration: Education and Storytelling. Registering your Actions for these intents allows your users to discover them in a simple, natural way through general requests to Google Assistant on Smart Displays. People will then be able to say "Hey Google, teach me something new" and they will be presented with a browsable selection of different education experiences. For stories, users can simply say "Hey Google, tell me a story".
We know you build personalized and premium experience for your users, and need to make it easy for them to connect their accounts to your Actions. To help streamline this process we opened two betas for improved account linking flows that will allow simple, streamlined authentication via apps.
Looking ahead, we will double down on enabling you, our developers and partners to build great experiences for GoogleAssistant and help you reach your users on the go and at home. You can expect to hear more from us on how we are improving the Google Assistant experience to make it easy for Android developers to integrate their Android app with Google Assistant and also help developers achieve success through discovery and monetization.
We are excited to see what you will build with these new features and tools. Thank you for being a part of the Google Assistant ecosystem. We can’t wait to launch even more features and tools for Android developers and Smart Display experiences in 2021.
Want to stay in the know with announcements from the Google Assistant team? Sign up for our monthly developer newsletter here.
Posted by Baris Gultekin and Payam Shodjai, Directors of Product Management
Top brands turn to Google Assistant every day to help their users get things done on their phones and on Smart Displays -- such as playing games, finding recipes or checking investments, just by using their voice. In fact, over the last year, the number of Actions completed by third-party developers has more than doubled.
We want to support our developer ecosystem as they continue building the best experiences for smart displays and Android phones. That’s why today at Google Assistant Developer Day, we introduced:
Now, every Android app developer can make it easier for their users to find what they're looking for, by fast forwarding them into the app’s key functionality, using just voice. With App Actions, top app developers such as Yahoo Mail, Fandango, and ColorNote, are currently creating these natural and engaging experiences for users by mapping their users' intents to specific functionality within their apps. Instead of having to navigate through each app to get tasks done, users can simply say “Hey Google” and the outcome they want - such as “find Motivation Mix on Spotify” using just their voice.
Here are a few updates we’re introducing today to App Actions.
Quickly open and search within apps with common intents
Every day, people ask Google Assistant to open their favorite apps. Today, we are building on this functionality to open specific pages within apps and also search within apps. Starting today, you can use the GET_THING intent to search within apps and the OPEN_APP_FEATURE intent to open specific pages in apps; offering more ways to easily connect users to your app through Assistant.
Many top brands such as eBay and Kroger are already using these intents. If you have the eBay app on your Android phone, try saying “Hey Google, find baseball cards on eBay” to try the GET_THING intent.
If you have the Kroger app on your Android phone, try saying “Hey Google, open Kroger pay” to try the OPEN_APP_FEATURE intent.
It's easy to implement all these common intents to your Android apps. You can simply declare support for these capabilities in your Actions.xml file to get started. For searching, you can provide a deep link that will allow Assistant to pass a search term into your app. For opening pages, you can provide a deep link with the corresponding name for Assistant to match users' requests.
Vertical specific built-in intents
For a deeper integration, we offer vertical-specific built-in intents (BII) that lets Google take care of all the Natural Language Understanding (NLU) so you don’t have to. We first piloted App Actions in some of the most popular app verticals such as Finance, Ridesharing, Food Ordering, and Fitness. Today, we are announcing that we have now grown our catalog to cover more than 60 intents across 10 verticals, adding new categories like Social, Games, Travel & Local, Productivity, Shopping and Communications
For example, Twitter and Wayfair have already implemented these vertical built in intents. So, if you have the Twitter app on your Android phone, try saying “Hey Google, post a Tweet” to see a Social vertical BII in action.
If you have the Wayfair app on your Android phone, try saying “Hey Google, buy accent chairs on Wayfair” to see a Shopping vertical BII in action.
Check out how you can get started with these built-in intents or explore creating custom intents today.
Custom Intents to highlight unique app experiences
Every app is unique with its own features and capabilities, which may not match the list of available App Actions built-in intents. For cases where there isn't a built-in intent for your app functionality, you can instead create a custom intent.Like BIIs, custom intents follow the actions.xml schema and act as connection points between Assistant and your defined fulfillments.
Snapchat and Walmart use custom intents to extend their app’s functionality to Google Assistant. For example, if you have the Snapchat app on your Android phone, just say, “Hey Google, send a Snap using the cartoon face lens” to try their Custom Intent.
Or, If you have the Walmart app on your Android phone, just say, “Hey Google, reserve a time slot with Walmart” to schedule your next grocery pickup.
With more common, built-in, and custom intents available, every Android developer can now enable their app to fulfill Assistant queries that tailor to exactly what their app offers. Developers can also use known developer tools such as Android Studio, and with just a few days of work, they can easily integrate their Android apps with the Google Assistant.
Suggestions and Shortcuts for improving user discoverability
We are excited about these new improvements to App Actions, but we also understand that it's equally important that people are able to discover your App Actions. We’re designing new touch points to help users easily learn about Android apps that support App Actions. For example, we’ll be recommending relevant Apps Actions even when the user doesn't mention the app name explicitly by showing suggestions. If you say broadly “Hey Google, show me Taylor Swift”, we’ll highlight a suggestion chip that will guide the user to open up the search result in Twitter. Google Assistant will also be suggesting apps proactively, depending on individual app usage patterns.
Android users will also be able to customize their experience, creating their own way to automate their most common tasks with app shortcuts, enabling people to set up quick phrases to enable app functions they frequently use. For example, you can create a MyFitnessPal shortcut to easily track their calories throughout the day and customize the query to say what you want - such as “Hey Google, check my calories.”
By simply saying "Hey Google, shortcuts", they can set up and explore suggested shortcuts in the settings screen. We’ll also make proactive suggestions for shortcuts throughout the Assistant mobile experience, tailored to how you use your phone.
Back in June, we launched new developer tools such as Actions Builder and Actions SDK, making it easier to design and build conversational Actions on Assistant, like games, for Smart Displays. Many partners have already been building with these, such as Cool Games and Sony. We’re excited to share new updates that not only enable developers to build more, higher quality native Assistant experiences with new game development frameworks and better testing tools, but we’ve also made user discovery of those experiences better than ever.
Improved voices
We’ve heard your feedback that you need better voices to match the quality of the experiences you’re delivering on the Assistant. We’ve released two new English voices that take advantage of an improved prosody model to make Assistant sound more natural. Give it a listen.
These voices are now available and you can leverage them in your existing Actions by simply making the change in the Actions Console.
But what can you build with these new voices? Last year, we introduced Interactive Canvas, an API that lets you build custom experiences for the Assistant that can be controlled via both touch and voice using simple technologies like HTML, CSS, and Javascript.
We’re expanding Interactive Canvas to Actions in the education and storytelling verticals; in addition to games. Whether you’re building an action that teaches someone to cook, explains the phases of the moon, helps a family member with grammar, or takes you through an interactive adventure, you’ll have access to the full visual power of Interactive Canvas.
Improved testing to deliver high quality experiences
Actions Testing API is a new programmatic way to test your critical user journeys and ensure there aren’t any broken conversation paths. Using this framework allows you to run end to end tests in an isolated preview environment, run regression tests, and add continuous testing to your arsenal. This API is being released to general availability soon.
New Dialogflow migration tool
For those of you who built experiences using Dialogflow, we want you to enjoy the benefits of the new platform without having to build from scratch. That’s why we’re offering a migration tool inside the Actions Console that automates much of the work to move projects to the improved platform.
New site for game developers
Game developers, we built a new resource hub just for you. Boost your game design expertise with full source code to games, design best practices, interviews with game developers, tools, and everything you need to create voice-enabled games for Smart Displays.
With more incredible experiences being built, we know it can be challenging to help users discover them and drive engagement. To make it easier for people to discover and engage with your experiences, we have invested in a slew of new discovery features:
New Built-in intents and the Learning Hub
We’ll soon be opening two new set Built-in intents (BIIs) for public registration: Education and Storytelling. Registering your Actions for these intents allows users to discover them in a simple, natural way through general requests to Google Assistant. These new BIIs cover a range of intents in the Education and Storytelling domains and join Games as principal areas of investment for the developer ecosystem.
People will then be able to say "Hey Google, teach me something new" and they will be presented with a Learning Hub where they can browse different education experiences. For stories, users can simply say "Hey Google, tell me a story". Developers can soon register for both new BIIs to get their experiences listed in these browsable catalog.
Household Authentication token and improving transactions
One of the exciting things about the Smart Display is that it’s an inherently communal device. So if you’re offering an experience that is meant to be enjoyed collaboratively, you need a way to share state between household members and between multiple devices. Let’s say you’re working on a puzzle and your roommate wants to help with a few pieces on the Smart Display. We’re introducing household authentication tokens so all users in a home can now share these types of experiences. This feature will be available soon via the Actions console.
Finally, we're making improvements to the transaction flow on Smart Displays. We want to make it easier for you to add seamless voice-based and display-based monetization capabilities to your experience. We've started by supporting voice-match as an option for payment authorization. And early next year, we'll also launch an on-display CVC entry.
Simplifying account linking and authentication
Once you build personalized and premium experiences, you need to make it as easy as possible to connect with existing accounts. To help streamline this process, we’re opening two betas: Link with Google and App Flip, for improved account linking flows to allow simple, streamlined authentication via apps.
Link with Google enables you to allow users to link to Google inside your Android app where they are already logged in to Google, in order to complete the linking flow with fewer clicks.
App Flip helps you build a better mobile account linking experience and decrease drop-off rates. If your app is installed on the user’s phone when they initiate account linking from a Google surface, they are seamlessly flipped to your app to provide authorization.
Assistant links
In addition to launching new channels of discovery for developer Actions, we also want to provide more control over how you and your users reach your Actions. Action links were a way to deep link to your conversational action that has been used with great success by partners like Sushiro, Caixa, and Giallo Zafferano. Now we are reintroducing this feature as Assistant links, which enable partners such as TD Ameritrade to deliver rich Google Assistant experiences in their websites as well as deep links to their Google Assistant integrations from anywhere on the web.
We are very excited about all these announcements - both across App Actions and native Assistant development. Whether you are exploring new ways to engage your users using voice via App Actions, or looking to build something new to engage users at home via Smart Displays, we hope you will leverage these new tools and features and share your feedback with us.
Posted by Jennifer Kohl, Program Manager, Developer Community Programs
On October 16-18, thousands of developers from all over the world are coming together for DevFest 2020, the largest virtual weekend of community-led learning on Google technologies.
As people around the world continue to adapt to spending more time at home, developers yearn for community now more than ever. In years past, DevFest was a series of in-person events over a season. For 2020, the community is coming together in a whole new way – virtually – over one weekend to keep developers connected when they may want it the most.
The magic of DevFest comes from the people who organize and speak at the events - developers with various backgrounds and skill levels, all with their own unique perspectives. In different parts of the world, you can find a DevFest session in many local languages. DevFest speakers are made up of various types of technologists, including kid developers , self-taught programmers from rural areas , and CEOs and CTOs of startups. DevFest also features a wide range of speakers from Google, Women Techmakers, Google Developer Experts, and more. Together, these friendly faces, with many different perspectives, create a unique and rich developer conference.
Hosted by Google Developer Groups, this year’s sessions include technical talks and workshops from the community, and a keynote from Google Developers. Through these events, developers will learn how Google technologies help them develop, learn, and build together.
Sessions will cover multiple technologies, such as Android, Google Cloud Platform, Machine Learning with TensorFlow, Web.dev, Firebase, Google Assistant, and Flutter.
For this reason, Google Developers supports the community-led efforts of Google Developer Groups and their annual tentpole event, DevFest. Google provides esteemed speakers from the company and custom technical content produced by developers at Google. The impact of DevFest is really driven by the grassroots, passionate GDG community organizers who volunteer their time. Google Developers is proud to support them.
During DevFest 2019, 138,000+ developers participated across 500+ DevFests in 100 countries. While 2020 is a very different year for events around the world, GDG chapters are galvanizing their communities to come together virtually for this global moment. The excitement for DevFest continues as more people seek new opportunities to meet and collaborate with like-minded, community-oriented developers in our local towns and regions.
Join the conversation on social media with #DevFest.
Sign up for DevFest at goo.gle/devfest.
Still curious? Check out these popular talks from DevFest 2019 events around the world...
Scalable Vector Graphics (SVG)
How to add Flutter to your app
The Flutter journey: production app
Getting started with microcontrollers
Chet Haase and Romaine Guy: quality
Building the ultimate Beatles app
Posted by Baris Gultekin, Director, Product Management Google Assistant and Payam Shodjai, Director, Product Management Google Assistant
More and more people turn to Google Assistant every day to help them get the most out of their phones and smart displays: From playing games to using their favorite app by voice, there are more opportunities than ever for developers to create new and engaging experiences for Google Assistant.
We welcome you to join us virtually at our Google Assistant Developer Day on Thursday, October 8, to learn more about new tools and features we’re building for developers to bring Google Assistant to mobile apps and Smart Displays and help drive discoverability and engagement via voice. This will also be a great chance to chat live with Google leaders and engineers on the team to get your questions answered.
You’ll hear from our product experts and partnership leads on best practices to integrate with Google Assistant to help users more easily engage with their favorite apps by voice. Other sessions will include in-depth conversations around native development on Google Assistant, with so much more.
We’ll also have guest speakers like: Garrett Gaudini, Head of Product at Postmates, Laurens Rutten, Founder & CEO of CoolGames, Corey Bozarth, VP of Product & Monetization at MyFitnessPal and many other, join us on stage to share their stories about how voice has transformed the way people interact with their apps and services.
Whether you build for mobile or smart home, these new tools will help make your content and services available to people who want to use their voice to get things done.
Registration is FREE! Head on over to the event website to register and check out the schedule.
Posted by Toni Klopfenstein, Developer Advocate
Easy account linking helps create more helpful user experiences with Google Assistant and your products and services. Today, we are launching OAuth-based App Flip, a new feature to help you build a better mobile account linking experience. App Flip allows your users to seamlessly link their accounts to Google without having to re-enter their credentials if they have already signed in to your app on their device. This streamlined authorization flow helps minimize users dropping out of the account linking process, and makes it easier for users to integrate your smart devices into their smart homes. This feature is now available for all Smart Home Actions and is available in beta for other Conversational Actions.
When App Flip is implemented within your application, the Google Assistant or Google Home app automatically flips to your app when users initiate the account linking process. Once users consent to link their accounts, your app then requests an authorization code from your server, and Google handles the remainder of the account linking flow.
Account Linking Flows
By flipping directly to your local app, users can skip logging in again and can simply choose to link their accounts.
You can implement App Flip by enabling your app to accept a deep link that allows Google to connect. We also have test tools available for both Android and iOS to help you verify your app integration with App Flip.
For more details on how to use App Flip with your integration, check out the docs, or the sample Android and iOS apps. For Conversational Actions, please sign up for the Beta program.
We want to hear from you, so continue sharing your feedback with us, and engage with other Action developers in the /r/GoogleAssistantDev community. Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. We can’t wait to see what you build!
Posted by the Assistant Developer Platform team
Since the launch of the Google Assistant, our developer ecosystem has been instrumental in delivering compelling voice experiences to more than 500 million active users. Today, we’re taking a major step forward in helping you build these custom voice apps and services by introducing a suite of new and improved developer tools: Actions Builder and Actions SDK. These tools make building Conversational Actions for the Assistant easier and more streamlined than ever.
Actions Builder is a web-based IDE that lets you develop, test, and deploy directly in the Actions console. The graphical interface lets you visualize the conversational flow, manage Natural Language Understanding (NLU) training data, and debug with advanced tools.
For those of you who prefer local IDEs, the updated Actions SDK provides a file based representation of your Actions project. This lets you author NLU training data and conversational flows locally as well as bulk import and export training data. We've also updated the CLI that accompanies Actions SDK, so you can build and manage Actions projects completely with code, using your favorite source control and continuous integration tools.
Together, Actions Builder and Actions SDK create a seamless, consolidated development experience. No matter what tool you start with, you can switch between them based on what works best for your workflow. For example, you can use Actions Builder to lay out conversational flows and provide NLU training data, Actions SDK to write fulfillment code, and the CLI to synchronize the two. These tools create an environment where all team members can contribute effectively and focus on what they do best: design and code.
A new, powerful interaction model lets you design conversations quickly and efficiently. Intents and scenes let you define robust NLU training data and behavior for specific conversational contexts. Using scenes as building blocks, you define active intents, declare context specific error handling, collect data through slot filling, and respond with prompts.
Scenes also separate conversational flow definitions from fulfillment logic, so you can reuse the same flows across multiple conversations. Transitions between scenes let you define when one conversational context switches to another. All your scenes and transitions describe a full conversational flow and all possible dialog turns.
You can express the entire interaction model with either the Actions Builder or Actions SDK. A typical way to develop is to use Actions Builder to view and edit your scenes and then use Actions SDK to sync changes to your local file system. This lets you version control your project, modify your project files, and build fulfillment in your favorite development environment.
Under the hood, we also made a lot of improvements that your users will appreciate. We sped up the Assistant runtime engine, so users get faster responses and a smoother experience. We’ve also made the runtime engine smarter, so your Actions can understand users better with the same amount of training data.
We've worked with Pretzel Labs and Galinha Pintadinha to test the capabilities of the new platform and to refine the interaction model and runtime engine improvements.
Pretzel Labs built Kids Court with Actions Builder, creating a full conversational flow with no code and added fulfilment for advanced functionality.
"Having the combination of a visual layout with webhook blocks for code helps us collaborate clearly and more efficiently. Something I liked very much about this was the separation between the designer and the developers' parts, making it very intuitive to make design changes without affecting backend logic." -- Adva Levin, founder of Pretzel Labs
Galinha Pintadinha runs one of the biggest YouTube channels and built one the most popular Conversational Actions in their country. Their development team migrated to the new platform to optimize their workflow and simplify future Action development. Galinha Pintadinha’s Actions now contain half the number of intents and have a radically simplified conversation tree. Using features like contextual error handling, they were able to improve the user experience and quality with little to no cost.
"Actions Builder is a robust and well designed toolbox for developing conversational apps. The concept of scenes and transitions helped us define the flow of our Action in a much more streamlined way."-- Mário Neto, engineer at Galinha Pintadinha
To learn more about Actions Builder and SDK and to start developing your next Actions, check out our new developer resources. Our codelabs will walk you through using the new tooling and interaction model. Samples for all major features are also available, so you can start playing with code immediately. See the full set of documentation to start building today.
Stay tuned for more platform updates and happy coding!
Posted by Payam Shodjai, Director of Product Management, Google Assistant
Today at VOICE Global, we shared our vision for Google Assistant to truly be the best way to get things done - and the important role that developers play in that vision, especially as voice experiences continue to evolve.
Google Assistant helps more than 500 million people every month in over 30 languages across 90 countries get things done at home and on the go. What’s at the heart of this growth is the simple insight that people want a more natural way to get what they need. That’s why we’ve invested heavily in making sure Google Assistant works seamlessly across devices and services and offers quick and accurate help.
Over the last few months, we’ve seen people’s needs shifting, and this is reflected in how Google Assistant is being used and the role that it can play to help navigate these changes. For example, to help people get accurate information on Search and Maps - like modified store hours or information on pick-up and delivery - we have been using Duplex conversational technology to contact businesses and update over half a million business listings.
We’ve also been working with our partners to bring great educational experiences into the home, so that families can continue learning in a communal setting. Bamboo Learning is bringing their voice-forward education platform to Google Assistant, with fun, new ways to learn history, math, and reading. Our hand-washing songs continue to be popular. The songs leverage WaveNet's natural expressiveness – allowing us to train Google Assistant to sing across numerous generated voices users can pick from.
Great experiences are at the core of what makes Google Assistant truly helpful. To help existing and aspiring developers build new experiences with ease, we are making some major improvements to our core platform and development tools. Rather than needing to hop back and forth between Actions Console and Dialogflow to build an Action, wouldn’t it be great if there were one integrated platform for building on Google Assistant?
Starting today, we’re releasing Actions Builder, a new web-based IDE that provides a graphical interface to show the entire conversation flow. It allows you to manage Natural Language Understanding (NLU) training data and provides advanced debugging tools. And, it is fully integrated into the Actions Console so you can now build, debug, test, release, and analyze your Actions - all in one place.
If you prefer to work in your own tools, you can use the updated Actions SDK. For the first time, you’ll have a file based representation of your action and the ability to use a local IDE. The SDK not only enables local authoring of NLU and conversation schemas, but it also allows bulk import and export of training data to improve conversation quality. The Actions SDK is accompanied by a command line interface, so you can build and manage an action fully in code using your favorite source control and continuous integration tools.
With these two releases, we are also introducing a new conversation model and improvements to the runtime engine. Now, it’s easier to design and build conversations and users will get faster and more accurate responses. We’re very excited about this suite of products which replaces Dialogflow as the preferred way to develop conversational actions on Google Assistant.
Based on feedback from developers, we’re also adding new functionality to build more interactive experience on Google Assistant with Home Storage, updated Media API and Continuous Match Mode.
One of the exciting things about speakers and smart displays is that they’re communal. Home Storage is a new feature that provides a communal storage solution for devices connected on the home graph and allows developers to save context for all individual users, such as the last saved point from a puzzle game.
Our updated Media APIs now support longer-form media sessions and lets users resume playback of content across surfaces. For example, users can start playback from a specific moment or resume where they dropped out of their previous session.
Sometimes you want to build experiences that enable users to speak more naturally with your action, without waiting for a change in mic states. Rolling out in the next few months, Continuous Match Mode allows Assistant to respond immediately to a user’s speech for more fluid experiences by recognizing defined words and phrases set by you. This is done transparently so that before the mic opens, Assistant will announce the mic will stay open temporarily so users know they can speak freely without waiting for additional prompts. For example, CoolGames is launching a game in a few weeks called, “Guess The Drawing” that uses Continuous Match Mode to allow the user to continuously guess what the drawing is until they get it right. The game is also built with Interactive Canvas for a more visual and immersive experience on smart displays.
In addition to making it easy for you to build new experiences for Google Assistant, we also want to bring the depth of great web content together with the simple and robust AMP framework to deliver new experiences on Smart Displays. AMP allows you to create compelling, smooth websites that have a great user experience. AMP compliant articles are coming to smart displays later this summer with News. Stay tuned for more updates in the coming months as we expand to enable more web content categories for Smart Displays.
With these tools, we want to empower developers to build helpful experiences of the future with Google Assistant, enabling people to get what they need more simply, while giving them time back to focus on what matters most.
Have you built an Action for the Google Assistant and wondered how many people are using it? Or how many of your users are returning users? In this blog post, we will dive into 5 improvements that the Actions on Google Console team has made to give you more insight into how your Action is being used.
We've updated three areas of the Actions Console for readability: Active Users Chart, Date Range Selection, and Filter Options. With these new updates, you can now better customize the data to analyze the usage of your Actions.
The labels at the top of the Active Users chart now read Daily, Weekly and Monthly, instead of the previous 1 Day, 7 Days and 28 Days labels. We also improved the readability of the individual date labels at the bottom of the chart to be more clear. You’ll also notice a quick insight at the bottom of the chart that shows the unique number of users during this time period.
Previously, the date range selectors applied globally to all the charts. These selectors are now local to each chart, allowing you more control over how you view your data.
The date selector provides the following ranges:
Previously when you added a filter, it was applied to all the charts on the page. Now, the filters apply only to the chart you're viewing. We’ve also enhanced the filtering options available for the ‘Surface’ filter, such as mobile devices, smart speakers, and smart display.
Before:
After:
The filter feature also lets you show data breakdowns over different dimensions. By default, the chart shows a single consolidated line, a result of all the filters applied. You can now select the ‘Show breakdown by’ option to see how the components of that data contribute to the totals based on the dimension you selected.
A brand new addition to analytics is the introduction of a retention metrics chart to help you understand how well your action is retaining users. This chart shows you how many users you had in a week and how many returned each week for up to 5 weeks. The higher the percentage week after week, the better your retention.
When you hover over each cell in the chart, you can see the exact number of users who have returned for that week from the previous week.
To learn more about what each metric means, you can check out our documentation.
Try out these new improvements to see how your Actions are performing with your users. You can also check out our documentation to learn more. Let us know if you have any feedback or suggestions in terms of metrics that you need to improve your Action. Thanks for reading! To share your thoughts or questions, join us on Reddit at r/GoogleAssistantDev.
Follow @ActionsOnGoogle on Twitter for more of our team's updates, and tweet using #AoGDevs to share what you’re working on. Can’t wait to see what you build!
Posted by Tomer Amarilio, Product Manager, Google Assistant
Headphones were one of the first devices optimized for the Google Assistant. With just your voice, you can ask the Assistant to make calls to friends or skip to the next song when you’re commuting on the subway to work or biking around on the weekend without having to always glance at your phone.
But as wireless Bluetooth devices like headphones and earbuds become more popular, we need to make it easier to have the same great Assistant experience across many headsets. We collaborated with Qualcomm to design a comprehensive, customizable development kit to provide all device makers with the building blocks needed to create a smart headset with the Google Assistant. The new Qualcomm Smart Headset Development Kit for the Google Assistant is powered by Qualcomm’s QCC5100-series Bluetooth audio chip and supports Google Fast Pair to make pairing Bluetooth accessories a hassle-free process.
To inspire device makers, we also built a Qualcomm Smart Headset Reference Design which delivers high quality audio, noise cancellation capabilities, and supports extended battery life and playback time. The reference design includes a push button to activate the Assistant and is just an example of what manufacturers can engineer.
Posted by Chris Turkstra, Director, Actions on Google
People are using the Assistant every day to get things done more easily, creating lots of opportunities for developers on this quickly growing platform. And we’ve heard from many of you that want easier ways to connect your content across the Assistant.
At I/O, we’re announcing new solutions for Actions on Google that were built specifically with you in mind. Whether you build for web, mobile, or smart home, these new tools will help make your content and services available to people who want to use their voice to get things done.
Help people with their “how to” questions
Every day, people turn to the internet to ask “how to” questions, like how to tie a tie, how to fix a faucet, or how to install a dog door. At I/O, we’re introducing support for How-to markup that lets you power richer and more helpful results in Search and the Assistant.
Adding How-to markup to your pages will enable the page to appear as a rich result on mobile Search and on Google Assistant Smart Displays. This is an incredibly lightweight way for web developers and creators to connect with millions of people, giving them helpful step-by-step instructions with video, images and text. You can start seeing How-to markup results on Search today, and your content will become available on the Smart Displays in the coming months.
Here’s an example where DIY Network added markup to their existing content on the web to provide a more helpful, interactive result on both Google Search and the Assistant:
For content creators that don’t maintain a website, we created a How-to Video Template where video creators can upload a simple spreadsheet with titles, text and timestamps for their YouTube video, and we’ll handle the rest. This is a simple way to transform your existing how-to videos into interactive, step-by-step tutorials across Google Assistant Smart Displays and Android phones.
Check out how REI is getting extra mileage out of their YouTube video:
How-to Video Templates are in developer preview so you can start building today, and your content will become available on Android phones and Smart Displays in the coming months.
Help people quickly get things done with App Actions
If you’re an app developer, people are turning to your apps every day to get things done. And we see people turn to the Assistant every day for a natural way to ask for help via voice. This offers an opportunity to use intents to create voice-based entry points from the Assistant to the right spot in your app.
Last year, we previewed App Actions, a simple mechanism for Android developers that uses intents from the Assistant to deep link to exactly the right spot in your app. At I/O, we are announcing the release of built-in intents for four new App Action categories: Health & Fitness, Finance and Banking, Ridesharing, and Food Ordering. Using these intents, you can integrate with the Assistant in no time.
If I wanted to track my run with Nike Run Club, I could just say “Hey Google, start my run in Nike Run Club” and the app will automatically start tracking my run. Or, let’s say I just finished dinner with my friend Chad and we're splitting the check. I can say "Hey Google, send $15 to Chad on PayPal" and the Assistant takes me right into Paypal, I log in, and all of my information is filled in – all I need to do is hit send.
Each of these integrations were completed in less than a day with the addition of an Actions.xml file that handles the mapping of intents between your app and the Actions platform. You can start building with these new intents today and deploy to Assistant users on Android in the coming months. This is a huge opportunity to offer your fans an effortless way to engage more frequently with your apps.
Take advantage of Smart Displays’ interactive screens
Last year, we saw the introduction of the Smart Display as a new device category. The interactive visual surface opens up many new possibilities for developers.
Today, we’re introducing a developer preview of Interactive Canvas which lets you create full-screen experiences that combine the power of voice, visuals and touch. Canvas works across Smart Displays and Android phones, and it uses open web technologies you’re likely already familiar with, like HTML, CSS and Javascript.
Here’s an example of what you can build when you can leverage the full screen of a Smart Display:
Interactive Canvas is available for building games starting today, and we’ll be adding more categories soon. Visit the Actions Console to be one of the first to try it out.
Enable smart home devices to communicate locally
There are now more than 30,000 connected devices that work with the Assistant across 3,500 brands, and today, we’re excited to announce a new suite of local technologies that are specifically designed to create an even better smart home.
Introducing a preview of the Local Home SDK which enables you to run your smart home code locally on Google Home Speakers and Nest Displays and use its radios to communicate locally with your smart devices. This reduces cloud hops and brings a new level of speed and reliability to the smart home. We’ve been working with some amazing partners including Philips, Wemo, TP-Link, and LIFX on testing this SDK and we’re excited to open it up for all developers next month.
Make setup more seamless
And, through the Local Home SDK, we’re improving the device setup experience by providing users with a seamless setup experience, something we launched in partnership with GE smart lights this past October. So far, people have loved the ability to set up their lights in less than a minute in the Google Home app. We’re now scaling this to more partners, so go here if you’re interested.
Make your devices smart with Assistant Connect
Also, at CES earlier this year we previewed Google Assistant Connect which leverages the Local Home SDK. Assistant Connect enables smart home and appliance developers to easily add Assistant functionality into their devices at low cost. It does this by offloading a lot of work onto the Assistant to complete Actions, display content and respond to commands. We've been hard at work developing the platform along with the first products built on it by Anker, Leviton and Tile. We can't wait to show you more about Assistant Connect later this year.
New device types and traits
For those of you creating Actions for the smart home, we’re also releasing 16 new device types and three new device traits including LockUnlock, ArmDisarm, and Timer. Head over to our developer documentation for the full list of 38 device types and 18 device traits, and check out our sample project on GitHub to start building.
Whether you’re looking to extend the reach of your content, drive more usage in your apps, or build custom Assistant-powered experiences, you now have more tools to do so.
If you want to learn more about how you can start building with these tools, check out our website to get started and our schedule so you can tune in to all of our developer talks that we’ll be hosting throughout the week.
We can’t wait to build together with you!
Posted by Mary Chen, Strategy Lead, Actions on Google
This year at Google I/O, the Actions on Google team is sharing new ways developers of all types can use the Assistant to help users get things done. Whether you’re making Android apps, websites, web content, Actions, or IoT devices, you’ll see how the Assistant can help you engage with users in natural and conversational ways.
Tune in to our announcements during the developer keynote, and then dive deeper with our technical talks. We listed the talks out below by area of interest. Make sure to bookmark them and reserve your seat if you’re attending live, or check back for livestream details if you’re joining us online.
In addition to these sessions, stay tuned for interactive demos and codelabs that you can try at I/O and at home. Follow @ActionsOnGoogle for updates and highlights before, during, and after the festivities.
See you soon!
Posted by Brad Abrams, Group Product Manager, Actions on Google
Before we look forward and discuss updates to Actions on Google for 2019, we wanted to recognize our global developer community for your tremendous work in 2018. We saw more than 4 times the number of projects created with Actions on Google this past year. And some of the most popular Action categories include Games and Trivia, Home Control, Music, Actions for Families, and Education – well done!
We hope to carry this enthusiasm forward, and at Mobile World Congress, we're announcing new tools so you can reach and engage with more people around the globe.
The Google Assistant's now available in more than 80 countries in nearly 30 languages, and you've been busy making your Actions accessible in many of those locales.
One of the most exciting things we've seen in the last couple of years is happening in places where the next billion users are coming online for the first time. In these fast-growing countries like India, Indonesia, Brazil, and Mexico, voice is often the primary way users interact with their devices because it's natural, universal, and the most accessible input method for people who are starting to engage with technology for the first time in their lives.
As more countries are coming online, we want to make it so you can reach and engage with these users as they're adopting the Google Assistant into their everyday lives with astonishing ease. There are tens of millions of users on Android Go and KaiOS in over 100 countries.
We'll be making your Actions available to Android Go and KaiOS devices in the next few months, so you should start thinking now about how to build for these platforms and users. Without any additional work required, your Actions will work on both operating systems at launch (unless of course, Action requires a screen with touch input). We'll also be launching a simulator so you can test your Actions to see how they look on entry-level Android Go smartphones and KaiOS feature phones.
A couple of partners have already built Actions with these new audiences in mind. Hello English, for example, created an Action to offer English lessons for users that speak Hindi, to create more opportunities for people through language learning. And Where is My Train? (WIMT) was built for the millions of Indians commuting daily, offering real-time locations and times for trains accessible by voice. Check out our developer docs for KaiOS and Android Go Edition, and start building for the next billion users.
And we're not just focused on a handful of emerging countries. We're always working to enable all of Actions on Google's tools so users can enjoy the best experience possible regardless of the country they live in or the language they speak—our work here never ends! Here's a snapshot of some of the progress we've made this past year:
We've already talked about how busy the development community was this past year, and we've been hard at work to keep up! If you're looking to reach and engage with millions—even billions more users—now's a good time to start thinking about how your Action can make a difference in people's lives around the globe.
Posted by Ilya Gelfenbeyn, Head of the Google Assistant Investments program
Last year, we announced the Google Assistant Investments program with the goal to help pioneering startups bring their ideas to life in the digital assistant ecosystem. Not only have we invested in some really great startups, we've also been working closely with these companies to make their services available to more users.
We're excited to be back to announce five new portfolio companies and catch up on the progress some of them have made this past year. With the next batch of investments, we're helping companies explore how digital assistants can improve the hospitality, insurance, fashion and education industries, and we have something for sports fans too.
First up, AskPorter. This London-based team was founded to make managing spaces simple, providing every property manager and occupant with a digital personal assistant. AskPorter is an AI-powered property management platform with a digital assistant called Porter. Porter assists and takes care of all aspects of property management such as guiding inspections arranging viewings, troubleshooting maintenance issues and chasing payments.
GradeSlam is an on-demand, chat-based, personalized learning and tutoring service available across all subject areas. Sessions are conducted via chat, creating a learning environment that allows students to interact freely and personally with qualified educators. The Montreal-based team is already used by more than 150,000 students, teachers and administrators.
Aiva Health puts smart speakers in hospitals and senior communities to reduce response times and improve satisfaction for patients, seniors, and caregivers alike. Aiva understands patient requests and routes them to the most appropriate caregiver so they can respond instantly via their mobile app. The Aiva platform provides centralized IoT management, powering Smart Hospitals and Smart Communities.
StyleHacks (formerly Maison Me) was founded with a goal of empowering people to take back control of their style and wardrobe. With a conversational interface and personalized AI-powered recommendations, they're helping people live their most stylish lives. The team has already launched the "StyleHacks" Action for phones and Smart Displays in December 2018, helping people decide what to wear by providing personalized recommendations based on the weather and preferences. And in the next few months, StyleHacks will also be able to help you shop for clothes you will actually wear. Just ask StyleHacks what to wear today
StatMuse turns the biggest sports stars into your own personal sports commentator. Powered by the personalities of more than 25 sports superstars including Peyton Manning, Jerry Rice and Scott Van Pelt, fans can get scores, stats and recaps for the NBA, NFL, NHL and MLB dating back to 1876. To try it out, just say, "Hey Google, talk to StatMuse."
It's been almost a year since we launched the Investments program and we're happy to see how some of these companies are already using voice to broaden the Google Assistant's capabilities. If you're working on new ways for people to use their voice to get things done, or building new hardware devices for digital assistants, we'd like to hear from you.
