Google Developers Blog

MediaPipe 3D Face Transform

2020-09-25T09:14:00.001-07:00

Posted by Kanstantsin Sokal, Software Engineer, MediaPipe team

Earlier this year, the MediaPipe Team released the Face Mesh solution, which estimates the approximate 3D face shape via 468 landmarks in real-time on mobile devices. In this blog, we introduce a new face transform estimation module that establishes a researcher- and developer-friendly semantic API useful for determining the 3D face pose and attaching virtual objects (like glasses, hats or masks) to a face.

The new module establishes a metric 3D space and uses the landmark screen positions to estimate common 3D face primitives, including a face pose transformation matrix and a triangular face mesh. Under the hood, a lightweight statistical analysis method called Procrustes Analysis is employed to drive a robust, performant and portable logic. The analysis runs on CPU and has a minimal speed/memory footprint on top of the original Face Mesh solution.

Figure 1: An example of virtual mask and glasses effects, based on the MediaPipe Face Mesh solution.

Introduction

The MediaPipe Face Landmark Model performs a single-camera face landmark detection in the screen coordinate space: the X- and Y- coordinates are normalized screen coordinates, while the Z coordinate is relative and is scaled as the X coordinate under the weak perspective projection camera model. While this format is well-suited for some applications, it does not directly enable crucial features like aligning a virtual 3D object with a detected face.

The newly introduced module moves away from the screen coordinate space towards a metric 3D space and provides the necessary primitives to handle a detected face as a regular 3D object. By design, you'll be able to use a perspective camera to project the final 3D scene back into the screen coordinate space with a guarantee that the face landmark positions are not changed.

Metric 3D Space

The Metric 3D space established within the new module is a right-handed orthonormal metric 3D coordinate space. Within the space, there is a virtual perspective camera located at the space origin and pointed in the negative direction of the Z-axis. It is assumed that the input camera frames are observed by exactly this virtual camera and therefore its parameters are later used to convert the screen landmark coordinates back into the Metric 3D space. The virtual camera parameters can be set freely, however for better results it is advised to set them as close to the real physical camera parameters as possible.

Figure 2: A visualization of multiple key elements in the metric 3D space. Created in Cinema 4D

Canonical Face Model

The Canonical Face Model is a static 3D model of a human face, which follows the 3D face landmark topology of the MediaPipe Face Landmark Model. The model bears two important functions:

Defines metric units: the scale of the canonical face model defines the metric units of the Metric 3D space. A metric unit used by the default canonical face model is a centimeter;
Bridges static and runtime spaces: the face pose transformation matrix is - in fact - a linear map from the canonical face model into the runtime face landmark set estimated on each frame. This way, virtual 3D assets modeled around the canonical face model can be aligned with a tracked face by applying the face pose transformation matrix to them.

Face Transform Estimation

The face transform estimation pipeline is a key component, responsible for estimating face transform data within the Metric 3D space. On each frame, the following steps are executed in the given order:

Face landmark screen coordinates are converted into the Metric 3D space coordinates;
Face pose transformation matrix is estimated as a rigid linear mapping from the canonical face metric landmark set into the runtime face metric landmark set in a way that minimizes a difference between the two;
A face mesh is created using the runtime face metric landmarks as the vertex positions (XYZ), while both the vertex texture coordinates (UV) and the triangular topology are inherited from the canonical face model.

Effect Renderer

The Effect Renderer is a component, which serves as a working example of a face effect renderer. It targets the OpenGL ES 2.0 API to enable a real-time performance on mobile devices and supports the following rendering modes:

3D object rendering mode: a virtual object is aligned with a detected face to emulate an object attached to the face (example: glasses);
Face mesh rendering mode: a texture is stretched on top of the face mesh surface to emulate a face painting technique.

In both rendering modes, the face mesh is first rendered as an occluder straight into the depth buffer. This step helps to create a more believable effect via hiding invisible elements behind the face surface.

Figure 3: An example of face effects rendered by the Face Effect Renderer.

Using Face Transform Module

The face transform estimation module is available as a part of the MediaPipe Face Mesh solution. It comes with face effect application examples, available as graphs and mobile apps on Android or iOS. If you wish to go beyond examples, the module contains generic calculators and subgraphs - those can be flexibly applied to solve specific use cases in any MediaPipe graph. For more information, please visit our documentation.

Follow MediaPipe

We look forward to publishing more blog posts related to new MediaPipe pipeline examples and features. Please follow the MediaPipe label on Google Developers Blog and Google Developers twitter account (@googledevs).

Acknowledgements

We would like to thank Chuo-Ling Chang, Ming Guang Yong, Jiuqiang Tang, Gregory Karpiak, Siarhei Kazakou, Matsvei Zhdanovich and Matthias Grundman for contributing to this blog post.

Announcing DevFest 2020

2020-09-23T10:06:00.000-07:00

Posted by Jennifer Kohl, Program Manager, Developer Community Programs

On October 16-18, thousands of developers from all over the world are coming together for DevFest 2020, the largest virtual weekend of community-led learning on Google technologies.

As people around the world continue to adapt to spending more time at home, developers yearn for community now more than ever. In years past, DevFest was a series of in-person events over a season. For 2020, the community is coming together in a whole new way – virtually – over one weekend to keep developers connected when they may want it the most.

The speakers

The magic of DevFest comes from the people who organize and speak at the events - developers with various backgrounds and skill levels, all with their own unique perspectives. In different parts of the world, you can find a DevFest session in many local languages. DevFest speakers are made up of various types of technologists, including kid developers , self-taught programmers from rural areas , and CEOs and CTOs of startups. DevFest also features a wide range of speakers from Google, Women Techmakers, Google Developer Experts, and more. Together, these friendly faces, with many different perspectives, create a unique and rich developer conference.

The sessions and their mission

Hosted by Google Developer Groups, this year’s sessions include technical talks and workshops from the community, and a keynote from Google Developers. Through these events, developers will learn how Google technologies help them develop, learn, and build together.

Sessions will cover multiple technologies, such as Android, Google Cloud Platform, Machine Learning with TensorFlow, Web.dev, Firebase, Google Assistant, and Flutter.

At our core, Google Developers believes community-led developer events like these are an integral part of the advancement of technology in the world.

For this reason, Google Developers supports the community-led efforts of Google Developer Groups and their annual tentpole event, DevFest. Google provides esteemed speakers from the company and custom technical content produced by developers at Google. The impact of DevFest is really driven by the grassroots, passionate GDG community organizers who volunteer their time. Google Developers is proud to support them.

The attendees

During DevFest 2019, 138,000+ developers participated across 500+ DevFests in 100 countries. While 2020 is a very different year for events around the world, GDG chapters are galvanizing their communities to come together virtually for this global moment. The excitement for DevFest continues as more people seek new opportunities to meet and collaborate with like-minded, community-oriented developers in our local towns and regions.

Join the conversation on social media with #DevFest.

Sign up for DevFest at goo.gle/devfest.

Still curious? Check out these popular talks from DevFest 2019 events around the world...

Scalable Vector Graphics (SVG)

How to add Flutter to your app

The Flutter journey: production app

Getting started with microcontrollers

Chet Haase and Romaine Guy: quality

Building the ultimate Beatles app

Google Nest Device Access Console now available for partners and individuals

2020-09-22T06:53:00.004-07:00

Posted by Gabriel Rubinsky, Senior Product Manager

Today, we’re excited to announce the Device Access Console is available.

The Device Access program lets individuals and qualified partners securely access and control Nest products with their apps and solutions.

At the heart of the Device Access program is the Smart Device Management API. Since we announced the program, Alarm.com, Control4, DISH, OhmConnect, NRG Energy, and Vivint Smart Home have successfully completed the Early Access Program (EAP) with Nest thermostat, camera, or doorbell traits. In the coming months, we expect additional devices to be supported and more smart home partners to launch their new integrations as well.

Enhanced privacy and security

The Device Access program is built on a foundation of privacy and security. The program requires partner submission of qualified use cases and completion of a security assessment before being allowed to utilize the Smart Device Management API for commercial use. The program process gives our users the confidence that commercial partners offering integrated Nest solutions have data protections and safeguards in place that meet our privacy and security standards.

Nest device access and control

The Device Access program currently allows qualified partners to integrate directly with Nest devices, enable control of thermostats, access and view camera feeds, and receive doorbell notifications with images. All qualified partner solutions and services will require end-user consent before being able to access, control, and manage Nest devices as part of their service offerings, either through a partner client app or service platform. Ultimately, this gives users more choice in how to control their home and their own generated data.

If you’re a developer or a Nest user interested in the Device Access program or access to the sandbox development environment,* you can find more information on our Device Access site.

Device Access for Commercial Developers
The Device Access program allows trusted partners to offer access, management, and control of Nest devices within the partner’s app, solution, and ecosystem. It allows developers to test all API traits in the sandbox environment, before moving forward with commercial integration. Learn more
Device Access for Individuals
For individual smart home developer enthusiasts, you can register to access the sandbox development environment, allowing you to directly control your own Nest devices through your private integrations and automations. Learn more

We’re doing the work to make Nest devices more secure and protect user privacy long into the future. This means expanding privacy and data security programs, and delivering flexibility for our customers to use thousands of products from partners to create a connected, helpful home.

* Registration consists of the acceptance of the Google API and Nest Device Access Sandbox Terms of Service, along with a one-time, non-refundable nominal fee per account

Google Pay picks Flutter to drive its global product development

2020-09-18T18:00:00.000-07:00

Posted by David Ko, Engineering Director; Jeff Lim, Software Engineer; Pankaj Gupta, Director of Engineering; Will Horn, Software Engineer

Three years ago, when we launched Google Pay India (then called Tez), our vision was to create a simple and secure payment app for everyone in India. We started with the premise of making payments simple and built a user interface that made making payments as easy as starting a conversation. The simplicity of the design resonated with users instantly and over time, we have added functionality to help users do more than just make payments. Today users can pay their bills, recharge their phones, get loans instantly through banks, buy train tickets and much more all within the app. Last year, we also launched the Spot Platform in India, which allows merchants to create branded experiences within the Google Pay app so they can connect with their customers in a more engaging way.

As we looked at scaling our learnings from India to other parts of the world, we wanted to focus on a fast and efficient development environment, which was modern and engaging with the flexibility needed to keep the UI clean. And more importantly one that enabled us to write once and be able to deploy to both iOS and Android reaching the wide variety of users.

It was clear that we would need to build it, and ensure that it worked across a wide variety of payment rails, infrastructure, and operating systems. But with the momentum we had for Google Pay in India, and the fast evolving product features - we had limited engineering resources to put behind this effort.

After evaluating various options, it was easy to pick Flutter as the obvious choice. The three things that made it click for us were:

We could write once in Dart and deploy on both iOS and Android, which led to a uniform best-in-class experience on both Android and iOS;
Just-in-Time compiler with hot reload during development enabled rapid iteration on UI which tremendously increased developer efficiency; and
Ahead-of-time compilation ensured high performance deployment.

Now the task was to get it done. We started with a small team of three software engineers from both Android and iOS. Those days were focused and intense. To start with we created a vertical slice of the app — home page, chat, and payments (with the critical native plugins for payments in India). The team first tried a hybrid approach, and then decided to do a clean rewrite as it was not scalable.

We ran a few small sprints for other engineers on the team to give them an opportunity to rewrite something in Flutter and provide feedback. Everyone loved Flutter — you could see the thrill on people’s faces as they talked about how fast it was to build a user interface. One of the most exciting things was that the team could get instant feedback while developing. We could also leverage the high quality widgets that Flutter provided to make development easier.

After carefully weighing the risks and our case for migration, we decided to go all in with Flutter. It was a monumental rewrite of a moving target, and the existing app continues to evolve while we were rewriting features. After many months of hard work, Google Pay Flutter implementation is now available in open beta in India and Singapore. Our users in India and Singapore can visit the Google Play Store page for Google Pay to opt into the beta program and experience the latest app built on Flutter. Next, we are looking forward to launching Google Pay on Flutter to everyone across the world on iOS and Android.

We hope this gives you a fair idea of how to approach and launch a complete rewrite of an active app that is used by millions of users and businesses of all sizes. It would not have been possible for us to deliver this without Flutter’s continued advances on the platform. Huge thanks to the Flutter team, as today, we are standing on their shoulders!

When fully migrated, Google Pay will be one of the largest production deployments on the Flutter platform. We look forward to sharing more learnings from our transition to Flutter in the future.

Doubling down on the edge with Coral's new accelerator

2020-09-16T09:28:00.000-07:00

Posted by The Coral Team

Moving into the fall, the Coral platform continues to grow with the release of the M.2 Accelerator with Dual Edge TPU. Its first application is in Google’s Series One room kits where it helps to remove interruptions and makes the audio clearer for better video meetings. To help even more folks build products with Coral intelligence, we’re dropping the prices on several of our products. And for those folks that are looking to level up their at home video production, we’re sharing a demo of a pose based AI director to make multi-camera video easier to make.

Coral M.2 Accelerator with Dual Edge TPU

The newest addition to our product family brings two Edge TPU co-processors to systems in an M.2 E-key form factor. While the design requires a dual bus PCIe M.2 slot, it brings enhanced ML performance (8 TOPS) to tasks such as running two models in parallel or pipelining one large model across both Edge TPUs.

The ability to scale across multiple edge accelerators isn’t limited to only two Edge TPUs. As edge computing expands to local data centers, cell towers, and gateways, multi-Edge TPU configurations will be required to help process increasingly sophisticated ML models. Coral allows the use of a single toolchain to create models for one or more Edge TPUs that can address many different future configurations.

A great example of how the Coral M.2 Accelerator with Dual Edge TPU is being used is in the Series One meeting room kits for Google Meet.

The new Series One room kits for Google Meet run smarter with Coral intelligence

Google’s new Series One room kits use our Coral M.2 Accelerator with Dual Edge TPU to bring enhanced audio clarity to video meetings. TrueVoice®, a multi-channel noise cancellation technology, minimizes distractions to ensure every voice is heard with up to 44 channels of echo and noise cancellation, making distracting sounds like snacking or typing on a keyboard a concern of the past.

Enabling the clearest possible communication in challenging environments was the target for the Google Meet hardware team. The consideration of what makes a challenging environment was not limited to unusually noisy environments, such as lunchrooms doubling as conference rooms. Any conference room can present challenging acoustics that make it difficult for all participants to be heard.

The secret to clarity without expensive and cumbersome equipment is to use virtual audio channels and AI driven sound isolation. Read more about how Coral was used to enhance and future-proof the innovative design.

Expanding the AI edge

Earlier this year, we reduced the prices of our prototyping devices and sensors. We are excited to share further price drops on more of our products. Our System-on-Module is now available for $99.99, and our Mini PCIe Accelerator, M.2 Accelerator A+E Key, and M.2 Accelerator B+M key are now available at $24.99. We hope this lower price will make our edge AI more accessible to more creative minds around the world. Later, this month our SoM offering will also expand to include 2 and 4GB RAM options.

Multi-cam with AI

As we expand our platform and product family, we continue to keep new edge AI use cases in mind. We are continually inspired by our developer community’s experimentation and implementations. When recently faced with the challenges of multicam video production from home, Markku Lepistö, Solutions Architect at Google Cloud, created this real-time pose-based multicam tool he so aptly dubbed, AI Director.

We love seeing such unique implementations of on-device ML and invite you to share your own projects and feedback at coral-support@google.com.

For a list of worldwide distributors, system integrators and partners, visit the Coral partnerships page. Please visit Coral.ai to discover more about our edge ML platform.

Applications are open for Google for Startups Accelerator in Japan

2020-09-15T09:00:00.001-07:00

Posted by Takuo Suzuki, Developer Relations Program Manager

The Google for Startups Accelerator helps founders across the globe solve for important economic and societal challenges, while helping them grow and scale their business. Due to the continued success of the program around the world, we are pleased to open up applications for our third Accelerator class in Japan, commencing January 2021. Applications will remain open until October 30, 2020.

This accelerator is designed for established startups across Japan using technology to help solve important social and environmental issues, and that contribute to the Japanese economy. This includes (but is not limited to) startups tackling:

Ageing society and declining workforce
Energy, environment, and sustainability
Rural revitalization
Medicine, health, and well-being
Education
Diversity, inclusion, and social equity

Google for Startups Accelerators provide support to later-stage companies that have already launched their product, and have strong market-fit and potential to scale rapidly in the future. Startups in the program benefit from tailored Google mentorship, product advice & credits, technical workshops, and by getting connected to other founders, VCs, and industry experts.

Each participating startup selected for the Google for Startups Accelerator program will join a 500+ company alumni network of startups from around the world, such as Selan, with their product Omister (Class #2 Japan), is improving education & childcare in Japan by providing bilingual instructors for children, and mDoc, (Class #1 Sustainability, Europe), a Nigerian startup helping people in West Africa with chronic diseases get treatment via their app.

In summary:

Suitable for startups solving for societal or environmental issues in Japan
Application open: September 15, 2020
Application close: October 30, 2020
Announcement of selected startups: December 2020
Program runs from late-January 2021 to end of April 2021 (planned)
Please refer to the website for further information and to apply.

Building solutions using the G Suite developer platform

2020-09-14T10:48:00.000-07:00

Posted by Charles Maxson, Developer Advocate, G Suite

Millions of users know G Suite as a collection of communication and productivity apps that enables teams to easily create, communicate, collaborate, and discover content to supercharge teamwork. Beneath the surface of this well-serving collection of apps is also an extensible platform that enables developers to build targeted custom experiences and integrations utilizing these apps, allowing G Suite’s vast user base to get even more value out of the platform.

At first glance, it may not be natural to think of the tools you use for day-to-day productivity and collaboration as a developer platform. But consider what makes up a developer platform; Languages, APIs, runtimes, frameworks, IDEs, ecosystem, etc; G Suite offers developers all of these things and more.

Let’s take a closer look at what makes up the G Suite developer platform and how you can use it.

G Suite as a Developer Platform

There are a lot of components that make up G Suite as a platform. As a developer, there is probably none more important than the data that your solution collects, processes and presents. As a platform, G Suite is both highly interoperable, secure, and also interestingly unique.

Being interoperable, G Suite lets you interact with your data--whether your data is in G Suite or elsewhere, no matter how you store it or how you want to analyze it. G Suite allows you to keep your data where it best suits your application, while offering you flexibility to access it easily. Some examples include rich integrations with sources like BigQuery or JDBC databases. Better yet, often little to no code is required to get you connected.

Where G Suite as a platform is unique regarding data is it can be used to store, or perhaps even more interesting, be used to produce data. For native storage, you may use Drive as a content repository, or store information in a Sheets spreadsheet, or collect it via Google Forms as a front end. Additionally, there are many scenarios where the content your users are engaging in (emails, chats, events, tasks, contacts, documents, identity, etc.) can be harnessed to create unique interactions with G Suite. Solutions that build off, or integrate with G Suite provide such unique business value, but regardless where your data resides, accessing it as a developer is a non-issue via the platform.

The core of the G Suite developer platform itself is composed of frameworks for developer features including G Suite Add-ons and Chatbots, as well as a comprehensive library of REST APIs. These allow you to interface with the full G Suite platform to create integrations, build extensions, add customizations, and access content or data.

G Suite Add-ons and Chatbots are frameworks specifically designed for G Suite that allow you to quickly and safely build experiences that enrich the way users interact within G Suite apps, while while the REST APIs give you essentially unlimited access to G Suite apps and data including Gmail, Classroom, Calendar, Drive, Docs, Sheets, Slides, Task, and more. What you build, and what you build with, including languages and dev environments is up to you!

The beauty of G Suite as a platform is how you can unlock complementary technologies like Google Cloud that expand the platform to be even more powerful. Think about a G Suite UI connecting to a Google Cloud Platform backend; the familiar interface of G Suite coupled with the phenomenal power and scale of GCP!

Building with GCP from G Suite, you have access to components like the AI platform. This enables scenarios like using Google Sheets as a front end to AI tools like the Vision, Natural Language and the Translation APIs. Imagine how you can change the way users interact with G Suite, your app and your data combined with the power of ML?

Another useful concept is how you can add natural conversational experiences to your app in G Suite with tools like DialogFlow. This way instead of writing complicated interfaces users have to learn, you could build a G Suite Chat bot that invokes Dialogflow to allow users to execute commands directly from within their team conversations in Chat. So for example, users could just ask a Chat bot to “Add a task to the project list” or “Assign this issue to Matt”. A recent example of this is DataQnA, a natural language interface for analyzing BigQuery data.

BigQuery is another GCP tool that works natively with G Suite to allow you to analyze and leverage larger, complicated data sets while producing unique custom reports that can be surfaced in a user friendly way. One of the ways to leverage BigQuery with G Suite is through Connected Sheets, which provides the power and scale of a BigQuery data warehouse in the familiar context of Sheets. With Connected Sheets, you can analyze billions of rows of live BigQuery data in Google Sheets without requiring SQL knowledge. You can apply familiar tools—like pivot tables, charts, and formulas—to easily derive insights from big data.

One relatively new addition to the Google Cloud family also worth mentioning here is AppSheet. AppSheet is a no-code tool that can be used to quickly build mobile and web apps. Being no-code, it may seem out of place in a discussion for a development platform, but AppSheet is a dynamic and agile tool that makes it great for building apps fast or envisioning prototypes, while also connecting to G Suite apps like Google Sheets, allowing you to access G Suite platform data with ease.

When you do need the power of writing custom code, one of the foundational components of the G Suite developer platform is Apps Script. For over a decade, Apps Script has been the server-less, JavaScript-based runtime that natively powers G Suite extensibility. Built directly into G Suite with its own IDE, Apps Script makes it super fast and easy to get started building solutions with nothing to install or configure, just open and start coding -- or you can even let the macro recorder write code for you! Apps Script masks a lot of complexities that developers face like handling user authentication, allowing you to focus on creating solutions quickly. Its native integration and relative simplicity also welcomes developers with diverse skill levels to build customized workflows, menus and UI, automations and more right inside G Suite.

While Apps Script is nimble and useful for many use cases, we know that many developers have preferences around tools, languages and development environments. G Suite is an open platform that encourages developers to choose options that makes them more productive. In continuing to build on that principle, we recently introduced Alternate Runtimes for G Suite Add-ons. This new capability allows you to create solutions using the G Suite Add-ons framework without being bound to Apps Script as a toolset, giving you the choice and freedom to leverage your existing preferences and investments in hosting infrastructure, development tools, source control, languages, and code libraries, etc.

Finally, what completes the vision of G Suite as a developer platform is that you have the confidence and convenience of an established platform that is broadly deployed and backed by tools like Google Identity Management and the G Suite Admin Console for administration and security. This enables you to build your solutions--whether its a customized solution for your internal users or an integration between your software platform and G Suite--and distribute them at a domain level or even globally via the G Suite Marketplace, which is an acquisition channel for developers and a discovery engine for end-users and enterprise admins alike.

Now that you can see how G Suite is a developer platform, imagine what you can build?

Visit the G Suite Developer homepage and get started on your journey today.

Instant Motion Tracking with MediaPipe

2020-08-31T09:53:00.000-07:00

Posted by Vikram Sharma, Software Engineering Intern; Jianing Wei, Staff Software Engineer; Tyler Mullen, Senior Software Engineer

Augmented Reality (AR) technology creates fun, engaging, and immersive user experiences. The ability to perform AR tracking across devices and platforms, without initialization, remains important for powering AR applications at scale.

Today, we are excited to release the Instant Motion Tracking solution in MediaPipe. It is built upon the MediaPipe Box Tracking solution we released previously. With Instant Motion Tracking, you can easily place fun virtual 2D and 3D content on static or moving surfaces, allowing them to seamlessly interact with the real world. This technology also powered MotionStills AR. Along with the library, we are releasing an open source Android application to showcase its capabilities. In this application, a user simply taps the camera viewfinder in order to place virtual 3D objects and GIF animations, augmenting the real-world environment.

Instant Motion Tracking in MediaPipe

Instant Motion Tracking

The Instant Motion Tracking solution provides the capability to seamlessly place virtual content on static or motion surfaces in the real world. To achieve that, we provide the six degrees of freedom tracking with relative scale in the form of rotation and translation matrices. This tracking information is then used in the rendering system to overlay virtual content on camera streams to create immersive AR experiences.

The core concept behind Instant Motion Tracking is to decouple the camera’s translation and rotation estimation, treating them instead as independent optimization problems. This approach enables AR tracking across devices and platforms without initialization or calibration. We do this by first finding the 3D camera translation using only the visual signals from the camera. This involves estimating the target region's apparent 2D translation and relative scale across frames. The process can be illustrated with a simple pinhole camera model, relating translation and scale of an object in the image plane to the final 3D translation.

By finding the change in relative size of our tracked region from view position V1 to V2, we can estimate the relative change in distance from the camera.

Next, we obtain the device’s 3D rotation from its built-in IMU (Inertial Measurement Unit) sensor. By combining this translation and rotation data, we can track a target region with six degrees of freedom at relative scale. This information allows for the placement of virtual content on any system with a camera and IMU functionality, and is calibration free. For more details on Instant Motion Tracking, please refer to our paper.

A MediaPipe Pipeline for Instant Motion Tracking

A diagram of Instant Motion Tracking pipeline is shown below, consisting of four major components: a Sticker Manager module, a Region Tracking module, a Matrices Manager module, and lastly a Rendering System. Each of the components consists of MediaPipe calculators or subgraphs.

Diagram of Instant Motion Tracking Pipeline

The Sticker Manager accepts sticker data from the application and produces initial anchors (tracked region information) based on user taps, and user gesture controls for every sticker object. Initial anchors are then sent to our Region Tracking module to generate tracked anchors. The Matrices Manager combines this data with our device’s rotation matrix to produce six degrees-of-freedom poses as model matrices. After integrating any user-specified transforms like asset scaling, our final poses are forwarded to the Rendering System to render all virtual objects overlaid on the camera frame to produce the output AR frame.

Using the Instant Motion Tracking Solution

The Instant Motion Tracking solution is easy to use by leveraging the MediaPipe cross-platform framework. With camera frames, device rotation matrix, and anchor positions (screen coordinates) as input, the MediaPipe graph produces AR renderings for each frame, providing engaging experiences. If you wish to integrate this Instant Motion Tracking library with your system or application, please visit our documentation to build your own AR experiences on any device with IMU functionality and a camera sensor.

Augmenting The World with 3D Stickers and GIFs

Instant Motion Tracking solution allows bringing both 3D stickers and GIF animations into Augmented Reality experiences. GIFs are rendered on flat 3D billboards placed in the world, introducing fun and immersive experiences with animated content blended into the real environment.Try it for yourself!

Demonstration of GIF placement in 3D

MediaPipe Instant Motion Tracking is already helping PixelShift.AI, a startup applying cutting-edge vision technologies to facilitate video content creation, to track virtual characters seamlessly in the view-finder for a realistic experience. Building upon Instant Motion Tracking’s high-quality pose estimation, PixelShift.AI enables VTubers to create mixed reality experiences with web technologies. The product is going to be released to the broader VTuber community later this year.

Instant Motion Tracking helps PixelShift.AI create mixed reality experiences

Follow MediaPipe

Acknowledgement

We would like to thank Vikram Sharma, Jianing Wei, Tyler Mullen, Chuo-Ling Chang, Ming Guang Yong, Jiuqiang Tang, Siarhei Kazakou, Genzhi Ye, Camillo Lugaresi, Buck Bourdon, and Matthias Grundman for their contributions to this release.

Guidance to developers affected by our effort to block less secure browsers and applications

2020-08-28T09:16:00.000-07:00

Posted by Lillan Marie Agerup, Product Manager

We are always working to improve security protections of Google accounts. Our security systems automatically detect, alert and help protect our users against a range of security threats. One form of phishing, known as “man-in-the-middle”, is hard to detect when an embedded browser framework (e.g., Chromium Embedded Framework - CEF) or another automation platform is being used for authentication. MITM presents an authentication flow on these platforms and intercepts the communications between a user and Google to gather the user’s credentials (including the second factor in some cases) and sign in. To protect our users from these types of attacks Google Account sign-ins from all embedded frameworks will be blocked starting on January 4, 2021. This block affects CEF-based apps and other non-supported browsers.

To minimize the disruption of service to our partners, we are providing this information to help developers set up OAuth 2.0 flows in supported user-agents. The information in this document outlines the following:

How to enable sign-in on your embedded framework-based apps using browser-based OAuth 2.0 flows.
How to test for compatibility.

Apps that use embedded frameworks

If you're an app developer and use CEF or other clients for authorization on devices, use browser-based OAuth 2.0 flows. Alternatively, you can use a compatible full native browser for sign-in.

For limited-input device applications, such as applications that do not have access to a browser or have limited input capabilities, use limited-input device OAuth 2.0 flows.

Browsers

Modern browsers with security updates will continue to be supported.

Browser standards

The browser must have JavaScript enabled. For more details, see our previous blog post.

The browser must not proxy or alter the network communication. Your browser must not do any of the following:

Server-side rendering
HTTPS proxy
Replay requests
Rewrite HTTP headers

The browser must have a reasonably complete implementation of web standards and browser features. You must confirm that your browser does not contain any of the following:

Headless browsers
Node.js
Text-based browsers

The browser must identify itself clearly in the User-Agent. The browser must not try to impersonate another browser like Chrome or Firefox.

The browser must not provide automation features. This includes scripts that automate keystrokes or clicks, especially to perform automatic sign-ins. We do not allow sign-in from browsers based on frameworks like CEF or Embedded Internet Explorer.

Test for compatibility

If you're a developer that currently uses CEF for sign-in, be aware that support for this type of authentication ends on January 4, 2021. To verify whether you'll be affected by the change, test your application for compatibility. To test your application, add a specific HTTP header and value to disable the allowlist. The following steps explain how to disable the allowlist:

Go to where you send requests to accounts.google.com.
Add Google-Accounts-Check-OAuth-Login:true to your HTTP request headers.

The following example details how to disable the allowlist in CEF.

Note: You can add your custom headers in CefRequestHandler#OnBeforeResourceLoad.

    CefRequest::HeaderMap hdrMap;
    request->GetHeaderMap(hdrMap);
    hdrMap.insert(std::make_pair("Google-Accounts-Check-OAuth-Login", "true"));

To test manually in Chrome, use ModHeader to set the header. The header enables the changes for that particular request.

Setting the header using ModHeader

ML Kit Pose Detection Makes Staying Active at Home Easier

2020-08-27T10:00:00.000-07:00

Posted by Kenny Sulaimon, Product Manager, ML Kit; Chengji Yan and Areeba Abid, Software Engineers, ML Kit

Two months ago we introduced the standalone version of the ML Kit SDK, making it even easier to integrate on-device machine learning into mobile apps. Since then we’ve launched the Digital Ink Recognition API, and also introduced the ML Kit early access program. Our first two early access APIs were Pose Detection and Entity Extraction. We’ve received an overwhelming amount of interest in these new APIs and today, we are thrilled to officially add Pose Detection to the ML Kit lineup.

A New ML Kit API, Pose Detection

Examples of ML Kit Pose Detection

ML Kit Pose Detection is an on-device, cross platform (Android and iOS), lightweight solution that tracks a subject's physical actions in real time. With this technology, building a one-of-a-kind experience for your users is easier than ever.

The API produces a full body 33 point skeletal match that includes facial landmarks (ears, eyes, mouth, and nose), along with hands and feet tracking. The API was also trained on a variety of complex athletic poses, such as Yoga positions.

Skeleton image detailing all 33 landmark points

Under The Hood

Diagram of the ML Kit Pose Detection Pipeline

The power of the ML Kit Pose Detection API is in its ease of use. The API builds on the cutting edge BlazePose pipeline and allows developers to build great experiences on Android and iOS, with little effort. We offer a full body model, support for both video and static image use cases, and have added multiple pre and post processing improvements to help developers get started with only a few lines of code.

The ML Kit Pose Detection API utilizes a two step process for detecting poses. First, the API combines an ultra-fast face detector with a prominent person detection algorithm, in order to detect when a person has entered the scene. The API is capable of detecting a single (highest confidence) person in the scene and requires the face of the user to be present in order to ensure optimal results.

Next, the API applies a full body, 33 landmark point skeleton to the detected person. These points are rendered in 2D space and do not account for depth. The API also contains a streaming mode option for further performance and latency optimization. When enabled, instead of running person detection on every frame, the API only runs this detector when the previous frame no longer detects a pose.

The ML Kit Pose Detection API also features two operating modes, “Fast” and “Accurate”. With the “Fast” mode enabled, you can expect a frame rate of around 30+ FPS on a modern Android device, such as a Pixel 4 and 45+ FPS on a modern iOS device, such as an iPhone X. With the “Accurate” mode enabled, you can expect more stable x,y coordinates on both types of devices, but a slower frame rate overall.

Lastly, we’ve also added a per point “InFrameLikelihood” score to help app developers ensure their users are in the right position and filter out extraneous points. This score is calculated during the landmark detection phase and a low likelihood score suggests that a landmark is outside the image frame.

Real World Applications

Examples of a pushup and squat counter using ML Kit Pose Detection

Keeping up with regular physical activity is one of the hardest things to do while at home. We often rely on gym buddies or physical trainers to help us with our workouts, but this has become increasingly difficult. Apps and technology can often help with this, but with existing solutions, many app developers are still struggling to understand and provide feedback on a user’s movement in real time. ML Kit Pose Detection aims to make this problem a whole lot easier.

The most common applications for Pose detection are fitness and yoga trackers. It’s possible to use our API to track pushups, squats and a variety of other physical activities in real time. These complex use cases can be achieved by using the output of the API, either with angle heuristics, tracking the distance between joints, or with your own proprietary classifier model.

To get you jump started with classifying poses, we are sharing additional tips on how to use angle heuristics to classify popular yoga poses. Check it out here.

Learning to Dance Without Leaving Home

Learning a new skill is always tough, but learning to dance without the aid of a real time instructor is even tougher. One of our early access partners, Groovetime, has set out to solve this problem.

With the power of ML Kit Pose Detection, Groovetime allows users to learn their favorite dance moves from popular short-form dance videos, while giving users automated real time feedback on their technique. You can join their early access beta here.

Groovetime App using ML Kit Pose Detection

Staying Active Wherever You Are

Our Pose Detection API is also helping adidas Training, another one of our early access partners, build a virtual workout experience that will help you stay active no matter where you are. This one-of-a-kind innovation will help analyze and give feedback on the user’s movements, using nothing more than just your phone. Integration into the adidas Training app is still in the early phases of the development cycle, but stay tuned for more updates in the future.

How to get started?

If you would like to start using the Pose Detection API in your mobile app, head over to the developer documentation or check out the sample apps for Android and iOS to see the API in action. For questions or feedback, please reach out to us through one of our community channels.

Join us for Google Assistant Developer Day on October 8

2020-08-26T09:00:00.000-07:00

Posted by Baris Gultekin, Director, Product Management Google Assistant and
Payam Shodjai, Director, Product Management Google Assistant

More and more people turn to Google Assistant every day to help them get the most out of their phones and smart displays: From playing games to using their favorite app by voice, there are more opportunities than ever for developers to create new and engaging experiences for Google Assistant.

We welcome you to join us virtually at our Google Assistant Developer Day on Thursday, October 8, to learn more about new tools and features we’re building for developers to bring Google Assistant to mobile apps and Smart Displays and help drive discoverability and engagement via voice. This will also be a great chance to chat live with Google leaders and engineers on the team to get your questions answered.

You’ll hear from our product experts and partnership leads on best practices to integrate with Google Assistant to help users more easily engage with their favorite apps by voice. Other sessions will include in-depth conversations around native development on Google Assistant, with so much more.

We’ll also have guest speakers like: Garrett Gaudini, Head of Product at Postmates, Laurens Rutten, Founder & CEO of CoolGames, Corey Bozarth, VP of Product & Monetization at MyFitnessPal and many other, join us on stage to share their stories about how voice has transformed the way people interact with their apps and services.

Whether you build for mobile or smart home, these new tools will help make your content and services available to people who want to use their voice to get things done.

Registration is FREE! Head on over to the event website to register and check out the schedule.

Helping the Haitian economy, one line of code at a time

2020-08-26T06:19:00.002-07:00

Posted by Jennifer Kohl, Program Manager, Developer Community Programs

Eustache Luckens Yadley at a GDG Port-au-Prince meetup

Meet Eustache Luckens Yadley, or “Yadley” for short. As a web developer from Port-au-Prince, Yadley has spent his career building web applications that benefit the local Haitian economy. Whether it’s ecommerce platforms that bring local sellers to market or software tools that help local businesses operate more effectively, Yadley has always been there with a technical hand to lend.

However, Yadley has also spent his career watching Haiti’s unemployment numbers rise to among the highest in the Caribbean. As he describes it,

“Every day, several thousand young people have no job to get by.”

So with code in mind and mouse in hand, Yadley got right to work. His first step was to identify a need in the economy. He soon figured out that Haiti had a shortage of delivery methods for consumers, making home delivery purchases of any kind extremely unreliable. With this observation, Yadley also noticed that there was a surplus of workers willing to deliver the goods, but no infrastructure to align their needs with that of the market’s.

Yadley watching a demo at a GDG Port-au-Prince meetup

In this moment, Yadley did what many good developers would do: build an app. He created the framework for what is now called “Livrezonpam,” an application that allows companies to post where and when they need a particular product delivered and workers to find the corresponding delivery jobs closest to them.

With a brilliant solution, Yadley’s last step was to find the right technical tools to build the concept out and make it a viable platform that users could work with to their benefit.

It was at this crucial step when Yadley found the Port-au-Prince Google Developer Group. With GDG Port-au-Prince, Yadley was able to bring his young app right into the developer community, run different demos of his product to experienced users, and get feedback from a wide array of developers with an intimate knowledge of the Haitian tech scene. The takeaways from working in the community translated directly to his work. Yadley learned how to build with the Google Cloud Platform Essentials, which proved key in managing all the data his app now collects. He also learned how to get the Google Maps Platform API working for his app, creating a streamlined user experience that helped workers and companies in Haiti locate one another with precision and ease.

This wide array of community technical resources, from trainings, to mentors, to helpful friends, allowed Yadley to grow his knowledge of several Google technologies, which in turn allowed him to grow his app for the Haitian community.

Today, Yadley is still an active member of the GDG community, growing his skills and those of the many friends around him. And at the same time, he is still growing Librezonpam on the Google Play App Store to help local businesses reach their customers and bring more jobs directly to the people of Haiti.

Ready to start building with a Google Developer Group near you? Find the closest community to you, here.

Introducing Spotlight: Women Techmakers series on career and professional development in tech

2020-08-25T09:36:00.000-07:00

Posted by Caitlin Morrissey, PgM for Women Techmakers

On July 9, Women Techmakers launched the first episode of “Spotlight”, a new global program focused on amplifying the stories and pathways of women in technology across the industry. The idea for this series came from our community who were looking for more help with navigating their careers, especially during this time. We quickly realized that with our extensive network of professional women in all different parts of the tech industry, we could play a part in scaling this kind of guidance and advice to women all over the world.

The first video features Priyanka Vergadia, a Developer Advocate for Google Cloud, who shared advice on taking risks, career paths in tech, and dealing with failure. “I wanted to be on the show because I am a firm believer that hearing someone's story can inspire you to create your own beautiful journey.”

Image of Priyanka Vergadia with Spotlight host Caitlin Morrissey

Since then, we’ve been releasing weekly episodes featuring women from all over the world on themes from finding the job that’s right for you and what’s needed to be successful in that role to overcoming imposter syndrome and leading with confidence. Each woman brings a unique perspective to the table -- some have their masters in Engineering, while others went through a coding bootcamp and got into tech as a second career. This diversity in experience and backgrounds brings a richness and variety to the advice shared across the episodes, providing valuable guidance a viewer can take away, regardless of where they are in their careers.

Image of Spotlight guests: Pujaa Rajan, Jessica Early-Cha and Jacquelle Horton

The response to the series has been overwhelmingly positive so far - “I am honestly blown away by the reception of the video,” said Priyanka, who received many heartfelt messages after her episode was published. And as Spotlight continues, we’re excited to tell more stories of more women in tech and build a library of career advice that anyone can use to help navigate the tech ecosystem.

You can check out all episodes of Spotlight here, and make sure to subscribe to the Women Techmakers channel so you don’t miss out on future interviews.

Building G Suite Add-ons with your favorite tech stack

2020-08-20T09:50:00.000-07:00

Posted by Jon Harmer, Product Manager and Steven Bazyl, Developer Advocate for G Suite

Let’s talk about the basics of G Suite Add-ons. G Suite Add-ons simplify how users get things done in G Suite by bringing in functionality from other applications where you need them. They provide a persistent sidebar for quick access, and they are context-aware -- meaning they can react to what you’re doing in context. For example, a CRM add-on can automatically surface details about a sales opportunity in response to an email based on the recipients or even the contents of the message itself.

Up until recently, G Suite Add-ons leaned on Apps Script to build Add-ons, but choice is always a good thing, and in some cases you may want to use another scripting language.. So let’s talk about how to build Add-ons using additional runtimes:

First, additional runtimes don't add any new capabilities to what you can build. What it does give you is more choice and flexibility in how you build Add-ons. We’ve heard feedback from developers that they would also like the option to use the tools and ecosystems they’ve already learned and invested in. And while there have always been ways to bridge Apps Script and other backends that expose APIs over HTTP/S, it isn't the cleanest of workarounds. .

So let’s look at a side by side comparison of what it looks like to build an Add-on with alternate runtimes:

function homePage() {
  let card = CardService.newCardBuilder()
    .addSection(CardService.newCardSection()
      .addWidget(CardService.newTextParagraph()
        .setText("Hello world"))
    ).build();
  return [card];
}

Here’s the hello world equivalent of an Add-on in Apps Script. Since Apps Script is more akin to a serverless framework like GCF, the code is straightforward -- a function that takes an event and returns the UI to render.

// Hello world Node.js
const express = require('express');
const app = express();
app.use(express.json());

app.post('/home', (req, res) => {
  let card = {
    sections: [{
      widgets: [{
        textParagraph: {
          text: 'Hello world'
        }
      }]
    }]
  };
  res.json({
    action: {
      navigations: [{
        pushCard:  card
      }]
    }
  });
}

This is the equivalent in NodeJS using express, a popular web server framework. It shows a little bit more of the underlying mechanics -- working with the HTTP request/response directly, starting the server, and so on.

The biggest difference is the card markup -- instead of using CardService, which under the covers builds a protobuf, we're using the JSON representation of the same thing.

function getCurrentMessage(event) {
  var accessToken = event.messageMetadata.accessToken;
  var messageId = event.messageMetadata.messageId;
  GmailApp.setCurrentMessageAccessToken(accessToken);
  return GmailApp.getMessageById(messageId);
}

Another area where things differ is accessing Google APis. In Apps Script, the clients are available in the global context -- the APIs mostly 'just work'. Moving to Node requires a little more effort, but not much.

Apps Script is super easy here. In fact, normally we wouldn't bother with setting the token when using more permissive scopes as it's done for us by Apps Script. We're doing it here to take advantage of the per-message scope that the add-on framework provides.

const { google } = require('googleapis');
const { OAuth2Client } = require('google-auth-library');
const gmail  = google.gmail({ version: 'v1' });

async function fetchMessage(event) {
    const accessToken = event.gmail.accessToken;
    const auth = new OAuth2Client();
    auth.setCredentials({access_token: accessToken});

    const messageId = event.gmail.messageId;
    const res = await gmail.users.messages.get({
        id: messageId,
        userId: 'me',
        headers: { 'X-Goog-Gmail-Access-Token': event.gmail.accessToken },
        auth
    });
    return res.data;
}

The NodeJS version is very similar -- a little extra code to import the libraries, but otherwise the same -- extract the message ID and token from the request, set the credentials, then call the API to get the message contents.

Your Add-on, Your way

One of the biggest wins for alternate runtimes is the testability that comes with using your favorite IDE, language, and framework--all of which helps you make developing add-ons more approachable.

Both Apps Script and alternate runtimes for G Suite Add-ons have important places in building Add-ons. If you’re getting into building Add-ons or if you want to prototype more complex ones, Apps Script is a good choice.. If you write and maintain systems as your full time job, though, alternate runtimes allow you to use those tools to build your Add-on, letting you leverage work, code and processes that you’re already using. With alternate runtimes for G Suite Add-ons, we want to make it possible for you to extend G Suite in a way that fits your needs using whatever tools you're most comfortable with.

And don't just take our word for it, hear from one of our early access partners. Shailesh Matariya, CTO at Gfacility has this to say about alternate runtimes: "We're really happy to use alternate runtimes in G Suite Add-ons. The results have been great and it's much easier to maintain the code. Historically, it would take 4-5 seconds to load data in our Add-on, whereas with alternate runtimes it's closer to 1 second, and that time and efficiency really adds up. Not to mention performance, we're seeing about a 50% performance increase and thanks to this our users are able to manage their workflows with just a few clicks, without having to jump to another system and deal with the hassle of constant updates."

Next Steps

Read the developer documentation for Alternate Runtimes and sign up for the early access program.

ChromeOS.dev — A blueprint to build world-class apps and games for Chrome OS

2020-08-12T09:30:00.000-07:00

Posted by Iein Valdez, Head of Chrome OS Developer Relations

This article originally appeared on ChromeOS.dev.

While people are spending more time at home than on the go, they’re relying increasingly on personal desktops and laptops to make everyday life easier. Whether they’re video-chatting with friends and family, discovering entertaining apps and games, multitasking at work, or pursuing a passion project, bigger screens and better performance have made all the difference.

This trend was clear from March through June 2020: Chromebook unit sales grew 127% year over year (YOY) while the rest of the U.S. notebook category increased by 40% YOY.¹ Laptops have become crucial to people at home who want to use their favorite apps and games, like Star Trek™ Fleet Command and Reigns: Game of Thrones to enjoy action-packed adventure, Calm to manage stress, or Disney+ to keep the whole family entertained.

To deliver app experiences that truly improve people’s lives, developers must be equipped with the right tools, resources, and best practices. That’s why we’re excited to introduce ChromeOS.dev — a dedicated resource for technical developers, designers, product managers, and business leaders.

ChromeOS.dev, available in English and Spanish (with other languages coming soon), features the latest news, product announcements, technical documentation, and code samples from popular apps. Whether you’re a web, Android, or Linux developer who’s just getting started or a certified expert, you’ll find all the information you need on ChromeOS.dev.

Hear from our experts at Google and Chrome OS, as well as a variety of developers, as they share practical tips, benefits, and the challenges of creating app experiences for today’s users. Plus, you can review the updated Chrome OS Layout and UX App Quality guidelines with helpful information on UI components, navigation, fonts, layouts, and everything that goes into creating world-class apps and games for Chrome OS.

Even better, as a fully open-source online destination, ChromeOS.dev is designed considering all the principles and methods for creating highly capable and reliable Progressive Web Apps (PWAs), ensuring developers always have quick, easy access to the information they need — even when they’re offline.

Check out a few of the newest updates and improvements below, and be sure to install the ChromeOS.dev PWA on your device to stay on top of the latest information.

New features for Chrome OS developers

Whether it’s developing Android, Linux, or web apps, every update on ChromeOS.dev is about making sure all developers can build better app experiences in a streamlined, easy-to-navigate environment.

Customizable Linux Terminal

The Linux (Beta) on Chrome OS Terminal now comes equipped with personalized features right out of the box, including:

Integrated tabs and shortcuts
Multitask with ease by using windows and tabs to manage different tasks and switch between multiple projects. You can also use familiar shortcuts such as Ctrl + T, Ctrl + W, and Ctrl + Tab to manage your tabs, or use the settings page to control if these keys should be used in your Terminal for apps like vim or emacs.
Themes
Customize your Terminal by selecting a theme to switch up the background, frame, font, and cursor color.
Redesigned Terminal settings
The settings tab has been reorganized to make it easier to customize all your Terminal options.

Developers can now start using these and other customizable features in the Terminal app.

Android Emulator support

Supported Chromebooks can now run a full version of the Android Emulator, which allows developers to test apps on any Android version and device without needing the actual hardware. Android app developers can simulate map locations and other sensor data to test how an app performs with various motions, orientations, and environmental conditions. With the Android Emulator support in Chrome OS, developers can optimize for different Android versions and devices — including tablets and foldable smartphones — right from their Chromebook.

Deploy apps directly to Chrome OS

Building and testing Android apps on a single machine is simpler than ever. Now, developers who are running Chrome OS M81 and higher can deploy and test apps directly on their Chromebooks — no need to use developer mode or to connect different devices physically via USB. Combined with Android Emulator support, Chrome OS is equipped to support full Android development.

Improved Project Wizard in Android Studio

An updated Primary/Detail Activity Template in Android Studio offers complete support to build experiences for larger screens, including Chromebooks, tablets, and foldables. This updated option provides multiple layouts for both phones and larger-screen devices as well as better keyboard/mouse scaffolding. This feature will be available in Android Studio 4.2 Canary 8.

Updated support from Android lint checks

We’ve improved the default checks in Android’s lint tool to help developers identify and correct common coding issues to improve their apps on larger screens, such as non-resizable and portrait-locked activities. This feature is currently available for testing in Canary channel.

Unlock your app’s full potential with Chrome OS

From day one, our goal has been to help developers at every skill level create simple, powerful, and secure app experiences for all platforms. As our new reality creates a greater need for helpful and engaging apps on large-screen devices, we’re working hard to streamline the process by making Chrome OS more versatile, customizable, and intuitive.

Visit ChromeOS.dev and install it on your Chromebook to stay on top of the latest resources, product updates, thought-provoking insights, and inspiring success stories from Chrome OS developers worldwide.

Sources:
¹The NPD Group, Inc., U.S. Retail Tracking Service, Notebook Computers, based on unit sales, April–June 2020 and March–June 2020.

Introducing Cast Connect: a better way to integrate Google Cast directly into your Android TV apps

2020-08-06T17:44:00.001-07:00

Posted by Meher Vurimi, Product Manager

For more than seven years, Google Cast has made it easy for users to enjoy your content on the big screen with Chromecast or a Chromecast built-in TV. We’re always looking to improve the casting experiences, which is why we’re excited to introduce Cast Connect. This new feature allows users to cast directly to your Android TV app while still allowing control from your sender app.

Why is this helpful?

With Google Cast your app is the remote control - helping users find, play, pause, seek, stop, and otherwise control what they’re watching. It enables you to extend video or audio from your Android, iOS, or Chrome app to a TV or sound system. With Android TV, partners are able to build apps that let users experience your app's immersive content on their TV screen and control with a remote.

With Cast Connect, we are combining the best of both worlds; first, by prioritizing playback via the Android TV app to deliver a richer and more immersive experience, and second, by allowing the user to still control their experience from your Android, iOS or Chrome app, and now, also directly using their Android TV’s remote control. Cast Connect helps the user easily engage with other content directly on the TV instead of only having to use your mobile device to browse for additional content.

Cast Connect User Journey on Stan

Availability

We’re working closely with a number of partners on bringing Cast Connect to their Apps and, most recently, we’re excited to announce that CBS and our Australian SVOD partner, Stan have launched Cast Connect. Starting today, the Cast Connect library is available on Android, iOS and Chrome. To get started with adding Cast Connect to your existing framework, head over to the Google Cast Developers site. Along the way, the Cast SDK team and the developer community are available to help you and answer questions on Stack Overflow by using the google-cast-connect tag.

Happy Casting!

Digital Ink Recognition in ML Kit

2020-08-06T10:00:00.000-07:00

Posted by Mircea Trăichioiu, Software Engineer, Handwriting Recognition

A month ago, we announced changes to ML Kit to make mobile development with machine learning even easier. Today we're announcing the addition of the Digital Ink Recognition API on both Android and iOS to allow developers to create apps where stylus and touch act as first class inputs.

Digital ink recognition: the latest addition to ML Kit’s APIs

Digital Ink Recognition is different from the existing Vision and Natural Language APIs in ML Kit, as it takes neither text nor images as input. Instead, it looks at the user's strokes on the screen and recognizes what they are writing or drawing. This is the same technology that powers handwriting recognition in Gboard - Google’s own keyboard app, which we described in detail in a 2019 blog post. It's also the same underlying technology used in the Quick, Draw! and AutoDraw experiments.

Handwriting input in Gboard

Turning doodles into art with Autodraw

With the new Digital Ink Recognition API, developers can now use this technology in their apps as well, for everything from letting users input text and figures with a finger or stylus to transcribing handwritten notes to make them searchable; all in near real time and entirely on-device.

Supports many languages and character sets

Digital Ink Recognition supports 300+ languages and 25+ writing systems including all major Latin languages, as well as Chinese, Japanese, Korean, Arabic, Cyrillic, and more. Classifiers parse written text into a string of characters

Recognizes shapes

Other classifiers can describe shapes, such as drawings and emojis, by the class to which they belong (circle, square, happy face, etc). We currently support an autodraw sketch recognizer, an emoji recognizer, and a basic shape recognizer.

Works offline

Digital Ink Recognition API runs on-device and does not require a network connection. However, you must download one or more models before you can use a recognizer. Models are downloaded on demand and are around 20MB in size. Refer to the model download documentation for more information.

Runs fast

The time to perform a recognition call depends on the exact device and the size of the input stroke sequence. On a typical mobile device recognizing a line of text takes about 100 ms.

How to get started

If you would like to start using Digital Ink Recognition in your mobile app, head over to the documentation or check out the sample apps for Android and iOS to see the API in action. For questions or feedback, please reach out to us through one of our community channels.

Automatic Deployment of Hugo Sites on Firebase Hosting and Drafts on Cloud Run

2020-08-04T09:21:00.000-07:00

Posted by James Ward, Developer Advocate

Recently I completed the migration of my blog from Wordpress to Hugo and I wanted to take advantage of it now being a static site by hosting it on a Content Delivery Network (CDN). With Hugo the source content is plain files instead of rows in a database. In the case of my blog those files are in git on GitHub. But when the source files change, the site needs to be regenerated and redeployed to the CDN. Also, sometimes it is nice to have drafts available for review. I setup a continuous delivery pipeline which deploys changes to my prod site on Firebase Hosting and drafts on Cloud Run, using Cloud Build. Read on for instructions for how to set all this up.

Step 1a) Setup A New Hugo Project

If you do not have an existing Hugo project you can create a GitHub copy (i.e. fork) of my Hugo starter repo:

Step 1b) Setup Existing Hugo Project

If you have an existing Hugo project you'll need to add some files to it:

.firebaserc

{
  "projects": {
    "production": "hello-hugo"
  }
}

cloudbuild-draft.yaml

steps:
- name: 'gcr.io/cloud-builders/git'
  entrypoint: '/bin/sh'
  args:
  - '-c'
  - |
    # Get the theme git submodule
    THEME_URL=$(git config -f .gitmodules --get-regexp '^submodule\..*\.url$' | awk '{ print $2 }')
    THEME_DIR=$(git config -f .gitmodules --get-regexp '^submodule\..*\.path$' | awk '{ print $2 }')
    rm -rf themes
    git clone $$THEME_URL $$THEME_DIR

- name: 'gcr.io/cloud-builders/docker'
  entrypoint: '/bin/sh'
  args:
  - '-c'
  - |
    docker build -t gcr.io/$PROJECT_ID/$REPO_NAME-$BRANCH_NAME:$COMMIT_SHA -f - . << EOF
    FROM klakegg/hugo:latest
    WORKDIR /workspace
    COPY . /workspace
    ENTRYPOINT hugo -D -p \$$PORT --bind \$$HUGO_BIND --renderToDisk --disableLiveReload --watch=false serve
    EOF
    docker push gcr.io/$PROJECT_ID/$REPO_NAME-$BRANCH_NAME:$COMMIT_SHA

- name: 'gcr.io/cloud-builders/gcloud'
  args:
  - run
  - deploy
  - --image=gcr.io/$PROJECT_ID/$REPO_NAME-$BRANCH_NAME:$COMMIT_SHA
  - --platform=managed
  - --project=$PROJECT_ID
  - --region=us-central1
  - --memory=512Mi
  - --allow-unauthenticated
  - $REPO_NAME-$BRANCH_NAME

cloudbuild.yaml

steps:
- name: 'gcr.io/cloud-builders/git'
  entrypoint: '/bin/sh'
  args:
  - '-c'
  - |
    # Get the theme git submodule
    THEME_URL=$(git config -f .gitmodules --get-regexp '^submodule\..*\.url$' | awk '{ print $2 }')
    THEME_DIR=$(git config -f .gitmodules --get-regexp '^submodule\..*\.path$' | awk '{ print $2 }')
    rm -rf themes
    git clone $$THEME_URL $$THEME_DIR

- name: 'gcr.io/cloud-builders/curl'
  entrypoint: '/bin/sh'
  args:
  - '-c'
  - |
    curl -sL https://github.com/gohugoio/hugo/releases/download/v0.69.2/hugo_0.69.2_Linux-64bit.tar.gz | tar -zxv
    ./hugo

- name: 'gcr.io/cloud-builders/wget'
  entrypoint: '/bin/sh'
  args:
  - '-c'
  - |
    # Get firebase CLI
    wget -O firebase https://firebase.tools/bin/linux/latest
    chmod +x firebase
    # Deploy site
    ./firebase deploy --project=$PROJECT_ID --only=hosting

firebase.json

{
  "hosting": {
    "public": "public"
  }
}

Step 2) Setup Cloud Build Triggers

In the Google Cloud Build console, connect to your newly forked repo: Select the newly forked repo: Create the default push trigger: Edit the new trigger: Set the trigger to only fire on changes to the ^master$ branch: Create a new trigger: Give it a name like drafts-trigger, specify the branch selector as .* (i.e. any branch), and the build configuration file type to "Cloud Build configuration file" with a value of cloudbuild-draft.yaml Setup permissions for the Cloud Build process to manage Cloud Run and Firebase Hosting by visiting the IAM management page, locate the member with the name ending with @cloudbuild.gserviceaccount.com, and select the "pencil" / edit button: Add a role for "Cloud Run Admin" and another for "Firebase Hosting Admin": Your default "prod" trigger isn't read to test yet, but you can test the drafts on Cloud Run by going back to the Cloud Build Triggers page, and clicking the "Run Trigger" button on the "drafts-trigger" line. Check the build logs by finding the build in the Cloud Build History. Once the build completes visit the Cloud Run console to find your newly created service which hosts the drafts version of your new blog. Note that the service name includes the branch so that you can see drafts from different branches.

Step 3) Setup Firebase Hosting

To setup your production / CDN'd site, login to the firebase console and select your project:

Now you'll need your project id, which can be found in the URL on the Firebase Project Overview page. The URL for my project is:

console.firebase.google.com/project/jw-demo/overview

Which means my project id is: jw-demo

Now copy your project id go into your GitHub fork, select the .firebaserc file and click the "pencil" / edit button:

Replace the hello-hugo string with your project id and commit the changes. This commit will trigger two new builds, one for the production site and one for the drafts site on Cloud Run. You can check the status of those builds on the Cloud Build History page. Once the default trigger (the one for Firebase hosting) finishes, check out your Hugo site running on Firebase Hosting by navigating to (replacing YOUR_PROJECT_ID with the project id you used above): https://YOUR_PROJECT_ID.web.app/

Your prod and drafts sites are now automatically deploying on new commits!

Step 4) (Optional) Change Hugo Theme

There are many themes for Hugo and they are easy to change. Typically themes are pulled into Hugo sites using git submodules. To change the theme, edit your .gitmodules file and set the subdirectories and url. As an example, here is the content when using the mainroad theme:

[submodule "themes/mainroad"]
 path = themes/mainroad
 url = https://github.com/vimux/mainroad.git

You will also need to change the theme value in your config.toml file to match the directory name in the themes directory. For example:

theme = "mainroad"

Note: At the time of writing this, Cloud Build does not clone git submodules so the cloudbuild.yaml does the cloning instead.

Step 5) (Optional) Setup Local Editing

To setup local editing you will first need to clone your fork. You can do this with the GitHub desktop app. Or from the command line:

git clone --recurse-submodules https://github.com/USER/REPO.git

Once you have the files locally, install Hugo, and from inside the repo's directory, run:

hugo -D serve

This will serve the drafts in the site. You can check out the site at: localhost:1313

Committing non-draft changes to master and pushing those changes to GitHub will kick off the build which will deploy them on your prod site. Committing draft to any branch will kick off the build which will deploy them on a Cloud Run site.

Hopefully that all helps you with hosting your Hugo sites! Let me know if you run into any problems.

Meet the global students solving local problems with code

2020-07-23T11:02:00.001-07:00

Posted by Erica Hanson, Developer Student Clubs Program Manager, Google

Google Developer Student Clubs (DSC) are university based community groups for students who are interested in Google’s developer technologies. Each year, Google puts a call out to the entire DSC global community, asking students to answer one simple question: Can you solve a local problem in your community by building with Google’s technologies?

This event is known as the DSC Solution Challenge and this year’s winners went above and beyond to answer the call - so much so that we couldn’t just pick one winner, we chose 10.

While we initially thought we were the ones sending out the challenge, these young developers instead flipped the script back on us. Through their innovative designs and uncompromised creative spirit, they’ve pushed our team here at Google to stretch our thinking about how developers can build a more hopeful future.

With this, we’re welcoming these passionate students and anyone interested to the virtual Solution Challenge Demo Day on August 26th, where the students will present their winning ideas in detail.

Ahead of the event, learn more about this incredible group of thinkers and their solutions below.

1. FreeSpeak - Technical University of Munich, Germany

Maria Pospelova, Wei Wen Qing, and Almo Gunadya Sutedjo developed FreeSpeak, a software that uses modern machine learning and video/audio analyzing tools by leveraging TensorFlow and Google Cloud’s Natural Language to analyze presentations and give individual feedback and tips as a “virtual coach.”

“We’ve loved connecting with talented people from around the world and exchanging ideas with them. We see that we can provide impact not only to our local neighborhood, but also around the world and help people. This motivates us to work a lot harder.”

2. CoronaAI - University of California Berkeley, United States

Anushka Purohit, Anupam Tiwari, and Neel Desai created CoronaAI, a TensorFlow based technology that helps examine COVID-19 data. Specifically, the device is made up of a band worn around a patient's chest that uses electrodes to extract real-time images of the lungs. From here, the band connects to a monitor that allows doctors to examine patients in real time without being near them.

“We're honestly huge fans of the Google Cloud Platform because of its simplicity, familiarity, and the large number of resources available. Developing this project was the best learning experience.”

3. Worthy Walk - National University of Computer & Emerging Sciences, Pakistan

Syed Moazzam Maqsood, Krinza Momin, Muhammad Ahmed Gul, and Hussain Zuhair built Worthy Walk: an Android and iOS app that provides its users a platform to achieve health goals by walking, running, or cycling. To encourage users, Worthy Walk provides an inbuilt currency called Knubs that can be redeemed as discounts from local businesses, shops, and startups.

“Being a part of DSC means friendship - sharing knowledge and resources - all while developing a social infrastructure that gives people the power to build a global community that works for all of us.”

4. Simhae : Deep sea of mind - Soonchunhyang University, South Korea

Yuna Kim, Young hoon Jo, Jeong yoon Joo, and Sangbeom Hwang created Simhae, a platform created with Flutter and Google Cloud that allows users to access basic information and activities to inspire them to attend self-help gatherings run by suicide prevention centers. They believe that this experience is an important point that can lead to solidarity of suicide survivors.

“It's so nice to have a chance to meet more diverse people. Through these communities, I can make up for my shortcomings and share more information with those who have different experiences than me - all while developing my own potential.”

5. Emergency Response Assistance - University of Education, Winneba (College of Technology Kumasi), Ghana

Elvis Antwi Sarfo, Yaw Barnieh Anane, Ampomah Ata Acheampong Prince, and Perditha Abena Acheampong constructed Emergency Response Assistance, an Android application to help health authorities post the latest first aid steps to educate the public and also help the victims report emergency cases with a click of a button. The Emergency Response team will also be able to track the exact location of the victims on the map.

“DSC is not just a community, it’s an inspiration. It’s outstanding how the platform has brought all of these students, lecturers, and teaching assistants, who are all so passionate about using tech to solve problems, together.”

6. Tulibot - Politeknik Elektronika Negeri Surabaya, Indonesia

Muhammad Alan Nur, Pravasta Caraka Bramastagiri, Eva Rahmadanti, and Namira Rizqi Annisa created Tulibot: an integrated assistive technology, built with the Google Speech API, that’s made to bridge communication between deaf people and society. The group made two main devices, Smart Glasses and Smart Gloves. Smart Glasses help with communication for the hearing impaired by showing real time answers directly on the glasses from its interlocutors. Smart Gloves transcribe gesture input into audio output by implementing gesture to text technology.

“This has been an amazing opportunity for us because with this challenge, we can learn many things like coding, management, business, and more. The special materials we can get directly from Google is so helpful.”

7. Picare - The Hong Kong University of Science & Technology, Hong Kong

Sze Yuk Yin, Kwok Ue Nam, Ng Chi Ting, Chong Cheuk Hei, and Silver Ng developed Picare, a healthcare matching platform built with Flutter and Google Machine Learning to help elderly people in Hong Kong. Users will be able to use the app to research, schedule, and pay caregivers directly through the app.

“Our community hosted several workshops ranging from design thinking to coding techniques. This boosted our development by introducing us to various state-of-the-art technologies, such as Machine Learning and Cloud computing, which helped us reach our development goals.”

8. Shareapy - Ho Chi Minh City University of Technology, Vietnam

Vo Ngoc Khanh Linh, Tran Lam Bao Khang, Nguyen Dang Huy, and Nguyen Thanh Nhan built Shareapy: a digitized support group app created with Android that helps bring people together who share similar problems regardless of their age, gender, religion, financial status, etc. After conducting an extremely rigorous user testing phase, this team had the chance to see all that TensorFlow and Firebase could do.

“My team loves Firebase so much. One of our team members now uses it to help do some of his homework problems.”

9. Capstone - Midlands State University, Zimbabwe

Victor Chinyavada, Marvellous Humphery Chirunga, and Lavender Zandile Tshuma started Capstone, a service hosted on the Google Cloud Platform that aims to combat plagiarism among students, authors, and researchers. In particular, the technology aims to develop more effective algorithms that will incorporate the latest in big data, artificial intelligence, and data mining. As a team, the group bonded over applying technologies from Google to their project, but their real takeaway was working together to solve problems.

“To submit our project on time, we started all night hackathons, which helped us finish all of our work while having fun and getting to know each other better.”

10. MiCamp - Dr. B.R. Ambedkar National Institute of Technology, India

Praveen Agrawal built MiCamp, an Android app that holds all the info students from his campus need. Features include a calendar with upcoming campus events, student profiles, a used book marketplace, hostel management, online food ordering, and more. As a team of one, Praveen needed to speed up his development, so he applied his new knowledge of Flutter to finish.

“I’d heard of technologies like Flutter, but never used them until joining DSC; they inspired us to use those technologies, which really improved my solution.”

________________________

Want to learn more about Developer Student Clubs? Join a club near you, here and stay tuned for our upcoming virtual Solution Challenge Demo Day on August 26th, here.

Summer updates from Coral

2020-07-22T11:04:00.000-07:00

Posted by the Coral Team

Summer has arrived along with a number of Coral updates. We're happy to announce a new partnership with balena that helps customers build, manage, and deploy IoT applications at scale on Coral devices. In addition, we've released a series of updates to expand platform compatibility, make development easier, and improve the ML capabilities of our devices.

Open-source Edge TPU runtime now available on GitHub

First up, our Edge TPU runtime is now open-source and available on GitHub, including scripts and instructions for building the library for Linux and Windows. Customers running a platform that is not officially supported by Coral, including ARMv7 and RISC-V can now compile the Edge TPU runtime themselves and start experimenting. An open source runtime is easier to integrate into your customized build pipeline, enabling support for creating Yocto-based images as well as other distributions.

Windows drivers now available for the Mini PCIe and M.2 accelerators

Coral customers can now also use the Mini PCIe and M.2 accelerators on the Microsoft Windows platform. New Windows drivers for these products complement the previously released Windows drivers for the USB accelerator and make it possible to start prototyping with the Coral USB Accelerator on Windows and then to move into production with our Mini PCIe and M.2 products.

New fresh bits on the Coral ML software stack

We’ve also made a number of new updates to our ML tools:

The Edge TPU compiler is now version 14.1. It can be updated by running sudo apt-get update && sudo apt-get install edgetpu, or follow the instructions here
Our new Model Pipelining API allows you to divide your model across multiple Edge TPUs. The C++ version is currently in beta and the source is on GitHub
New embedding extractor models for EfficientNet, for use with on-device backpropagation. Embedding extractor models are compiled with the last fully-connected layer removed, allowing you to retrain for classification. Previously, only Inception and MobileNet were available and now retraining can also be done on EfficientNet
New Colab notebooks to retrain a classification model with TensorFlow 2.0 and build C++ examples

Balena partners with Coral to enable AI at the edge

We are excited to share that the Balena fleet management platform now supports Coral products!

Companies running a fleet of ML-enabled devices on the edge need to keep their systems up-to-date with the latest security patches in order to protect data, model IP and hardware from being compromised. Additionally, ML applications benefit from being consistently retrained to recognize new use cases with maximum accuracy. Coral + balena together, bring simplicity and ease to the provisioning, deployment, updating, and monitoring of your ML project at the edge, moving early prototyping seamlessly towards production environments with many thousands of devices.

Read more about all the benefits of Coral devices combined with balena container technology or get started deploying container images to your Coral fleet with this demo project.

New version of Mendel Linux

Mendel Linux (5.0 release Eagle) is now available for the Coral Dev Board and SoM and includes a more stable package repository that provides a smoother updating experience. It also brings compatibility improvements and a new version of the GPU driver.

New models

Last but not least, we’ve recently released BodyPix, a Google person-segmentation model that was previously only available for TensorFlow.JS, as a Coral model. This enables real-time privacy preserving understanding of where people (and body parts) are on a camera frame. We first demoed this at CES 2020 and it was one of our most popular demos. Using BodyPix we can remove people from the frame, display only their outline, and aggregate over time to see heat maps of population flow.

Here are two possible applications of BodyPix: Body-part segmentation and anonymous population flow. Both are running on the Dev Board.

We’re excited to add BodyPix to the portfolio of projects the community is using to extend our models far beyond our demos—including tackling today’s biggest challenges. For example, Neuralet has taken our MobileNet V2 SSD Detection model and used it to implement Smart Social Distancing. Using the bounding box of person detection, they can compute a region for safe distancing and let a user know if social distance isn’t being maintained. The best part is this is done without any sort of facial recognition or tracking, with Coral we can accomplish this in real-time in a privacy preserving manner.

We can’t wait to see more projects that the community can make with BodyPix. Beyond anonymous population flow there’s endless possibilities with background and body part manipulation. Let us know what you come up with at our community channels, including GitHub and StackOverflow.

________________________

We are excited to share all that Coral has to offer as we continue to evolve our platform. For a list of worldwide distributors, system integrators and partners, including balena, visit the Coral partnerships page. Please visit Coral.ai to discover more about our edge ML platform and share your feedback at coral-support@google.com.

Discontinuing support for JSON-RPC and global HTTP batch endpoints

2020-07-13T13:00:00.000-07:00

UPDATE:

August 19, 2020: Revised guidance for Google Cloud client libraries and Apache Beam/Cloud Dataflow

August 12, 2020: Added guidance for Dataproc hadoop connector

July 13, 2020: Enumerate endpoints for JSON-RPC and Global HTTP Batch. Include examples of Non-Global HTTP Batch endpoints for contrast.

July 8, 2020: Limit usage of JSON-RPC and Global HTTP batch endpoints to existing projects only. Starting July 15 (JSON-RPC) and July 16 (Global HTTP Batch) we will no longer allow new projects to call these two endpoints. Projects with calls in the last 4 weeks will continue to work until the deadline of Aug 12, 2020.

Apr 23, 2020: gcloud min version has been updated

Apr 22, 2020: error injection planned for Apr 28 is CANCELLED, JSON-RPC and Global HTTP batch endpoint will perform normally. Next error injection window will be on May 26 as scheduled.

We have invested heavily in our API and service infrastructure to improve performance and security and to add features developers need to build world-class APIs. As we make changes we must address features that are no longer compatible with the latest architecture and business requirements.

The JSON-RPC protocol (http://www.jsonrpc.org/specification) and Global HTTP Batch (example) are two such features. Our support for these features was based on an architecture using a single shared proxy to receive requests for all APIs. As we move towards a more distributed, high performance architecture where requests go directly to the appropriate API server we can no longer support these global endpoints.

We had originally planned to decommission these features by Mar 25, 2019. However, it came to our attention that few highly impacted customers might not have received the earlier notification.

As a result, we are extending the deprecation timeline to Aug 12, 2020, when we will discontinue support for both these features. Please note the timeline extension to Aug 12, 2020 does not apply to Places API. Please visit Places SDK for iOS migration for information on Places API migration.

Starting February 2020 and running through August 2020, we will periodically inject errors for short windows of time. Please see below for exact details and schedule of planned downtime windows.

We know that these changes have customer impact and have worked to make the transition steps as clear as possible. Please see the guidance below which will help ease the transition.

Planned downtime

To enable customers to identify systems that depend on these deprecated features, before the final turndown date is reached, there will be scheduled downtime for Global Batch and JSON-RPC starting from February 2020 and running through August 2020.

Below are details and schedule of periodic error injection windows, details of the downtime in June and July will be confirmed nearer the time, so please check back this blog post for the latest schedule.

How do I know if I should migrate ?

JSON-RPC

To identify whether you use JSON-RPC, you can check whether you send requests to "https://www.googleapis.com/rpc" or "https://content.googleapis.com/rpc". If you do, you should migrate.

HTTP batch

A batch request is homogenous if the inner requests are addressed to the same API, even if addressed to different methods of the same API. Homogenous batching will still be supported but through API specific batch endpoints. If you are currently forming homogeneous batch requests, using Google API Client Libraries or using non-Google API client libraries or no client library (i.e making raw HTTP requests), you should migrate.

A batch request is heterogeneous if the inner requests go to different APIs. Heterogeneous batching will not be supported after the turn down of the Global HTTP batch endpoint. If you are currently forming heterogeneous batch requests, change your client code to send only homogenous batch requests, i.e you should migrate.

What do you need to do to migrate ?

Clients will need to make the changes outlined below to migrate.

JSON-RPC

Endpoints

The following JSON-RPC endpoints will no longer be supported.

Using Client Libraries

If you are using JSON-RPC client libraries (either the Google published libraries or other libraries), switch to REST client libraries and modify your application to work with REST client libraries.

Example code for Javascript
Before

// json-rpc request for the list method
gapi.client.rpcRequest('zoo.animals.list', 'v2',
  {name:'giraffe'}).execute(x=>console.log(x))

After

// json-rest request for the list method
gapi.client.zoo.animals.list({name:'giraffe'}).then(x=>console.log(x))

Not Using Client Libraries i.e making raw HTTP requests

If you are not using client libraries (i.e. making raw HTTP requests):

Use the REST URLs, and

Change how you form the request and parse the response.

Example code

Before

// Request URL (JSON-RPC)
POST https://content.googleapis.com/rpc?alt=json&key=xxx
// Request Body (JSON-RPC)
[{
  "jsonrpc":"2.0",
  "id":"gapiRpc",
  "method":"zoo.animals.list",
  "apiVersion":"v2",
  "params":{"name":"giraffe"}
}]

After

// Request URL (JSON-REST)
GET 
https://content.googleapis.com/zoo/v2/animals?name=giraffe&key=xxx

HTTP batch

Endpoints

The following Global HTTP Batch endpoints will no longer be supported.

https://www.googleapis.com/batch
https://content.googleapis.com/batch
https://clients6.google.com/batch

Non-Global HTTP Batch endpoints that include the API name in the URI will continue to be supported. Examples includes:

https://www.googleapis.com/batch/compute/v1
https://content.googleapis.com/batch/compute/v1
https://clients6.google.com/batch/compute/v1
https://compute.googleapis.com/batch
https://content-compute.googleapis.com/batch
https://compute.clients6.google.com/batch

Heterogeneous batch requests

If you are currently forming heterogeneous batch requests, change your client code to send only homogenous batch requests.

Example code
The example demonstrates how we can split a heterogeneous batch request for 2 apis (urlshortener and zoo) into 2 homogeneous batch requests.

Before

// heterogeneous batch request example.

// Notice that the outer batch request contains inner API requests
// for two different APIs.

// Request to urlshortener API
request1 = gapi.client.urlshortener.url.get({"shortUrl": 
"http://goo.gl/fbsS"});

// Request to zoo API
request2 = gapi.client.zoo.animals.list();

// Request to urlshortener API
request3 = gapi.client.urlshortener.url.get({"shortUrl":
"https://goo.gl/XYFuPH"});

// Request to zoo API
request4 = gapi.client.zoo.animal().get({"name": "giraffe"});

// Creating single heterogeneous batch request object
heterogeneousBatchRequest = gapi.client.newBatch();
// adding the 4 batch requests
heterogeneousBatchRequest.add(request1);
heterogeneousBatchRequest.add(request2);  
heterogeneousBatchRequest.add(request3);
heterogeneousBatchRequest.add(request4);
// print the heterogeneous batch request
heterogeneousBatchRequest.then(x=>console.log(x));

After

// Split heterogeneous batch request into two homogenous batch requests

// Request to urlshortener API
request1 = gapi.client.urlshortener.url.get({"shortUrl": "http://goo.gl/fbsS"});

// Request to zoo API
request2 = gapi.client.zoo.animals.list();

// Request to urlshortener API
request3 = gapi.client.urlshortener.url.get({"shortUrl": "http://goo.gl/fbsS"})

// Request to zoo API
request4 = gapi.client.zoo.animals.list();

// Creating homogenous batch request object for urlshorterner
homogenousBatchUrlshortener = gapi.client.newBatch();
// adding the 2 batch requests for urlshorterner
homogenousBatchUrlshortener.add(request1); 
homogenousBatchUrlshortener.add(request3); 

// Creating homogenous batch request object for zoo
homogenousBatchZoo = gapi.client.newBatch();

// adding the 2 batch requests for zoo
homogenousBatchZoo.add(request2); 
homogenousBatchZoo.add(request4);
// print the 2 homogenous batch request
Promise.all([homogenousBatchUrlshortener,homogenousBatchZoo])
    .then(x=>console.log(x));

Homogeneous batch requests

Google API client libraries

If you are using Google API Client Libraries, these libraries have been regenerated to no longer make requests to the global HTTP batch endpoint. Recommendation for clients using these libraries is to upgrade to the latest version if possible. Please see the language specific guidance below for minimum Google API Client Library to upgrade to.

1. Java

Minimum Google Client Library Version supported is 1.23.1.
Consumer Steps
1. Update all Google API Service packages (`com.google.apis`) to a version where the supporting library version is 1.23.1 or higher. For example, upgrade `com.google.apis:google-api-services-drive` from version `v3-rev159-1.22.0` to `v3-rev20190620-1.30.1`.
2. Please ensure you are creating your BatchRequest (i.e) com.google.api.client.googleapis.batch.BatchRequest instance via your service client. The BatchRequest constructor is now deprecated and will not work in the future. Instead, use your service client’s batch() method to create a new BatchRequest using below example
3. For global batch, confirm that your code does not explicitly specify the Global Batch endpoint.
4. Rebuild

2. Python

Minimum Google Client Library Version supported is 1.6.6.
Consumer Steps
- Update to version 1.6.6 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild
Please note, a warning is included if users try to construct a request with the legacy batch endpoint. See Release notes for reference.

3. PHP

Minimum Google Client Library Version supported is 0.86.
Consumer Steps
- Update to version 0.86 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild

4. .NET

Minimum Google Client Library Version supported is 1.32.2.
Consumer Steps
- Update to version 1.32.2 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild

5. Javascript

If you are using Javascript client lib, please reach out to us with your questions and we will do our best to support

6. Objective-C

Minimum Google Client Library Version supported is 1.3.0.
Consumer Steps
- Update to version 1.3.0 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild

7. Dart

The Google API Dart Client Library does not support these features.

8. Ruby

Minimum Google Client Library Version supported is 0.11.0.
Consumer Steps
- Update to version 0.11.0 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild

9. Node.js

Minimum Google Client Library Version supported is 1.0.
Consumer Steps
- Update to version 1.0 or higher
- Confirm that your code does not explicitly make a call to a Global HTTP Batch endpoint (e.g. www.googleapis.com/batch). These should be made to Non-Global HTTP Batch endpoints (e.g. www.googleapis.com/batch/compute/v1).
- Rebuild

10. Go

The Google API Go Client Library does not support these features.

11. C++

The Google C++ API Client Library has been deprecated due to unrelated reasons. However, should you need to use the C++ client with batching in the short term, see consumer steps below.
Consumer Steps
- Change app code to explicitly pass the non-global batch URL to the C++ client library i.e pass the API-version specific endpoint (e.g. "www.googleapis.com/batch/compute/v1") to the overload method that takes the batch endpoint as an explicit parameter; merely updating the C++ client library to the latest version will not help.
  Example code
  Before
```
HttpRequestBatch batch(service->transport());
```
  After
```
HttpRequestBatch batch(service->transport(), 
service->batch_url());
```
- Link, rebuild and redeploy

Google Cloud Client Libraries

Java
Upgrade to v1.39.0 or later of the Google Cloud Java Client for Storage (released in v0.57.0 of the Google Cloud Client Library).
Python
Upgrade to v1.9.0 or later of the GCS client.

GCloud

Older versions of gcloud used global batch calls in some cases. You should update your version of gcloudl. GCloud from v148.0.0 onwards is compatible with this deprecation.

In general this can be done by running "gcloud components update".

Apache Beam/ Cloud Dataflow

Users with Dataflow pipelines written using the Beam (or Dataflow) Python or Java SDKs 2.4 and below should update their pipelines to a later SDK (than 2.4), ideally to the latest one available here.

Dataproc hadoop connector

Users who run Apache Hadoop or Apache Spark jobs directly on data in Cloud Storage using gcs-connector should update their connector to 1.6.3 or later.

Non-Google API client libraries or no client library

If you are currently forming homogeneous batch requests and using non-Google API client libraries or no client library (i.e making raw HTTP requests), then:

Change the endpoint from www.googleapis.com/batch to www.googleapis.com/batch/<api>/<version>.
Or, simply read the value of 'batchPath' from the API's discovery doc and use that value.

Google Pay plugin for Magento 2

2020-07-13T09:15:00.000-07:00

Posted by Soc Sieng, Developer Advocate

Google Pay plugin for Magento 2

We are pleased to announce the launch of the official Google Pay plugin for Magento 2. The Google Pay plugin can help increase conversions by enabling a simpler and more secure checkout experience in your Magento website. When you integrate with Google Pay, your customers can complete their purchases quickly using the payment methods they’ve securely saved to their Google Accounts.

Google Pay in action.

The Google Pay plugin is built in collaboration with Unbound Commerce, is free to use, and integrates with popular payment service providers including Adyen, BlueSnap, Braintree, FirstData - Payeezy & Ucom, Moneris, Stripe, and Vantiv.

Installation

The Google Pay plugin can be installed from the Magento Marketplace using this link or by searching the Magento Marketplace for “Google Pay”.

Refer to the Magento Marketplace User Guide for more installation instructions.

Getting started

To get started with the Google Pay plugin, you will need your Google Pay merchant identifier which can be found in the Google Pay Business Console.

Your Merchant ID can be found in the Google Pay Business Console.

Configuring the Google Pay plugin

Once installed, you can configure the plugin in your site’s Magento administration console by navigating to Stores > Configuration > Sales > Payment Methods and selecting the Configure button next to Google Pay.

Click on the Configure button to start the setup process.

Testing out Google Pay can be achieved in three easy steps:

Google Pay credentials: enter your Google Pay merchant ID (available from the Google Pay Business Console) and merchant name.
Payment gateway credentials: select your payment gateway from the list of payment gateways supported by the Google Pay plugin.
1. Choose the Sandbox environment for testing purposes.
2. Enter your payment gateway’s credentials into their respective form fields.
Google Pay settings: enable Google Pay and choose the card networks that you would like to accept.

You can optionally try out some of the advanced settings that provide the ability to customize the color and type of Google Pay button as well as enabling Minicart integration, which is recommended.

Checkout the Advanced Settings to further customize how and where the Google Pay button is presented in your store.

If your payment provider isn’t listed as an option in the payment gateway list, check to see if your payment provider’s plugin has built-in support for Google Pay.

Launching Google Pay for your website

When you’ve completed your testing, submit your website integration in the Google Pay Business Console. You will need to provide your website’s URL and screenshots to complete the submission.

Summing it up

Integrating Google Pay into your website is a great way to increase conversions and to improve the purchasing experience for your customers.

Find out more about Google Pay and the Google Pay plugin for Magento.

What do you think?

Do you have any questions? Let us know in the comments below or tweet using #AskGooglePayDev.

Introducing an easier way to design your G Suite Add-on

2020-07-10T09:33:00.000-07:00

Posted by Kylie Poppen, Senior Interaction Designer, G Suite and Akshay Potnis, Interaction Designer, G Suite

You’ve just scoped out an awesome new way to solve for your customer’s next challenge, but wait, what about the design? Building an integration between your software platform and another comes with a laundry list of things to think about: your vision, your users, their experience, your partners, APIs, developer docs, and so on. Caught between two different platforms, many constraints, and limited time, you're probably wondering: how might we build the most intuitive and powerful user experience?

Imagine making a presentation, with Google Slides you have all sorts of templates to get you started, and you can build a great deck easily. But, to build a seamless integration between two software platforms, those pre-built templates don’t exist and you basically have to start from scratch. In the best case scenario, you’d create your own components and layer them on top of each other with the goal of making the UI seem just about right. But this takes time. Hours longer than you want it to. Without design guidelines, you're stuck guessing what is or is not possible, looking to other apps and emulating what they've already done. Which leads us to the reality that some add-ons have a suboptimal experience, because time is limited, and you're left to build only for what you know you can do, rather than what's actually possible.

To simplify all of this, we’re introducing the G Suite Add-ons UI Design Kit, now live on Figma. With it you can browse all of the components of G Suite Add-ons’ card-based interface, learn best practices, and simply drag-and-drop to create your own unique designs. Save the time spent recreating what an add-on will look like, so that you can spend that time thinking about how your add-on will work .

While the UI Design Kit has only been live for a little over a month, we’ve already been hearing feedback from our partners about its impact.

“Zapier connects more than 2,000 apps, allowing businesses to automate repetitive, time-consuming tasks. When building these integrations, we want to ensure a seamless experience for our customers,” said Ryan Powell, Product Manager at Zapier. “However, a partner’s UI can be difficult to navigate when starting from scratch. G Suite’s UI Design Kit allows us to build, test and optimize integrations because we know from the start what is and is not possible inside of GSuite’s UI.”

Here’s how to use the UI Design Kit:

Step 1

Find and duplicate design kit

Search for G suite on Figma community or use this link
Open G Suite Add Ons UI Design Kit
Just click the duplicate button.

Step 2

Choose a template to begin

Go to UI templates page
Select a template from the list of templates

Step 3

Copy the template and detach from symbols to start editing

Helpful Hints: Features to help you iterate quickly

Build with auto layout, you don’t need to worry about the details.

Copy paste maintains layout padding & structure.

Maintained padding & structure while editing.

Built in fixed footer and peek cards.

Visualize your design against G-Suite surfaces easily.

Documentation built right into the template.

Go to the component page (e.g section)
Find layout + documentation / api links on respective pages

Next Steps to Consider:

With G Suite Add-ons, users and admins can seamlessly get their work done, across their favorite workplace applications, without needing to leave G Suite. With this UI Design Kit, you too can focus your time on building a great user experience inside of G Suite, while simplifying and accelerating the design process. Follow these steps to get started today:

Download the UI Design Kit

Get started with G Suite Add-ons

Hopefully this will inspire you to build more add-ons using the Cards Framework! To learn more about building for G Suite, check out the developer page, and please register for Next OnAir, which kicks off July 14th.

Teaching the art of great documentation

2020-07-09T09:00:00.002-07:00

Posted by James Scott, Technical writer

Technical writing is simple - you merely have to explain brutally complex technologies to relentlessly unforgiving audiences. It's unsurprising that so many engineers find writing documentation is the most painful part of their job. If you would like to teach your colleagues to become writers, the good news is Google's fun and interactive technical writing course materials are free and available for everyone to use! Alternatively, if you're a developer who would like to learn how to write more clearly, you can read through the course work for yourself or convince a colleague to teach the course at your organisation!

We researched documentation extensively, and it turns out that the best sentences in the world consist primarily of words. Our self-paced and facilitator-led courses will not only help software engineers choose the right words but also help to make the whole writing process a lot less scary. Perhaps software engineers won't become William Shakespeare or even William Shatner overnight, but hopefully they will gain the confidence to write something worth publishing. As working from home becomes more common, good documentation has never been more important in enabling software engineers to work independently.

Courses overview

Google introduced the technical writing courses, Technical Writing One and Technical Writing Two, in 2015. Since then, thousands of Google software engineers and product managers have taken and enjoyed the courses. In February 2020, we released the courses to the world.

The classes have the following structure:

Students complete self-study work before attending the live class. The self-study work is valuable on its own, even for students who will never attend the live class.
A facilitator guides students through a live class. The live class features practical exercises, class discussion, and extensive peer-to-peer feedback. Note that Google does not lead these live courses but provides extensive material to help facilitators prepare to lead them.

Organizations can choose to host the live classes virtually or in-person.

Technical Writing One

The first course, Technical Writing One, covers the basics of technical writing. Students learn to start thinking about their audience before even putting pen to paper. For example, in one exercise, students are challenged to write instructions for putting toothpaste on a toothbrush. That might sound relatively simple, but here's the catch - your audience has never brushed their teeth before. That's not to say they have bad oral hygiene, but they don't even know what a toothbrush is. The exercise aims to get students to think about documenting a completely new technology.

Another important lesson that Technical Writing One teaches you is how to shorten the sentence length in your documentation and how to edit unnecessarily long sentences. Hopefully once you have taken the course, you might edit the preceding sentence down to something like the following: ~~Another important lesson that~~ Technical Writing One teaches you is to shorten sentences ~~length in your documentation and how to edit unnecessarily long sentences~~.

The course also advocates using lists instead of walls of text, so here, in list form, are some other topics it covers:

Using active voice instead of passive voice.
Revising text into clear paragraphs.
Learning various self-editing techniques.

Technical Writing Two

Technical Writing Two builds on the techniques from the first course and is for those who already know verbs from adverbs. The course encourages students to express their creative side. For example, in one exercise, students find the best way to illustrate technical concepts. Spoiler alert: can you spot any issues with the following diagram?

Figure 1: Finding a website through DNS

Other intermediate techniques the course covers include:

Organizing large doc sets.
Revising and reorganizing text.
Writing accurate descriptions.
Creating tutorials for beginners.

Students take part in interactive exercises and peer review with a lab partner. Technical Writing Two also includes class discussions on documentation types and how to write the dreaded first draft.

Want to know more?

If you would like to teach the courses at your own organization, see the facilitator guides. To review the pre-work and read through the training materials, see the course overviews.

New user features and developer tools to build the helpful home

2020-07-08T10:03:00.000-07:00

Posted by Michele Turner, Director of Product and Smart Home Ecosystem for Google Nest

To create a helpful home experience, we have focused on foundational features necessary to make it easier for people to manage their smart devices. But as people spend more and more time at home during these challenging times, it’s important that we invest in additional ways to work with developers to build a more useful connected home.

Today, at the "Hey Google" Smart Home Virtual Summit, we gave updates on our latest smart home initiatives, talked more in-depth about the new smart home controls in Android 11, and previewed some platform tools that we're investing in to make devices easier to set up and work with Google Assistant.

Smart Home for Entertainment Device support with Google Assistant

As many of us continue to stay home, smart devices are being used a lot more. With the biggest growth coming from entertainment devices, we’re increasing our support in this area with our Smart Home API.

Last year, we launched Google Assistant support for Smart Home for Entertainment Device (SHED) device types and traits, including TVs, remotes, set-top boxes, speakers, soundbars, and even game consoles from top brands like Xbox, Roku, Dish, and LG. And now, we are making these APIs public for any Smart TV, set-top box or game developers to use. SHED gives users the ability to control their favorite entertainment devices from any Assistant-enabled smart display, smart speaker or mobile device.

Smart Home controls in Android 11

With the release of Android 11, coming out later this year, we are introducing a dedicated space for Smart Home controls that users can find quickly, and access any time. We’ve redesigned the power menu to make devices linked to Google Assistant just a button-press away.

Users with the Home App can choose all, or just their favorite controls to be in the space. For partners, you get this for free - there’s no new development work required. We’ll have sliders which will allow you to adjust specific settings, like the temperature of your thermostat in the morning, or how far to open the blinds. You can also customize what devices are visible from the control space and whether these devices can be accessible in your lockscreen.

Improved state reporting and reliability

With Android 11, we want to give users a quick and easy way to check or adjust the devices in their home. And as we continue to add new surfaces for device control, it becomes more critical to ensure we have accurate state. In the coming months, we’ll be introducing tools to measure your reliability and latency to help improve and debug state reporting. Once you hit key targets for reliability and latency, we will shift from a default of querying your state to using report state to render stateful controls. This will reduce query volume on your servers and improve the user experience with accurate device state across multiple surfaces.

In addition to state accuracy, the best user experience comes with strong reliability and low latency. To help achieve both, we launched local execution with the Local Home SDK back in April. As part of the Smart Home platform, local fulfillment can extend your Smart Home Action and routes commands to devices through the local network, benefitting users with reduced latency and higher reliability by removing an additional cloud hop.

To ease the development process, the Local Home platform supports both Chrome and Node.js runtime environments, as well as building and testing of apps on local development machines or personal servers. Once you've deployed your local fulfillment app, users will benefit immediately without having to upgrade hardware or manually update firmware. Nanoleaf and Yeelight have already enabled local execution for their devices. It’s available to all developers through the Actions on Google Console.

Improving linking

Implementing a high quality integration is important - it reduces churn and delights users. Yet, it’s still challenging to get users to discover these features, and we’re doing a couple of things on our end to increase the funnel of users linked to your action. We are excited to launch OAuth-based App Flip on the developer console today. With AppFlip, we streamline the standard account linking flow by flipping users from the Google Home App to the Partner app to gather consent without requiring the users to re-enter their credentials.

To increase awareness of your Action, you will soon be able to initiate the account linking flow within your app. There will also be more opportunities to increase awareness through feature promotion and in-app notification using your app, and we will have more details on discovery and linking opportunities later this year.

Robust monitoring, logging, analytics tools

We know that visibility into the behavior of your smart home integrations is critical, from debugging in early development to detailed analytics in production. To enhance developer productivity, we've integrated with the powerful monitoring and troubleshooting tools available in Google Cloud Platform to provide detailed event logs and usage metrics for smart home projects.

We’ve also recently launched new tools to help developers improve the reliability of their integrations and aid in debugging and resolving issues quickly. You can view aggregate metrics directly in the developer console, or build logs-based metrics to find trends and gain deeper insights into common problems. Google Cloud Platform also enables developers to build custom alerts to quickly identify production issues.

You can also find a new Smart Home Analytics Dashboard accessible from the developer console and pre-populated with charts for common metrics such as Daily Active Users and Request Breakdown — giving you an overall picture of your integration's behavior. This dashboard is powered by new usage and performance metrics in Google Cloud Monitoring, giving you the power to set alerts and be notified if your integration has an issue. Get started today by going to the “Analytics” tab in the Actions console or the Google Cloud console to check out these new logs, metrics, and alerting capabilities for your projects!

Updates to Device Access program

Last year, we announced that we’re moving from the Works with Nest program to Works with Google Assistant and build on a foundation of privacy and data security to ensure users have confidence in how Google and our partners are protecting the consumer’s home data.

As part of that effort, we created the Device Access program to provide a way for partners to integrate directly with Nest devices. To support the Device Access program, we will soon launch the Device Access Console, a self-serve console that guides commercial developers through the different project phases: development, certification and pilot testing, and finally production.

For a commercial developer the console allows them to manage your various projects and integrations. It also provides development guides and trait documentation for all supported Nest devices. Individuals who want to create their own automations with their Nest devices will be able to do so with this console, but only for the homes they are a member of.

Expanding routines

One of the most popular features with Nest users is the ability to automatically trigger routines based on whether users are Home or Away. Later this year, similar functionality will be available with Google Assistant through occupancy detection.

Sleep is also a critical part in maintaining our overall well-being as we stay more at home. Last year we launched the Gentle Sleep & Wake feature with Philips Hue, which slowly brightens or dims the lights at a specific time or can be tied to your morning alarms. Just say, “Turn on Gentle Wake up” to your bedroom speaker to ‘set it and forget it.’ The Light Effects trait is now public so all developers can integrate their native Sleep or Wake experiences - in fact LIFX has recently launched! We encourage you to build and integrate your own unique experiences. We’ll have a larger launch moment later this year when we launch emulated Sleep and Wake effects so that it’ll work out of the box for any smart light!

Another way partners will be able to innovate on our platform and provide more helpful experiences to users is by extending personal routines with custom routines designed by partners, available in the coming months. Developers will be able to create and suggest routines, not just for their devices, but that can work with other devices in a customer’s home. You’ll be able to create solutions for your customers that are based on your core business and bring value to your customers - whether it’s wellness, cleaning, or entertainment. Users will be able to browse and opt-in to approved routines and choose to have Nest and other devices react and participate in that routine.

Our Smart Home efforts have grown significantly over the past several years. We now have integrations with thousands of partners covering all the major connected product categories and devices, and will continue our ambitious goal to build deeper in-home integrations. Be sure to review our docs/samples/videos to learn about all the cool new stuff, and connect with us on our dev communities.

Google Developers Blog

MediaPipe 3D Face Transform

Introduction

Metric 3D Space

Canonical Face Model

Face Transform Estimation

Effect Renderer

Using Face Transform Module

Follow MediaPipe

Acknowledgements

Announcing DevFest 2020

The speakers

The sessions and their mission

The attendees

Google Nest Device Access Console now available for partners and individuals

Google Pay picks Flutter to drive its global product development

Doubling down on the edge with Coral's new accelerator

Applications are open for Google for Startups Accelerator in Japan

Building solutions using the G Suite developer platform

G Suite as a Developer Platform

Instant Motion Tracking with MediaPipe

Instant Motion Tracking

A MediaPipe Pipeline for Instant Motion Tracking

Using the Instant Motion Tracking Solution

Augmenting The World with 3D Stickers and GIFs

Follow MediaPipe

Acknowledgement

Guidance to developers affected by our effort to block less secure browsers and applications

Apps that use embedded frameworks

Browsers

Browser standards

Test for compatibility

Related content

ML Kit Pose Detection Makes Staying Active at Home Easier

A New ML Kit API, Pose Detection

Under The Hood

Real World Applications

Learning to Dance Without Leaving Home

Staying Active Wherever You Are

How to get started?

Join us for Google Assistant Developer Day on October 8

Helping the Haitian economy, one line of code at a time

Introducing Spotlight: Women Techmakers series on career and professional development in tech

Building G Suite Add-ons with your favorite tech stack

Your Add-on, Your way

Next Steps

ChromeOS.dev — A blueprint to build world-class apps and games for Chrome OS

New features for Chrome OS developers

Customizable Linux Terminal

Android Emulator support

Deploy apps directly to Chrome OS

Improved Project Wizard in Android Studio

Updated support from Android lint checks

Unlock your app’s full potential with Chrome OS

Introducing Cast Connect: a better way to integrate Google Cast directly into your Android TV apps

Why is this helpful?

Availability

Digital Ink Recognition in ML Kit

Supports many languages and character sets

Recognizes shapes

Works offline

Runs fast

How to get started

Automatic Deployment of Hugo Sites on Firebase Hosting and Drafts on Cloud Run

Step 1a) Setup A New Hugo Project

Step 1b) Setup Existing Hugo Project

Step 2) Setup Cloud Build Triggers

Step 3) Setup Firebase Hosting

Step 4) (Optional) Change Hugo Theme

Step 5) (Optional) Setup Local Editing

Meet the global students solving local problems with code

1. FreeSpeak - Technical University of Munich, Germany

2. CoronaAI - University of California Berkeley, United States

3. Worthy Walk - National University of Computer & Emerging Sciences, Pakistan

4. Simhae : Deep sea of mind - Soonchunhyang University, South Korea

5. Emergency Response Assistance - University of Education, Winneba (College of Technology Kumasi), Ghana

6. Tulibot - Politeknik Elektronika Negeri Surabaya, Indonesia

7. Picare - The Hong Kong University of Science & Technology, Hong Kong

8. Shareapy - Ho Chi Minh City University of Technology, Vietnam

9. Capstone - Midlands State University, Zimbabwe