Posted by Terence Zhang – Developer Relations Engineer
At Google I/O, we unveiled a vision for Android, reimagined with AI at its core. As Android developers, you are at the forefront of this exciting shift. By embracing Generative AI (Gen AI), you are creating a new breed of Android apps that deliver unparalleled experiences and amazing features to your users.
Gemini models power new generative AI apps, both via the cloud and directly on the device. You can now build with Gen AI using our most capable models via the cloud with the Google AI client SDK or Vertex AI for Firebase in your Android apps. For on the device, Gemini Nano is our recommended model. We have also integrated Gen AI into developer tools – Gemini in Android Studio increases the productivity of your developers.
Let's take a look at the most important announcements for AI on Android of this year's I/O sessions in more detail!
#1: Build AI apps using cloud-based Gemini models
To start your Gen AI journey, design the prompts for your use case using Google AI Studio. Once you’re happy with your prompts, use the Gemini API directly in your app to access Google’s latest models, including Gemini 1.5 Pro and 1.5 Flash, both with one million token context windows (with two million available via the waitlist for Gemini 1.5 Pro).
If you’re interested in learning more about and experimenting with the Gemini API, the Google AI SDK for Android is a great place to start. To integrate Gemini into your production app, consider using Vertex AI for Firebase (currently in Preview, with a full release planned for Fall 2024). This platform provides a streamlined way to build and deploy generative AI functions.
We are also launching the first Gemini API Developer Contest (Terms and Conditions apply). This is the best time to build an app that integrates the Gemini API and win incredible prizes! Custom Delorean anyone?
#2: Use Gemini Nano for Gen AI on the device
While cloud-based models are very capable, on-device inference enables offline inference and low latency responses, and ensures that data does not leave the device.
At I/O, we announced that Gemini Nano will gain multimodal capabilities, allowing devices to understand context beyond text – such as images, sounds, and spoken language. This will help powerful experiences like Talkbackto help people who are blind or have low vision interact with their devices using touch and voice feedback. Gemini Nano with Multimodality will be available later this year, starting with Google Pixel devices.
We also shared more about AICore, a system service that manages on-device foundation models, enabling Gemini Nano to perform on-device inference. AICore provides developers with a streamlined API to execute Gen AI workloads with virtually no impact on binary size, while centralizing runtime, delivery, and critical security components for Gemini Nano. This eliminates the need for developers to maintain their own models and allows many applications to share access to Gemini Nano on the same device.
Gemini Nano is already transforming key Google apps, including Messages and recorder to enable Smart Compose and recording summarization capabilities, respectively. Outside of Google apps, we’re actively working with developers who have compelling on-device Gen AI use cases and have signed up for our Early Access Program (EAP), including Patreon, Grammarly, and Adobe.
Adobe is one of these pioneers, and they are exploring Gemini Nano to enable on-device processing for part of their AI assistant in Acrobat, enabling one-click summaries and enabling users to interact with documents. By strategically combining Gen AI models on-device and in the cloud, Adobe optimizes performance, cost, and accessibility. Simpler tasks like summarizing and suggesting initial questions are handled on-device, enabling offline access and cost savings. More complex tasks like answering user questions are processed in the cloud, enabling an efficient and seamless user experience.
This is just the beginning – later this year we’ll be investing heavily to enable even more developers to get started with them.
For more information about building with Gen AI, check out the I/O talks Android on-device GenAI under the hood And Add Generative AI to Your Android App with the Gemini APItogether with our new documentation.
#3: Use Gemini in Android Studio to be more productive
In addition to directly controlling features in your app, we've also integrated Gemini into developer tools. Gemini in Android Studio is your Android coding partner, bringing the power of Gemini to your developer workflow. Thanks to your feedback since previewing it as Studio Bot at Google I/O last year, we've continued to evolve our models, expanding them to over 200 countries and territoriesand now incorporating this experience into stable builds of Android Studio.
During Google I/O, we previewed a number of features that are available to try in the Android Studio Koala preview release, such as natural language code suggestions and AI-powered analysis for App Quality Insights. We also got a Early preview of multimodal input with Gemini 1.5 Prowhich allows you to upload images as part of your AI queries. This way, Gemini can help you build fully functional Compose UIs from a wireframe sketch.
You can read more about the updates hereand make sure you check it out What's new in Android developer tools?.