How to effectively A/B test energy consumption for your Android app features

How to effectively A/B test energy consumption for your Android app features


Posted by Mayank Jain – Product Manager, and Yasser Dbeis – Software Engineer; Android studio

Android developers have told us that they are looking for tools to optimize power consumption for various devices on Android.

The new Power Profiler in Android Studio helps Android developers by displaying energy consumption on devices while the app is being used. Understanding energy consumption on Android devices can help Android developers identify and resolve energy consumption issues in their apps. They can run A/B tests to compare the power consumption of different algorithms, features, or even different versions of their app.

The new Power Profiler in Android Studio

The new Power Profiler in Android Studio

Apps optimized for lower power consumption lead to improved battery and thermal performance of the device, which means an improved user experience on Android.

This energy consumption data is made available through the On Device Power Monitor (ODPM) on Pixel 6+ devices, segmented by each subsystem called 'Power Rails'. To see Profileable conductor rails for a list of supported subsystems.

The Power Profiler can help app developers detect problems in several areas:

    • Detecting unoptimized code that consumes more power than necessary.
    • Find background tasks that cause unnecessary CPU usage.
    • Identify wakelocks that keep the device awake when they are not needed.

Once a power consumption issue has been identified, the Power Profiler can be used to test different hypotheses to understand why the app is consuming excessive power. For example, if the problem is caused by background tasks, the developer can try to avoid running the tasks unnecessarily or for extended periods of time. And if the problem is caused by wakelocks, the developer can try to release the wakelocks when the resource is not in use, or use them more wisely. Then compare the power consumption before/after the change using the Power Profiler.

In this blog post, we'll demonstrate a technique that uses A/B testing to understand how your app's power consumption characteristics can change with different versions of the same feature – and how to measure it effectively.

A real-world example of how the Power Profiler can be used to improve the battery life of an app.

Let's assume you have an app that allows users to purchase their favorite movies.

Sample app to demonstrate A/B testing for measuring energy consumption

Sample app to demonstrate A/B testing for measuring energy consumption
Video (c) copyright Blender Foundation | www.bigbuckbunny.org

As your app becomes popular and used by more users, you realize that a high-quality 4K video takes a very long time to load every time the app is launched. Due to its large size, you will want to understand its impact on the device's power consumption.

Originally, this video was in 4K quality with the best intentions, to show the best possible movie highlights to your customers.

This makes you think…

    • Do you really need a 4K video banner on the home screen?
    • Does it make sense to load a 4K video over the network every time your app runs?
    • How will the power consumption characteristics of your app change if you replace the 4K video with something of lower quality (while still maintaining the vibrant look and feel of the video)?

This is a perfect scenario to A/B test for power consumption

With an A/B test, you can test two slightly different variants of the video banner feature and choose the one with the better power consumption characteristics.

Scenario A: Run the app with a 4K video banner on the screen and measure energy consumption

Scenario B: Run the app with a lower resolution video banner on the screen and measure energy consumption

A/B test setup

Let's take a moment and set up our Android Studio profiler to run this A/B test. We need to launch the app and attach the CPU profiler to it and activate a system trace (where the Power Profiler appears).

Step 1

Create a custom “Run Configuration” by clicking the three-dot menu > Edit

Custom execution configuration

Custom execution configuration

Step 2

Then select the “Profiling” tab and make sure “Start this recording at startup” and CPU Activity > System Tracking is selected. Then click “Apply”.

Edit configuration settings

Edit configuration settings

Now simply run the “Low Overhead Profile App Startup Profiling” when you want to run this app from scratch and attach the CPU profiler to it.

Note on precision

The following example scenarios use the entire app startup to estimate energy consumption for the purpose of this blog. However, you can use more advanced techniques to obtain an even more accurate power measurement. Some techniques to try are:

    • Isolate and measure video playback power consumption only after a tap event on the video player
    • Use the tracemarks API to mark the start and stop time for the power measurement timeline – and then measure energy consumption only within that highlighted window

Scenario A

In this scenario, we run the app while 4K video is playing and measure energy consumption during the first 30 seconds. Optionally, we can also run scenario A several times and average the measured values. Once the system trace appears in Android Studio, select the time range of 0-30 seconds in the timeline selection panel and record it as a screenshot to compare with scenario B

Power consumption in scenario A: Playing a 4K video

Power consumption in scenario A: Playing a 4K video

As you can see, the average power consumption of WLAN, CPU cores and memory together is about 1,352 mW (milliwatt)

Now let's compare and contrast how this energy consumption changes in scenario B

Scenario B

In this scenario, we run the app with low-quality video and measure energy consumption during the first 30 seconds. As before, we can optionally run scenario B multiple times and average out the energy consumption measurements. Again, once the system trace appears in Android Studio, select the time range of 0-30 seconds in the Timeline selection panel.

Power consumption in scenario B: Play lower quality video

Power consumption in scenario B: Play lower quality video

The total power consumption by WLAN, CPU Little, CPU Big and CPU Mid & Memory is approximately 741 mW (milliwatt)

Conclusion

All else being equal, Scenario B (with lower quality video) consumed 741 mW of power compared to Scenario A (with 4K video) which required 1,352 mW of power.

Scenario B (lower quality video) consumed 45% less power than Scenario A (4K), while lower quality video produces little to no visual difference in the perceived quality of the app's screen.

As a result of this A/B power consumption test, you conclude that replacing the 4K video with a lower quality video on the home screen of our app not only reduces energy consumption by 45%, but also reduces the required network bandwidth and possibly also the thermal performance of the devices.

If your app's business logic still requires the 4K video to be displayed on the app's screen, you can explore strategies such as:

    • Caching the 4K video during successive runs of the app.
    • Load video via a user step.
    • Load an image initially and only load the video after the screen is fully displayed (delayed loading).

The total power consumption figures in the A/B testing scenario above may seem small, but it shows the techniques app developers can use to effectively A/B test power consumption for their app's features using the Power Profiler in Android Studio.

Next steps

The new Power Profiler is available in Android Studio Hedgehog further. If you would like to know more, please visit the official documentation.