Android Macrobenchmark: Compare Releases Without Debug-Build Noise

Choose an externally observable journey

Use the Macrobenchmark guide to configure the benchmark module, target package and supported build. Select a journey with a clear start, action and completion condition.

Avoid benchmarking a debug build and presenting the result as production performance. The profiling guide explains release/profileable versus debuggable trade-offs.

Control state explicitly

A benchmark that waits on an unpredictable production endpoint can measure server variation rather than your code change.

Make the test wait for real readiness

Use stable UI selectors and meaningful completion conditions. A fixed sleep can either waste time or finish before a slow device is ready, producing inconsistent results.

Keep test-only data and accounts isolated from real users and production transactions.

Inspect traces as well as summaries

A regression summary tells you that performance changed; a trace helps explain where. Retain representative slow iterations instead of discarding them as inconvenient outliers without investigation.

Compare distributions and normal noise before declaring a small difference meaningful.

Use benchmarks as release evidence

Add the journey to a repeatable review or CI process where hardware conditions can be managed. APKLint's Performance Checker can help define the scenario and acceptance questions, but it does not run Macrobenchmark. A defensible result includes the test code, target artifact, environment and measured output, with any uncontrolled variables clearly stated.