For #6931: Add Performance Build Variants to README.

This commit is contained in:
Michael Comella 2019-12-20 11:38:12 -08:00 committed by Michael Comella
parent faa4136685
commit f272a54d59
1 changed files with 49 additions and 1 deletions

View File

@ -83,7 +83,55 @@ which app id and settings to use. Here is a description of what each means:
- **fenixBeta** is a release build with beta signing which uses the org.mozilla.fenix.beta app id for beta releases to Google Play
- **fenixProduction** is a release build with release signing which uses the org.mozilla.fenix app id for production releases to Google Play
- **fennecProduction** is an experimental build with release signing which uses the org.mozilla.firefox app id and supports upgrading the older Firefox. **WARNING** Pre-production versions of this may result in data loss.
- **forPerformanceTest** is a release build with the debuggable flag set and test activities enabled for running Raptor performance tests
- **forPerformanceTest**: see "Performance Build Variants" below.
#### Performance Build Variants
For accurate performance measurements, read this section!
If you want to analyze performance during **local development**:
- Recommendation: use a `forPerformanceTest` variant with local Leanplum, Adjust, & Sentry API keys: contact the front-end perf group for access to them
- Rationale: `forPerformanceTest` is a release variant with `debuggable` set to true. There are numerous performance-impacting differences between debug and release variants so we need a release variant. To profile, we also need debuggable, which is disabled on other release variants. If API keys are not provided, the SDKs may change their behavior in non-trivial ways.
- Caveats:
- debuggable has a non-trivial & variable impact on performance but is needed to take profiles.
- Random experiment opt-in & feature flags may impact performance (see [perf-frontend-issues#45](https://github.com/mozilla-mobile/perf-frontend-issues/issues/45) for mitigation).
If you want to run **performance tests/benchmarks** in automation or locally:
- Recommendation: production builds. If debuggable is required, use recommendation above but note the caveat above. If your needs are not met, please contact the front-end perf group to identify a new solution.
- Rationale: like the rationale above, we need release variants so the choice is based on the debuggable flag.
For additional context on these recommendations, see [the perf build variant analysis](https://docs.google.com/document/d/1aW-m0HYncTDDiRz_2x6EjcYkjBpL9SHhhYix13Vil30/edit#).
You will **need to sign `forPerformanceTest` variants.** For local development, our recommendation is to add the following configuration to `app/build.gradle`:
```groovy
android { // this line already exists
signingConfigs {
forPerformanceTest {
storeFile file("/Users/<user-name>/.android/debug.keystore")
storePassword "android"
keyAlias "androiddebugkey"
keyPassword "android"
}
}
buildTypes { // this line already exists
// ...
forPerformanceTest releaseTemplate >> { // this line already exists.
// ...
signingConfig signingConfigs.forPerformanceTest
}
}
}
```
Where `<user-name>` is your OS user name. This sample assumes you are using macOS: if you're on another OS, please update the path appropriately.
This recommendation will let you use AS just like you do with debug builds but **please do not check in these changes.**
See [perf-frontend-issues#44](https://github.com/mozilla-mobile/perf-frontend-issues/issues/44) for efforts to make performance signing easier.
## Pre-push hooks
To reduce review turn-around time, we'd like all pushes to run tests locally. We'd