47 lines
3.6 KiB
Markdown
47 lines
3.6 KiB
Markdown
# Crash Monitoring
|
|
|
|
## Important
|
|
* The main goal here is not to file an issue for every single distinct crash report, but to find regressions of new problems that need to be addressed.
|
|
* Once you're familiar with the process this should not take more than 10 mins in the morning.
|
|
* Quick links that you should check everyday [Sentry Query](https://sentry.io/organizations/mozilla/issues/?groupStatsPeriod=auto&page=0&project=6295546&query=is%3Aunresolved+level%3Afatal+firstSeen%3A-1w&sort=freq&statsPeriod=14d) and [Socorro Query](https://crash-stats.mozilla.org/topcrashers/?product=Fenix&days=3&_range_type=build&process_type=any)
|
|
|
|
## Things to note before starting:
|
|
* Since we are focused on Java crashes, Sentry currently is the better choice.
|
|
* Ignore `level:info` issues (blue labels in Sentry). These issues are informational only.
|
|
|
|
## What to do when monitoring crashes:
|
|
* Look at Sentry Fenix-nightly overview. Go though trending issues. [Sentry Dashboard](https://sentry.io/organizations/mozilla/projects/fenix-nightly/?project=6295546).
|
|
* Look at Sentry custom search [Sentry Query](https://sentry.io/organizations/mozilla/issues/?groupStatsPeriod=auto&page=0&project=6295546&query=is%3Aunresolved+level%3Afatal+firstSeen%3A-1w&sort=freq&statsPeriod=14d).
|
|
* Sign up for https://groups.google.com/a/mozilla.org/g/stability to get a daily email on the stability of Fenix.
|
|
|
|
## How to determine a crash requires actions:
|
|
* Crashes that have level either Fatal or Error.
|
|
* Crashes that are occurring on latest Fenix and A-C versions.
|
|
* Crashes that are spiking.
|
|
* Crashes that are new.
|
|
* Crashes that happen repeatedly and often.
|
|
* Crashes that are happening to multiple users.
|
|
|
|
## When a crash report requires actions:
|
|
* Is this a crash due to a recent change? If so, contact the developer.
|
|
* The histogram on the right side can help determine this along with checking the Firefox-Beta and Firefox Sentry products.
|
|
* Triage the crash to determine if the issue is real and requires a Bugzilla issue to track it.
|
|
* When filing an issue add a link to it as a comment in the Sentry crash for the products (nightly, beta, release) where the crash appears.
|
|
* Notify the relevant teams on Slack/Matrix that there's a new crash in Nightly that needs urgent attention, e.g. **#synced-client-integrations** for all things involving application services (A-S), **#nimbus-rust-sdk** for Nimbus, and **[GeckoView on Matrix](https://chat.mozilla.org/#/room/#geckoview:mozilla.org)**.
|
|
|
|
## What can you do to help when not monitoring crashes
|
|
* If you recently landed a new module / change that is significant, contact the crash monitor so they are aware of it.
|
|
|
|
## Crash monitoring with Socorro
|
|
* Look at [Top Crashers for Fenix Nightly](https://crash-stats.mozilla.org/topcrashers/?product=Fenix&days=3&_range_type=build&process_type=any) for reports on Nightly builds.
|
|
* This will return zero results if GV build ID is greater than 3 days old. Change to the 7 day view and ask #releaseduty-mobile in Slack about the GV upgrade task being broken.
|
|
* Use Sentry and Bugzilla to determine if the crash has already been reported. If a Bugzilla bug has been filed for a crash, a link to the bug should be listed in Socorro's "Bugzilla IDs" column.
|
|
* If the crash is new and the volume is high, then consider filing an issue using the crash-stats Bugzilla tab from a crash ID.
|
|
|
|
## How to file bugs in Bugzilla from Socorro
|
|
* Go to the crash report.
|
|
* "Reports" tab.
|
|
* Select a report.
|
|
* Select the "Bugzilla" tab.
|
|
* Just above the header "Related Bugs", you will see a "Create a bug" option. Select the product that matches the crash report.
|