Source code
Revision control
Copy as Markdown
Other Tools
# Debugging Firefox with Pernosco
This page is intended to help Firefox/Gecko developers get started using [Pernosco](https://pernos.co/)
to debug Firefox.
Mozilla employees have a paid subscription to Pernosco, and are encouraged to use it.
comes with 5 uploads to see if it is suitable for your workflow.
## Prerequisites
- You need the ability to produce an {code}`rr` trace. See {doc}`debugging_firefox_with_rr`
- Employees, you can also generate traces for Pernosco for some CI job failures via
Treeherder.
## Workflow
At a high level, working with Pernosco consists of recording a trace then uploading
the trace for processing.
### Install your Pernosco configuration
If you are using your own Pernoso account, follow the instructions
Mozilla employees, see the [Pernosco install and configuration doc](https://docs.google.com/document/d/1cb-mkNyJYMudpS0a-R6zxy0kOMIRUSKrxxL-E9zFGKE/edit)
for credentials.
In general, you will want:
- Your email in {code}`~/.config/pernosco/user`
- Your group in {code}`~/.config/pernosco/group`
- Your key in {code}`~/.config/pernosco/user_secret_key`
### Gather a Local Trace
In general, to gather a trace follow the instruction for recording {code}`rr` traces
in {doc}`debugging_firefox_with_rr`. However, to gather a local rr trace that’s
compatible with Pernosco you need to disable some incompatible CPU features (currently
only AVX-512)
On an Intel machine, you can disable this at the command line. To simplify things a
shell script {code}`pernosco-record` can be created like so,
```bash
#!/bin/bash
rr record --disable-avx-512 "$@"
```
replacing plain {code}`rr` in command lines ({code}`mach --debugger pernosco-record`)
On AMD Zen Machines, you can disable AVX-512 and 5 level page tables at boot.
On a Debian derived distro this involves editing {code}`/etc/default/grub` and
adding {code}`clearcpuid=304,no5lvl` to {code}`GRUB_CMDLINE_LINUX_DEFAULT`, then
running {code}`sudo update-grub` after.
### Upload the trace
Find the trace you’re interested in {code}`~/.local/share/rr/<trace>`, and call {code}`pernosco-submit`
```bash
pernosco-submit upload ~/.local/share/rr/<trace> PATH_TO_CENTRAL --title "Name Of Trace"
```
where {code}`PATH TO CENTRAL` is the path to the source of your build, checked out
to the revision matching the build.
The last trace recorded is always symlinked, so you can use {code}`~/.local/share/rr/latest-trace` too.
### Wait for email
After the trace is processed, you'll get an email with a link to your trace. Good
hunting!
## Treeherder Workflow
You can also try to get traces generated via the Treeherder interface. On a failing
test that is eligible, in "Artifacts and Debugging Tools", you'll see a link to
```{image} ../img/record-this-failure.png
:width: 35 %
```
This link brings you to the self-service reproducer:
```{image} ../img/self-service.png
:width: 60 %
```
Select a test and a count, and whether or not chaos-mode should be used, then click
reproduce. At this point there is a **long** wait; the page will notify you when
reproduction has started with an {code}`alert` dialog.