Source code
Revision control
Copy as Markdown
Other Tools
# Remote debugging backward compatibility tests
The DevTools backward compatibility tests are automated smoke tests which
verify that a client Firefox can connect to and debug a server Firefox which has
either the `same` version, or is older (`beta`, `release`, …).
## Technical overview
The tests are written as regular aboutdebugging browser-chrome mochitests. They
can be found in `devtools/client/aboutdebugging/test/browser/browser_backward*`.
When run as regular mochitests however, they early return without failing. To
properly run the tests, you need to use `./mach devtools-compat-test`, which
will setup the server Firefox and then run the client test.
The server is setup by the python module found in
`devtools/client/aboutdebugging/test/backward_compat_test_server`. It handles
the logic to download, install and start the server. This server will be driven
by Marionette, and a few command wrappers are exposed through control.py.
## Running the tests
Locally you can run the whole test suite with:
```
./mach devtools-compat-test
```
By default this will create a Firefox server using your current Firefox build.
You can run the suite against our other channels: `release`, `beta`,
`devedition`, `nightly`, `local` (the default).
```
./mach devtools-compat-test --server {channel}
```
In this case, the devtools-compat-test task will download Firefox for the
server. The downloaded and installed binaries are cached to speedup successive
runs.
Use `--headless` to hide all browser windows.
```
./mach devtools-compat-test --headless
```
Additional parameters are forwarded to the mochitest harness, such as `--repeat`,
or specific test path:
```
./mach devtools-compat-test \
devtools/client/aboutdebugging/test/browser/browser_backward_compat_connect.js \
--repeat 5
```
Running the tests with `./mach mochitest` or `./mach test` is not an error, but
as mentioned earlier the test will simply early return. Make sure to explicitly
use `./mach devtools-compat-test` to run the tests locally.
## Daily cron on CI
Three tasks run once a day on mozilla-central, through the
`devtools-backward-compat` cron job:
| Treeherder symbol | Server |
|---|---|
| `DTbc(same)` | the build under test, same version |
| `DTbc(beta)` | latest Firefox Beta |
| `DTbc(release)` | latest Firefox Release |
They will be scheduled in regular mozilla-central jobs. You can look for
previous runs by [filtering on devtools-compat](https://treeherder.mozilla.org/jobs?repo=mozilla-central&searchStr=devtools-compat).
## Manual Try push
If you want to push to try manually, you can use `./mach try fuzzy --full` and
select the `devtools-compat-*` jobs, or simply:
```
./mach try fuzzy --full -q devtools-compat
```
## Adding new tests
The tests are very similar to regular DevTools browser chrome mochitests, and
share the same helpers as the other aboutdebugging tests. To add a new test,
simply add it to devtools/client/aboutdebugging/test/browser/browser_backward_compat.toml.
## Adding a server-side action
Server-side actions are commands on the control channel. They allow to drive the
server browser via Marionette commands (examples: `_cmd_navigate`,
`_cmd_install_extension`).
To create a new action, add a `_cmd_<name>` method to `ControlServer` in
`backward_compat_test_server/control.py`, then call it from a test with
`runDevToolsServerCommand(config, "<name>", args)`.
For instance for `"install-extension"`:
```
const { addonId } = await runDevToolsServerCommand(
config,
"install-extension"
);
```