Source code

Revision control

Other Tools

1
Task Kinds
2
==========
3
4
This section lists and documents the available task kinds.
5
6
build
7
-----
8
9
Builds are tasks that produce an installer or other output that can be run by
10
users or automated tests. This is more restrictive than most definitions of
11
"build" in a Mozilla context: it does not include tasks that run build-like
12
actions for static analysis or to produce instrumented artifacts.
13
14
build-fat-aar
15
-------------
16
17
Build architecture-independent GeckoView AAR (Android ARchive) files. This build-like tasks is an
18
artifact build (ARMv7, but this is arbitrary) that itself depends on arch-specific Android build
19
jobs. It fetches arch-specific AAR files, extracts arch-specific libraries and preference files,
20
and then assembles a multi-architecture "fat AAR". Downstream consumers are expected to use
21
per-ABI feature splits to produce arch-specific APKs.
22
23
If you want to run this task locally, you need to specify these environment variable:
24
- MOZ_ANDROID_FAT_AAR_ARCHITECTURES: must be a comma-separated list of architecture.
25
Eg: "armeabi-v7a,arm64-v8a,x86,x86_64".
26
- each of MOZ_ANDROID_FAT_AAR_ARM64_V8A, MOZ_ANDROID_FAT_AAR_ARMEABI_V7A,
27
MOZ_ANDROID_FAT_AAR_X86, MOZ_ANDROID_FAT_AAR_X86_64 must be a path relative to
28
MOZ_FETCHES_DIR.
29
30
build-signing
31
-------------
32
33
Many builds must be signed. The build-signing task takes the unsigned `build`
34
kind artifacts and passes them through signingscriptworker to a signing server
35
and returns signed results.
36
37
artifact-build
38
--------------
39
40
This kind performs an artifact build: one based on precompiled binaries
41
discovered via the TaskCluster index. This task verifies that such builds
42
continue to work correctly.
43
44
hazard
45
------
46
47
Hazard builds are similar to "regular' builds, but use a compiler extension to
48
extract a bunch of data from the build and then analyze that data looking for
49
hazardous behaviors.
50
51
l10n
52
----
53
54
The l10n kind takes the last published nightly build, and generates localized builds
55
from it. You can read more about how to trigger these on the `wiki
57
58
nightly-l10n
59
------------
60
61
The nightly l10n kind repacks a specific nightly build (from the same source code)
62
in order to provide localized versions of the same source.
63
64
nightly-l10n-signing
65
--------------------
66
67
The nightly l10n signing kind takes artifacts from the nightly-l10n kind and
68
passes them to signing servers to have their contents signed appropriately, based
69
on an appropriate signing format. One signing job is created for each nightly-l10n
70
job (usually chunked).
71
72
source-test
73
-----------
74
75
Source-tests are tasks that run directly from the Gecko source. This can include linting,
76
unit tests, source-code analysis, or measurement work. While source-test tasks run from
77
a source checkout, it is still possible for them to depend on a build artifact, though
78
often they do not.
79
80
code-review
81
-----------
82
83
Publish issues found by source-test tasks on Phabricator.
84
This is a part of Release Management code review Bot.
85
86
upload-symbols
87
--------------
88
89
Upload-symbols tasks run after builds and upload the symbols files generated by
90
build tasks to Socorro for later use in crash analysis.
91
92
upload-generated-sources
93
------------------------
94
95
Upload-generated-sources tasks run after builds and upload source files that were generated as part of the build process to an s3 bucket for later use in links from crash reports or when debugging shipped builds.
96
97
valgrind
98
--------
99
100
Valgrind tasks produce builds instrumented by valgrind.
101
102
searchfox
103
---------
104
105
Searchfox builds generate C++ index data for Searchfox.
106
107
static-analysis-autotest
108
------------------------
109
110
Static analysis autotest utility in order to be sure that there is no regression
111
when upgrading utilities that impact static-analysis.
112
113
toolchain
114
---------
115
116
Toolchain builds create the compiler toolchains used to build Firefox. These
117
will eventually be dependencies of the builds themselves, but for the moment
118
are run manually via try pushes and the results uploaded to tooltool.
119
120
spidermonkey
121
------------
122
123
Spidermonkey tasks check out the full gecko source tree, then compile only the
124
spidermonkey portion. Each task runs specific tests after the build.
125
126
test
127
----
128
129
The ``desktop-test`` kind defines tests for builds. Its ``tests.yml`` defines
130
the full suite of desktop tests and their particulars, leaving it to the
131
transforms to determine how those particulars apply to the various platforms.
132
133
The process of generating tests goes like this, based on a set of YAML files
134
named in ``kind.yml``:
135
136
* For each build task, determine the related test platforms based on the build
137
platform. For example, a Windows 2010 build might be tested on Windows 7
138
and Windows 10. Each test platform specifies "test sets" indicating which
139
tests to run. This is configured in the file named
140
``test-platforms.yml``.
141
142
* Each test set is expanded to a list of tests to run. This is configured in
143
the file named by ``test-sets.yml``. A platform may specify several test
144
sets, in which case the union of those sets is used.
145
146
* Each named test is looked up in the file named by ``tests.yml`` to find a
147
test description. This test description indicates what the test does, how
148
it is reported to treeherder, and how to perform the test, all in a
149
platform-independent fashion.
150
151
* Each test description is converted into one or more tasks. This is
152
performed by a sequence of transforms defined in the ``transforms`` key in
153
``kind.yml``. See :doc:`transforms`: for more information on these
154
transforms.
155
156
* The resulting tasks become a part of the task graph.
157
158
.. important::
159
160
This process generates *all* test jobs, regardless of tree or try syntax.
161
It is up to a later stages of the task-graph generation (the target set and
162
optimization) to select the tests that will actually be performed.
163
164
docker-image
165
------------
166
167
Tasks of the ``docker-image`` kind build the Docker images in which other
168
Docker tasks run.
169
170
The tasks to generate each docker image have predictable labels:
171
``build-docker-image-<name>``.
172
173
Docker images are built from subdirectories of ``taskcluster/docker``, using
174
``docker build``. There is currently no capability for one Docker image to
175
depend on another in-tree docker image, without uploading the latter to a
176
Docker repository.
177
178
balrog
179
------
180
181
Balrog tasks are responsible for submitting metadata to our update server (Balrog).
182
They are typically downstream of a beetmover job that moves signed MARs somewhere
183
(eg: beetmover and beetmover-l10n for releases, beetmover-repackage for nightlies).
184
185
beetmover
186
---------
187
188
Beetmover, takes specific artifacts, "Beets", and pushes them to a location outside
189
of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the
190
process determines the final location and a "pretty" name (versioned product name)
191
192
beetmover-l10n
193
--------------
194
195
Beetmover L10n, takes specific artifacts, "Beets", and pushes them to a location outside
196
of Taskcluster's task artifacts, (archive.mozilla.org as one place) and in the
197
process determines the final location and a "pretty" name (versioned product name)
198
This separate kind uses logic specific to localized artifacts, such as including
199
the language in the final artifact names.
200
201
beetmover-repackage
202
-------------------
203
204
Beetmover-repackage is beetmover but for tasks that need an intermediate step
205
between signing and packaging, such as OSX. For more details see the definitions
206
of the Beetmover kind above and the repackage kind below.
207
208
release-beetmover-push-to-release
209
---------------------------------
210
211
release-beetmover-push-to-release publishes promoted releases from the
212
candidates directory to the release directory. This is part of release
213
promotion.
214
215
beetmover-snap
216
--------------
217
Beetmover-source publishes Ubuntu's snap. This is part of release promotion.
218
219
beetmover-source
220
----------------
221
Beetmover-source publishes release source. This is part of release promotion.
222
223
beetmover-geckoview
224
-------------------
225
Beetmover-geckoview publishes the Android library called "geckoview".
226
227
condprof
228
--------
229
condprof creates and updates realistic profiles.
230
231
release-source-checksums-signing
232
--------------------------------
233
release-source-checksums-signing take as input the checksums file generated by
234
source-related beetmover task and sign it via the signing scriptworkers.
235
Returns the same file signed and additional detached signature.
236
237
beetmover-checksums
238
-------------------
239
Beetmover, takes specific artifact checksums and pushes it to a location outside
240
of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the
241
process determines the final location and "pretty" names it (version product name)
242
243
release-beetmover-source-checksums
244
----------------------------------
245
Beetmover, takes source specific artifact checksums and pushes it to a location outside
246
of Taskcluster's task artifacts (archive.mozilla.org as one place) and in the
247
process determines the final location and "pretty" names it (version product name)
248
249
push-apk
250
--------
251
PushApk publishes Android packages onto Google Play Store. Jobs of this kind take
252
all the signed multi-locales (aka "multi") APKs for a given release and upload them
253
all at once.
254
255
push-apk-checks
256
---------------
257
Runs the checks done in push-apk to ensure APKs are sane before submitting them
258
259
release-balrog-submit-toplevel
260
------------------------------
261
Toplevel tasks are responsible for submitting metadata to Balrog that is not specific to any
262
particular platform+locale. For example: fileUrl templates, versions, and platform aliases.
263
264
Toplevel tasks are also responsible for updating test channel rules to point at the Release
265
being generated.
266
267
release-secondary-balrog-submit-toplevel
268
----------------------------------------
269
Performs the same function as `release-balrog-submit-toplevel`, but against the beta channel
270
during RC builds.
271
272
release-balrog-scheduling
273
-------------------------
274
Schedules a Release for shipping in Balrog. If a `release_eta` was provided when starting the Release,
275
it will be scheduled to go live at that day and time.
276
277
release-secondary-balrog-scheduling
278
-----------------------------------
279
Performs the same function as `release-balrog-scheduling`, except for the beta channel as part of RC
280
Releases.
281
282
release-binary-transparency
283
---------------------------
284
Binary transparency creates a publicly verifiable log of binary shas for downstream
286
287
release-snap-repackage
288
----------------------
289
Generate an installer using Ubuntu's Snap format.
290
291
release-snap-push
292
-----------------
293
Pushes Snap repackage on Snap store.
294
295
release-secondary-snap-push
296
---------------------------
297
Performs the same function as `release-snap-push`, except for the beta channel as part of RC
298
Releases.
299
300
release-notify-av-announce
301
--------------------------
302
Notify anti-virus vendors when a release is likely shipping.
303
304
release-notify-push
305
-------------------
306
Notify when a release has been pushed to CDNs.
307
308
release-notify-ship
309
-------------------
310
Notify when a release has been shipped.
311
312
release-secondary-notify-ship
313
-----------------------------
314
Notify when an RC release has been shipped to the beta channel.
315
316
release-notify-promote
317
----------------------
318
Notify when a release has been promoted.
319
320
release-notify-started
321
----------------------
322
Notify when a release has been started.
323
324
release-bouncer-sub
325
-------------------
326
Submits bouncer updates for releases.
327
328
release-bouncer-sub-nazgul
329
--------------------------
330
Submits bouncer updates for releases, using new implementation of bouncer-admin (Nazgul).
331
332
release-mark-as-shipped
333
-----------------------
334
Marks releases as shipped in Ship-It v1
335
336
release-bouncer-aliases
337
-----------------------
338
Update Bouncer's (download.mozilla.org) "latest" aliases.
339
340
cron-bouncer-check
341
------------------
342
Checks Bouncer (download.mozilla.org) uptake.
343
344
bouncer-locations
345
-----------------
346
Updates nightly bouncer locations for version bump
347
348
release-bouncer-check
349
---------------------
350
Checks Bouncer (download.mozilla.org) uptake as part of the release tasks.
351
352
release-generate-checksums
353
--------------------------
354
Generate the per-release checksums along with the summaries
355
356
release-generate-checksums-signing
357
----------------------------------
358
Sign the pre-release checksums produced by the above task
359
360
release-generate-checksums-beetmover
361
------------------------------------
362
Submit to S3 the artifacts produced by the release-checksums task and its signing counterpart.
363
364
release-final-verify
365
--------------------
366
Verifies the contents and package of release update MARs.
367
368
release-secondary-final-verify
369
------------------------------
370
Verifies the contents and package of release update MARs for RC releases.
371
372
release-push-langpacks
373
-------------------------------
374
Publishes language packs onto addons.mozilla.org.
375
376
release-beetmover-signed-langpacks
377
----------------------------------
378
Publishes signed langpacks to archive.mozilla.org
379
380
release-beetmover-signed-langpacks-checksums
381
--------------------------------------------
382
Publishes signed langpacks to archive.mozilla.org
383
384
release-update-verify
385
---------------------
386
Verifies the contents and package of release update MARs.
387
release-secondary-update-verify
388
-------------------------------
389
Verifies the contents and package of release update MARs.
390
391
release-update-verify-next
392
--------------------------
393
Verifies the contents and package of release and updare MARs from the previous ESR release.
394
395
release-update-verify-config
396
----------------------------
397
Creates configs for release-update-verify tasks
398
399
release-secondary-update-verify-config
400
--------------------------------------
401
Creates configs for release-secondary-update-verify tasks
402
403
release-update-verify-config-next
404
---------------------------------
405
Creates configs for release-update-verify-next tasks
406
407
release-updates-builder
408
-----------------------
409
Top level Balrog blob submission & patcher/update verify config updates.
410
411
release-version-bump
412
--------------------
413
Bumps to the next version.
414
415
release-source
416
--------------
417
Generates source for the release
418
419
release-source-signing
420
----------------------
421
Signs source for the release
422
423
release-partner-repack
424
----------------------
425
Generates customized versions of releases for partners.
426
427
release-partner-repack-chunking-dummy
428
-------------------------------------
429
Chunks the partner repacks by locale.
430
431
release-partner-repack-signing
432
------------------------------
433
Internal signing of partner repacks.
434
435
release-partner-repack-repackage
436
--------------------------------
437
Repackaging of partner repacks.
438
439
release-partner-repack-repackage-signing
440
----------------------------------------
441
External signing of partner repacks.
442
443
release-partner-repack-beetmover
444
--------------------------------
445
Moves the partner repacks to S3 buckets.
446
447
release-partner-repack-bouncer-sub
448
----------------------------------
449
Sets up bouncer products for partners.
450
451
452
release-early-tagging
453
---------------------
454
Utilises treescript to perform tagging that should happen near the start of a release.
455
456
release-eme-free-repack
457
-----------------------
458
Generates customized versions of releases for eme-free repacks.
459
460
release-eme-free-repack-signing
461
-------------------------------
462
Internal signing of eme-free repacks
463
464
release-eme-free-repack-repackage
465
---------------------------------
466
Repackaging of eme-free repacks.
467
468
release-eme-free-repack-repackage-signing
469
-----------------------------------------
470
External signing of eme-free repacks.
471
472
release-eme-free-repack-beetmover
473
---------------------------------
474
Moves the eme-free repacks to S3 buckets.
475
476
release-eme-free-repack-beetmover-checksums
477
-------------------------------------------
478
Moves the beetmover checksum for eme-free repacks to S3 buckets.
479
480
repackage
481
---------
482
Repackage tasks take a signed output and package them up into something suitable
483
for shipping to our users. For example, on OSX we return a tarball as the signed output
484
and this task would package that up as an Apple Disk Image (.dmg)
485
486
repackage-l10n
487
--------------
488
Repackage-L10n is a ```Repackage``` task split up to be suitable for use after l10n repacks.
489
490
491
repackage-signing
492
-----------------
493
Repackage-signing take the repackaged installers (windows) and signs them.
494
495
repackage-signing-l10n
496
----------------------
497
Repackage-signing-l10n take the repackaged installers (windows) and signs them for localized versions.
498
499
mar-signing
500
-----------
501
Mar-signing takes the complete update MARs and signs them.
502
503
mar-signing-l10n
504
----------------
505
Mar-signing-l10n takes the complete update MARs and signs them for localized versions.
506
507
mar-signing-autograph-stage
508
---------------------------
509
These tasks are only to test autograph-stage, when the autograph team asks for their staging environment to be tested.
510
511
repackage-msi
512
-------------
513
Repackage-msi takes the signed full installer and produces an msi installer (that wraps the full installer)
514
Using the ```./mach repackage``` command
515
516
repackage-signing-msi
517
---------------------
518
Repackage-signing-msi takes the repackaged msi installers and signs them.
519
520
repo-update
521
-----------
522
Repo-Update tasks are tasks that perform some action on the project repo itself,
523
in order to update its state in some way.
524
525
pipfile-update
526
--------------
527
Pipfile-update tasks generate update Pipfile.lock for in-tree Pipfiles, and attach
528
patches with the updates to Phabricator.
529
530
partials
531
--------
532
Partials takes the complete.mar files produced in previous tasks and generates partial
533
updates between previous nightly releases and the new one. Requires a release_history
534
in the parameters. See ``mach release-history`` if doing this manually.
535
536
partials-signing
537
----------------
538
Partials-signing takes the partial updates produced in Partials and signs them.
539
540
post-balrog-dummy
541
-----------------
542
Dummy tasks to consolidate balrog dependencies to avoid taskcluster limits on number of dependencies per task.
543
544
post-beetmover-dummy
545
--------------------
546
Dummy tasks to consolidate beetmover dependencies to avoid taskcluster limits on number of dependencies per task.
547
548
post-beetmover-checksums-dummy
549
------------------------------
550
Dummy tasks to consolidate beetmover-checksums dependencies to avoid taskcluster limits on number of dependencies per task.
551
552
post-langpack-dummy
553
-------------------
554
Dummy tasks to consolidate language pack beetmover dependencies to avoid taskcluster limits on number of dependencies per task.
555
556
fetch
557
-----
558
Tasks that obtain something from a remote service and re-expose it as a
559
task artifact. These tasks are used to effectively cache and re-host
560
remote content so it is reliably and deterministically available.
561
562
packages
563
--------
564
Tasks used to build packages for use in docker images.
565
566
diffoscope
567
----------
568
Tasks used to compare pairs of Firefox builds using https://diffoscope.org/.
569
As of writing, this is mainly meant to be used in try builds, by editing
570
taskcluster/ci/diffoscope/kind.yml for your needs.
571
572
addon
573
-----
574
Tasks used to build/package add-ons.
575
576
openh264-plugin
577
---------------
578
Tasks used to build the openh264 plugin.
579
580
openh264-signing
581
----------------
582
Signing for the openh264 plugin.
583
584
webrender
585
---------
586
Tasks used to do testing of WebRender standalone (without gecko). The
587
WebRender code lives in gfx/wr and has its own testing infrastructure.
588
589
github-sync
590
------------
591
Tasks used to do synchronize parts of Gecko that have downstream GitHub
592
repositories.
593
594
instrumented-build
595
------------------
596
Tasks that generate builds with PGO instrumentation enabled. This is an
597
intermediate build that can be used to generate profiling information for a
598
final PGO build. This is the 1st stage of the full 3-step PGO process.
599
600
generate-profile
601
----------------
602
Tasks that take a build configured for PGO and run the binary against a sample
603
set to generate profile data. This is the 2nd stage of the full 3-step PGO
604
process.
605
606
geckodriver-repack
607
------------------
608
Tasks to repackage the geckodriver binary from a build tasks's common
609
test archive into it's own archive.
610
611
geckodriver-signing
612
-------------------
613
Signing for geckodriver binary.
614
615
visual-metrics
616
--------------
617
Tasks that compute visual performance metrics from videos and images captured
618
by other tasks.
619
620
visual-metrics-dep
621
------------------
622
Tasks that compute visual performance metrics from videos and images captured
623
by another task that produces a jobs.json artifact
624
625
iris
626
----
627
Iris testing suite
628
629
maybe-release
630
-------------
631
A shipitscript task that does the following:
632
633
1. Checks if automated releases are disabled
634
2. Checks if the changes between the current revision and the previous releases
635
revision are considered "worthwhile" for a new release.
636
3. Triggers the release via ship-it, which will then create an action task.
637
638
l10n-bump
639
---------
640
Cron-driven tasks that bump l10n-changesets files in-tree, using data from the l10n dashboard.
641
642
system-symbols
643
--------------
644
Generate missing macOS and windows system symbols from crash reports.
645
646
system-symbols-upload
647
---------------------
648
Upload macOS and windows system symbols to tecken.