index.rst - mozsearch

Enable keyboard shortcuts

================================

Vendoring Third Party Components

================================

The firefox source tree vendors many third party dependencies. The build system

provides a normalized way to keep track of:

1. The upstream source license, location and revision

2. (Optionally) The upstream source modification, including

   1. Mozilla-specific patches

   2. Custom update actions, such as excluding some files, moving files around

      etc.

This is done through a descriptive ``moz.yaml`` file added to the third

party sources, and the use of:

.. code-block:: sh

    ./mach vendor [options] ./path/to/moz.yaml

to interact with it.

Template ``moz.yaml`` file

==========================

.. code-block:: yaml

    # All fields are mandatory unless otherwise noted

    schema: 1

    # Version of this schema

    bugzilla:

        # Bugzilla product and component for this directory and subdirectories.

        product: product name

        component: component name

    origin:

        name: name of the package

        # eg. nestegg

        description: short (one line) description

        url: package's homepage url

        # Usually different from repository url

        release: identifier

        # Human-readable identifier for this version/release

        # Generally "version NNN", "tag SSS", "bookmark SSS"

        revision: sha

        # Revision to pull in

        # Must be a long or short commit SHA (long preferred)

        license: MPL-2.0

        # Multiple licenses can be specified (as a YAML list)

        # Where possible using the mnemonic from https://spdx.org/licenses/

        # From a list of approved licenses (to be determined)

        # A "LICENSE" file must exist in the destination directory after patches are applied

        license-file: COPYING

        # optional

        # explicit name of the License file if it's not one of the supported

        # hard-coded value.

    vendoring:

        # optional

        # Information needed to update the library automatically.

        url: source url (generally repository clone url)

        # eg. https://github.com/kinetiknz/nestegg.git

        # Any repository host can be specified here, however initially we'll only support

        # automated vendoring from github.com and googlesource.com, to be extended later.

        source-hosting: gitlab

        # name of the infrastructure used to host the sources. Can be one of

        # gitlab, angle, googlesource, codeberg, github or git. The later is

        # more generic but less efficient.

        patches:

            - file

            - path/to/file

            - path/*.patch

        # optional

        # List of patch files to apply after vendoring. Applied in the order specified, and

        # alphabetically if globbing is used. Patches must apply cleanly before changes are

        # pushed

        # All patch files are implicitly added to the keep file list.

        keep:

            - file

            - path/to/file

            - another/path

            - *.mozilla

        # optional

        # List of files in mozilla-central that are not deleted while vendoring

        # Implicitly contains "moz.yaml", any files referenced as patches

        exclude:

            - file

            - path/to/file

            - another/path

            - docs

            - src/*.test

        # optional

        # Files/paths that will not be vendored from source repository

        # Implicitly contains ".git", and ".gitignore"

        include:

            - file

            - path/to/file

            - another/path

            - docs/LICENSE.*

        # optional

        # Files/paths that will always be vendored, even if they would

        # otherwise be excluded by "exclude".

        # If neither "exclude" or "include" are set, all files will be vendored

        # Files/paths in "include" will always be vendored, even if excluded

        # eg. excluding "docs/" then including "docs/LICENSE" will vendor just the LICENSE file

        # from the docs directory

        # All three file/path parameters ("keep", "exclude", and "include") support filenames,

        # directory names, and globs/wildcards.

        update-actions:

            - action: move-file

              from: '{vendor_dir}/origin'

              to: '{vendor_dir}/dest'

            - action: move-dir

              from: '{vendor_dir}/origin'

              to: '{vendor_dir}/dest'

            - action: copy-file

              from: '{vendor_dir}/origin'

              to: '{vendor_dir}/dest'

            - action: delete-path

              path: "src/unused"

            - action: replace-in-file

              pattern: '@REVISION@'

              with: '{revision}'

              file: '{yaml_dir}/vcs_version.h'

            - action: replace-in-file-regex

              file: '{vendor_dir}/lib/arm/armopts.s'

              pattern: '@HAVE_ARM_ASM_((EDSP)|(MEDIA)|(NEON))@'

              with: '1'

            - action: run-script

              script: '{yaml_dir}/update.sh'

              args: ['{revision}']

              cwd: '{cwd}'

        # optional

        # In-tree actions to be executed after vendoring but before pushing.

Common Vendoring Operations

===========================

Update to the latest upstream revision:

.. code-block:: sh

   ./mach vendor /path/to/moz.yaml

Check for latest revision, returning no output if it is up-to-date, and a

version identifier if it needs to be updated:

.. code-block:: sh

   ./mach vendor /path/to/moz.yaml --check-for-update

Vendor a specific revision:

.. code-block:: sh

   ./mach vendor /path/to/moz.yaml -r $REVISION --force

In the presence of patches, two steps are needed:

1. Vendor without applying patches (patches are applied *after*

   ``update-actions``) through ``--patch-mode none``

2. Apply patches on updated sources through ``--patch -mode only``

In the absence of patches, a single step is needed, and no extra argument is

required.