Update etc/taskcluster/README.md for Community-TC migration

This commit is contained in:
Simon Sapin 2019-11-13 19:56:21 +01:00
parent eb05af27d2
commit 2d8ebc57dd
2 changed files with 93 additions and 87 deletions

View file

@ -1,5 +1,22 @@
# Testing Servo on Taskcluster # Testing Servo on Taskcluster
## In-tree and out-of-tree configuration
Where possible, we prefer keeping Taskcluster-related configuration and code in this directory,
set up CI so that testing of a given git branch uses the version in that branch.
That way, anyone can make changes (such installing a new system dependency
[in a `Dockerfile`](#docker-images)) in the same PR that relies on those changes.
For some things however that is not practical,
or some deployment step that mutates global state is required.
That configuration is split between the [mozilla/community-tc-config] and
[servo/taskcluster-config] repositories,
managed by the Taskcluster team and the Servo team repsectively.
[mozilla/community-tc-config]: https://github.com/mozilla/community-tc-config/blob/master/config/projects.yml
[servo/taskcluster-config]: https://github.com/servo/taskcluster-config/tree/master/config
## Homu ## Homu
When a pull request is reviewed and the appropriate command is given, When a pull request is reviewed and the appropriate command is given,
@ -31,10 +48,10 @@ So a common pattern is to have it only run a single initial task called a *decis
that can have complex logic based on code and data in the repository that can have complex logic based on code and data in the repository
to build an arbitrary [task graph]. to build an arbitrary [task graph].
[GitHub integration service]: https://docs.taskcluster.net/docs/manual/using/github [GitHub integration service]: https://community-tc.services.mozilla.com/docs/manual/using/github
[as a GitHub App]: https://github.com/apps/taskcluster [as a GitHub App]: https://github.com/apps/community-tc-integration/
[`.taskcluster.yml`]: https://docs.taskcluster.net/docs/reference/integrations/taskcluster-github/docs/taskcluster-yml-v1 [`.taskcluster.yml`]: https://community-tc.services.mozilla.com/docs/reference/integrations/taskcluster-github/docs/taskcluster-yml-v1
[task graph]: https://docs.taskcluster.net/docs/manual/using/task-graph [task graph]: https://community-tc.services.mozilla.com/docs/manual/using/task-graph
## Servos decision task ## Servos decision task
@ -56,7 +73,7 @@ as it only needs to clone the repository and run Python.
[`taskcluster-bootstrap-docker-images`]: https://github.com/servo/taskcluster-bootstrap-docker-images/ [`taskcluster-bootstrap-docker-images`]: https://github.com/servo/taskcluster-bootstrap-docker-images/
## In-tree Docker images ## Docker images
[Similar to Firefox][firefox], Servos decision task supports running other tasks [Similar to Firefox][firefox], Servos decision task supports running other tasks
in Docker images built on-demand, based on `Dockerfile`s in the main repository. in Docker images built on-demand, based on `Dockerfile`s in the main repository.
@ -105,13 +122,13 @@ The logic for all this is in [`decision_task.py`](decision_task.py)
and can be modified in any pull request. and can be modified in any pull request.
[web-platform-tests]: https://github.com/web-platform-tests/wpt [web-platform-tests]: https://github.com/web-platform-tests/wpt
[artifact]: https://docs.taskcluster.net/docs/manual/using/artifacts [artifact]: https://community-tc.services.mozilla.com/docs/manual/using/artifacts
## Log artifacts ## Log artifacts
Taskcluster automatically save the `stdio` output of a task as an artifact, Taskcluster automatically save the `stdio` output of a task as an artifact,
and as special support for seeing and streaming that output while the task is still running. and has special support for showing and streaming that output while the task is still running.
Servos decision task additionally looks for `*.log` arguments to its taskss commands, Servos decision task additionally looks for `*.log` arguments to its taskss commands,
assumes they instruct a program to create a log file with that name, assumes they instruct a program to create a log file with that name,
@ -145,10 +162,6 @@ Scopes that start with `assume:` are special,
they expand to the scopes defined in the matching roles. they expand to the scopes defined in the matching roles.
In this case, the [`repo:github.com/servo/servo:branch:*`][branches] role matches. In this case, the [`repo:github.com/servo/servo:branch:*`][branches] role matches.
Servo admins have scope `auth:update-role:repo:github.com/servo/*` which allows them
to edit that role in the web UI and grant more scopes to these tasks
(if that person has the new scope themselves).
The [`project:servo:decision-task/base`][base] The [`project:servo:decision-task/base`][base]
and [`project:servo:decision-task/trusted`][trusted] roles and [`project:servo:decision-task/trusted`][trusted] roles
centralize the set of scopes granted to the decision task. centralize the set of scopes granted to the decision task.
@ -159,14 +172,19 @@ Only the `base` role is granted to tasks executed when a pull request is opened.
These tasks are less trusted because they run before the code has been reviewed, These tasks are less trusted because they run before the code has been reviewed,
and anyone can open a PR. and anyone can open a PR.
[Scopes]: https://docs.taskcluster.net/docs/manual/design/apis/hawk/scopes Members of the [@servo/taskcluster-admins] GitHub team are granted
[web UI]: https://tools.taskcluster.net/ the scope `assume:project-admin:servo`, which is necessary to deploy changes
[credentials]: https://tools.taskcluster.net/credentials to those roles from the [servo/taskcluster-config] repository.
[Roles]: https://docs.taskcluster.net/docs/manual/design/apis/hawk/roles
[expand]: https://docs.taskcluster.net/docs/reference/platform/taskcluster-auth/docs/roles [Scopes]: https://community-tc.services.mozilla.com/docs/manual/design/apis/hawk/scopes
[branches]: https://tools.taskcluster.net/auth/roles/repo%3Agithub.com%2Fservo%2Fservo%3Abranch%3A* [web UI]: https://community-tc.services.mozilla.com/
[base]: https://tools.taskcluster.net/auth/roles/project%3Aservo%3Adecision-task%2Fbase [credentials]: https://community-tc.services.mozilla.com/profile
[trusted]: https://tools.taskcluster.net/auth/roles/project%3Aservo%3Adecision-task%2Ftrusted [Roles]: https://community-tc.services.mozilla.com/docs/manual/design/apis/hawk/roles
[expand]: https://community-tc.services.mozilla.com/docs/reference/platform/taskcluster-auth/docs/roles
[branches]: https://community-tc.services.mozilla.com/auth/roles/repo%3Agithub.com%2Fservo%2Fservo%3Abranch%3A*
[base]: https://community-tc.services.mozilla.com/auth/roles/project%3Aservo%3Adecision-task%2Fbase
[trusted]: https://community-tc.services.mozilla.com/auth/roles/project%3Aservo%3Adecision-task%2Ftrusted
[@servo/taskcluster-admins]: https://github.com/orgs/servo/teams/taskcluster-admins
## Daily tasks ## Daily tasks
@ -182,43 +200,55 @@ Scopes available to the daily decision task need to be both requested in the hoo
and granted through the [`hook-id:project-servo/daily`] role. and granted through the [`hook-id:project-servo/daily`] role.
Because they do not have something similar to GitHub statuses that link to them, Because they do not have something similar to GitHub statuses that link to them,
daily tasks are indexed under the [`project.servo.servo.daily`] namespace. daily tasks are indexed under the [`project.servo.daily`] namespace.
[`project.servo.servo.daily`]: https://tools.taskcluster.net/index/project.servo.servo.daily [`project.servo.daily`]: https://tools.taskcluster.net/index/project.servo.daily
[`project-servo/daily`]: https://github.com/servo/taskcluster-config/blob/master/config/hooks.yml
[`project-servo/daily`]: https://tools.taskcluster.net/hooks/project-servo/daily [Hooks service]: https://community-tc.services.mozilla.com/docs/manual/using/scheduled-tasks
[Hooks service]: https://docs.taskcluster.net/docs/manual/using/scheduled-tasks [`hook-id:project-servo/daily`]: https://github.com/servo/taskcluster-config/blob/master/config/roles.yml
[`hook-id:project-servo/daily`]: https://tools.taskcluster.net/auth/roles/hook-id%3Aproject-servo%2Fdaily
## AWS EC2 workers ## Servos worker pools
Tasks scheduled with the `servo-docker-worker` worker type run in a Linux environment, Each task is assigned to a “worker pool”.
in a Docker container, on an AWS EC2 virtual machine. Servo has several, for the different environments a task can run in:
These machines are short-lived “spot instances”. * `docker` and `docker-untrusted` provide a Linux environment with full `root` privileges,
They are started automatically as needed by the [AWS provisioner] in a Docker container running a [Docker image](#docker-images) of the tasks choice,
when the existing capacity is insufficient to execute queued tasks. in a short-lived virtual machine,
They terminate themselves after being idle without work for a while, on Google Cloud Platform.
or unconditionally after a few days.
Because these workers are short-lived,
we dont need to worry about evicting old entries from Cargos or rustups download cache,
for example.
Servo admins can view and edit the [worker type definition] which configures the provisioner, Instances are started automatically as needed
in particular with the types of EC2 instances to be used. when the existing capacity is insufficient to execute queued tasks.
They terminate themselves after being idle without work for a while,
or unconditionally after a few days.
Because these workers are short-lived,
we dont need to worry about evicting old entries from Cargos or rustups download cache,
for example.
[AWS provisioner]: https://docs.taskcluster.net/docs/reference/integrations/aws-provisioner/references/api [The Taskcluster team manages][mozilla/community-tc-config]
[worker type definition]: https://tools.taskcluster.net/aws-provisioner/servo-docker-worker/view the configuration and VM image for these two pools.
The latter has fewer scopes. It runs automated testing of pull requests
as soon as theyre opened or updated, before any review.
* `win2016` runs Windows Server 2016 on AWS EC2.
Like with Docker tasks, workers are short-lived and started automatically.
The configuration and VM image for this pool
is [managed by the Servo team][servo/taskcluster-config].
## Other worker types Tasks run as an unprivileged user.
Because creating an new the VM image is slow and deploying it mutates global state,
when a tool does not require system-wide installation
we prefer having each task obtain it as needed by extracting an archive in a directory.
See calls of `with_directory_mount` and `with_repacked_msi` in
[`decision_task.py`](decision_task.py) and [`decisionlib.py`](decisionlib.py).
See respective `README.md` files for: * `macos` runs, you guessed it, macOS.
Tasks run on dedicated hardware provided long-term by Macstadium.
* [Windows](windows/README.md), also short-lived workers on EC2 The system-wide configuration of those machines
* [macOS](macos/README.md), Mac Minis hosted by Macstadium is [managed by the Servo team][servo/taskcluster-config] through SaltStack.
* [Non-virtualized Linux](packet.net/README.md), hosted by Packet.net There is a task-owned (but preserved across tasks) install of Homebrew,
with `Brewfile`s [in this repository](macos/).
## Taskcluster Treeherder integration ## Taskcluster Treeherder integration
@ -226,37 +256,15 @@ See respective `README.md` files for:
See [`treeherder.md`](treeherder.md). See [`treeherder.md`](treeherder.md).
## Self-service, Bugzilla, and IRC ## Self-service, IRC, and Bugzilla
Taskcluster is designed to be “self-service” as much as possible, Taskcluster is designed to be “self-service” as much as possible.
with features like in-tree `.taskcluster.yml` Between this repository [servo/taskcluster-config] and [mozilla/community-tc-config],
or the web UI for modifying the worker type definitions. anyone should be able to submit PRs for any part of the configuration.
However some changes like adding a new worker type still require Taskcluster admin access.
For those, file requests on Bugzilla under [Taskcluster :: Service Request][req].
For asking for help less formally, try the `#servo` or `#taskcluster` channels on Mozilla IRC. Feel free to ask for help on the `#servo` or `#taskcluster` channels on Mozilla IRC.
[req]: https://bugzilla.mozilla.org/enter_bug.cgi?product=Taskcluster&component=Service%20Request For issue reports or feature requests on various bits of Taskcluster *software*,
file bugs [in Mozillas Bugzilla, under `Taskcluster`][bug].
[bug]: https://bugzilla.mozilla.org/enter_bug.cgi?product=Taskcluster
## Configuration recap
We try to keep as much as possible of our Taskcluster configuration in this repository.
To modify those, submit a pull request.
* The [`.taskcluster.yml`][tc.yml] file,
for starting decision tasks in reaction to GitHub events
* The [`etc/ci/decision_task.py`](decision_task.py) file,
defining what other tasks to schedule
However some configuration needs to be handled separately.
Modifying those requires Servo-project-level administrative access.
* The [`aws-provisioner/servo-docker-worker`][worker type definition] worker type definition,
for EC2 instances configuration
* The [`project-servo/daily`] hook definition,
for starting daily decision tasks
* The [`hook-id:project-servo/daily`] role,
for scopes granted to those tasks
* The [`repo:github.com/servo/servo:branch:*`][branches] role,
for scopes granted to tasks responding to a GitHub push to the repository (includin by Homu)

View file

@ -53,7 +53,7 @@ Note that you need to add at least one “Binding”,
or the “Start Listening” button wont do anything. or the “Start Listening” button wont do anything.
[Pulse]: https://wiki.mozilla.org/Auto-tools/Projects/Pulse [Pulse]: https://wiki.mozilla.org/Auto-tools/Projects/Pulse
[Pulse Inspector]: https://tools.taskcluster.net/pulse-inspector [Pulse Inspector]: https://community-tc.services.mozilla.com/pulse-messages
### Push data ### Push data
@ -69,13 +69,13 @@ Treeherder consumes messages from the `exchange/taskcluster-github/v1/push` exch
In Pulse Inspector, these messages for the Servo repository can be seen In Pulse Inspector, these messages for the Servo repository can be seen
by specifying the [`primary.servo.servo`] routing key pattern. by specifying the [`primary.servo.servo`] routing key pattern.
[taskcluster-github]: https://github.com/taskcluster/taskcluster-github [taskcluster-github]: https://github.com/taskcluster/taskcluster/tree/master/services/github
[enabled]: https://github.com/apps/taskcluster [enabled]: https://github.com/apps/community-tc-integration/
[webhooks]: https://developer.github.com/webhooks/ [webhooks]: https://developer.github.com/webhooks/
[Pulse messages]: https://docs.taskcluster.net/docs/reference/integrations/taskcluster-github/references/events [Pulse messages]: https://community-tc.services.mozilla.com/docs/reference/integrations/github/exchanges
[`api.js`]: https://github.com/taskcluster/taskcluster-github/blob/master/src/api.js [`api.js`]: https://github.com/taskcluster/taskcluster/blob/master/services/github/src/api.js
[`push_loader.py`]: https://github.com/mozilla/treeherder/blob/master/treeherder/etl/push_loader.py [`push_loader.py`]: https://github.com/mozilla/treeherder/blob/master/treeherder/etl/push_loader.py
[`primary.servo.servo`]: https://tools.taskcluster.net/pulse-inspector?bindings%5B0%5D%5Bexchange%5D=exchange%2Ftaskcluster-github%2Fv1%2Fpush&bindings%5B0%5D%5BroutingKeyPattern%5D=primary.servo.servo [`primary.servo.servo`]: https://community-tc.services.mozilla.com/pulse-messages?bindings%5B0%5D%5Bexchange%5D=exchange%2Ftaskcluster-github%2Fv1%2Fpush&bindings%5B0%5D%5BroutingKeyPattern%5D=primary.servo.servo
### (Taskcluster) job data ### (Taskcluster) job data
@ -84,9 +84,8 @@ The Taskcluster Queue generates a number of [Pulse messages about tasks].
Each value of the `routes` array in the task definition, with a `route.` prefix prepended, Each value of the `routes` array in the task definition, with a `route.` prefix prepended,
is additional routing key for those messages. is additional routing key for those messages.
The [taskcluster-treeherder] integration service reads those messages Treeherder reads those messages
if they have an appropriate route ([see in Pulse inspector][inspector1]), if they have an appropriate route ([see in Pulse inspector][inspector1]),
and converts them into [other messages] ([see in Pulse inspector][inspector2]).
However, it will drop an incoming message However, it will drop an incoming message
if the `extra.treeherder` object in the task definition doesnt conform to the [schema]. if the `extra.treeherder` object in the task definition doesnt conform to the [schema].
Such schema validation errors are logged, but those logs are not easy to access. Such schema validation errors are logged, but those logs are not easy to access.
@ -96,10 +95,9 @@ Finally, Treeherder reads that latter kind of message in [`job_loader.py`].
[Pulse messages about tasks]: https://docs.taskcluster.net/docs/reference/platform/taskcluster-queue/references/events [Pulse messages about tasks]: https://community-tc.services.mozilla.com/docs/reference/platform/taskcluster-queue/references/events
[taskcluster-treeherder]: https://github.com/taskcluster/taskcluster-treeherder/ [taskcluster-treeherder]: https://github.com/taskcluster/taskcluster-treeherder/
[other messages]: https://docs.taskcluster.net/docs/reference/integrations/taskcluster-treeherder#job-pulse-messages [other messages]: https://community-tc.services.mozilla.com/docs/reference/integrations/taskcluster-treeherder#job-pulse-messages
[schema]: https://schemas.taskcluster.net/treeherder/v1/task-treeherder-config.json [schema]: https://schemas.taskcluster.net/treeherder/v1/task-treeherder-config.json
[`job_loader.py`]: https://github.com/mozilla/treeherder/blob/master/treeherder/etl/job_loader.py [`job_loader.py`]: https://github.com/mozilla/treeherder/blob/master/treeherder/etl/job_loader.py
[inspector1]: https://tools.taskcluster.net/pulse-inspector?bindings%5B0%5D%5Bexchange%5D=exchange%2Ftaskcluster-queue%2Fv1%2Ftask-defined&bindings%5B0%5D%5BroutingKeyPattern%5D=route.tc-treeherder.%23 [inspector1]: https://tools.taskcluster.net/pulse-inspector?bindings%5B0%5D%5Bexchange%5D=exchange%2Ftaskcluster-queue%2Fv1%2Ftask-defined&bindings%5B0%5D%5BroutingKeyPattern%5D=route.tc-treeherder.%23
[inspector2]: https://tools.taskcluster.net/pulse-inspector?bindings%5B0%5D%5Bexchange%5D=exchange%2Ftaskcluster-treeherder%2Fv1%2Fjobs&bindings%5B0%5D%5BroutingKeyPattern%5D=tc-treeherder.servo-auto._