Output test results as GitHub comments
After filtering intermittents, output the results as JSON. Update the
GitHub workflow to aggregate this JSON data into an artifact and use the
aggregated data to generate a GitHub comment with details about the try
run. The idea here is that this comment will make it easier to track
intermittent tests and notice when a change affects a test marked as
intermittent -- either causing it to permanently fail or fixing it.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they modify the CI infrastructure.
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
After filtering intermittents, output the results as JSON. Update the
GitHub workflow to aggregate this JSON data into an artifact and use the
aggregated data to generate a GitHub comment with details about the try
run. The idea here is that this comment will make it easier to track
intermittent tests and notice when a change affects a test marked as
intermittent -- either causing it to permanently fail or fixing it.
wpt: Make WPT result formatting logic independent of ServoHandler
This will allow results to be formatted by other parts of the code (such as the intermittent filtering code). Previously formatting was handled in ServoHandler, which was a bit strange as it's really only necessary for GroupingFormatter and the intermittent filtering code. This also allows the results to be properly typed by the Python typing system.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are part of the testing infrastructure that is generally untested.
This will allow results to be formatted by other parts of the code (such
as the intermittent filtering) code. Previously, formatting was handled
in ServoHandler, which was a bit strange as it's really only necessary
for GroupingFormatter and the intermittent filtering code. This also
allows the results to be properly typed by the Python typing system.
Integrate filter-intermittents into test-wpt
This change integrates the `filter-intermittents` command into `test-wpt`. This is in preparation for future work on tracking intermittent failures. This change also:
- Removes `ServoJsonFormatter` and replaces it with a generic WPT log handler which tracks unexpected results.
- The intermittent filter is now controlled via environment variables and the GitHub version requires a token instead of credentials.
- Output is saved to a single file and is always text.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes do not require tests because they are changes to build infrastructure.
- Also updates raqote to latest with an upgrade of font-kit to 0.11
applied on as a patch
- Update lyon_geom to the latest version
Major change:
- All matrices are now stored in row major order. This means that
parameters to rotation functions no longer should be negated.
- `post_...()` functions are now named `then()`. `pre_transform()` is removed,
so `then()` is used and the order of operations changed.
This change integrates the filter-intermittents command into test-wpt.
This is in preparation for future work on tracking intermittent
failures. This change also:
- Removes the SrvoJson logger and replaces it with a generic WPT log
handler which tracks unexpected results.
- The intermittent filter is now controlled via environment variables
and the GitHub version requires a token instead of credentials.
- Output is saved to a single file and is always text.
Fix absolute descendents being replaced when retreiving root
<!-- Please describe your changes on the following line: -->
The cause of #16410 is because calling `try_get_layout_root` will replace its `abs_descendants` even itself isn't absolute positioned. So I pushed them instead if the root isn't absolute. Also add a few docs and comments to help understand how block size of absolute flow is being calculated.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#16410 (GitHub issue number if applicable)
<!-- Either: -->
- [x] There are tests for these changes:
Reduced testcase from the issue.
```html
<style>
html {
min-height: 100%;
position: relative;
}
div {
position: absolute;
bottom: 0;
width: 100%;
height: 25px;
border: 1px solid black;
}
</style>
<div>
</div>
```
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Allow displaying content with "application/json" mime type
For me this allows the WPT test
performance-timeline/tentative/include-frames-one-remote-child.sub.html
to match expected results. The wptserver is sending a 404 JSON response
because the URL that the test requests is not found.
<!-- Please describe your changes on the following line: -->
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] These changes fix#29136.
- [x] There are tests for these changes.
<!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.-->
<!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. -->
Stringify unknown JavaScript objects in global exception handlers
When turning DOM exceptions into `ErrorInfo` always try to stringify
the JavaScript value, even if it's an object that isn't a `DOMException`
or native exception. This means that exceptions that extend the `Error`
prototype are now stringified. The result is that test output for WPT
global assertion failures is more useful. For instance for the test
include-frames-from-child-same-origin-grandchild.sub.html:
Before:
```
uncaught exception: unknown (can't convert to string)
```
After:
```
uncaught exception: Error: assert_equals: expected 4 but got 3
```
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [x] There are tests for these changes.
For me this allows the WPT test
performance-timeline/tentative/include-frames-one-remote-child.sub.html
to match expected results. The wptserver is sending a 404 JSON response
because the URL that the test requests is not found.
Combine `run.py` and `update.py` into `servowpt.py` in order to allow
them to share code. Import them directly into the mach script to avoid
having to call `compile` and `exec` on the code. This makes it clearer
how they are executed. In addition, move all of the setup into
`setupwpt.py` to avoid differences between tests executed via mach and
not. Finally, be more ambitious when detecting the build to use. If none
was specified, try to use the one that exists between "release" and
"debug."
When turning DOM exceptions into `ErrorInfo` always try to stringify
the JavaScript value, even if it's an object that isn't a `DOMException`
or native exception. This means that exceptions that extend the `Error`
prototype are now stringified. The result is that test output for WPT
global assertion failures is more useful. For instance for the test
include-frames-from-child-same-origin-grandchild.sub.html:
Before:
```
uncaught exception: unknown (can't convert to string)
```
After:
```
uncaught exception: Error: assert_equals: expected 4 but got 3
```
wpt: update README instructions about running tests manually
I wanted to run a single test many times in parallel to smoke out some timeouts, but this was surprisingly tricky. This involved running multiple instances of `mach test-wpt`, which in turn involved convincing wptrunner not to start a WPT server for each instance, which is not currently exposed as an option. I also eventually realised I needed to install a custom testharnessreport.js, but the instructions were very outdated, so most of the paths and commands etc had changed.
This patch adds a section about running `mach test-wpt` with an external WPT server, and updates the section about running tests manually.
---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `___` with appropriate data: -->
- [ ] ~~`./mach build -d` does not report any errors~~
- [ ] ~~`./mach test-tidy` does not report any errors~~
- [ ] These changes fix #___ (GitHub issue number if applicable)
<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because they affect the wpt readme only
Wait until a poster image is cached to in order to unblock document load. If
not, the document may continue loading before the image is ready to use,
leading to intermittency in test output. Now load is unblocked when
getting the ImageResponse from the cache, which allows the code to
properly unblock the load when the entire load fails or succeeds.
This reveals several false passes in the `object-view-box` test suite
which were very flaky.
Fixes#29204.
Fixes#29188.
Fixes#29179.
Signed-off-by: Martin Robinson <mrobinson@igalia.com>
