In rust-url 1.0 the `Url` struct is going to have private fields, and there
is no way to to create an aribitrary one without going through the parser.
The plugin never had a clear demonstrated performance benefit,
it was made mostly because it was possible and relatively easy at the time.
implements data-url fetching
Fixes#10165
jdm mentioned of decoupling the, payload loading mechanism to data_loader.rs. So accordingly a `decoder` method has been added to data_loader.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10272)
<!-- Reviewable:end -->
Add Http Global State Object
This adds a new HttpState object which holds common http state(#10175). This reduces the amount of work that is required to add extra things to the Http state.
The HttpState object currently holds:
```
hsts_list: Arc::new(RwLock::new(HSTSList::new())),
cookie_jar: Arc::new(RwLock::new(CookieStorage::new())),
auth_cache: Arc::new(RwLock::new(HashMap::new())),
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10188)
<!-- Reviewable:end -->
Replace hsts_list, auth_cache, and cookie_jar with http_state
Reverted a few changes
Moved http_state back to factory
Removed unnecessary Arc
Removed Arc for http_state
Fix#10147: Correctly handle flag local_urls_only
In function Request::fetch_main, flag local_urls_only (if set)
should allow fetching local urls only. Before this change, the flag had
the inverse behaviour.
Fixes#10147.
Test with: `./mach test-unit -p net fetch::test_fetch_with_local_urls_only`
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10154)
<!-- Reviewable:end -->
add auth cache to resources + set auth header from it if url does not have creds
initial attempt of
- in resource_thread.rs, define an HTTP authorization cache storage (username, password, URL) and instantiate it like the cookie_storage member (inside an Arc<Rwlock<>> value, to enable sharing it between threads)
- in modify_request_headers in http_loader.rs, implement the remaining pieces of step 12(13?) of the appropriate specification using this new authorization cache.
for the NCSU student project Implement HTTP authorization UI and persistent sessions.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10111)
<!-- Reviewable:end -->
When fetching about:blank, response body should be the empty byte
sequence.
Spec: https://fetch.spec.whatwg.org/#concept-basic-fetch
Before this change, response body would be set to `ResponseBody::Empty`,
and then fetching would result in an infinite loop at step 19 in fn
`main_fetch` (methods.rs).
In function Request::fetch_main, flag local_urls_only (if set)
should allow fetching local urls only. Before this change, the flag had
the inverse behaviour.
Add Tests For Asynchronous Fetch Calls With Filtered Responses
I've added two tests using async_fetch intended to create filtered responses. I caught and fixed a bug where waiting for response.body to be completed would never pass for Opaque or OpaqueRedirect, since their response.body is always `Empty`.
As always, I'd appreciate review and feedback! I don't plan to write new tests, but I will gladly patch up any gaps in the current tests.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9907)
<!-- Reviewable:end -->
correctly send secure cookies after hsts url match
Fixes#8100, where sites in the hsts list were not recieving secure
cookies if the site was originally loading using a plain http url.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9780)
<!-- Reviewable:end -->
Validate MIME pattern checking
The MIME detector for a HTML file contains an additional byte in the mask. While it doesn't hurt, it is not used in the byte matching code (since it is zipped with the `pattern` iterator).
I'm not clear how to test this within the `servo` code. The fix doesn't change any externally visible behavior. A test to validate the `ByteMatcher` structures would require the private ByteMatcher structures to be visible to the test. I could use a sub-module, as described at https://doc.rust-lang.org/book/testing.html#the-tests-module but this pattern doesn't appear anywhere else in the `servo` code.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9631)
<!-- Reviewable:end -->
This adds a size to the test webp file, since the error fixed occurs
when the test object is the same length as the matched pattern, and
is not equal to the pattern.
Implementation of HTTP Redirect Fetch step
I've made a first draft of a complete implementation of HTTP Redirect Fetch, most of which is just refactored out of HTTP Fetch. I've also made some minor changes in a few other steps, all collected in the second commit, based on recent changes to the Fetch Standard. Since HTTP Redirect Fetch is so new, I figured now would be a fine time to make those other changes.
The biggest thing on my mind right now is how the spec says[1] "This algorithm will be used by HTML's "navigate" algorithm in addition to HTTP fetch above." This makes me think that this function, as well as HTTP Fetch, need to public, or at least have a public-facing function- since each Fetch function takes an Rc<Request>, which might be weird to require callers to supply.
[1] https://fetch.spec.whatwg.org/#http-redirect-fetch
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9608)
<!-- Reviewable:end -->
Test filtered responses and implement Cors Check Fetch step
I've been writing tests for creating filtered responses. So far I have three of the four types being made (namely, Basic, CORS, and Opaque), and just need to figure out how to make an OpaqueRedirect filtered response, since it's handled separately from the others. I will also add more tests to ensure the content of the filtered responses matches the limitations placed by the specification.
Along the way I implemented Cors Check, since it's required for the CORS filtered response. @jdm suggested I handle it in here, since it's such a small step, compared to other parts of Fetch.
Since all the tests currently pass, and I've spent a while adding the Cors Check and other pieces, I figured now would be a good time to start having it reviewed.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9525)
<!-- Reviewable:end -->
Partial implementation of Main Fetch step
I started out with Step 11 of Main Fetch so I could test creating filtered responses, which broke my tests that compare the Fetch result to a message on a server. I realized that if I got the tests to work, I'd likely end up breaking them again with the next step of Main Fetch I added, so I went ahead and did as much of Main Fetch as I could figure out.
Some steps I'm sure I could implement, I just don't know how. Such as when the spec says to "wait for response", or how to implement a Runnable object (which iirc is what I need to use) to run everything after Step 8 in parallel.
The fetch tests are still not running correctly, but I sure it's because they're getting a filtered response which doesn't have the body of response. I'm not sure how to handle that, whether it means a change needed in the tests or in the Fetch code. Like always, I look forward to feedback on my work!
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9427)
<!-- Reviewable:end -->
