implements data-url fetching
Fixes#10165
jdm mentioned of decoupling the, payload loading mechanism to data_loader.rs. So accordingly a `decoder` method has been added to data_loader.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10272)
<!-- Reviewable:end -->
Add Http Global State Object
This adds a new HttpState object which holds common http state(#10175). This reduces the amount of work that is required to add extra things to the Http state.
The HttpState object currently holds:
```
hsts_list: Arc::new(RwLock::new(HSTSList::new())),
cookie_jar: Arc::new(RwLock::new(CookieStorage::new())),
auth_cache: Arc::new(RwLock::new(HashMap::new())),
```
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10188)
<!-- Reviewable:end -->
Replace hsts_list, auth_cache, and cookie_jar with http_state
Reverted a few changes
Moved http_state back to factory
Removed unnecessary Arc
Removed Arc for http_state
Fix#10147: Correctly handle flag local_urls_only
In function Request::fetch_main, flag local_urls_only (if set)
should allow fetching local urls only. Before this change, the flag had
the inverse behaviour.
Fixes#10147.
Test with: `./mach test-unit -p net fetch::test_fetch_with_local_urls_only`
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10154)
<!-- Reviewable:end -->
add auth cache to resources + set auth header from it if url does not have creds
initial attempt of
- in resource_thread.rs, define an HTTP authorization cache storage (username, password, URL) and instantiate it like the cookie_storage member (inside an Arc<Rwlock<>> value, to enable sharing it between threads)
- in modify_request_headers in http_loader.rs, implement the remaining pieces of step 12(13?) of the appropriate specification using this new authorization cache.
for the NCSU student project Implement HTTP authorization UI and persistent sessions.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/10111)
<!-- Reviewable:end -->
When fetching about:blank, response body should be the empty byte
sequence.
Spec: https://fetch.spec.whatwg.org/#concept-basic-fetch
Before this change, response body would be set to `ResponseBody::Empty`,
and then fetching would result in an infinite loop at step 19 in fn
`main_fetch` (methods.rs).
In function Request::fetch_main, flag local_urls_only (if set)
should allow fetching local urls only. Before this change, the flag had
the inverse behaviour.
correctly send secure cookies after hsts url match
Fixes#8100, where sites in the hsts list were not recieving secure
cookies if the site was originally loading using a plain http url.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9780)
<!-- Reviewable:end -->
Set response.body Asynchronously In Fetch
Following having finished making Fetch asynchronous, response.body should be set asynchronously, since it's the major goal of calling Fetch. So far, I've made the body wrapped in Arc<Mutex<>>, and I've wrapped a new thread around the part where it's set. I've also discovered that the fetch_async function makes step 8 of Main Fetch obsolete, and I've commented it appropriately.
I'm currently having a hard time with the thread for setting response.body, though. @jdm suggested I have the body set continually, block by block, but my implementation for that runs so slow that I can't finish running my fetch test suite in reasonable time. @KiChjang pointed out that a lot of the lag is due to how response.body currently stores everything inside a Vec. Changing the storage container seems to be both necessary and beyond the scope of the time I have to work on this.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9850)
<!-- Reviewable:end -->
Validate MIME pattern checking
The MIME detector for a HTML file contains an additional byte in the mask. While it doesn't hurt, it is not used in the byte matching code (since it is zipped with the `pattern` iterator).
I'm not clear how to test this within the `servo` code. The fix doesn't change any externally visible behavior. A test to validate the `ByteMatcher` structures would require the private ByteMatcher structures to be visible to the test. I could use a sub-module, as described at https://doc.rust-lang.org/book/testing.html#the-tests-module but this pattern doesn't appear anywhere else in the `servo` code.
<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/servo/9631)
<!-- Reviewable:end -->
This adds a size to the test webp file, since the error fixed occurs
when the test object is the same length as the matched pattern, and
is not equal to the pattern.
