Certain tests involving HTTP requests were sometimes hanging
indefinitely, so this uses Mockito for HTTP mocking. This seemingly
resolves the issue and makes these tests more reliable.
The existing can_fail_404_links test has been renamed to
can_fail_unresolved_links, to represent what actually occurs in the
test. The can_fail_404_links test now deals with a proper 404
response.
Just to be clear, the check_site test in the site component will
still create outgoing HTTP requests (due to the URLs used in the
test_site), so this commit only uses HTTP mocking where possible.
The can_fail_404_links() test doesn't encounter a 404 response in
actuality, since the google.comys domain doesn't resolve. When the
test is updated such that the response's status code is a 404, the
test fails because the check_url() function doesn't handle
non-success responses how the test's assertions expect. This commit
updates check_url() to handle non-success responses, treating them
much like errors.
* Add anchor existant checking to link_checker component
* Oops, forgot some changes
* Drop scraper dependency and rework tests
* Handle name attributes
As mentioned in #381, crates.io 404's any request without an Accept:
text/html header. It 200's any request with one, but at least
false-successes don't prevent checking any other links.
This also makes it easier to add a custom User-Agent if desired.
rustfmt and fix a clippy nit (unnecessary return) while I'm here.
Sam Ford