URL library for Rust, based on the URL Standard.
Please see UPGRADING.md if you are upgrading from a previous version.
URL parser for Rust
Home Page: https://docs.rs/url/
License: Apache License 2.0
URL library for Rust, based on the URL Standard.
Please see UPGRADING.md if you are upgrading from a previous version.
extern crate url;
use url::Url;
fn main(){
let mut url = Url::parse("http://example.com").unwrap();
url.set_query_from_pairs([("q", "rust")].iter().map(|&(k,v)| (k,v)));
println!("{}", url);
}
expected:
http://example.com?q=rust
found:
http://example.com/?
For example, this url does not parse:
postgres://postgres@%2Fvar%2Frun%2Fpostgresql
This is hindering rust-postgres's move to rust-url, and @sfackler points out page 20 of http://tools.ietf.org/html/rfc3986#section-3.2.2 which seems to allow for this.
I'm working on Teepee https://github.com/teepee/teepee/, and we want to use rust-url to implement parsers for some HTTP headers. There are four headers that may contain URLs, or parts of them.
The Location header must be an absolute URL, and so must fail when the value is relative.
The Content-Location and Referer headers may be relative, but because all parsers for HTTP headers must have the same function signature, we would prefer not to have to specify a base URL. The algorithm specified by http://url.spec.whatwg.org/ seems to work this way. We would prefer to have an enum that tells us whether a URL is absolute or relative. Something like this, perhaps:
enum TypedUrl {
Absolute(Url),
Relative(Url)
}
Completely different types would also work.
Finally, the Host header has just a hostname or IP address. I'm currently using your Host type for that.
The code is available at https://github.com/db48x/teepee/blob/parse-some-headers/src/httpcommon/headers/url_based.rs.
CC @SimonSapin
Currently, there is no method to extract the origin of a URL as String
It would be better if there is a method which does this, otherwise one has to use url.scheme, url.domain() and url.port() to construct the origin.
In rust-postgres, there's a test that generates a url like so:
let mut url = Url::from_file_path(&Path::new(unix_socket_directory)).unwrap();
*url.username_mut().unwrap() = "postgres".to_string();
url.scheme = "postgres".to_string();
When printed, the url is:
postgres://postgres@/var/run/postgresql
But when this url is parsed, it parses to a url with an empty host (a parser error).
I'm running into lifetime issues trying to get the domain() from a url.
This code:
let url_r = Url::parse(raw_url);
match url_r {
Ok(url) => {
match url.domain() {
Some(d) => {
println!("{}", d)
}
None => {}
}
}
Err(e) => {
println!("{}", e)
}
}
raises an error: error: url
does not live long enough
and I'm confused about what the right thing to do is or how to do it. As I understand it unwrap() will panic if the url doesn't parse.
If you change the scheme from a non-relative one to a relative one without regenerating the scheme data, the Url object stops working properly. For example, parsing view-source+http://google.com
and then using slice()
on url.scheme
to make it http again does not change the corresponding scheme data.
Perhaps there should be a set_scheme()
method for setting the scheme safely?
Otherwise, the library cannot be built.
While this library does indeed parse urls, it also helps encode them, the description on crates.io doesn't suggest it though which lead me to skip over it when first searching. A package description including url encoding would help.
Ideally, it should be compiled in using rustdoc --html-before-content ./github.png
to be a permanent fixture on the page.
rustdoc
options don't seem to be supported by cargo
yet.
Compiling url v0.2.30
C:\Users\nathan\.cargo\registry\src\github.com-1ecc6299db9ec823\url-0.2.30\src\lib.rs:999:33: 999:47 error: unresolved n
ame `percent_decode`
C:\Users\nathan\.cargo\registry\src\github.com-1ecc6299db9ec823\url-0.2.30\src\lib.rs:999 match String::from_utf
8(percent_decode(path_part.as_bytes())) {
^~~~~~~~~~~~~~
error: aborting due to previous error
...
Build failed, waiting for other jobs to finish...
Could not compile `url`.
rustc --version
rustc 1.0.0-nightly (2baf34825 2015-04-21) (built 2015-04-22)
main.rs
#![feature(hash)]
extern crate url;
use url::Url;
use std::hash::{hash, SipHasher};
fn main() {
let issue_list_url = Url::parse(
"https://github.com/rust-lang/rust/issues?labels=E-easy&state=open"
).unwrap();
println!("{}", hash::<_, SipHasher>(&issue_list_url));
}
Terminal session:
~/Desktop/rust $ cargo new --bin test
~/Desktop/rust $ cd test
~/Desktop/rust/test $ vim src/main.rs # edit main.rs per above
~/Desktop/rust/test $ echo '[dependencies]' >> Cargo.toml
~/Desktop/rust/test $ echo 'url = "0.2.34"' >> Cargo.toml
~/Desktop/rust/test $ cargo run
Updating registry `https://github.com/rust-lang/crates.io-index`
Compiling rustc-serialize v0.3.15
Compiling matches v0.1.2
Compiling url v0.2.35
Compiling test v0.1.0 (file:///home/aidanhs/Desktop/rust/test)
Running `target/debug/test`
10598872291636267386
~/Desktop/rust/test $ sed -i 's/35/34/' Cargo.lock
~/Desktop/rust/test $ cargo run
Compiling url v0.2.34
Compiling test v0.1.0 (file:///home/aidanhs/Desktop/rust/test)
Running `target/debug/test`
11128727563018172050
Note the editing Cargo.lock by hand - url = "0.2.34"
in Cargo.toml is not sufficient.
This is a pretty surprising change for a patch release! I'd expect a more significant version bump.
Possibly went unnoticed though? There don't seem to be any hashing tests.
Currently the path of URLs is represented by a Vec<String>
. The empty vector represents the path /
. In HTTP one needs to reconstruct the effective URL. rust-url supports almost all cases but it does make a difference between no-path and root-path. Server-wide OPTIONS requests use a special path of *
and proxys must relay requests such as OPTIONS http://www.example.org:8001 HTTP/1.1
as OPTIONS * HTTP/1.1
to the final server. With a trailing slash like in OPTIONS http://www.example.org:8001/ HTTP/1.1
the meaning of the request changes but both URLs are parsed as the same URL by rust-url.
I propose to change the type of RelativeSchemeData.path
to Option<Vec<String>>
to be able to represent all URLs commonly used in HTTP. I think this is possible without changing the signature and meaning of many functions and preserving the current functionality but be able to check if there was a path in the URL or not if necessarry.
For instance, I'm not clear on when path
and friends return None
, or, what the value is for a url ending in /
versus ending in something else or what path
returns when the request is to /
, etc.
Add a path_mut method, which provides a mutable reference to the requested path. This is important for things like a Mounting middleware, which need to be able to modify the url.
When building cargo with (very) recent rustc, then errors like below are reported:
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:59:35: 59:45 error: type `&str` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:59 if !domain.as_slice().is_ascii() {
^~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:66:54: 66:72 error: type `collections::string::String` does not implement any method in scope named `into_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:66 Ok(Host::Domain(domain.into_string().into_ascii_lower()))
^~~~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:164:35: 164:51 error: type `&str` does not implement any method in scope named `to_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:164 input.slice_to(i).to_ascii_lower(),
^~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:172:40: 172:56 error: type `&'a str` does not implement any method in scope named `to_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:172 Context::Setter => Some((input.to_ascii_lower(), "")),
^~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/form_urlencoded.rs:78:47: 78:57 error: type `&[u8]` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/form_urlencoded.rs:78 if !(encoding_override.is_utf8() || input.is_ascii()) {
^~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/punycode.rs:144:14: 144:24 error: type `char` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/punycode.rs:144 if c.is_ascii() { Some(c as u8) } else { None }
This seems to be the case since rust-lang/rust@070ab63 , and the cause of the error is also present in current HEAD of rust-url.
(This is very similar to alexcrichton/curl-rust#45 and dtolnay/semver#27 )
When port 80 is in the passed in &str
the resulting Url
object shows a port value of None
.
These tests fail https://github.com/w3c/web-platform-tests/blob/master/url/urltestdata.txt#L346-L351 because non-relative URLs can't be constructed using a base URL.
At the moment if you want to parse a URL in the format databasedriver://hostname:port/whatever
you need to write your own scheme handler function. It's not super annoying but it's definitely something people might miss when they expect this thing to parse them a hostname.
Since nobody does URL joining anyways would it make sense to introduce an interface that provides a parser that is RelativeScheme with a dummy port for all schemes by default?
The iron framework constructs an URL with RelativeSchemeData { port: 42, default_port: 42, ... }
(see iron/iron#400). The url crate should treat this as equal to RelativeSchemeData { port: None, default_port: 42 }
.
At least this behavior would make sense for HTTP URLs.
the fixes on master haven't been pushed
% cargo publish
Updating registry `https://github.com/rust-lang/crates.io-index`
Packaging url v0.2.0 (file:///home/simon/projects/rust-url)
Verifying url v0.2.0 (file:///home/simon/projects/rust-url)
Updating registry `https://github.com/rust-lang/crates.io-index`
Compiling url v0.2.0 (file:///home/simon/projects/rust-url)
Uploading url v0.2.0 (file:///home/simon/projects/rust-url)
It doesn’t take that much time, but it might be slightly wasteful.
I'd expect a method called path() to return a String for the path, not a list containing the path's parts. For what it's worth.
The current implementation of the Hash
trait calls serialize
, which allocates a String
and then hashes the contents of that. Cargo's use case, at least, involves hashing quite a few URLs, and I've often seen this function very high up in the profiles of Cargo.
On Mac OS X 10.10.3 with Rust rustc 1.0.0-beta.2, compilation of the package fails, because the IntoIterator has not been declared with use.
Can be fixed by including use std::iter::IntoIterator;
in form_urlencoded.rs.
Compiling url v0.1.0 (https://github.com/servo/rust-url#275c4a8e)
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:474:27: 474:28 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:474 pub fn from_file_path<T: ToUrlPath>(path: &T) -> Result<Url, ()> {
^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:496:32: 496:33 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:496 pub fn from_directory_path<T: ToUrlPath>(path: &T) -> Result<Url, ()> {
^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:541:25: 541:26 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:541 pub fn to_file_path<T: FromUrlPath>(&self) -> Result<T, ()> {
^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:836:25: 836:26 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:836 pub fn to_file_path<T: FromUrlPath>(&self) -> Result<T, ()> {
For example, this program will print /
extern crate url;
use url::Url;
fn main() {
let url = Url::parse("http://postgres@localhost/").unwrap();
println!("{}", url.to_file_path::<Path>().unwrap().display());
}
Is this expected? If so, is there a way to differentiate paths with hosts from those with paths? It also looks like this may be relevant
My use case: the Host
http header.
I'm parsing into a struct Host { hostname: String, port: Option<Port> }
.
When I'm building ironframework, it stops on building url.
Compiling url v0.2.34
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:94:10: 94:22 error: use of undeclared trait name `IntoIterator`
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:94 where I: IntoIterator, I::Item: Borrow<(K, V)>, K: AsRef<str>, V: AsRef<str> {
^~~~~~~~~~~~
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:117:10: 117:22 error: use of undeclared trait name `IntoIterator`
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:117 where I: IntoIterator, I::Item: Borrow<(K, V)>, K: AsRef<str>, V: AsRef<str> {
^~~~~~~~~~~~
error: aborting due to 2 previous errors
Could not compile `url`.
Rust version
101 user@localhost ~/test (git)-[master] % rustc -V
rustc 1.0.0-dev (built 2015-05-12)
Inspired by sfackler/rust-postgres-macros which is a compile time SQL parser
There are ton of instances in servo, where we do something like Url::parse("http://mozilla.com").unwrap()
. It'd be great if we could do that validation at compile time with a macro (e.g. url!("http://mozilla.com")
).
I have no idea how feasible this is, just throwing around the idea.
Coming from a URI perspective (RFC 3986) and a webserver perspective, there are strings which contain (path,query,fragment) but omit (scheme,username,password,host,port). These are relative to a web server. I'm working on an OAuth2 library and an REST API library, both of which are server side or have server-side components, and work with such strings.
The way the URL struct is is broken down (path being way down in RelativeSchemeData) makes it difficult to parse/represent (path,query,fragment) type strings. I'm having brief moments of insanity where I'm on the verge of writing another differently structured URI rust library to address this. Could rust-url address my concerns... or is the data structure layout pretty much settled?
As an aside, what is the status/acceptance of http://url.spec.whatwg.org/ ? Is it really the standard that the majority of software providers aspire to for URLs? And whence went URIs? Seems a drastic break from when mention of URLs nearly disappeared from IETF standards, as this one almost fails to mention URIs.
According to this fixme it is unimplemented for a windows path to be converted to a URL.
I ran across the wikipedia article for this today, and notably it looks like this conversion happens:
path: c:\path\to\the file.txt
url: file:///c:/path/to/the%20file.txt
Is this a "correct enough" spec to implement? I wouldn't mind dabbling in this, but I want to make sure that this is a reasonable thing to do first.
E.g. from a relative scheme to a non-relative scheme. See discussion in #83.
#![feature(plugin)]
#![plugin(afl_coverage_plugin)]
extern crate afl_coverage;
extern crate url;
use std::io::{self, Read};
fn main() {
let mut input = String::new();
let result = io::stdin().read_to_string(&mut input);
if result.is_ok() {
if let Some(puny) = url::punycode::decode_to_string(&input) {
assert!(url::punycode::encode_str(&puny).is_some());
}
}
}
This is encoded as base64. Decode it before feeding it into the program.
MDAw872AgDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwLQ==
root@afl-rust:~/afl-staging-area# cargo run < test
Running `target/debug/afl-staging-area`
thread '<main>' panicked at 'assertion failed: url::punycode::encode_str(&puny).is_some()', src/main.rs:15
An unknown error occurred
To learn more, run the command again with --verbose.
We're given a string that we want to decode as punycode. decode_str
returns Some
on this input, indicating we successfully decoded without errors. Upon attempting to encode that decoded result, it returns an Err
.
This is a pretty obscure issue. This might not be an actual issue. Feel free to close this.
This bug was found using https://github.com/kmcallister/afl.rs 👍
Defining a web server would likely be defining a bunch of paths, and it'd be nice to be able to parse them to ensure they are correct, as well as being able to modify the Path
, such as path.join(otherPath)
, path.add_query_param()
, etc.
Currently serialize() is difficult to use for all users who don't want to drain vectors of string slices, but prefer keep iteration strictly read only.
Maybe, in the servo project it's uncommon to do that, but at least I am making my case and believe read-only operation should be the default.
url::form_urlencoded::serialize([("client_id", "bogus"),
("scope", "A B C"].iter().map(|&(a,b)| (a, b)));
To my mind, the .map()
call should not be required.
Maybe my code is the exception though, and not the rule, but when implementing protocols one will usually have a pre-determined set of key-value pairs.
Errors are often printed like println!("could not parse: {}", e)
. With ParseError
this will show things like could not parse: Empty host
, which looks weird. Displaying an error occurred: empty host
seems nicer.
(I imagine, completely nonscientifically, that most uses would prefer a leading lower case letter.)
Since the std::path
reform, that module switched to having a single Path
type with conditionally-compiled platform-specific behavior instead of separate std::path::unix::Path
and std::path::windows::Path
types and only a platform-specific re-export as std::path::Path
.
rust-url also has some platform specific behavior for converting between Path
and file://
URLs. Previously, we could implement traits for both Path
types and have that code exist and all platforms. Now that it’s a single type, we have to use #[cfg(…)]
to do conditional compilation too.
This is a long way to say that we now have Windows-specific code that is not compiled on Unix. Since I don’t use Windows and Windows is not available on Travis-CI, this code often gets broken without me noticing: #89, #96, #101.
src/host.rs
, and a new file, possibly src/nameprep.rs
The following lines added to src/urltestdata.txt
make the tests fail
http://:passw%[email protected] s:http pass:pass/word h:www.example.com p:/
http://us%40er:[email protected] s:http u:us@er pass:password h:www.example.com p:/
As per the spec:
The user name (and password), if present, are followed by a
commercial at-sign "@". Within the user and password field, any ":",
"@", or "/" must be encoded.
There seems to be unescaping happening there, though.
https://github.com/servo/rust-url/blob/master/src/parser.rs#L348
I notice that currently these traits aren't implemented for Url
, and I was curious if this was done on purpose? If not, would it be possible to add them?
I just downloaded 0.2.3 and it looks like it doesn't have commit 1c738d2 which fixes those pesky Unicode warnings :(
Thanks for your work on the library!
This code:
"file:...".parse::<Url>();
will yield this error:
test tests::bad_file_url ... FAILED
failures:
---- tests::bad_file_url stdout ----
thread 'tests::bad_file_url' panicked at 'arithmetic operation overflowed', src/parser.rs:287
failures:
tests::bad_file_url
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured
Using:
rustc 0.13.0-nightly (c38e73fef 2014-12-10 20:11:27 +0000)
cargo 0.0.1-pre-nightly (0f6667c 2014-12-08 21:19:01 +0000)
Installed as part of Hyper: https://github.com/hyperium/hyper
Is there a right way to specify a version of Rust that works with this package?
Here's what happens when I cargo run
:
✗ cargo run
Compiling url v0.2.2
Compiling time v0.1.1
Compiling openssl-sys v0.2.3
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:187:40: 187:51 error: use of moved value: `scheme_type`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:187 remaining, Context::UrlParser, scheme_type, parser));
^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:186:29: 187:62 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185:76: 185:87 note: `scheme_type` moved here because it has type `SchemeType`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185 let (host, port, default_port, remaining) = try!(parse_host(remaining, scheme_type, parser));
^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185:49: 185:98 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:243:59: 243:70 error: use of moved value: `scheme_type`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:243 let scheme_data = SchemeData::Relative(if scheme_type == SchemeType::FileLike {
^~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:242:67: 242:78 note: `scheme_type` moved here because it has type `SchemeType`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:242 &[], input.slice_from(1), Context::UrlParser, scheme_type, parser));
^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:241:41: 242:89 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:357:37: 357:56 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:357 USERINFO_ENCODE_SET, &mut username);
^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:375:37: 375:56 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:375 USERINFO_ENCODE_SET, &mut password);
^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:546:41: 546:59 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:546 DEFAULT_ENCODE_SET, &mut path_part);
^~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:599:37: 599:54 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:599 SIMPLE_ENCODE_SET, &mut scheme_data);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:648:48: 648:64 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:648 Ok((percent_encode(query_bytes.as_slice(), QUERY_ENCODE_SET), remaining))
^~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:660:37: 660:54 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:660 SIMPLE_ENCODE_SET, &mut fragment);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:55:47: 55:66 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:55 utf8_percent_encode_to(input, USERNAME_ENCODE_SET, username);
^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:67:47: 67:66 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:67 utf8_percent_encode_to(input, PASSWORD_ENCODE_SET, &mut new_password);
^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 error: use of moved value: `encoding_override`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140 byte_serialize(name, &mut output, encoding_override);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 note: `encoding_override` moved here because it has type `encoding::EncodingOverride`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140 byte_serialize(name, &mut output, encoding_override);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:142:44: 142:61 error: use of moved value: `encoding_override`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:142 byte_serialize(value, &mut output, encoding_override);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 note: `encoding_override` moved here because it has type `encoding::EncodingOverride`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140 byte_serialize(name, &mut output, encoding_override);
^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:130:44: 130:70 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:130 percent_encode_to(&[byte], FORM_URLENCODED_ENCODE_SET, output)
^~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:938:56: 938:74 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:938 Ok(self.components().map(|c| percent_encode(c, DEFAULT_ENCODE_SET)).collect())
^~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:956:49: 956:67 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:956 path.push(percent_encode(component, DEFAULT_ENCODE_SET));
^~~~~~~~~~~~~~~~~~
url crate doesn’t seem to exist anymore?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.