servo / rust-url Goto Github PK

View Code? Open in Web Editor NEW

1.3K 27.0 314.0 3.77 MB

URL parser for Rust

Home Page: https://docs.rs/url/

License: Apache License 2.0

Rust 99.40% Python 0.60%

rust-url's Introduction

rust-url

URL library for Rust, based on the URL Standard.

Documentation

Please see UPGRADING.md if you are upgrading from a previous version.

rust-url's People

Contributors

Stargazers

Watchers

Forkers

dmski seanmonstar zarvox michaelsproul mahkoh tildeio reem nathantypanski cgaebel catethysis alexcrichton ljrmorgan gtolle rubys steveklabnik jgillich nickel-org kyledewey chris-morgan canndrew kevinburke zirak spearhex akosthekiss julyip kanetkarster tetsuharuohzeki gchp frewsxcv rgs1 bkearns isc-labs fhartwig liamfraser fuchsnj nox blaenk kichjang pcwalton divarvel notriddle metajack untitaker svmk flashbuckets chandler goffrie dlrobertson solidsnack yoava333 zofrex asajeffrey carols10cents jezell codecommunity mark-buer emilio avadacatavra canova manishearth amikhalev erickt hsivonen kybishop cbrewster wfraser christophebiocca alexxnica froydnj nnethercote therustmonk brson enet4 stanislav-tkach tomecki behnam hackeryarn toomanybees shouren imor nlincoln tmccombs luisbg theduke eijebong andygauge juggle-tux kichristensen golddranks rillian ignatenkobrain rap2hpoutre mgeisler nyantec copyninja mbrubeck valenting marmistrz tailhook sebasmagri

rust-url's Issues

Windows paths into URLs

According to this fixme it is unimplemented for a windows path to be converted to a URL.

I ran across the wikipedia article for this today, and notably it looks like this conversion happens:

path: c:\path\to\the file.txt
url: file:///c:/path/to/the%20file.txt

Is this a "correct enough" spec to implement? I wouldn't mind dabbling in this, but I want to make sure that this is a reasonable thing to do first.

`port: None` should be equal to `port: default_port`

The iron framework constructs an URL with RelativeSchemeData { port: 42, default_port: 42, ... } (see iron/iron#400). The url crate should treat this as equal to RelativeSchemeData { port: None, default_port: 42 }.

At least this behavior would make sense for HTTP URLs.

crates.io version is out of date

the fixes on master haven't been pushed

Add IPv4 regression test

#116 (comment)

Large ports cannot be parsed

https://github.com/w3c/web-platform-tests/blob/11f3aee19205a2ae97efa1cbeb211a2395192ea8/url/urltestdata.txt#L19

rust-url doesn't build on 0.13 nightly

Using:
rustc 0.13.0-nightly (c38e73fef 2014-12-10 20:11:27 +0000)
cargo 0.0.1-pre-nightly (0f6667c 2014-12-08 21:19:01 +0000)

Installed as part of Hyper: https://github.com/hyperium/hyper

Is there a right way to specify a version of Rust that works with this package?

Here's what happens when I cargo run:

✗ cargo run
   Compiling url v0.2.2
   Compiling time v0.1.1
   Compiling openssl-sys v0.2.3
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:187:40: 187:51 error: use of moved value: `scheme_type`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:187         remaining, Context::UrlParser, scheme_type, parser));
                                                                                                                                     ^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:186:29: 187:62 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185:76: 185:87 note: `scheme_type` moved here because it has type `SchemeType`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185     let (host, port, default_port, remaining) = try!(parse_host(remaining, scheme_type, parser));
                                                                                                                                                                         ^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:185:49: 185:98 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:243:59: 243:70 error: use of moved value: `scheme_type`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:243                 let scheme_data = SchemeData::Relative(if scheme_type == SchemeType::FileLike {
                                                                                                                                                        ^~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:242:67: 242:78 note: `scheme_type` moved here because it has type `SchemeType`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:242                     &[], input.slice_from(1), Context::UrlParser, scheme_type, parser));
                                                                                                                                                                ^~~~~~~~~~~
<std macros>:1:1: 8:2 note: in expansion of try!
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:241:41: 242:89 note: expansion site
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:357:37: 357:56 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:357                                     USERINFO_ENCODE_SET, &mut username);
                                                                                                                                  ^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:375:37: 375:56 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:375                                     USERINFO_ENCODE_SET, &mut password);
                                                                                                                                  ^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:546:41: 546:59 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:546                                         DEFAULT_ENCODE_SET, &mut path_part);
                                                                                                                                      ^~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:599:37: 599:54 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:599                                     SIMPLE_ENCODE_SET, &mut scheme_data);
                                                                                                                                  ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:648:48: 648:64 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:648     Ok((percent_encode(query_bytes.as_slice(), QUERY_ENCODE_SET), remaining))
                                                                                                                                             ^~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:660:37: 660:54 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/parser.rs:660                                     SIMPLE_ENCODE_SET, &mut fragment);
                                                                                                                                  ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:55:47: 55:66 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:55                 utf8_percent_encode_to(input, USERNAME_ENCODE_SET, username);
                                                                                                                                             ^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:67:47: 67:66 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/urlutils.rs:67                 utf8_percent_encode_to(input, PASSWORD_ENCODE_SET, &mut new_password);
                                                                                                                                             ^~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 error: use of moved value: `encoding_override`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140         byte_serialize(name, &mut output, encoding_override);
                                                                                                                                                 ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 note: `encoding_override` moved here because it has type `encoding::EncodingOverride`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140         byte_serialize(name, &mut output, encoding_override);
                                                                                                                                                 ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:142:44: 142:61 error: use of moved value: `encoding_override`
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:142         byte_serialize(value, &mut output, encoding_override);
                                                                                                                                                  ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140:43: 140:60 note: `encoding_override` moved here because it has type `encoding::EncodingOverride`, which is non-copyable
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:140         byte_serialize(name, &mut output, encoding_override);
                                                                                                                                                 ^~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:130:44: 130:70 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/form_urlencoded.rs:130                 percent_encode_to(&[byte], FORM_URLENCODED_ENCODE_SET, output)
                                                                                                                                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:938:56: 938:74 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:938         Ok(self.components().map(|c| percent_encode(c, DEFAULT_ENCODE_SET)).collect())
                                                                                                                                                  ^~~~~~~~~~~~~~~~~~
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:956:49: 956:67 error: cannot move out of static item
/Users/AdamCanady/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.2/src/lib.rs:956             path.push(percent_encode(component, DEFAULT_ENCODE_SET));
                                                                                                                                           ^~~~~~~~~~~~~~~~~~

resolving a fragment against any scheme does not succeed

These tests fail https://github.com/w3c/web-platform-tests/blob/master/url/urltestdata.txt#L346-L351 because non-relative URLs can't be constructed using a base URL.

Potential punycode encoding or decoding issue

Code

#![feature(plugin)]
#![plugin(afl_coverage_plugin)]

extern crate afl_coverage;

extern crate url;

use std::io::{self, Read};

fn main() {
    let mut input = String::new();
    let result = io::stdin().read_to_string(&mut input);
    if result.is_ok() {
        if let Some(puny) = url::punycode::decode_to_string(&input) {
            assert!(url::punycode::encode_str(&puny).is_some());
        }
    }
}

Input

This is encoded as base64. Decode it before feeding it into the program.

MDAw872AgDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwLQ==

Error

root@afl-rust:~/afl-staging-area# cargo run < test
     Running `target/debug/afl-staging-area`
thread '<main>' panicked at 'assertion failed: url::punycode::encode_str(&puny).is_some()', src/main.rs:15
An unknown error occurred

To learn more, run the command again with --verbose.

Summary

We're given a string that we want to decode as punycode. decode_str returns Some on this input, indicating we successfully decoded without errors. Upon attempting to encode that decoded result, it returns an Err.

This is a pretty obscure issue. This might not be an actual issue. Feel free to close this.

This bug was found using https://github.com/kmcallister/afl.rs 👍

Add a no path case to urls

Currently the path of URLs is represented by a Vec<String>. The empty vector represents the path /. In HTTP one needs to reconstruct the effective URL. rust-url supports almost all cases but it does make a difference between no-path and root-path. Server-wide OPTIONS requests use a special path of * and proxys must relay requests such as OPTIONS http://www.example.org:8001 HTTP/1.1 as OPTIONS * HTTP/1.1 to the final server. With a trailing slash like in OPTIONS http://www.example.org:8001/ HTTP/1.1 the meaning of the request changes but both URLs are parsed as the same URL by rust-url.

I propose to change the type of RelativeSchemeData.path to Option<Vec<String>> to be able to represent all URLs commonly used in HTTP. I think this is possible without changing the signature and meaning of many functions and preserving the current functionality but be able to check if there was a path in the URL or not if necessarry.

`Fork me` banner collapses with crate description

Ideally, it should be compiled in using rustdoc --html-before-content ./github.png to be a permanent fixture on the page.

rustdoc options don't seem to be supported by cargo yet.

Need to keep up with std::ascii reform

When building cargo with (very) recent rustc, then errors like below are reported:

/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:59:35: 59:45 error: type `&str` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:59             if !domain.as_slice().is_ascii() {
                                                                                                                       ^~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:66:54: 66:72 error: type `collections::string::String` does not implement any method in scope named `into_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/host.rs:66                 Ok(Host::Domain(domain.into_string().into_ascii_lower()))
                                                                                                                                          ^~~~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:164:35: 164:51 error: type `&str` does not implement any method in scope named `to_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:164                 input.slice_to(i).to_ascii_lower(),
                                                                                                                          ^~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:172:40: 172:56 error: type `&'a str` does not implement any method in scope named `to_ascii_lower`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/parser.rs:172         Context::Setter => Some((input.to_ascii_lower(), "")),
                                                                                                                               ^~~~~~~~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/form_urlencoded.rs:78:47: 78:57 error: type `&[u8]` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/form_urlencoded.rs:78     if !(encoding_override.is_utf8() || input.is_ascii()) {
                                                                                                                                              ^~~~~~~~~~
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/punycode.rs:144:14: 144:24 error: type `char` does not implement any method in scope named `is_ascii`
/home/akiss/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.7/src/punycode.rs:144         if c.is_ascii() { Some(c as u8) } else { None }

This seems to be the case since rust-lang/rust@070ab63 , and the cause of the error is also present in current HEAD of rust-url.

(This is very similar to alexcrichton/curl-rust#45 and dtolnay/semver#27 )

Invalid IPv4 addresses are not rejected

https://github.com/w3c/web-platform-tests/blob/11f3aee19205a2ae97efa1cbeb211a2395192ea8/url/urltestdata.txt#L325

use of undeclared trait name `IntoIterator` in form_urlencoded.rs

On Mac OS X 10.10.3 with Rust rustc 1.0.0-beta.2, compilation of the package fails, because the IntoIterator has not been declared with use.

Can be fixed by including use std::iter::IntoIterator; in form_urlencoded.rs.

Implement Ord/PartialOrd?

I notice that currently these traits aren't implemented for Url, and I was curious if this was done on purpose? If not, would it be possible to add them?

Method to extract origin from a URL

Currently, there is no method to extract the origin of a URL as String

eg: from https://example.com:490/samplefolder/samplefile.txt, extract https://example.com:490/

It would be better if there is a method which does this, otherwise one has to use url.scheme, url.domain() and url.port() to construct the origin.

Add path_mut

Add a path_mut method, which provides a mutable reference to the requested path. This is important for things like a Mounting middleware, which need to be able to modify the url.

Compile-time url parsing 'url!' macro

Inspired by sfackler/rust-postgres-macros which is a compile time SQL parser

There are ton of instances in servo, where we do something like Url::parse("http://mozilla.com").unwrap(). It'd be great if we could do that validation at compile time with a macro (e.g. url!("http://mozilla.com")).

I have no idea how feasible this is, just throwing around the idea.

The code doesn't compile with master nightly rust: rustc 1.0.0-nightly (44a287e6e 2015-01-08 17:03:40 -0800)

Misleading Description

While this library does indeed parse urls, it also helps encode them, the description on crates.io doesn't suggest it though which lead me to skip over it when first searching. A package description including url encoding would help.

Url set_query_from_pairs not working

extern crate url;
use url::Url;
fn main(){
  let mut url = Url::parse("http://example.com").unwrap();
  url.set_query_from_pairs([("q", "rust")].iter().map(|&(k,v)| (k,v)));
  println!("{}", url);
}

expected:

http://example.com?q=rust

found:
http://example.com/?

fails to compile with rust nightly on Windows (4/22)

   Compiling url v0.2.30
C:\Users\nathan\.cargo\registry\src\github.com-1ecc6299db9ec823\url-0.2.30\src\lib.rs:999:33: 999:47 error: unresolved n
ame `percent_decode`
C:\Users\nathan\.cargo\registry\src\github.com-1ecc6299db9ec823\url-0.2.30\src\lib.rs:999         match String::from_utf
8(percent_decode(path_part.as_bytes())) {

  ^~~~~~~~~~~~~~
error: aborting due to previous error
...
Build failed, waiting for other jobs to finish...
Could not compile `url`.

rustc --version
rustc 1.0.0-nightly (2baf34825 2015-04-21) (built 2015-04-22)

Cannot parse usernames in path urls.

In rust-postgres, there's a test that generates a url like so:

let mut url = Url::from_file_path(&Path::new(unix_socket_directory)).unwrap();
*url.username_mut().unwrap() = "postgres".to_string();                         
url.scheme = "postgres".to_string();

When printed, the url is:

postgres://postgres@/var/run/postgresql

But when this url is parsed, it parses to a url with an empty host (a parser error).

What to do when URLUtils.protocol setter change the scheme "type"

E.g. from a relative scheme to a non-relative scheme. See discussion in #83.

'/' is not allowed in domains

For example, this url does not parse:

postgres://postgres@%2Fvar%2Frun%2Fpostgresql

This is hindering rust-postgres's move to rust-url, and @sfackler points out page 20 of http://tools.ietf.org/html/rfc3986#section-3.2.2 which seems to allow for this.

URL Parsing for Databases and other things

At the moment if you want to parse a URL in the format databasedriver://hostname:port/whatever you need to write your own scheme handler function. It's not super annoying but it's definitely something people might miss when they expect this thing to parse them a hostname.

Since nobody does URL joining anyways would it make sense to introduce an interface that provides a parser that is RelativeScheme with a dummy port for all schemes by default?

Rust doesn’t appear to ship url crate anymore

url crate doesn’t seem to exist anymore?

Escaped chars in username / password are not unescaped

The following lines added to src/urltestdata.txt make the tests fail

http://:passw%[email protected]  s:http pass:pass/word h:www.example.com p:/
http://us%40er:[email protected]  s:http u:us@er pass:password h:www.example.com p:/

As per the spec:

 The user name (and password), if present, are followed by a
 commercial at-sign "@". Within the user and password field, any ":",
 "@", or "/" must be encoded.

There seems to be unescaping happening there, though.

https://github.com/servo/rust-url/blob/master/src/parser.rs#L348

Compile Error: private type in exported type parameter bound

Compiling url v0.1.0 (https://github.com/servo/rust-url#275c4a8e)
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:474:27: 474:28 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:474     pub fn from_file_path<T: ToUrlPath>(path: &T) -> Result<Url, ()> {
                                                                                                           ^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:496:32: 496:33 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:496     pub fn from_directory_path<T: ToUrlPath>(path: &T) -> Result<Url, ()> {
                                                                                                                ^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:541:25: 541:26 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:541     pub fn to_file_path<T: FromUrlPath>(&self) -> Result<T, ()> {
                                                                                                         ^
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:836:25: 836:26 error: private type in exported type parameter bound
/home/robin/.cargo/git/checkouts/rust-url-1e22af4233079a1e/master/src/lib.rs:836     pub fn to_file_path<T: FromUrlPath>(&self) -> Result<T, ()> {

Url struct layout makes processing URIs in servers difficult

Coming from a URI perspective (RFC 3986) and a webserver perspective, there are strings which contain (path,query,fragment) but omit (scheme,username,password,host,port). These are relative to a web server. I'm working on an OAuth2 library and an REST API library, both of which are server side or have server-side components, and work with such strings.

The way the URL struct is is broken down (path being way down in RelativeSchemeData) makes it difficult to parse/represent (path,query,fragment) type strings. I'm having brief moments of insanity where I'm on the verge of writing another differently structured URI rust library to address this. Could rust-url address my concerns... or is the data structure layout pretty much settled?

As an aside, what is the status/acceptance of http://url.spec.whatwg.org/ ? Is it really the standard that the majority of software providers aspire to for URLs? And whence went URIs? Seems a drastic break from when mention of URLs nearly disappeared from IETF standards, as this one almost fails to mention URIs.

Expose parse method to parse hostname:port

My use case: the Host http header.

I'm parsing into a struct Host { hostname: String, port: Option<Port> }.

Implement Nameprep for IDNA

Relevant spec: https://url.spec.whatwg.org/#idna, http://www.unicode.org/reports/tr46/#Processing
Code: src/host.rs, and a new file, possibly src/nameprep.rs

`cargo publish` updates the registry twice

% cargo publish
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Packaging url v0.2.0 (file:///home/simon/projects/rust-url)
   Verifying url v0.2.0 (file:///home/simon/projects/rust-url)
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling url v0.2.0 (file:///home/simon/projects/rust-url)
   Uploading url v0.2.0 (file:///home/simon/projects/rust-url)

It doesn’t take that much time, but it might be slightly wasteful.

Changing the scheme can break things

If you change the scheme from a non-relative one to a relative one without regenerating the scheme data, the Url object stops working properly. For example, parsing view-source+http://google.com and then using slice() on url.scheme to make it http again does not change the corresponding scheme data.

Perhaps there should be a set_scheme() method for setting the scheme safely?

Consider lower-casing the first letter of the ParseError `Show` impl

Errors are often printed like println!("could not parse: {}", e). With ParseError this will show things like could not parse: Empty host, which looks weird. Displaying an error occurred: empty host seems nicer.

(I imagine, completely nonscientifically, that most uses would prefer a leading lower case letter.)

Optimize Url::hash

The current implementation of the Hash trait calls serialize, which allocates a String and then hashes the contents of that. Cargo's use case, at least, involves hashing quite a few URLs, and I've often seen this function very high up in the profiles of Cargo.

Provide Path, like old url crate

Defining a web server would likely be defining a bunch of paths, and it'd be nice to be able to parse them to ensure they are correct, as well as being able to modify the Path, such as path.join(otherPath), path.add_query_param(), etc.

http://doc.rust-lang.org/url/struct.Path.html

Submit test for #124 to wpt

CC @SimonSapin

Better docs for edge-cases on getters

For instance, I'm not clear on when path and friends return None, or, what the value is for a url ending in / versus ending in something else or what path returns when the request is to /, etc.

Windows build regularly gets broken

Since the std::path reform, that module switched to having a single Path type with conditionally-compiled platform-specific behavior instead of separate std::path::unix::Path and std::path::windows::Path types and only a platform-specific re-export as std::path::Path.

rust-url also has some platform specific behavior for converting between Path and file:// URLs. Previously, we could implement traits for both Path types and have that code exist and all platforms. Now that it’s a single type, we have to use #[cfg(…)] to do conditional compilation too.

This is a long way to say that we now have Windows-specific code that is not compiled on Unix. Since I don’t use Windows and Windows is not available on Travis-CI, this code often gets broken without me noticing: #89, #96, #101.

Add documentation example that doesn't involve unwrap

I'm running into lifetime issues trying to get the domain() from a url.

This code:

    let url_r = Url::parse(raw_url);
    match url_r {
        Ok(url) => {
            match url.domain() {
                Some(d) => {
                    println!("{}", d)
                }
                None => {}
            }
        }
        Err(e) => {
            println!("{}", e)
        }
    }

raises an error: error: url does not live long enough

and I'm confused about what the right thing to do is or how to do it. As I understand it unwrap() will panic if the url doesn't parse.

Integer Overflow on odd `file` URL

This code:

"file:...".parse::<Url>();

will yield this error:

test tests::bad_file_url ... FAILED

failures:

---- tests::bad_file_url stdout ----
        thread 'tests::bad_file_url' panicked at 'arithmetic operation overflowed', src/parser.rs:287



failures:
    tests::bad_file_url

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured

serialized_path() is unintuitive

I'd expect a method called path() to return a String for the path, not a list containing the path's parts. For what it's worth.

FromUrlPath and ToUrlPath traits need to be public

Otherwise, the library cannot be built.

Url hash values have changed between 0.2.34 and 0.2.35

main.rs

#![feature(hash)]
extern crate url;

use url::Url;
use std::hash::{hash, SipHasher};

fn main() {
    let issue_list_url = Url::parse(
        "https://github.com/rust-lang/rust/issues?labels=E-easy&state=open"
    ).unwrap();
    println!("{}", hash::<_, SipHasher>(&issue_list_url));
}

Terminal session:

~/Desktop/rust $ cargo new --bin test
~/Desktop/rust $ cd test
~/Desktop/rust/test $ vim src/main.rs # edit main.rs per above
~/Desktop/rust/test $ echo '[dependencies]' >> Cargo.toml
~/Desktop/rust/test $ echo 'url = "0.2.34"' >> Cargo.toml
~/Desktop/rust/test $ cargo run
    Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling rustc-serialize v0.3.15
   Compiling matches v0.1.2
   Compiling url v0.2.35
   Compiling test v0.1.0 (file:///home/aidanhs/Desktop/rust/test)
     Running `target/debug/test`
10598872291636267386
~/Desktop/rust/test $ sed -i 's/35/34/' Cargo.lock
~/Desktop/rust/test $ cargo run                                                                                                                                                                 
   Compiling url v0.2.34
   Compiling test v0.1.0 (file:///home/aidanhs/Desktop/rust/test)
     Running `target/debug/test`
11128727563018172050

Note the editing Cargo.lock by hand - url = "0.2.34" in Cargo.toml is not sufficient.

This is a pretty surprising change for a patch release! I'd expect a more significant version bump.
Possibly went unnoticed though? There don't seem to be any hashing tests.

to_file_path() on localhost returns Path::new("/")

For example, this program will print /

extern crate url;                                                 

use url::Url;                                                     

fn main() {                                                       
    let url = Url::parse("http://postgres@localhost/").unwrap();  
    println!("{}", url.to_file_path::<Path>().unwrap().display());
}

Is this expected? If so, is there a way to differentiate paths with hosts from those with paths? It also looks like this may be relevant

requirements for parsing HTTP headers containing URLs

I'm working on Teepee https://github.com/teepee/teepee/, and we want to use rust-url to implement parsers for some HTTP headers. There are four headers that may contain URLs, or parts of them.

The Location header must be an absolute URL, and so must fail when the value is relative.

The Content-Location and Referer headers may be relative, but because all parsers for HTTP headers must have the same function signature, we would prefer not to have to specify a base URL. The algorithm specified by http://url.spec.whatwg.org/ seems to work this way. We would prefer to have an enum that tells us whether a URL is absolute or relative. Something like this, perhaps:

enum TypedUrl {
  Absolute(Url),
  Relative(Url)
}

Completely different types would also work.

Finally, the Host header has just a hostname or IP address. I'm currently using your Host type for that.

The code is available at https://github.com/db48x/teepee/blob/parse-some-headers/src/httpcommon/headers/url_based.rs.

tag 0.2.3 release, maybe release 0.2.4?

I just downloaded 0.2.3 and it looks like it doesn't have commit 1c738d2 which fixes those pesky Unicode warnings :(

Thanks for your work on the library!

Url 0.2.34: build failed

When I'm building ironframework, it stops on building url.

   Compiling url v0.2.34
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:94:10: 94:22 error: use of undeclared trait name `IntoIterator`
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:94 where I: IntoIterator, I::Item: Borrow<(K, V)>, K: AsRef<str>, V: AsRef<str> {
                                                                                                         ^~~~~~~~~~~~
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:117:10: 117:22 error: use of undeclared trait name `IntoIterator`
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/url-0.2.34/src/form_urlencoded.rs:117 where I: IntoIterator, I::Item: Borrow<(K, V)>, K: AsRef<str>, V: AsRef<str> {
                                                                                                          ^~~~~~~~~~~~
error: aborting due to 2 previous errors
Could not compile `url`.

Rust version

101 user@localhost ~/test (git)-[master] % rustc -V                                                                                                                                                                                              
rustc 1.0.0-dev (built 2015-05-12)

`url::form_urlencoded::serialize` is moving iterated key-value pairs

Currently serialize() is difficult to use for all users who don't want to drain vectors of string slices, but prefer keep iteration strictly read only.
Maybe, in the servo project it's uncommon to do that, but at least I am making my case and believe read-only operation should be the default.

Example

url::form_urlencoded::serialize([("client_id", "bogus"),
                                 ("scope", "A B C"].iter().map(|&(a,b)| (a, b)));

To my mind, the .map() call should not be required.

Maybe my code is the exception though, and not the rule, but when implementing protocols one will usually have a pre-determined set of key-value pairs.

Fails to parse port 80 only

When port 80 is in the passed in &str the resulting Url object shows a port value of None.