Code Monkey home page Code Monkey logo

rust-phf's Introduction

Rust-PHF

CI Latest Version

Documentation

Rust-PHF is a library to generate efficient lookup tables at compile time using perfect hash functions.

It currently uses the CHD algorithm and can generate a 100,000 entry map in roughly .4 seconds.

MSRV (minimum supported rust version) is Rust 1.61.

Usage

PHF data structures can be constructed via either the procedural macros in the phf_macros crate or code generation supported by the phf_codegen crate.

To compile the phf crate with a dependency on libcore instead of libstd, enabling use in environments where libstd will not work, set default-features = false for the dependency:

[dependencies]
# to use `phf` in `no_std` environments
phf = { version = "0.11", default-features = false }

phf_macros

use phf::phf_map;

#[derive(Clone)]
pub enum Keyword {
    Loop,
    Continue,
    Break,
    Fn,
    Extern,
}

static KEYWORDS: phf::Map<&'static str, Keyword> = phf_map! {
    "loop" => Keyword::Loop,
    "continue" => Keyword::Continue,
    "break" => Keyword::Break,
    "fn" => Keyword::Fn,
    "extern" => Keyword::Extern,
};

pub fn parse_keyword(keyword: &str) -> Option<Keyword> {
    KEYWORDS.get(keyword).cloned()
}
[dependencies]
phf = { version = "0.11", features = ["macros"] }

Note

Currently, the macro syntax has some limitations and may not work as you want. See #183 or #196 for example.

phf_codegen

To use phf_codegen on build.rs, you have to add dependencies under [build-dependencies]:

[build-dependencies]
phf = { version = "0.11.1", default-features = false }
phf_codegen = "0.11.1"

Then put code on build.rs:

use std::env;
use std::fs::File;
use std::io::{BufWriter, Write};
use std::path::Path;

fn main() {
    let path = Path::new(&env::var("OUT_DIR").unwrap()).join("codegen.rs");
    let mut file = BufWriter::new(File::create(&path).unwrap());

    write!(
        &mut file,
        "static KEYWORDS: phf::Map<&'static str, Keyword> = {}",
        phf_codegen::Map::new()
            .entry("loop", "Keyword::Loop")
            .entry("continue", "Keyword::Continue")
            .entry("break", "Keyword::Break")
            .entry("fn", "Keyword::Fn")
            .entry("extern", "Keyword::Extern")
            .build()
    )
    .unwrap();
    write!(&mut file, ";\n").unwrap();
}

and lib.rs:

#[derive(Clone)]
enum Keyword {
    Loop,
    Continue,
    Break,
    Fn,
    Extern,
}

include!(concat!(env!("OUT_DIR"), "/codegen.rs"));

pub fn parse_keyword(keyword: &str) -> Option<Keyword> {
    KEYWORDS.get(keyword).cloned()
}

rust-phf's People

Contributors

abonander avatar aidanhs avatar alexcrichton avatar benesch avatar bhgomes avatar bobo1239 avatar chris-morgan avatar cuviper avatar erichdongubler avatar gwenn avatar gz avatar jf2048 avatar johntitor avatar kazurin-775 avatar kmcallister avatar kornelski avatar manishearth avatar mati865 avatar nox avatar nrc avatar pickfire avatar ryman avatar sfackler avatar simonsapin avatar skyfloogle avatar spaceface16518 avatar swatinem avatar techcable avatar turbocool3r avatar upsuper avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust-phf's Issues

Allow hashers other than SipHasher

With a compile-time hashmap, having a cryptographically secure hash is less important since there are no pathological cases to worry about -- everything is done at compile time.

IIRC SipHasher isn't that secure anyway, but there are options like FNV available too.

Perhaps we should allow for these to be selected instead, using defaulted generics?

Upload a final version of phf_mac with a deprecation message in its description

Looking at the package page for phf_mac, there's no indication that it's been deprecated in favor of phf_macros. While I do support this rename, I had to read through a few old issue reports before I understood what had happened.

I suggest uploading a final version of phf_mac, with a description that tells visitors it's deprecated. That'll avoid confusion for people who've been using the package before (like me), or had discovered the package via search.

Support non literal keys

Hello,

is it possible to support keys that are no literals? I try to implement parsing from string to enum variants (works with phf), and formatting the enum variants back to string also with phf (does not work).

static LANGUAGE_FROM_STR: phf::Map<&'static str, Language> = phf_map! {
    "ar" => Language::Ar,
    "de" => Language::De,
    "en" => Language::En,
    "fr" => Language::Fr,
    "sl" => Language::Sl,
};

// The signature is probably wrong
static STR_FROM_LANGUAGE: phf::Map<Language, &'static str> = phf_map! {
    Language::Ar => "ar",
    Language::De => "de",
    Language::En => "en",
    Language::Fr => "fr",
    Language::Sl => "sl",
};

The error:

src/language.rs:26:5: 26:17 error: expected a literal
src/language.rs:26     Language::Ar => "ar",
                       ^~~~~~~~~~~~
src/lib.rs:1:1: 462:1 note: in expansion of phf_map!
src/language.rs:25:62: 31:3 note: expansion site

...

phf_shared-0.5.0/src/lib.rs:2:1: 2:11 error: no_std is experimental

Hello

Extremely new to rust. I'm trying to add emojicons="*" to my project. Emojicons use rust-phf dependencies.

When building with cargo :

$ cargo build --verbose                                                                                                                                                                               [15:25:37]
   Compiling phf_shared v0.5.0
     Running `rustc /Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/phf_shared-0.5.0/src/lib.rs --crate-name phf_shared --crate-type lib -g -C metadata=43fbde1a8f1dfa04 -C extra-filename=-43fbde1a8f1dfa04 --out-dir /Users/bti/code/rust/lessons/guessing_game/target/debug/deps --emit=dep-info,link -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps --cap-lints allow`
       Fresh regex-syntax v0.2.2
       Fresh winapi v0.2.5
       Fresh winapi-build v0.1.1
       Fresh libc v0.2.4
       Fresh memchr v0.1.7
       Fresh aho-corasick v0.4.0
       Fresh advapi32-sys v0.1.2
   Compiling regex v0.1.44
     Running `rustc /Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/regex-0.1.44/src/lib.rs --crate-name regex --crate-type lib -g --cfg feature=\"pattern\" -C metadata=14c7acbc4abf7a40 -C extra-filename=-14c7acbc4abf7a40 --out-dir /Users/bti/code/rust/lessons/guessing_game/target/debug/deps --emit=dep-info,link -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps --extern regex_syntax=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps/libregex_syntax-6d9e9f3bd76497d4.rlib --extern aho_corasick=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps/libaho_corasick-399b4a6bb6d46cc8.rlib --extern memchr=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps/libmemchr-49395bf503c0fa9c.rlib --cap-lints allow`
       Fresh rand v0.3.12
/Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/phf_shared-0.5.0/src/lib.rs:2:1: 2:11 error: no_std is experimental
/Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/phf_shared-0.5.0/src/lib.rs:2 #![no_std]
                                                                                         ^~~~~~~~~~
error: aborting due to previous error
Build failed, waiting for other jobs to finish...
/Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/regex-0.1.44/src/lib.rs:403:34: 403:51 error: #[feature] may not be used on the stable release channel
/Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/regex-0.1.44/src/lib.rs:403 #![cfg_attr(feature = "pattern", feature(pattern))]
                                                                                                                        ^~~~~~~~~~~~~~~~~
error: aborting due to previous error
Could not compile `phf_shared`.

Caused by:
  Process didn't exit successfully: `rustc /Users/bti/.cargo/registry/src/github.com-0a35038f75765ae4/phf_shared-0.5.0/src/lib.rs --crate-name phf_shared --crate-type lib -g -C metadata=43fbde1a8f1dfa04 -C extra-filename=-43fbde1a8f1dfa04 --out-dir /Users/bti/code/rust/lessons/guessing_game/target/debug/deps --emit=dep-info,link -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps -L dependency=/Users/bti/code/rust/lessons/guessing_game/target/debug/deps --cap-lints allow` (exit code: 101)

I've tried cargo update without success. Rust version rustc --version 1.5.0 (3d7cd77e4 2015-12-04).

Any ideas ?

Cheers

Panic in get_index if Map is empty

I was playing around with phf_builder and tried creating an empty Map. Like this:

extern crate phf;
extern crate phf_builder;
fn main() {
    let map: phf::map::Map<String, String> = phf_builder::Map::new().build();
    map.get("aaa");
}

It panics in get:
thread '' panicked at 'attempt to calculate the remainder with a divisor of zero', ~/.cargo/registry/src/github.com-1ecc6299db9ec823/phf_shared-0.7.21/src/lib.rs:46:26

Drop support for scalar keys

rustc is able to optimize matches over scalars just fine, and it'll probably perform significantly better than a table generated by rust-phf. We should focus on strings and byte arrays.

Hash for ASCII data

Hello.

I need lookup for ASCII data only. I think that &str has some overhead, but I can't test it. Simply changing &'static str to &'static [u8] produce lots of compiler errors. Also it writes data to codegen.rs as array, not as b"text".

It it possible?

PS: I'm using phf_codegen on stable rustc.

Performance

From the bench it seems like there is no performance gain compared to hash maps and matching is still faster.

Have you done benchmarks that illustrate a performance gain? I couldn't find anything on this in the Readme or the docs.

build issue

Hi,

I'm trying to build rust-phf on the current version of rustc, and I'm seeing a number of errors like this:

src/map.rs:46:9: 53:14 error: failed to resolve. Could not find `iter` in `std`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...
src/map.rs:46:9: 53:14 error: unresolved name `std::iter::IntoIterator::into_iter`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...
src/map.rs:46:9: 53:14 error: failed to resolve. Could not find `iter` in `std`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...
src/map.rs:46:9: 53:14 error: unresolved name `std::iter::Iterator::next`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...
src/map.rs:46:13: 46:19 error: failed to resolve. Could not find `option` in `std`
src/map.rs:46         for (k, v) in self.entries() {
                          ^~~~~~
src/map.rs:46:13: 46:19 error: unresolved enum variant, struct or const `Some`
src/map.rs:46         for (k, v) in self.entries() {
                          ^~~~~~
src/map.rs:46:9: 53:14 error: failed to resolve. Could not find `option` in `std`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...
src/map.rs:46:9: 53:14 error: unresolved enum variant, struct or const `None`
src/map.rs:46         for (k, v) in self.entries() {
src/map.rs:47             if !first {
src/map.rs:48                 try!(write!(fmt, ", "));
src/map.rs:49             }
src/map.rs:50             try!(write!(fmt, "{:?}: {:?}", k, v));
src/map.rs:51             first = false;
              ...

Is that expected? I really don't understand the error at all.

phf_macros does not compile with beta2

I got the following error

src/lib.rs:34:1: 34:52 error: unstable feature
src/lib.rs:34 #![feature(plugin_registrar, quote, rustc_private)]
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
note: this feature may not be used in the beta release channel
error: aborting due to previous error
Could not compile `phf_macros`.

I guess it's because it uses heavily macros and compiler plugins wich are not in the "stable" part of rust for 1.0.0

Can't use [u8] as a key with phf_codegen

if I use phf_codegen to do the following:

let mut set = phf_codegen::Set::new();
set.entry(b"hello".to_vec());

Then its output looks like this:

entries: phf::Slice::Static(&[
    ([104, 101, 108, 108, 111], ()),
]),

Which doesn't work because there needs to be a & before the array.

Errors on latest nightly

I got here via https://github.com/dinfuehr/dora-rust/ which seems to be up-to-date. Is my nightly too recent?

Compiling phf_macros v0.7.9
/home/odroid/.cargo/registry/src/github.com-48ad6e4054423464/phf_macros-0.7.9/src/lib.rs:84:12: 84:75 error: no method named `span_note` found for type `&mut syntax::ext::base::ExtCtxt<'_>` in the current scope
/home/odroid/.cargo/registry/src/github.com-48ad6e4054423464/phf_macros-0.7.9/src/lib.rs:84         cx.span_note(sp, &format!("PHF generation took {} seconds", time));
                                                                                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/odroid/.cargo/registry/src/github.com-48ad6e4054423464/phf_macros-0.7.9/src/lib.rs:306:16: 306:55 error: no method named `span_note` found for type `&mut syntax::ext::base::ExtCtxt<'_>` in the current scope
/home/odroid/.cargo/registry/src/github.com-48ad6e4054423464/phf_macros-0.7.9/src/lib.rs:306             cx.span_note(*span, "one occurrence here");
                                                                                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
error: aborting due to 2 previous errors

Tests fail on phf

% cargo test
   Compiling phf v0.0.0 (file:///Users/lazarus/dev/rust-phf/phf)
error: internal compiler error: unexpected failure
note: the compiler hit an unexpected failure path. this is a bug.
note: we would appreciate a bug report: http://doc.rust-lang.org/complement-bugreport.html
note: run with `RUST_BACKTRACE=1` for a backtrace
task 'rustc' failed at 'index out of bounds: the len is 56 but the index is 74', /Users/rustbuild/src/rust-buildbot/slave/nightly-mac/build/src/libsyntax/lib.rs:1


Could not compile `phf`.

phf_map! on &'static [T; N] broken

I guess this is a result of issue #76 but it's no longer possible to use phf_map! with b"foo" syntax to get a phf::Map<&'static [u8; 3], T>. Using &'static [1, 2, 3] gives an ICE. [1, 2, 3] works but loses readability for ASCII byte blobs.

#![feature(plugin)]
#![plugin(phf_macros)]

extern crate phf;
extern crate phf_macros;

/* used to work, now type error */
static MAP1: phf::Map<&'static [u8; 3], usize> = phf_map! {
    b"foo" => 2,
};

/* it would be nice if this would work */
static MAP2: phf::Map<[u8; 3], usize> = phf_map! {
    *b"foo" => 2,
};

/* error: internal compiler error: Error constructed but not emitted */
static MAP3: phf::Map<&'static [u8; 3], usize> = phf_map! {
    &'static [4, 3, 2] => 2,
};

/* works */
static MAP4: phf::Map<[u8; 3], usize> = phf_map! {
    [4, 3, 2] => 2,
};

The enum is not big enough ? why the match is fast than phf?

The result:

 D/c/w ╍ cargo bench
   Compiling w v0.1.0 (file:///home/mxo/Downloads/cache/w)
    Finished release [optimized] target(s) in 1.59 secs
     Running /home/mxo/.cache/mozilla/cargo/release/deps/w-ebf44af63816e0a3

running 3 tests
test hashmap ... bench:         109 ns/iter (+/- 4)
test match_  ... bench:          25 ns/iter (+/- 1)
test phf_    ... bench:          75 ns/iter (+/- 5)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured; 0 filtered out

 D/c/w ╍                                                                                                                                           (2s 781ms) 
 D/c/w ╍ cargo bench
    Finished release [optimized] target(s) in 0.0 secs
     Running /home/mxo/.cache/mozilla/cargo/release/deps/w-ebf44af63816e0a3

running 3 tests
test hashmap ... bench:         110 ns/iter (+/- 54)
test match_  ... bench:          23 ns/iter (+/- 0)
test phf_    ... bench:          75 ns/iter (+/- 1)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured; 0 filtered out

 D/c/w ╍                                                                                                                                           (1s 150ms) 
 D/c/w ╍ cargo bench
    Finished release [optimized] target(s) in 0.0 secs
     Running /home/mxo/.cache/mozilla/cargo/release/deps/w-ebf44af63816e0a3

running 3 tests
test hashmap ... bench:         113 ns/iter (+/- 9)
test match_  ... bench:          23 ns/iter (+/- 1)
test phf_    ... bench:          78 ns/iter (+/- 2)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured; 0 filtered out

My code:

#![feature(plugin)]
#![plugin(phf_macros)]
#![feature(test)]
extern crate test;

#[macro_use]
extern crate lazy_static;

extern crate phf;

#[derive(Clone)]
pub enum Keyword {
    Loop,
    Continue,
    Break,
    Fn,
    Extern,
}

fn match_parse_keyword(kw: &str) -> Option<Keyword> {
    match kw {
        "loop" => Some(Keyword::Loop),
        "continue" => Some(Keyword::Continue),
        "break" => Some(Keyword::Break),
        "fn" => Some(Keyword::Fn),
        "extern" => Some(Keyword::Extern),
        _ => None,
    }
}

#[bench]
fn match_(b: &mut test::Bencher) {
    b.iter(|| {
        vec!["loop", "continue", "break", "fn", "extern"]
            .into_iter()
            .for_each(|e| assert!(match_parse_keyword(e).is_some()))
    })
}

static KEYWORDS: phf::Map<&'static str, &'static Keyword> = phf_map! {
    "loop" => &Keyword::Loop,
    "continue" => &Keyword::Continue,
    "break" => &Keyword::Break,
    "fn" => &Keyword::Fn,
    "extern" => &Keyword::Extern,
};

pub fn phf_parse_keyword(keyword: &str) -> Option<&'static Keyword> {
    KEYWORDS.get(keyword).map(|x|*x )
}
#[bench]
fn phf_(b: &mut test::Bencher) {
    b.iter(|| {
        vec!["loop", "continue", "break", "fn", "extern"]
            .into_iter()
            .for_each(|e| assert!(phf_parse_keyword(e).is_some()))
    })
}

fn hashmap_parse_keyword(kw: &str)-> Option<&'static Keyword> {
lazy_static!{
        static ref MAP: std::collections::HashMap<&'static str, Keyword>= vec![("loop" ,Keyword::Loop),("continue", Keyword::Continue), ("break" , Keyword::Break), ("fn" , Keyword::Fn), ("extern" , Keyword::Extern) ].into_iter().collect();
    }
    MAP.get(kw)
}
#[bench]
fn hashmap(b: &mut test::Bencher) {
    b.iter(|| {
        vec!["loop", "continue", "break", "fn", "extern"]
            .into_iter()
            .for_each(|e| assert!(hashmap_parse_keyword(e).is_some()))
    })
}

fn main() {
    println!("Hello, world!");
}

Support for syntex in phf code generation on stable

The crate serde-rs/syntex provides basically the same functionality as libsyntax but can be used with stable Rust, so that more tidy macros could be expanded by a unified build script instead of current codegen approach, when the entries are hard-coded into build.rs itself.

Can the phf_macros crate possibly be made to use syntex on stable Rust?

UPDATE: It appears that serde actually uses syntex in serde_codegen, not serde_macros crate. So maybe it's plausible to use syntex to make phf_codegen parse the same input files as with phf_macros.

add a specialized StringMap type for smaller code, especially in shared libraries

The current representation of phf::Map for string keys forces each entry in the map to have a pointer to some string data, which requires relocations in shared libraries and may force the entries to be larger than they need be, depending on alignment requirements. (This is a common problem in C; I'm less sure if it applies to Rust, because I'm less familiar with what Rust does in this case.)

gperf includes the --pic option for dealing with this; entries in the map then contain integer offsets into a string pool, rather than character pointers. The relocations are eliminated, resulting in smaller binaries, slightly improved startup performance, and potentially improving data sharing. Additionally, the integer offset type can then be sized appropriately to the total size of the string pool, which may result in it being u8, u16, or u32 as appropriate. This last part is probably harder to do for phf, but I have found it useful for one-off tables.

Using C instead of Rust, the alignment requirements come about because you have the equivalent of:

struct entry {
  char* key;
  <enum type> value;
};

On x86-64, this requires 16 bytes regardless of the size of enum type. If you can use an integer offset instead, you have:

struct entry {
  uint32_t key_offset;
  <enum type> value;
};

which fits into 8 bytes instead. Depending on the precise size of enum type, you might be able to use a smaller type for key_offset and get the entry size down to 4 or so. Maybe the wins for Rust would be more significant here, because &'static str are larger than a pointer? Not sure on this.

Generic key type prevents finding with non-static slices

PhfMap<&'static str, Keyword> has K set to &'static str. This is an absolute killer for rust-phf, negating (without unsafe code) the whole point of the library.

fn find<'a>(&'a self, key: &K) -> Option<&'a V>

In other words, this is

fn find<'a>(&'a self, key: &&'static str) -> Option<&'a Keyword>

See the problem? I must pass the key as a reference to a static string.

If I wanted to do it based on user input—by far the most common case, I expect—I cannot do that, for it is not &'static str.

As it stands, I must cheat by transmuting the lifetime for the duration of the call, knowing it to be safe.

We evidently need something else. Straight generics don’t cut it.

Rustc crashes with invalid syntax when creating phf using Map

I reported this bug to rust-lang repo, and I was told to open issue here, since it seems to be a bug in phf.

src/main.rs

#![feature(plugin)]
#![plugin(phf_macros)]

extern crate phf;

pub static TEST: phf::Map<&'static str, ()> = phf_map! {
    Signature:: => ()
};

fn main() {}

Cargo.toml

[package]
name = "reproducing_bug"
version = "0.1.0"
authors = ["Hannes Karppila <[email protected]>"]

[dependencies]
phf = "0.7.*"
phf_macros = "0.7.*"

I expected to see this happen: No crash, but a syntax error

Instead, this happened: Rustc crashed

Meta

rustc --version --verbose:

rustc 1.16.0-nightly (4ecc85beb 2016-12-28)
binary: rustc
commit-hash: 4ecc85beb339aa8089d936e450b0d800bdf580ae
commit-date: 2016-12-28
host: x86_64-apple-darwin
release: 1.16.0-nightly
LLVM version: 3.9

Backtrace: ($ RUST_BACKTRACE=1 cargo run --verbose)

       Fresh phf_shared v0.7.20
       Fresh libc v0.2.18
       Fresh phf v0.7.20
       Fresh rand v0.3.15
       Fresh phf_generator v0.7.20
       Fresh phf_macros v0.7.20
   Compiling reproducing_bug v0.1.0 (file:///Users/Hannes/codes/rust/2/minimal_reproduce)
     Running `rustc --crate-name reproducing_bug src/main.rs --crate-type bin -g -C metadata=876b28311dab6b77 -C extra-filename=-876b28311dab6b77 --out-dir /Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps --emit=dep-info,link -L dependency=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps --extern phf_macros=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps/libphf_macros-af33d2f4f2cea9b3.dylib --extern phf=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps/libphf-b7085cc37d7a6d9f.rlib`
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: run with `RUST_BACKTRACE=1` for a backtrace

thread 'rustc' panicked at 'called `Result::unwrap()` on an `Err` value: Diagnostic { level: Fatal, message: "expected identifier, found `=>`", code: None, span: MultiSpan { primary_spans: [Span { lo: BytePos(136), hi: BytePos(138), expn_id: ExpnId(4294967295) }], span_labels: [] }, children: [] }', /Users/rustbuild/src/rust-buildbot/slave/nightly-dist-rustc-mac/build/src/libcore/result.rs:845
stack backtrace:
   1:        0x1075de57c - std::sys::imp::backtrace::tracing::imp::write::h36c82bddc32ee014
   2:        0x1075eabee - std::panicking::default_hook::{{closure}}::h5befc2c9c6395cdf
   3:        0x1075ea793 - std::panicking::default_hook::hc066c0e0f8dcf194
   4:        0x1075eb0b6 - std::panicking::rust_panic_with_hook::hb2f44ed7df6389fd
   5:        0x1075eaf54 - std::panicking::begin_panic::h98e44359ade7bd0a
   6:        0x1075eae72 - std::panicking::begin_panic_fmt::hb5af1180f1c36e1d
   7:        0x1075eadd7 - rust_begin_unwind
   8:        0x1076272b0 - core::panicking::panic_fmt::h37d320d6d0d8b0d4
   9:        0x10823d3c8 - core::result::unwrap_failed::h876af85011fc2863
  10:        0x10822d8f7 - <core::result::Result<T, E>>::unwrap::hedb659003fcc10ed
  11:        0x1082a2a66 - phf_macros::parse_map::hb30158bf62d13777
  12:        0x1082a15fc - phf_macros::expand_phf_map::he99af7a44deaaf58
  13:        0x103940b35 - <F as syntax::ext::base::TTMacroExpander>::expand::h506e0c549ee35b59
  14:        0x106fdbf0c - syntax::ext::expand::MacroExpander::expand_invoc::h73bee7937f28f5a7
  15:        0x106fd8bb0 - syntax::ext::expand::MacroExpander::expand::h688156ccf1139ecc
  16:        0x106fd8149 - syntax::ext::expand::MacroExpander::expand_crate::h0fd8b508eb533b15
  17:        0x102c5586b - rustc_driver::driver::phase_2_configure_and_expand::{{closure}}::h50ad681d9c143607
  18:        0x102c4dc43 - rustc_driver::driver::phase_2_configure_and_expand::h7572e3a30b4fb045
  19:        0x102c4795b - rustc_driver::driver::compile_input::h288ea94170be9057
  20:        0x102c907eb - rustc_driver::run_compiler::ha8e57199bffb1089
  21:        0x102ba0158 - std::panicking::try::do_call::h4527918aedb10721
  22:        0x1075edcaa - __rust_maybe_catch_panic
  23:        0x102bca7e3 - <F as alloc::boxed::FnBox<A>>::call_box::hf4f3e505436bf553
  24:        0x1075e9d24 - std::sys::imp::thread::Thread::new::thread_start::h0986e95fd3be13c5
  25:     0x7fff91e7a99c - _pthread_body
  26:     0x7fff91e7a919 - _pthread_start

error: Could not compile `reproducing_bug`.

Caused by:
  process didn't exit successfully: `rustc --crate-name reproducing_bug src/main.rs --crate-type bin -g -C metadata=876b28311dab6b77 -C extra-filename=-876b28311dab6b77 --out-dir /Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps --emit=dep-info,link -L dependency=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps --extern phf_macros=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps/libphf_macros-af33d2f4f2cea9b3.dylib --extern phf=/Users/Hannes/codes/rust/2/minimal_reproduce/target/debug/deps/libphf-b7085cc37d7a6d9f.rlib` (exit code: 101)

method `Iter` is not a member of trait `IntoIterator`

Compiling phf v0.6.5
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/map.rs:133:10: 133:35 error: method Iter is not a member of trait IntoIterator
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/map.rs:133 type Iter = Entries<'a, K, V>;
^~~~~~~~~~~~~~~~~~~~~~~~~
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/set.rs:104:10: 104:29 error: method Iter is not a member of trait IntoIterator
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/set.rs:104 type Iter = Iter<'a, T>;
^~~~~~~~~~~~~~~~~~~
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/ordered_map.rs:152:10: 152:35 error: method Iter is not a member of trait IntoIterator
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/ordered_map.rs:152 type Iter = Entries<'a, K, V>;
^~~~~~~~~~~~~~~~~~~~~~~~~
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/ordered_set.rs:115:10: 115:29 error: method Iter is not a member of trait IntoIterator
/home/gra/.cargo/registry/src/github.com-1ecc6299db9ec823/phf-0.6.5/src/ordered_set.rs:115 type Iter = Iter<'a, T>;
^~~~~~~~~~~~~~~~~~~
error: aborting due to 4 previous errors

Efficient lookup by zero-terminated C string

In some use cases (like returning a pointer by a symbol in FFI scenario) it is needed to look up a value by a given *const c_char, without assuming anything about the encoding except that a single NUL byte always ends the sequence. In this regard it could be similar to a binary lookup, but without having the length of the input sequence as an immediate value beforehand.

Is it reasonable to provide a specialized routine for incremental comparison to save cycles by not performing the length count in a separate pass?

Runtime generation of PHF maps?

Does generating PHF maps at runtime fit the scope of this library? I would like to experiment with the idea, even if it ends up in a different crate. Any pointers on where to begin?

phf memory-mapped files?

I'd love to have phf generate at compile-time a file that it can then memory-map at runtime and use as an efficient perfect hash. That would support programs that want to efficiently store and check a large set of words, but want to allow updating that set at runtime without recompiling the program.

The usage in the build.rs script would look similar, except that instead of writing out a .rs file, it would write out a data file. phf (or some crate depending on it) could then provide a function to open that file, which would memory-map the file and use it as a perfect hash.

Does this seem reasonable?

Integer literal keys

Do integer literal keys not work? I tried to change the example the readme to:

static KEYWORDS: phf::Map<u8, Keyword> = phf_map! {
    3 => Keyword::Loop,
    4 => Keyword::Continue,
    5 => Keyword::Break,
    6 => Keyword::Fn,
    7 => Keyword::Extern,
};

But it yields errors:

src/main.rs:16:5: 16:6 error: unsupported literal type
src/main.rs:16     3 => Keyword::Loop,
                   ^
src/main.rs:1:1: 28:1 note: in expansion of phf_map!
src/main.rs:15:42: 21:2 note: expansion site
src/main.rs:17:5: 17:6 error: unsupported literal type
src/main.rs:17     4 => Keyword::Continue,
                   ^
src/main.rs:1:1: 28:1 note: in expansion of phf_map!
src/main.rs:15:42: 21:2 note: expansion site
src/main.rs:18:5: 18:6 error: unsupported literal type
src/main.rs:18     5 => Keyword::Break,
                   ^
src/main.rs:1:1: 28:1 note: in expansion of phf_map!
src/main.rs:15:42: 21:2 note: expansion site
src/main.rs:19:5: 19:6 error: unsupported literal type
src/main.rs:19     6 => Keyword::Fn,
                   ^
src/main.rs:1:1: 28:1 note: in expansion of phf_map!
src/main.rs:15:42: 21:2 note: expansion site
src/main.rs:20:5: 20:6 error: unsupported literal type
src/main.rs:20     7 => Keyword::Extern,
                   ^
src/main.rs:1:1: 28:1 note: in expansion of phf_map!
src/main.rs:15:42: 21:2 note: expansion site
error: aborting due to 5 previous errors

Am I doing something wrong or is it genuinely not supported? If the latter, can it be documented which key types are supported? I could not find it in the documentation (though I apologize if it's there and I missed it).

[Servo release build] Could not find `symbol` in `syntax`

error[E0432]: unresolved import syntax::symbol::Symbol --> /Users/User/Documents/Sources/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/phf_macros-0.7.20/src/lib.rs:57:5 | 57 | use syntax::symbol::Symbol; | ^^^^^^^^^^^^^^^^^^^^^^ Could not find symbol in syntax

error[E0432]: unresolved import syntax::symbol::InternedString --> /Users/User/Documents/Sources/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/phf_macros-0.7.20/src/util.rs:9:5 | 9 | use syntax::symbol::InternedString; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Could not find symbol in syntax

error: no method named as_str found for type &syntax::parse::token::InternedString in the current scope --> /Users/User/Documents/Sources/servo/.cargo/registry/src/github.com-1ecc6299db9ec823/phf_macros-0.7.20/src/lib.rs:242:64 | 242 | ast::LitKind::Str(ref s, _) => Some(Key::Str(s.as_str())), | ^^^^^^

Just giving a heads up.

Update unicase to 2

There is new release of unicase (2.x), so it would be nice if you can make an update.

Generally, only one change in api UniCase("xxx") -> UniCase::new("xxx").. I would send PR myself, but they added support for different encodings, so you can either use case from above or Ascii:new("xxx") and I'm not sure if phf should stick with ascii or not.

Static Hash Values

I have a need to generate hashes of string and $ident values at compile time. Would it be possible to register an additional macro simply called sip_hash to your system which sip::hash and returns the u64?

Thanks

phf_codegen overflows the stack

The phf_codegen crate causes the following error whenever a large number of items (around 500 on my computer) is added to its Map builder:

thread 'rustc' has overflowed its stack
Could not compile 'myprogram'

I do not have this issue when using the phf_map! macro, but I can't use that on stable Rust.

phf_map! cannot be used with &'static [u8] keys

tests/test.rs:20:63: 22:6 error: mismatched types:
 expected `phf::map::Map<&'static [u8], &'static str>`,
    found `phf::map::Map<&[u8; 9], &str>`
(expected slice,
    found array of 9 elements) [E0308]
tests/test.rs:20     static SLICE_KEY: phf::Map<&'static [u8], &'static str> = phf_map!(
tests/test.rs:21         b"camembert" => "delicious",
tests/test.rs:22     );

wrong number of lifetime parameters in src/map.rs line 130

rust-postgres has a dependency on rust-phf, and when I try to compile my code that uses rust-postgres I get this error:

src/map.rs:130:11: 130:78 error: wrong number of lifetime parameters: expected 0, found 1 [E0107]
src/map.rs:130     iter: iter::Map<'a, &'a (K, V), (&'a K, &'a V), slice::Items<'a, (K, V)>>,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/map.rs:130:11: 130:78 error: wrong number of type arguments: expected 4, found 3
src/map.rs:130     iter: iter::Map<'a, &'a (K, V), (&'a K, &'a V), slice::Items<'a, (K, V)>>,
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I don't really know how lifetime parameters work. I'm having a hard time figuring out why this happens.

phf_map! (sort of) on stable with macro_rules + proc_macro_derive

You may be interested in this: https://github.com/servo/rust-cssparser/blob/48905424c5214b22a545eb7135cf671fbd63c312/src/lib.rs#L158-L211

I’ve managed to make macro_rules! macro similar to phf_map!, that uses a derive procedural macro internally instead of a compiler plugin,and therefore runs on Rust 1.15 stable.

Differences are:

  • The macro expands to an item, not an expression. (In the phf_map plugin, it expands to a const expression used to initialize a static.) Extra work is needed to support pub v.s. not, doc-comments, etc.
  • Values are given as strings that contain Rust syntax for const expressions, rather than literally as const expressions.

Both are limitations due to busing derive rather than having a proper functional procedural macro (which are not stable yet).

(This particular macro also does case-insensitive matching, but that part can easily be removed.)

Integrate into compiler

The match clause accepts any type of literal, which is syntaxically similar to phf maps. I feel we could simply code it in the compiler to make an intelligent match.

Simplify number literals

I would like to suggest an enhancement, if it is possible, to simplify how number literals are specified. I would like to specify 3 instead of 3u16, basically for readability.

static MYMAP: phf::Map<u16, &'static str> = phf_map! {
    3u16 => "three",
    12u16 => "twelve",
    2222u16 => "two thousand two hundred twenty two",
};

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.