Code Monkey home page Code Monkey logo

serde-yaml's Introduction

Serde YAML

github crates.io docs.rs build status

Rust library for using the Serde serialization framework with data in YAML file format. (This project is no longer maintained.)

Dependency

[dependencies]
serde = "1.0"
serde_yaml = "0.9"

Release notes are available under GitHub releases.

Using Serde YAML

API documentation is available in rustdoc form but the general idea is:

use std::collections::BTreeMap;

fn main() -> Result<(), serde_yaml::Error> {
    // You have some type.
    let mut map = BTreeMap::new();
    map.insert("x".to_string(), 1.0);
    map.insert("y".to_string(), 2.0);

    // Serialize it to a YAML string.
    let yaml = serde_yaml::to_string(&map)?;
    assert_eq!(yaml, "x: 1.0\ny: 2.0\n");

    // Deserialize it back to a Rust type.
    let deserialized_map: BTreeMap<String, f64> = serde_yaml::from_str(&yaml)?;
    assert_eq!(map, deserialized_map);
    Ok(())
}

It can also be used with Serde's derive macros to handle structs and enums defined in your program.

[dependencies]
serde = { version = "1.0", features = ["derive"] }
serde_yaml = "0.9"

Structs serialize in the obvious way:

use serde::{Serialize, Deserialize};

#[derive(Debug, PartialEq, Serialize, Deserialize)]
struct Point {
    x: f64,
    y: f64,
}

fn main() -> Result<(), serde_yaml::Error> {
    let point = Point { x: 1.0, y: 2.0 };

    let yaml = serde_yaml::to_string(&point)?;
    assert_eq!(yaml, "x: 1.0\ny: 2.0\n");

    let deserialized_point: Point = serde_yaml::from_str(&yaml)?;
    assert_eq!(point, deserialized_point);
    Ok(())
}

Enums serialize using YAML's !tag syntax to identify the variant name.

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, PartialEq, Debug)]
enum Enum {
    Unit,
    Newtype(usize),
    Tuple(usize, usize, usize),
    Struct { x: f64, y: f64 },
}

fn main() -> Result<(), serde_yaml::Error> {
    let yaml = "
        - !Newtype 1
        - !Tuple [0, 0, 0]
        - !Struct {x: 1.0, y: 2.0}
    ";
    let values: Vec<Enum> = serde_yaml::from_str(yaml).unwrap();
    assert_eq!(values[0], Enum::Newtype(1));
    assert_eq!(values[1], Enum::Tuple(0, 0, 0));
    assert_eq!(values[2], Enum::Struct { x: 1.0, y: 2.0 });

    // The last two in YAML's block style instead:
    let yaml = "
        - !Tuple
          - 0
          - 0
          - 0
        - !Struct
          x: 1.0
          y: 2.0
    ";
    let values: Vec<Enum> = serde_yaml::from_str(yaml).unwrap();
    assert_eq!(values[0], Enum::Tuple(0, 0, 0));
    assert_eq!(values[1], Enum::Struct { x: 1.0, y: 2.0 });

    // Variants with no data can be written using !Tag or just the string name.
    let yaml = "
        - Unit  # serialization produces this one
        - !Unit
    ";
    let values: Vec<Enum> = serde_yaml::from_str(yaml).unwrap();
    assert_eq!(values[0], Enum::Unit);
    assert_eq!(values[1], Enum::Unit);

    Ok(())
}

License

Licensed under either of Apache License, Version 2.0 or MIT license at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this crate by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

serde-yaml's People

Contributors

asomers avatar atouchet avatar cad97 avatar colltoaction avatar cuviper avatar davidkorczynski avatar dtolnay avatar ignatenkobrain avatar jturner314 avatar jturner314-nrl avatar kamilaborowska avatar killercup avatar knkski avatar lindenk avatar luke-biel avatar macisamuele avatar manuel-woelker avatar masinc avatar mgeisler avatar oli-obk avatar p4l1ly avatar pedrocr avatar razrfalcon avatar rukai avatar sfackler avatar sinkuu avatar stephanbuys avatar tshepang avatar vitiral avatar vroland avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

serde-yaml's Issues

Make ErrorImpl public

To present more advanced and convenient error messages like hints for the mismatched type errors, one would need to have the full error implementation, particularly the position and path, which are currently hidden for inspection inside a yaml::Error as a ErrorImpl::Message(_, Option<Pos>)

/// This type represents all possible errors that can occur when serializing or
/// deserializing YAML data.
pub struct Error(Box<ErrorImpl>);

/// This type represents all possible errors that can occur when serializing or
/// deserializing a value using YAML.
#[derive(Debug)]
pub enum ErrorImpl {
    Message(String, Option<Pos>),
    // Other errors
}

Why is the Box<ErrorImpl> not exposed as a pub member in yaml::Error while it is a crate-public enum?

Should I make a PR changing it or there are API design concerns to take into account?

Real numbers should be parsed and `visit_f64` instead of `visit_str`

I ran into a problem implementing a deserialize_with for a type based on f32. Source (important part here is the impl Visitor for FloatVisitor):

impl DeserializeFromF32 for Size {
    fn deserialize_from_f32<D>(deserializer: &mut D) -> ::std::result::Result<Self, D::Error>
        where D: serde::de::Deserializer
    {
        use std::marker::PhantomData;

        struct FloatVisitor<__D> {
            _marker: PhantomData<__D>,
        }

        impl<__D> ::serde::de::Visitor for FloatVisitor<__D>
            where __D: ::serde::de::Deserializer
        {
            type Value = f32;

            fn visit_f32<E>(&mut self, value: f32) -> ::std::result::Result<Self::Value, E>
                where E: ::serde::de::Error
            {
                Ok(value)
            }
        }

        deserializer
            .deserialize_f32(FloatVisitor::<D>{ _marker: PhantomData })
            .map(|v| Size::new(v))
    }
}

Strangely, the config was failing to parse saying it expected a &str. The problem is in the yaml deserialize impl:

    fn deserialize<V>(&mut self, mut visitor: V) -> Result<V::Value>
        where V: de::Visitor,
    {
        match *self.doc {
            Yaml::Integer(i) => visitor.visit_i64(i),
            Yaml::Real(ref s) |
            Yaml::String(ref s) => visitor.visit_str(s),

Floats end up being passed to visit_str! Can we change this to parse the float and pass it to visit_f64?

this serializer isn't working with serde_yaml but it does work with serde_json

I may be doing something wrong, but I wrote this serializer today and tried it with both JSON and YAML.

#[derive(Clone, Eq, PartialEq, Debug)]
pub struct NonEmptyWithCursor<T> {
    most: Vec<T>,
    head: T,
    cursor: usize,
}

impl<T> Serialize for NonEmptyWithCursor<T>
    where T: Serialize
{
    fn serialize<S>(&self, serializer: &mut S) -> Result<(), S::Error>
        where S: Serializer
    {
        let mut state = serializer.serialize_map(Some(2))?;
        serializer.serialize_map_key(&mut state, "cursor")?;
        serializer.serialize_map_value(&mut state, self.cursor)?;
        serializer.serialize_map_key(&mut state, "data")?;
        let mut seqstate = serializer.serialize_seq(Some(self.len()))?;
        serializer.serialize_seq_elt(&mut seqstate, &self.head)?;
        for elt in self.most.iter() {
            serializer.serialize_seq_elt(&mut seqstate, elt)?;
        }
        serializer.serialize_seq_end(seqstate)?;
        serializer.serialize_map_end(state)
    }
}

With JSON output, it emits something like this: {"cursor": 0, "data": [...]}. But with YAML output, it emits the cursor: 0 but not any data.

Is there something else I should be doing to emit a sequence as a value associated with a key? Or is this actually a bug in serde_yaml?

Don't override formatted error messages

Currently, if there is an error while deserializing yaml, serde-yaml will override some errors' formatted messages with their debug messages rather than letting them pass through.

This happens most notably with yaml_rust::scanner::ScanError.

Alias to Scalar should be deserializable as string

While any scalar value can be deserialised as a string, intending to deserialise an alias to a scalar as a string fails.

The following test case (to be added to test_de.rs) fails:

#[test]
fn test_number_alias_as_string() {
    #[derive(Deserialize, PartialEq, Debug)]
    struct Num {
        version: String,
        value: String,
    }
    let yaml = unindent("
        ---
        version: &a 1.10
        value: *a");
    let expected = Num { version: "1.10".to_owned(), value: "1.10".to_owned() };
    test_de(&yaml, &expected);
}

The aliased value is deserialized as a float instead of as a string.

If I understand the code right, that should be easy to fix. If you don't mind I will provide a pull-request.

Remove public dependency on yaml_rust

The public dependency right now is these three impls:

impl From<yaml_rust::Yaml> for serde_yaml::Value { /* ... */ }
impl From<yaml_rust::emitter::EmitError> for serde_yaml::Error { /* ... */ }
impl From<yaml_rust::scanner::ScanError> for serde_yaml::Error { /* ... */ }

Making the dependency entirely private would allow us to freely update yaml_rust versions, switch to a yaml_rust fork if necessary, and most importantly release serde_yaml 1.0 along with serde 1.0 before yaml_rust reaches 1.0.

Should be able to store a Value in a struct

In the process of moving from serde_json to serde_yaml and I am trying to store a Value in a struct

use serde_yaml::Value;

#[derive(Debug, Serialize, Deserialize)]
pub struct GenericInstructions {
    #[serde(rename="type")]
    pub _type: String,
    pub config: Value,
}

but I get

the trait bound `serde_yaml::Yaml: serde::Serialize` is not satisfied

This works fine using serde_json.

{Des,S}erialize not implemented for LinkedHashMap

I'm using serde-yaml to deserialize a custom struct. When I switch to serde 0.9 and serde_yaml 0.6, I get the following sort of error:

error[E0277]: the trait bound `linked_hash_map::LinkedHashMap<serde_yaml::Value, serde_yaml::Value>: serde::Deserialize` is not satisfied
 --> src/tests/spec.rs:9:17
  |
9 | #[derive(Debug, Deserialize, Serialize)]
  |                 ^^^^^^^^^^^ the trait `serde::Deserialize` is not implemented for `linked_hash_map::LinkedHashMap<serde_yaml::Value, serde_yaml::Value>`
  |
  = note: required by `serde::de::SeqVisitor::visit`

I tested against a fork of serde-yaml, and I believe the fix is simply to change the linked-hash-map dependency in Cargo.toml to:

linked-hash-map = { version = "0.4", features = ["serde_impl"] }

(I believe the version bump from 0.3 is worthwhile / harmless, but the serde_impl feature is the key bit here.)

Out of bounds access in output generated by PyYAML

Moved from #49 (comment).


Hi, I ran into the Out of bounds access in the wild with some output generated by PyYAML. I minimized it to a pretty simple example:

---
content: "Foo\
    Bar"

Is this just a case of missing support for line continuations? Should I file a separate ticket for that?


@radix

Release Package to track Serde 1.0

I see that you have already started tracking 1.0 of serde which is awesome. I also pulled down a copy of the code and see that you are tracking the master branch of serde. All tests pass and the project successfully compiles.

The README.md still states serde="0.9" and https://crates.io/crates/serde_yaml shows ^0.9.2. How can I help release a new serde_yaml package that reports a dependency of serde=1.0.0 or equiv.

Happy to help make it happen.

Provide better error messages when deserializing

Currently when deserializing, if a serde error occurs, the source line and column where the error originated from is not included in the resulting serde-yaml::Error. I would propose that this information should be part of serde-yaml errors.

Fork yaml-rust

The yaml-rust maintainer has been very slow to respond to pull requests and it is harmful to this library. It's time for a fork.

Invalid code generation for HashMap keys

In YAML, map keys are allowed to be complex types, such as arrays or maps. For example, a Rust HashMap<Vec<u32>, String> could be written in YAML as [1, 2, 3]: foo.

However, serializing HashMap<Vec<u32>, String> with serde-yaml 0.7 gives back this string: [+ 1 , 2 , 3]: foo. The + sign is a problem, because + 1 parses as a string, while +1 parses as a number, which means the YAML can't be deserialized correctly.

serde-yaml 0.5.0 is harder to test because keys are no longer alphabetical

Thank you for serde_yaml! We're using it almost everywhere in cage now, and it's great.

I tried upgrading to 0.5.0 this morning, however, and a lot of my YAML-related test suites started failing. These test suites often take the form of round-trip YAML parsing and serialization tests:

macro_rules! assert_roundtrip {
    ( $ty:ty, $yaml:expr ) => {
        {
            let yaml: &str = $yaml;
            let data: $ty = serde_yaml::from_str(&yaml).unwrap();
            let yaml2 = serde_yaml::to_string(&data).unwrap();
            assert_eq!(normalize_yaml(yaml), normalize_yaml(&yaml2));
        }
    }
}

#[test]
fn extends_can_be_roundtripped() {
    let yaml = r#"---
"file": "bar/docker-compose.yml"
"service": "foo"
"#;
    assert_roundtrip!(Extends, yaml);
}

These test cases fail because they no longer output YAML keys in an easily predictable order:

---- v2::extends_can_be_roundtripped stdout ----
    thread 'v2::extends_can_be_roundtripped' panicked at 'assertion failed: `(left == right)` (left: `"---\n\"file\": \"bar/docker-compose.yml\"\n\"service\": \"foo\""`, right: `"---\n\"service\": \"foo\"\n\"file\": \"bar/docker-compose.yml\""`)', src/v2/extends.in.rs:53
note: Run with `RUST_BACKTRACE=1` for a backtrace.

If keys are being output in unstable orders, this also means that we won't be able to usefully diff the YAML files output by cage export, which is helpful if we need to debug a configuration change to our production servers.

Would you expect the new order of serialized YAML keys to be predictable from run to run?

Thank you very much for maintaining serde_yaml, and for any advice you can provide!

Remove from_iter

This was added to serde_json years ago and is superseded by from_reader.

Deserialization of optional types is still required by serde-yaml

Take the following example:

#[derive(Deserialize)]
struct Data {
  a: Option<f64>,
  b: Option<String>,
  c: bool
}

and the following YAML:

b:
c: true

For this I would expect an object initialized like the following:

Data {
  a: None,
  b: Some("".to_string()),
  c: true
}

Instead serde_yaml throws an error that a is not defined.

Serialize large u64 values.

I'm not sure if the issue here lies with yaml-rust or with the serde wrapper.

But serializing large u64 values results in negative numbers.
I haven't been able to determine where exactly the cutoff is.

Minimal reproduction:

Rust: rustc 1.15.0-nightly (d9bdc636d 2016-11-24)
serde: 0.8
serde_yaml: 0.5

#![feature(proc_macro)]

#[macro_use] extern crate serde_derive;
extern crate serde_yaml;

#[derive(Serialize)]
struct T {
  pub x: u64,
}

fn main() {
  let t = T { x: ::std::u64::MAX };
  let yaml = ::serde_yaml::to_string(&t).unwrap();
  println!("{}", yaml);
}

This results in x: -1.

Option to remove quotes

Hello.
Is there an option to remove the quotes from the yaml output?
As far as I know, quotes are not really used in YAML:

Strings (scalars) are ordinarily unquoted, but may be enclosed in double-quotes ("), or single-quotes (').

Thanks.

Parsing a unit enum value as a String rather than a HashMap

Currently, when parsing an enum type, any Values must be a HashMap:

main.rs:

#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]

#[macro_use]
extern crate serde;
extern crate serde_yaml;

use std::fs::File;


#[derive(Serialize, Deserialize, Debug)]
enum Beans {
    Blue,
    Black,
    Chamomile
}

#[derive(Serialize, Deserialize, Debug)]
struct Food {
    beans : Beans
}

fn main() {
    let f = File::open("test_yaml.yaml").unwrap();
    let test : Food = serde_yaml::from_reader(f).unwrap();

    println!("{:?}", test);
}

test_yaml.yaml:

beans:
    Blue: ()

This would will parse Blue into test. However, it is inconvenient to require all unit enum values to have associated non-existent data. Instead, if the value is a unit value, it should also be parsable as a string.

beans: Blue

FmtToIoWriter flushes on every tiny string written

Thank you for the serde_yaml support! We're making heavy use of this in cage.

There's a call to flush in FmtToIoWriter that slows down YAML output by making a large number of system calls:

Sample strace output:

write(3, "\"", 1)                       = 1
write(3, "\n", 1)                       = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, "\"", 1)                       = 1
write(3, "ENV_NAME", 8)                 = 8
write(3, "\"", 1)                       = 1
write(3, ": ", 2)                       = 2
write(3, "\"", 1)                       = 1
write(3, "development", 11)             = 11
write(3, "\"", 1)                       = 1
write(3, "\n", 1)                       = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, " ", 1)                        = 1
write(3, "\"", 1)                       = 1

Since we do a large amount of YAML output, these syscalls are taking up a large amount of our program's runtime.

I'd be happy to try and prepare PR if you're interested.

Remove the flush in to_writer

Added in 5df9007 to preserve backward compatibility with 0.4.0 which always flushed the writer. We should be able to leave flushing to the caller like serde_json does.

cc @emk

`DeserializeSeed` support

Sometimes, it's necessary for a type to only implement DeserializeSeed, and not Deserialize. serde_yaml should support this use case, especially with the functions related to from_str. I think the type should look something like this:

fn from_str_seed<T>(seed: T, s: &str) -> Result<T::Value>
    where T: DeserializeSeed

Document all public items

The goal is to add #[deny(missing_docs)] to lib.rs and have it compile.

It seems that currently, mapping.rs is the only mod with no docs. And we can probably copy them together from the collections in std, as they are mostly the same methods.

Expose the Serializer and Deserializer types

This is required for composition with other libraries like serde_ignored.

This is harder than just slapping pub on the thing because the Deserializer currently borrows some data owned by the serde_yaml::from_str method:

struct Deserializer<'a> {
    events: &'a [(Event, Marker)],
    /// Map from alias id to index in events.
    aliases: &'a BTreeMap<usize, usize>,
    pos: &'a mut usize,
    path: Path<'a>,
}

May need to refactor into a public owned Deserializer and an internal borrowed Deserializer.

The Serializer is also hard because it doesn't write to an underlying io::Write like JSON's serializer does.

Omit empty Vec

Hello.
It would be nice to be able to omit empty vectors in the output and also being able to parse an input with a missing field to the empty vector.

For instance, the following code:

    let point = Point { x: 1.0, y: 2.0, children: vec![] };

could produce:


---
"x": 1
"y": 2

instead of:


---
"x": 1
"y": 2
"children": []

Thanks.

Provide entry API for Value

Also useful internally in a few places:

// TODO: use entry() once LinkedHashMap supports entry()
// https://github.com/contain-rs/linked-hash-map/issues/5

Release 0.6.0

I published 0.6.0-rc1 with support for serde 0.9 and a rewritten parser that shows line and column for errors. Some examples from the test suite:

invalid value: string "str", expected an integer at line 2 column 7

missing field `w` at line 2 column 2

invalid length 2, expected a tuple of size 3 at line 2 column 1

I don't have any projects myself that use serde_yaml so @sfackler or @jwilm would you be interested in updating one of your projects to rc1 as a sanity check?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.