Question: deserialize_option_number_from_string is not compatible with serde_json::Value::String about serde-aux HOT 8 CLOSED

I am sorry to say that once I tried, I realised that it is just plain wrong to do, even though we can. See, there are several issues:

1. When deserialising into a serde_json::Value, all the macros of the MyStruct are simply ignored, as those are only going to be used when deserialising directly into it, rather than any other struct, like serde_json::Value.
2. When we add a branch to deserialise into serde_json::Value, it will always be deserialised, removing all the sense of deserialising into a custom-wrapped struct. This is wrong due to having other types which may be able to be deserialised into their own types and then into a string, which are similar to JSON. In my case, I got the serde_qs value deserialised into the serde_json value, bringing all sorts of issues. I fixed those, but I realised that having such a generic variant simply removes all sense in this annotation (not to mention it is wrong) and even breaks it from working properly for other implementations.

That said, unfortunately, I will not add this variant to this crate, as this crate is supposed to be generic, and it simply won't be if I add it. I can't think of any other way to allow it to work for you as well, except for what you said and manually getting the value from the serde_json::Value object yourself.

from serde-aux.

Hi! Thank you for reporting an issue! This is definitely not intended behaviour. I wonder why it happens to you, as this example is from the tests which pass every night and have been for a long time.

What is Value in your code example?

    fn serde_json_value_eq(s: Value, result: Option<f32>) {
        let a: MyStruct = serde_json::from_value(s).unwrap();
        assert_eq!(a.option_num, result);
        assert_eq!(a.missing, None);
    }
    let json_as_str = r#" { "option_num": "1" } "#;
    let json_as_value = serde_json::from_str::<Value>(json_as_str).unwrap();

In the documentation, we use a string slice:

fn serde_json_eq(s: &str, result: Option<f32>) {
    let a: MyStruct = serde_json::from_str(s).unwrap();
    assert_eq!(a.option_num, result);
    assert_eq!(a.missing, None);
}
fn serde_json_err(s: &str) {
    assert!(serde_json::from_str::<MyStruct>(s).is_err());
}

serde_json_eq(r#" { "option_num": "1" } "#, Some(1.0));
serde_json_eq(r#" { "option_num": "-1" } "#, Some(-1.0));
serde_json_eq(r#" { "option_num": "0.1" } "#, Some(0.1));
serde_json_eq(r#" { "option_num": "-0.1" } "#, Some(-0.1));

And deserialise directly into the struct, not Value (I presume this is serde_json::Value).

from serde-aux.

sorry, my example is not exactly clear. the issue is that serde_json::Value::String is not correct deserialised to the number.

    #[derive(Debug, Deserialize)]
    struct MyStruct {
        #[serde(deserialize_with = "deserialize_option_number_from_string")]
        option_num: Option<f32>,
        #[serde(default, deserialize_with = "deserialize_option_number_from_string")]
        missing: Option<i32>
    }
    fn serde_json_value_eq(s: &str, result: Option<f32>) {
        let json_as_value = serde_json::from_str::<serde_json::Value>(s).unwrap();
        println!("JSON Value: {:?}", json_as_value);
        let a: MyStruct = serde_json::from_value(json_as_value).unwrap();
        assert_eq!(a.option_num, result);
        assert_eq!(a.missing, None);
    }
    serde_json_value_eq(r#" { "option_num": "1" } "#, Some(1.0));

JSON Value: Object {"option_num": String("1")}
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("data did not match any variant of untagged enum NumericOrNull", line: 0, column: 0)'

if I change the NumericOrNull implementation to this then it works:

    #[derive(Deserialize)]
    #[serde(untagged)]
    enum NumericOrNull<'a, T> {
        Str(&'a str),
        FromStr(T),
        Null,
        SerdeString(serde_json::Value),
    }

    match NumericOrNull::<T>::deserialize(deserializer)? {
        NumericOrNull::Str(s) => match s {
            "" => Ok(None),
            _ => T::from_str(s).map(Some).map_err(serde::de::Error::custom),
        },
        NumericOrNull::FromStr(i) => Ok(Some(i)),
        NumericOrNull::Null => Ok(None),
        NumericOrNull::SerdeString(value) => match value {
            Value::String(s) => match s.as_str() {
                "" => Ok(None),
                _ => T::from_str(s.as_str())
                    .map(Some)
                    .map_err(serde::de::Error::custom),
            },
            _ => Err(serde::de::Error::custom(value)),
        },
    }

here are my dependencies

[dependencies]
serde = { version = "1.0.159", features = ["derive"] }
serde_json = "1.0.95"
serde-aux = "4.2.0"

from serde-aux.

Yes, so this is neither intentional nor unintentional. The deserialise annotation works with structs, and you deserialise it into a serde_json::Value, which is incorrect, for it is a separate data structure. To make it correct, you would have to provide all the implementations for all the types possible, as you did with SerdeString(serde_json::Value). Nobody deserialises json into serde_json::Value as it just doesn't make sense - you would have to traverse the enum and extract the value anyway, with all the conversions done manually; the main use-case with custom deserializer is to extract data into user data structures, like MyStruct in your example. So, if I may say so, you are just using it wrong. The serde_json::Value isn't MyStruct and you deserialise the json into the first, not into the second.

from serde-aux.

Thank you for the explanation, that makes sense.

I ended up in this situation because I am reading a JSONB column from the database - which is deserialised as a serde_json::Value and I am attempting to deserialise it to a custom struct. But I suppose I should either deserialise the column to a string or convert the value to a string before attempting to deserialise it to a custom struct.

from serde-aux.

Actually, I think, for the serde_json::Value, we can provide this implementation just as you did. Do you mind if I borrow your example code here?

from serde-aux.

Yes, feel free to use my code sample. Thank you!

from serde-aux.

thanks for the explanation. I agree that this crate is supposed to be generic and deserialising into serde_json::Value violates that. appreciate the effort.

from serde-aux.