Cursor iterator returns Error with InvalidResponse "invalid type: unit value, expected a string" about mongo-rust-driver HOT 13 CLOSED

I am also facing this issue using async API with find_one which seems to be using find under the hood, but in my case Im passing non-empty valid document filter which works well in Compass and Im getting little different error, but looks like its thrown from the same place

Error { 
  kind: InvalidResponse { 
    message: "invalid type: map, expected a string"
  }, 
  labels: { }
}

from mongo-rust-driver.

ok, so the issue comes from serde trying to serialize/deserialize the struct and in my case struct had _id: String which should have been _id: bson::oid::ObjectId for serde to correctly unpack the BSON into struct

from mongo-rust-driver.

Hi @vexy, thank you for the additional information! To clarify, the purpose of using the batch_size option in this context is to isolate which document being returned is causing deserialization issues. When batch_size is set to 1, each document will be deserialized individually which allows you to isolate which document is causing the deserialization errors.

Setting batch_size(1) in FindOptions parameter and passing it to a regular find(...) method may either succeed or fail randomly.

Are you seeing InvalidResponse failures similar to the ones detailed above? Have you been able to determine which documents are causing these errors?

I'd recommend attempting the same find operations you've detailed above on a Collection<Document>. This will allow you to inspect the un-deserialized documents being returned from your database to verify that they match the schema you've defined. You can also try to convert them to your type using from_document. Something along the lines of:

let find_options = FindOptions::builder().batch_size(1).build();
let document_collection = collection.clone_with_type::<Document>();
let cursor = document_collection.find(None, find_options).await?;
while let Some(doc) = cursor.try_next().await? {
    // inspect the document returned and attempt to convert it to your defined type
}

from mongo-rust-driver.

Hi @belohnung, the error you're receiving likely indicates that the data returned from your collection does not match the schema of the ASNEntry type you've defined. Have you inspected the data in the collection prior to attempting to deserialize it into ASNEntry? To do this you can convert your collection to a Collection<Document> via Collection::clone_with_type and call find to inspect the data in Document format.

from mongo-rust-driver.

i tried deserializing it manually by letting find return me values and deserializing with serde_json::from_value
and it worked

from mongo-rust-driver.

@belohnung do you have any further questions or comments regarding this issue? In addition to the debugging strategies mentioned above, you can change the batch_size configured for your find operation to 1 in order to deserialize the documents returned from your collection one at a time.

from mongo-rust-driver.

well this is still a bug in the batch deserializer as it seems

from mongo-rust-driver.

@belohnung Can you elaborate? Does the deserialization work when you change batch_size to 1?

from mongo-rust-driver.

i don't have much time right now, once i find some i will try

from mongo-rust-driver.

Following up with my findings on the same issue:

Pay attention to ObjectId field.

I've received similar error messages simply because my _id type was left out both from struct declaration and from any serde-ing.

use mongodb::bson::oid::ObjectId

#[derive(Serialize, Deserialize, Debug)]
pub struct Students {
    #[serde(rename = "_id")]
    pub id: ObjectId,
    pub name: String,
    // - snip -
}

These were some of error debug messages along the adventure:

Error {
    kind: InvalidResponse {
        message: "missing field `name`",
    },
    labels: {},
}

OR

Error { 
  kind: InvalidResponse { 
    message: "invalid type: map, expected a string"
  }, 
  labels: { }
}

OR

Error { 
  kind: InvalidResponse { 
    message: "invalid type: unit value, expected a string"
  }, 
  labels: { }
}

... depending on your struct, these messages may vary. (@belohnung, @tommycodebox)

batch_size usage in FindOptions

Setting batch_size(1) in FindOptions parameter and passing it to a regular find(...) method may either succeed or fail randomly.
Considering the following:

    //prepare find options
    let custom_find_options = FindOptions::builder()
            .batch_size(1)
            .read_concern(mongodb::options::ReadConcern::majority())    //optional
            .build();

    // dump entire collection here
    let result_cursor = self.collection.find(None, custom_find_options).await;
    while let Some(object) = result_cursor.try_next().await? {
        // - something -
    }

This approach works randomly (I'd say depending on the total amount of documents returned), but may fail with following:

thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value:
SendError(Sender { inner: Some(Inner { state: State { is_complete: false, is_closed: false, is_rx_task_set: false, is_tx_task_set: false } }) })',
~/.cargo/registry/src/github.com-1ecc6299db9ec823/mongodb-2.1.0/src/cmap/connection_requester.rs:39:34

@isabelatkinson, this is what I was getting when tried batch_size(100), batch_size(10), batch_size(50). 😕

I would agree with @belohnung, in thinking:

well this is still a bug in the batch deserializer as it seems

...although I'm not sure when and under which conditions query sometimes succeeds.

from mongo-rust-driver.

Hello @isabelatkinson, I've finally had some time to tryout your suggestion.

It turned out that my problem and weird error messages were due to the nature of the underlying document fields. Just as you suggested 💪

Long story short some document fields had to be structured as Option<Type> instead of just Type.
More specifically, this is what was causing trouble (pseudo-struct):

struct NoGo {
    id: ObjectId,
    name: String,  //if there are fields in document that do not have 'name'
}

struct WillWork {
    id: ObjectId,
    name: Option<String>,  //wrap missing fields with Option<T> type
}

Anyhow, thank you for your support 🙏

from mongo-rust-driver.

There has not been any recent activity on this ticket, so we are marking it as stale. If we do not hear anything further from you, this issue will be automatically closed in one week.

from mongo-rust-driver.

There has not been any recent activity on this ticket, so we are closing it. Thanks for reaching out and please feel free to file a new issue if you have further questions.

from mongo-rust-driver.