Comments (23)
Do you happen to have a simple test case? Which compression are you using?
from cdrs-tokio.
I have a service in rust which received the compressed data from a python code. I use lz4. I was able to insert the same exact data using python driver.
from cdrs-tokio.
Can you share your code how you insert the data (with the types used)?
from cdrs-tokio.
I cannot share my code since it for my company.
I use type of blob
in the Cassandra table.
In the code I insert it as Vec<u8>
it is something like this:
struct CassandraItem {
id_number: String,
blob: BinaryData,
}
impl CassandraItem {
fn into_query_values(self) -> QueryValues {
query_values!(
self.id_number,
self.blob.inner().to_vec(), // convert the Bytes obj to Vec<u8>
)
}
}
async fn insert_row(
session: Arc<CurrentSession>,
row: CassandraItem,
) -> Result<Envelope, String> {
match session
.query_with_values("INSERT INTO key_space.table1 (id_number,blob) VALUES (?,?)", row.into_query_values())
.await
{
Ok(res) => Ok(res),
Err(err) => {
...
}
}
}
from cdrs-tokio.
Thanks for the code snippet - now it's clear why it's failing. Cassandra contains the list type which is represented by Vec
in Rust. When you pass a Vec<u8>
as a query value, you are creating a list of u8 values, not a blob of binary data. You should either use the Blob
type from the crate, or implement Into<Bytes>
for the type you wish to pass as a query value.
Can you confirm it works by doing that?
from cdrs-tokio.
Also, please update to the latest patch release due to a bug in compressing large envelopes.
from cdrs-tokio.
Thank you for the clarification.
when I do:
impl CassandraItem {
fn into_query_values(self) -> QueryValues {
query_values!(
self.id_number,
Blob::from(self.blob.inner())
)
}
}
it stack, doesn't panic but also doesn't return back, looks like infinite loop.
For the other option, how can I implementing into<Bytes>
if it is private and not in the same trait as Bytes?
from cdrs-tokio.
Are you sure you're using the patched version from today? It fixed some problems with compressed messages. As for Bytes
- it's public: https://docs.rs/cdrs-tokio/latest/cdrs_tokio/types/value/struct.Bytes.html
from cdrs-tokio.
Sorry I didn't realized it is pub.
So, I tried Bytes::new(self.blob.inner().to_vec()),
still stack.
Since I update to 7.0.1 it doesn't work - it appears to be getting into a infinite loop and doesn't return at all.
The problem is not in the db since I can contact it with python driver.
Any thoughts?
from cdrs-tokio.
The tests at https://github.com/krojew/cdrs-tokio/blob/master/cdrs-tokio/tests/compression.rs send 5MB blob to the db and read it back, with compression. Therefore it generally works, so something is specific in your case, but it's hard to tell what without details. Let's try to dig deeper.
- How do you know it's an infinite loop?
- Can you run these tests with your db configuration and see if they work?
- Are there any logs present?
- Can you use a debugger and see where it's stuck?
from cdrs-tokio.
apparently 7.0.1 broke my code, when I used 7.0.0 it worked. Any thoughts?
Using Blob::from(self.blob.inner())
worked. Thank you so much!
from cdrs-tokio.
Can you run those linked tests with your db setup on 7.0.1?
from cdrs-tokio.
Please let me know how I should run library built in test
from cdrs-tokio.
- Run your cluster in the configuration you're having problems with.
- Edit
compression.rs
and change cluster settings to the ones you're having problems with. - Run
cargo test --test compression encode_decode_test_v4 --no-fail-fast --all-features -- --test-threads=1
andcargo test --test compression encode_decode_test_v5 --no-fail-fast --all-features -- --test-threads=1
.
from cdrs-tokio.
I tried it, and I don't get response as before.
from cdrs-tokio.
That's good - that confirms something is specific with your cluster configuration. Can you paste your changes to compression.rs
and how you run your cluster?
from cdrs-tokio.
I just changed the session creation. Unfortunately, I cannot paste it here.
it is something like that:
let cluster_config = NodeTcpConfigBuilder::new()
.with_contact_points(seeds)
.with_authenticator_provider(Arc::new(authenticator))
.build()
.await
.unwrap();
let session =
TcpSessionBuilder::new(RoundRobinLoadBalancingStrategy::new(), cluster_config).build().unwrap();
from cdrs-tokio.
What kind of authenticator are you using? Maybe the cluster is stuck on authentication.
from cdrs-tokio.
let authenticator = StaticPasswordAuthenticatorProvider::new("username", "password");
from cdrs-tokio.
Can you disable authentication and try again without an authenticator?
from cdrs-tokio.
I have to provide one
from cdrs-tokio.
Just tested with an static password authentication and everything works. I ran the cluster with:
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
The only changes to compression.rs
tests are:
.with_authenticator_provider(Arc::new(StaticPasswordAuthenticatorProvider::new(
"cassandra",
"cassandra",
)))
Therefore something is probably wrong with your cluster. Unless you can provide more details, e.g. how do you know it's in infinite loop or the cluster configuration, I cannot provide a solution.
from cdrs-tokio.
Works now. dont know why it didnt work before. Thank you!
from cdrs-tokio.
Related Issues (20)
- Keyspace setup falls Transport to infinite loop HOT 1
- schema change event is never received HOT 23
- Add support for HTTP proxy HOT 2
- error UnexpectedWriteType("CAS")) with multiple tokio tasks HOT 10
- Trouble with dropped connections on Session HOT 5
- Add non exhaustive to proto enums HOT 1
- TLS Example HOT 1
- Paging HOT 13
- Support for tracing informations HOT 4
- Range end index 88 out of range for slice of length 83 HOT 2
- Session can hit errors if used immediately
- rustc stack overflow when deriving IntoCdrsValue and TryFromRow if struct contains static string reference HOT 4
- Case where a machine is up but unreachable HOT 44
- High CPU usage for cdrs_tokio::transport::AsyncTransport::start_processing HOT 19
- exec_with_params is returning ErrorType::Unprepared. HOT 7
- Row to serde_json::Value HOT 12
- ExponentialReconnectionPolicy panics after several attempts HOT 1
- Protocol v5 tests HOT 3
- Bug in transport.rs HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cdrs-tokio.