Verify Vector Clocks are sent about riak-dotnet-client HOT 12 CLOSED

Killing this for now - this 'feature' isn't the business of the low level client in CorrugatedIron but, instead, belongs in an O-R/M type tool.

from riak-dotnet-client.

How DO you set the vector clock on PUT requests? obj.VectorClock has a get-accessor only.

from riak-dotnet-client.

PS, I'm getting a BASE64-encoded clock back in a header, which I want to add to the PUT request. It's not in the content byte array; or otherwise, if it has to be, how do I put it in there?

from riak-dotnet-client.

Hello @haf,

Apologies for the slow response. We didn't expose this as part of the Put as we assume that people have always done a Get to hydrate the object prior to calling Put. Basho themselves state that this is recommended practice. Without doing so, you can't guarantee that the object you're attempting to represent has all the data in it that you need. For example, if you set the vclock you're telling Riak that you're modifying a value with respect to a particular point. If you don't also set all the indexes, links, etc as well then you'll lose all of those important details.

I recommend not doing what you're attempting to do, and instead calling Get via CI before calling Put.

Hope that helps.
OJ

from riak-dotnet-client.

Hi,

I am writing a very simple Riak proxy (because riak barfs on some HTTP headers otherwise), so I do do a GET before I do a PUT, but I need the ability to set the vclock data again. Doesn't this qualify for a good use-case?

Although, you did mention things I didn't know about: indicies, links, etc; why not provide a convenient SetMetaData(vclock, indicies, links, etc) method that I can use to set all things, so that I don't forget anything?

Regards,
Henrik

from riak-dotnet-client.

Hello,

I'd have though that it would have made more sense to look at something like nginx and configuration to strip out the headers for you. Then you can just plug the two together and off you go. Seems like a lot of work just to remove some headers!

if you do a GET before a PUT, why do you need to set the Vclock again, don't you already have it?

The reason we don't provide a SetMetaData()-like function is because the format and usage of all the metadata in Riak isn't simple. It's quite complex and we want to hide that detail away from the users with an API that's clean and makes sense. If we were to attempt this we'd have to deal with parsing things in the myriad of ways that the data could be expressed, which isn't a small job. We already do a lot of parsing to handle the known and proper case.

Other than this specific use case, which I think could perhaps be better solved with an off the shelf reverse-proxy, I can't think of a need to set those values in bulk like that, especially when you do a GET in CI before the PUT.

Cheers.
OJ

from riak-dotnet-client.

Hi,

I investigated nginx before I took this path. The problem with nginx is 1) you require a custom-built module to support the stripping of headers, 2) the http_headers_more module doesn't seem to support the white-listing of properties, but only black-listing. It seems an arbitrary header can make Riak barf (or there is no way I can prove this is not the case, unless I send all headers possible to riak, or alternatively spend a lot of time digging through its source code).

No, I don't have it in the proxy service, because that proxy service is stateless. The client of the proxy has it. I'm not making the API proxy stateful with all the problems that bring with it.

If you can show me the nginx configuration and garantuee that it works, that is an OK solution, otherwise, why not just let me set the vclock? I don't understand why I would be dropping links or misc data -- I'm not even interested in that functionality right now -- so there won't be any exotic links or "etc" data in there...

Kind regards,
Henrik

from riak-dotnet-client.

As of Riak v1.0, the sane behavior is for clients to pass vclock handling over to the server. This is documented in Client vs Vnode Vector Clocks. This should be the default on any new Riak cluster built. If you've upgraded from an older version it might be different. You can verify this in the app.config for any node - look for {vnode_vclocks, true} in the riak_kv section of the file. It should be on or around line 172. If you indeed do have vnode_vclocks set to true, you have nothing to worry about.

From a client design standpoint, we've opted to err on the side of making things safe and sane for users while hiding some of the difficult bits that come along with a distributed database like Riak. There's no reason for a client to be setting vclocks in all but the rarest of use cases. A proxy server does not qualify as rare.

from riak-dotnet-client.

@peschkaj
I belive you are mistaken in your interpretation of how vector clocks are used, or I might be misunderstanding you. My assumption is to use the system as it was originally designed, with sibling values and multi-master vnodes.

Vector clocks are both logical clocks and participant identifiers. Previously clients were participants in the distributed computation from an id-in-vclock perspective, now, only vnodes are.

However, and this is the big if, I want to defer merges to reads as I would have to in an Available system with multiple masters, I am forced to pass along with a PUT request, a vector clock that can be used by the vnode to know whether I have seen a previous merge (event) or not. If I don't pass the vector clock along with the PUT, then the vnode cannot handle the request semantically with respect to previously seen events, so it will create a sibling value, always (unless the key has never been seen before).

Hence, I need to pass vector clocks along. You suggest I do a GET before I do a PUT; and yes, I could do this inside the handler of the PUT, inside my proxy, but then I would in an instant, not only double the number of messages, but also push a lot more data (all of the value, alongside the key) over the network (optimisable to HEAD requests, I presume, but I don't know if the .Net client supports that, since again, I can't set the needed headers.)

So either you are saying; do a "double tap in the HEAD" (pun intended), or you are telling me to use laster-writer-wins (except of course, there's no global time, meaning basically 'an arbitrary writer wins').

On the other hand, you tell me I could use nginx, but I'm saying that I can't, because I don't know a way to filter headers properly.

Finally, I have another use-case that requires client ids (and we are designing our API to be able to generate stateless [no state inside API that is] client ids). In the long run we'd like to make use of the CRDT functionality in riak_dt: because we are using Riak to build read models, it is a simple task to define commutative and associative functions that merge sibling values/events and for this we need to be able to have VClocks.

I hope this addressed all of your points.

Kind regards,
Henrik

from riak-dotnet-client.

If you'd started with CRDTs, this issue would've been reopened 19 days ago.

from riak-dotnet-client.

Hi all,

Jeremiah's changes have been merged into the develop branch now. The RiakObject implements a new interface called IWriteableVclock in an explicit manner. This interface allows the vclock to be set. It requires the user to cast the object to the interface before it can be called as we didn't want this exposed to everyone.

We have not yet added this to the documentation.

OJ

from riak-dotnet-client.

Sweet!

On 19 maj 2013, at 00:38, OJ Reeves [email protected] wrote:

Hi all,

Jeremiah's changes have been merged into the develop branch now. The RiakObject implements a new interface called IWriteableVclock in an explicit manner. This interface allows the vclock to be set. It requires the user to cast the object to the interface before it can be called as we didn't want this exposed to everyone.

We have not yet added this to the documentation.

OJ

—
Reply to this email directly or view it on GitHub.

from riak-dotnet-client.