whoahbot / dm-redis-adapter Goto Github PK

The logic is flipped here:

https://github.com/whoahbot/dm-redis-adapter/blob/master/lib/dm-redis-adapter/adapter.rb#L350

It should be unless instead of if. With it the way it is (using if), the :path variable is essentially renamed to :db, which is what you want when you have a database name - not a socket path.

Hi,

there is still a problem with natural keys. If there is more than one key
defined, the common query does not work anymore.

    it "should find elements with two natural keys" do
      class Foo
        include DataMapper::Resource
        property :key1,   String, :key => true
        property :key2, String, :key => true
      end
      DataMapper.finalize

      Foo.create(:key1 => "value1", :key2 => 'value2')
      Foo.first(:key1 => "value1").key2.should == 'value2'
    end

+1332267068.573186 (db 15) "select" "15"
+1332267068.573313 (db 15) "incr" "foos:key1:key2:serial"
+1332267068.573524 (db 15) "sadd" "foos:key1:key2:all" "value1value2"
+1332267068.573687 (db 15) "hmset" "foos:value1value2" "key1" "value1" "key2" "value2"
+1332267068.574434 (db 15) "sismember" "foos:key1:key2:all" "value1"
+1332267068.575936 (db 15) "select" "15"
+1332267068.576032 (db 15) "flushdb"

Trying this out now. Looking good. One thing I've run into though:

The dm 'last' finder is returning the first item in a collection (i.e. User.last is returning the first User). I'm not sure if this is a limitation of redis.

thanks!

Hello,

I got this when I accidentally connected to a running mongodb instance (I used wrong port):

ruby-1.9.2-p180 :004 > a.update(someid: 123)
SystemStackError: stack level too deep 
from /Users/user/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/irb/workspace.rb:80
Maybe IRB bug!!

It seems that Enums don't work for dm-redis-adapter.
Creating an object with #new and assigning an enum works, but upon #save it just is reflected as nil.

It seem validations aren't working properly.

I've added the following to my initializer:
require 'dm-validations'

and the following to my User model:
property :name, String, :required => true, :index => true

running:
User.create

doesn't raise an exception.

I added an :index => true to a property after already adding some records. Using Person.all(:name => 'test') does then return nothing, I guess because a key lookup is used but then key has not been set before. So I tried running the rake tasks automigrate and autouprade but both dont seem to do anything (according to redis monitor).

So I wonder how to add/ remove an index to existing records? :)

The gem currently has Jeweller and RSpec as primary dependencies, meaning these are installed every time the gem is installed.

Jeweller has an additional 8 dependencies, at least four of which don't need to be included in the average Ruby project.

Is it possible to put these two gems into a :test group so that they don't need to be installed all the time? I'm happy to do this and issue a PR, I'm just wondering if there's a specific reason why they're included.

I updated my fork to use the mset and mget commands. I was able to detect a 15% reduction in time spent just on the tests. All of that was basically from the mget since the tests usually only are setting one value at a time.

My fork

Please excuse me if this is actually not a problem as I am a bit new to both DataMapper and Redis. However I am following the example code in the DataMapper Associations documentation (http://datamapper.org/docs/associations.html) for creating a Twitter-like follower + followed self-referential association in my User model.

This pastie should illustrate the problem: http://pastie.org/2943832

As you can see, when the association is first created, user1 ends up following himself along with the user he was intended to follow, user2 believes she has one follower, but can't seem to find any links_to_followers. Furthermore, unfollowing seems to leave an orphan association on the followed user's side of the association, and just generally seems to make a mess of things.

It seems that the association isn't being stored in/queried from Redis properly, or perhaps it simply isn't possible to store such a relationship in Redis?

When I try to call #count on my DM resource, I receive the following exception:

NoMethodError: undefined method `aggregate' for #DataMapper::Adapters::RedisAdapter:0x007fb60e074f60

Any idea?

i cant make the name string of a simple user model ( id & name ) unique.
so i am able to insert multiple users by the same name.

tried
property :name, String, :key => true
and
validates_is_unique :name

Finding an object with Klass.first(:property => 'value') results in an instance that believes itself destroyed!

require 'rubygems'
require 'dm-core'
require 'dm-redis-adapter'
require 'dm-migrations'

DataMapper.setup(:redis,  "redis://localhost/15")
DataMapper.setup(:other, "sqlite3:/tmp/test.db")

class RedisModel
  include DataMapper::Resource
  def self.default_repository_name
    :redis
  end
  property :id, Serial
  property :foo, String, :index => true
end

class OtherModel
  include DataMapper::Resource
  def self.default_repository_name
    :other
  end
  property :id, Serial
  property :foo, String, :index => true
end
OtherModel.auto_migrate!

RedisModel.create(:foo => 'bar')
OtherModel.create(:foo => 'bar')

puts "Redis", RedisModel.first(:foo => 'bar').destroyed? # => true                                                                                                                                          
puts "Other", OtherModel.first(:foo => 'bar').destroyed? # => false                                                                                                                                         

The error one gets when trying to set properties on such an object is, of course, DataMapper::ImmutableError.

What's the deal? Can you reproduce?

I have three clases, basically it goes like this:

• A user has many instances
• An instance has many workspaces

class User
    include DataMapper::Resource
        ...
    has n, :instances
end

class Instance
    include DataMapper::Resource
        ...
    belongs_to :user
    has n, :workspaces
end

class Workspace
    include DataMapper::Resource
        ...
    belongs_to :instance
end

But when I'm trying to save a new Instance:

u = User.first(:id => id)
i = Instance.new(:name => name, :user => u)

i.save

I get the following error:

NoMethodError at 
undefined method `save' for 2:Fixnum
file: resource.rb location: block in save_children line: 1054

Any help with this please?

When I try User.first I get the first User element. User.last however returns the same element. I'm guessing this is a bug?

http://stackoverflow.com/q/10145851/111363

I have these classes defined:

class Zone

include DataMapper::Resource

property :id, String, :key => true, :default => lambda { |x,y| UUID.new.generate }
property :preview_mode, Boolean, :default => false

timestamps :at

has 1, :campaign
end

and

class Campaign

include DataMapper::Resource

property :id, String, :key => true, :default => lambda { |x,y| UUID.new.generate }
property :name, String

timestamps :at

belongs_to :zone
has n, :rules

validates_presence_of :name
end

I'm able to do Campaign.first.zone but not Zone.first.campaign.

I would like to be able to do the lookups in both directions.

When I change the id to a serial type, the lookups work in both directions.

FYI
about the naming problem:

dk: yannick_: all DM adapters should match the naming convention: dm--adapter
[10:57] yannick_: so then the require should be require 'dm-redis-adapter' ?
[10:58] dk: yannick_: we want to get it so there's no need to explicitly require the adapter in the code. DataMapper.setup() should load it
[10:58] yannick_: ok
[10:58] dk: yannick_: as of right not it would require a file named "redis_adapter.rb" in the lib folder
nlj: dkubb: The problem with redis_adapter is it gets 'classified' to RediAdapter
dk: namelessjon: oh, good call on that. will need to update the logic in DataMapper.setup to treat the name mostly as-is
[11:11] dk: namelessjon: is there a ticket for that?

see http://datamapper.lighthouseapp.com/projects/20609-datamapper/tickets/890-classification-of-do-adapter-names

Hey,

Having a .gemspec in the repo/gem is cool for bundler, so it can bundle edge sources. I can provide a patch but I thought it'd be reasonably simple?

Thx
Martin

Any ideas on the below? Using dm-core/serializer/etc 1.0.2, I do not know if this was happening with previous versions as I just started playing around with Redis today.

Item.energy.purchasable
=> [#<Item @id=48 @name="Red Bull" @type="Energy" @category="Consumables" @description= @required_level=1 @purchasable=true @enabled=true @image_file_name=nil @image_content_type=nil @image_file_size=nil @image_updated_at=nil>

Item.energy.purchasable.to_json
DataMapper::ImmutableError: Immutable resource cannot be lazy loaded

Whether this gem support transactions?

When I try to switch from psql to redis to check performance I've got this error.

#<DataMapper::Transaction:0xc8ef768>#each_adapter(:connect_adapter, [:log_fatal_transaction_breakage]) failed with #<NoMethodError: undefined method `transaction_primitive' for #<DataMapper::Adapters::RedisAdapter:0x9ba2ddc>>: /home/mtfk/.rvm/gems/ruby-1.9.3-p286/gems/dm-transactions-1.2.0/lib/dm-transactions.rb:303:in `connect_adapter'

I have no idea which of any of those is the cause for the changes, my guess it all comes down to datamapper, but half the tests fail because the code that saves records uses different keys than the code that reads records.

set heffalump:[0]:id 1
get heffalump:0:id

I changed the writing code to join the key together on writing so it matches the read. There was also an issue with the uid being 1 off from what was actually saved. I corrected this by taking 1 off when incrementing the uid.

Changes can be found here

If there is something else that should be done to fix it let me know. I don't really know what the intended key pattern was, so I did what made all the tests pass.

When using redis-rb 2.1.1 and redis 2.0.3, calling the smembers method on the redis adapter causes each member key to be wrapped in brackets, like "[1]". dm-redis-adapter is expecting to_i to work on this, which it will not.

A monkeypatch-style fix is described in the following gist: https://gist.github.com/728471

(I didn't fork and send a pull request because I'm unsure of if this is a bug with dm-redis-adapter or just with redis-rb)

Hey,

i played around with the redis adapter, because i am thinking about changing from a tuplespace (rinda) to redis. But i have trouble with inheritance for queries - like described here: http://datamapper.org/docs/misc.html

So far as i understood the adapter code - I think that the redis adapter does not consider the InclusionComparison correctly in the perform_query function.

Can anyone help me to fix this for the redis adapter?

Attached the spec that shows the problem - it actually only returns Woman (First test) and Mother (second test) but not all of them, which i espected to happed for the first test.

require 'spec_helper'

describe DataMapper::Adapters::RedisAdapter do
  before(:all) do
    @adapter = DataMapper.setup(:default, {
        :adapter  => "redis",
        :db => 15
    })
    @repository = DataMapper.repository(@adapter.name)
    @redis = Redis.new(:db => 15)
  end
  describe 'Inheritance' do

    before :all do

      class Person
        include DataMapper::Resource

        property :name, String
        property :job,  String,        :length => 1..255
        property :type, Discriminator
        property :id, Serial
      end

      class Male   < Person; end
      class Father < Male;   end
      class Son    < Male;   end

      class Woman    < Person; end
      class Mother   < Woman;  end
      class Daughter < Woman;  end

      DataMapper.finalize
    end

    it 'should select all women, mothers, and daughters based on Woman query' do
      w = Woman.create(:name => "woman")
      m = Mother.create(:name => "mother")
      d = Daughter.create(:name => "daughter")

      r = Woman.all
      r.to_set.should == [w,m,d].to_set
      r.size.should == [w,m,d].size
    end

    it 'should select all women, mothers, and daughters based on Person query' do
      w = Woman.create(:name => "woman")
      m = Mother.create(:name => "mother")
      d = Daughter.create(:name => "daughter")
      p = Person.all
      p.to_set.should == [w,m,d].to_set
      p.size.should == [w,m,d].size
    end

    it 'should select all mothers' do
      w = Woman.create(:name => "woman")
      m = Mother.create(:name => "mother")
      d = Daughter.create(:name => "daughter")

      mo = Mother.all
      mo.to_set.should == [m].to_set
      mo.size.should == [m].size
    end

  end
  after(:each) do
    @redis.flushdb
  end
end

Hi. I hit undefined symbol: rb_thread_select error on ruby 2.2.0 with hiredis v0.4.5.
It seems that ruby 2.2.x removed rb_thread_select and hiredis supports it since v0.6.0.
redis/hiredis-rb@f85e8c6

I'm sorry if I'm doing something wrong. Thanks.

I'm having some trouble saving associated objects properly. This pastie should explain the issue:
http://pastie.org/1041141

Any idea what's going on?

To fix the issue I ended up with this monkey patch:

module DataMapper
  module Adapters
    class RedisAdapter
      def delete(collection)
-       collection.count.times do |x|
-         record = collection[x]
+       collection.each do |record|
          @redis.del("#{collection.query.model.to_s.downcase}:#{record[redis_key_for(collection.query.model)]}")
          @redis.srem(key_set_for(collection.query.model), record[redis_key_for(collection.query.model)])
          record.model.properties.select {|p| p.index}.each do |p|
            @redis.srem("#{collection.query.model.to_s.downcase}:#{p.name}:#{encode(record[p.name])}", record[redis_key_for(collection.query.model)])
          end
        end
      end
    end
  end
end

Hi,

there is still a problem with natural keys. If there is more than one key
defined, the common query does not work anymore.

    it "should find elements with two natural keys" do
      class Foo
        include DataMapper::Resource
        property :key1,   String, :key => true
        property :key2, String, :key => true
      end
      DataMapper.finalize

      Foo.create(:key1 => "value1", :key2 => 'value2')
      Foo.first(:key1 => "value1").key2.should == 'value2'
    end

+1332267068.573186 (db 15) "select" "15"
+1332267068.573313 (db 15) "incr" "foos:key1:key2:serial"
+1332267068.573524 (db 15) "sadd" "foos:key1:key2:all" "value1value2"
+1332267068.573687 (db 15) "hmset" "foos:value1value2" "key1" "value1" "key2" "value2"
+1332267068.574434 (db 15) "sismember" "foos:key1:key2:all" "value1"
+1332267068.575936 (db 15) "select" "15"
+1332267068.576032 (db 15) "flushdb"

When adding objects to has n, :through association,
they are not found later, after the record is fetched again.

https://gist.github.com/870925

Any possibiity to pass a namespace in datamapper setup? I need to do something like:

DataMapper.setup(:default, {
:adapter => "redis",
:host => redis_config['host'],
:port => redis_config['port'],
:namespace => redis_config['namespace']

})

Is that even possible?

hope someone can help me.

thanks a lot

When using non-Serial keys for models (like a Text, for example), models are able to be built and persisted just fine. However, when reading back out, the key property becomes a string "0".

Take this example program:

jof@jof ~/tmp % cat foomodel.rb 
#!/usr/bin/env ruby

require 'rubygems'
require 'dm-core'
require 'dm-redis-adapter'

DataMapper.setup(:default, :adapter => :redis)
class Foo
  include DataMapper::Resource
  property :hostname,   Text, :key => true
  property :ip_address, Text
end
DataMapper.finalize

(0..10).each do |i|
  Foo.create(:hostname => "hostname#{i}", :ip_address => '127.0.0.1')
end

Foo.all.each { |f| p f.hostname }

Contrary to what you might expect, this prints:

jof@jof ~/tmp % ./foomodel.rb
"0"
"0"
"0"
"0"
"0"
"0"
"0"
"0"
"0"
"0"
"0"

However, the keys persisted into Redis just fine:

jof@jof ~ % redis-cli hgetall foo:hostname0
1) "hostname"
2) "hostname0"
3) "ip_address"
4) "127.0.0.1"
jof@jof ~ % redis-cli hgetall foo:hostname1
1) "hostname"
2) "hostname1"
3) "ip_address"
4) "127.0.0.1"

Using a simple Person.first seems to load all people in the database and then only use the first record.

Output from redis monitor when doing a Person.first with 3 records in the db:

+1311105369.469164 "smembers" "person:id:all"
+1311105369.469654 "hgetall" "person:1"
+1311105369.470109 "hgetall" "person:2"
+1311105369.470555 "hgetall" "person:3"

With 0.4.0, require 'dm_redis' doesn't seem to work for me (which is what the README has), but require 'dm-redis-adapter' does work.

The adapter seems to do too much enconding for foreign_keys.

I have a Person which has many Tasks. When creating a new task for person with id 1 the foreign_key gets set to "MQ==", which is acutally a base64 encoded "1".

+1311105952.070343 "smembers" "task:person_id:MQ=="

This seems completely unecessary, because in different places the adapter uses the key directly, without any base64 encoding:

+1311105952.067318 "hgetall" "person:1"

This might just be because implementing something like this in Redis is difficult, but I'm finding that

Obj.first

works as expected but

Obj.first(:foo => 'bar')

doesn't return the first object with property "foo" == "bar" it just return nil. It will return the first object in the collection but only if it also has the property "foo" == "bar".

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

Hello, I am using this adapter and after some updates and codification, I got a error trying to remove some items in a has_many relation. below follows the error stack:

undefined method `aggregate' for #DataMapper::Adapters::RedisAdapter:0x007febfb268ba0

dm-aggregates (1.0.2) lib/dm-aggregates/repository.rb:6:in aggregate' dm-aggregates (1.0.2) lib/dm-aggregates/aggregate_functions.rb:185:inaggregate'

I am using dm-core, dm-validations, dm-constraints to do dependent destroy AR work.

It seems this error is happening when I did some has 1 relation between my models. Something like:

https://gist.github.com/thiagocifani/5236698

please, if you have any idea what should I do to fix this error, let me know

have a nice day