zendesk / active_record_shards Goto Github PK

View Code? Open in Web Editor NEW

250.0 250.0 26.0 846 KB

Support for sharded databases and replicas for ActiveRecord

License: MIT License

Ruby 100.00%

active_record_shards's People

Contributors

Stargazers

Watchers

active_record_shards's Issues

Fallback to default DB to fallback db if shard is down

We encountered one issue, one of our shard is down due to heavy load due to which all instances gets down as every instance has same database.yml is it possible that we add one fallback DB if any shard is down when we sent the fallback db for that so that it will impact user of only that shard and things will work for other shard also we Add some configuration to send notification in case of fallback mechanism

Any help will be appreciated

How to create a connection model?

I'm trying to create a connection to another database server using an abstract connection model like so:

class MyConnection < ActiveRecord::Base
  self.establish_connection( configurations['my_connection'][Rails.env] )
  self.abstract_class = true
end

This doesn't seem to work. The models that use this connection model are not connected to the other database, they are still using the main default db. I want to use this gem for the slaves and possibly the sharding but need to be able to establish a connection to another db. How do I do that using this gem? How do I connect to another database as I normally would do?

Thanks in advance

rake tasks are broken on rails4

➜  zendesk_core_application git:(staugaard/rails4) bundle exec rake app:db:create app:db:migrate --trace
** Invoke load_app (first_time)
** Execute load_app
** Invoke app:db:create (first_time)
** Invoke app:db:load_config (first_time)
** Execute app:db:load_config
** Execute app:db:create
rake aborted!
NoMethodError: undefined method `create_database' for main:Object
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta3/lib/active_record_shards/tasks.rb:32:in `block (3 levels) in <top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta3/lib/active_record_shards/tasks.rb:30:in `each'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta3/lib/active_record_shards/tasks.rb:30:in `block (2 levels) in <top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:240:in `call'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:240:in `block in execute'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:235:in `each'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:235:in `execute'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:179:in `block in invoke_with_call_chain'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/2.1.0/monitor.rb:211:in `mon_synchronize'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:172:in `invoke_with_call_chain'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/task.rb:165:in `invoke'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:150:in `invoke_task'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:106:in `block (2 levels) in top_level'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:106:in `each'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:106:in `block in top_level'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:115:in `run_with_threads'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:100:in `top_level'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:78:in `block in run'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:75:in `run'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/bin/rake:33:in `<top (required)>'
/opt/boxen/rbenv/versions/2.1.1/bin/rake:23:in `load'
/opt/boxen/rbenv/versions/2.1.1/bin/rake:23:in `<main>'
Tasks: TOP => app:db:create

Inconsistent readonly object

Hi there,

The readme indicates there are two kinds of usage of on_replica.

ActiveRecord::Base.on_replica do
  Account.find_by_big_expensive_query
end

Account.on_replica.find_by_big_expensive_query

I think they should both return readonly objects, but it turned out only the latter one is readonly.

ActiveRecord::Base.on_replica do
  Account.find_by_big_expensive_query.readonly?
end
=> false

Account.find_by_big_expensive_query.readonly?
=> true

Did I miss something here?

Environment:

active_record_shards: git master
rails: 4.2

How to define multiple slaves?

How does one define multiple slaves in database.yml? Specifically asking since we have multiple follower databases in heroku which we wanna specify as slaves in database.yml.

Thread Safe : (RuntimeError) Can't add a new key into hash during iteration

Hi,

Very rarely we are getting Runtime Error. Please help us in finding the solution

class HardWorker < BaseWorker

  sidekiq_options :queue => :default, :retry => 0, :backtrace => true, :failures => :exhausted

  def perform(id)
    ActiveRecord::Base.on_shard(:shard_1) do
        10.times do 
          User.find(rand(1..10))
        end
    end
end

#Trace

~/.rvm/gems/ruby-1.9.3-p547@gemset1/gems/activerecord-2.3.18/lib/active_record/connection_adapters/abstract/connection_pool.rb:296:in `[]='
~/.rvm/gems/ruby-1.9.3-p547@gemset1/gems/activerecord-2.3.18/lib/active_record/connection_adapters/abstract/connection_pool.rb:296:in `establish_connection'
~/.rvm/gems/ruby-1.9.3-p547@gemset1/gems/active_record_shards-2.7.0/lib/active_record_shards/connection_switcher.rb:182:in `establish_shard_connection'
~/.rvm/gems/ruby-1.9.3-p547@gemset1/gems/active_record_shards-2.7.0/lib/active_record_shards/connection_switcher.rb:153:in `switch_connection'
~/.rvm/gems/ruby-1.9.3-p547@gemset1/gems/active_record_shards-2.7.0/lib/active_record_shards/connection_switcher.rb:15:in `on_shard'
~/project/app/workers/hard_worker.rb:6:in `perform'

We are planning to use the active_record_shards gem for sharding the database with sidekiq.

active_record_shard is Thread Safe ?

Rails - 2.318
Ruby - 1.9.3-p547
active_record_shards - 2.7.0
sidekiq - 3.0.0

Thanks

Overriding rake tasks

ActiveRecord Shards overrides a handful of db related rake tasks. Would it be a problem if I implemented overriding schema load and structure load?

Related to that question is how ActiveRecord Shards currently calls the original rake tasks, previous to #38 ActiveRecord Shards used to call directly to ActiveRecord::Tasks::DatabaseTasks but now calls to connections. What caused this change?

While trying to implement schema and structure load I want to use ActiveRecord::Tasks::DatabaseTasks if that makes since.

Conflicting with another database

Rails 5.2.3
gem 'active_record_shards', '~> 4.0.0.beta9'
I am using gem 'clickhouse-activerecord', '~> 0.5.7'
When I do self.establish_connection CLICKHOUSE_DB, it still connects with default mysql db instead of clickhouse db.
I tried the same after removing this gem and it started working as expected.
Any idea on what can I do? Am i doing something wrong?

Change the about text to say 'replicas' instead of 'slaves.

Currently the about text which appears in e.g. Explore Repositories says Support for sharded databases and slaves for ActiveRecord. The word 'slave' used in this context can be disturbing and replica or follower is a more descriptive term. The readme says ActiveRecord Shards is an extension for ActiveRecord that provides support for sharded database and replicas so this change would also improve consistency.

Schema Cache issue

Hi,

I am using rails 3.2.18 and resque 1.25.0 and ours is shared. We are facing the problem with resque fork processes. When the new process forks, its trying the get schema once again. To avoid this, I tried

  Resque.before_first_fork do
    ["shard1","shard2"].each do |shard|
      ActiveRecord::Base.on_shard(shard) do
        ActiveRecord::Base.send(:subclasses).each { |klass| klass.columns }
      end
    end
  end

For shard1 its go through
https://github.com/rails/rails/blob/3-2-stable/activerecord/lib/active_record/connection_adapters/abstract_mysql_adapter.rb#L424

For shard2 its not evening going through that columns method itself. Because of this successive calls for shard2 tables schema is loaded repeatedly.

Can you please help us on this?

Thanks

Question - Is it possible to switch connection to a database which doesn't exists in database.yml ?

We have been using this gem in production for some time now without any issues. However, every time we add a new shard in database.yml we have to redeploy our service. I was curious if its possible to switch connection to a database which doesn't exists in database.yml by directly passing in the database name something like below?

ActiveRecord::Base.on_shard(database_name)

I have seen other solutions online which suggests to use establish_connection in application_controller before_action but before implementing that I would like to know if this gem already supports this functionality or if it could be extended to support it. I am willing to contribute PR if someone can provide little guidance in the right direction.

Thanks!

How to run migrations on sharded database (v4.0.0)?

Hello!

How can I run my migrations on a sharded database? I'm using v4.0.0.beta9, and I can't find any information on how to create tables on my shard.

Thanks!

New user usability: do not return scope from block / raise on empty block

Issue 1, queries sent to the replica have missing conditions

When I use .on_slave in my local dev environment (I have two databases set up locally, one replicating from the other) I don't get the results I expect. For instance this query:

=> UserActivity.where("id > 2001").on_slave.count
20012

runs this query on the replica:

2020-06-24 21:25:36.059 BST [86763] LOG:  execute <unnamed>: SELECT COUNT(*) FROM "user_activities"

You can see that it's missing the where clause. If I re-run the query without the .on_slave call:

=> UserActivity.where("id > 2001").count
18025

The call is sent to the primary as expected, but this time with the clause intact:

2020-06-24 21:25:16.601 BST [87228] LOG:  execute <unnamed>: SELECT COUNT(*) FROM "user_activities" WHERE (id > 2001)

If I use the block syntax then the query is sent to the replica and maintains the condition as expected and returns the correct result:

=> ActiveRecord::Base.on_slave { UserActivity.where("id > 2001").count }
18025

2020-06-24 21:34:46.614 BST [86763] LOG:  execute <unnamed>: SELECT COUNT(*) FROM "user_activities" WHERE (id > 2001)

Issue 2, queries sent to the wrong server

If I make a query like this (note on_slave now appears before the .where):

UserActivity.on_slave.where("id > 2001").count
18025

The result is correct, but the query gets sent to the primary and not the replica:

2020-06-24 21:45:45.708 BST [88178] LOG:  execute <unnamed>: SELECT COUNT(*) FROM "user_activities" WHERE (id > 2001)

Local setup

This is the database.yml I have in place:

development: &default
  adapter: postgresql
  database: connector_development
  encoding: utf8
  min_messages: warning
  pool: <%= Integer(ENV["DB_POOL"] || 5) %>
  reaping_frequency: <%= Integer(ENV["DB_REAPING_FREQUENCY"] || 10) %>
  timeout: 5000
  variables:
    statement_timeout: 5000 # ms
  connect_timeout: 1
  checkout_timeout: 1
  prepared_statements: false
  slave:
    host: 127.0.0.1
    port: 5433

We are using Ruby 2.6.5 and Rails 5.0 and active_record_shards 3.17.0.

rake task broken

@osheroff @lukkry @bquorning @grosser

I upgraded from to 2.8.0 to 3.0.0.beta2 and now I get this:

➜  zendesk_inbox git:(master) ✗ rake --trace
rake aborted!
NoMethodError: undefined method `clean' for <Rake::Task db:drop => [load_config, rails_env]>:Rake::Task
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta2/lib/active_record_shards/tasks.rb:4:in `block in <top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta2/lib/active_record_shards/tasks.rb:3:in `each'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/active_record_shards-3.0.0.beta2/lib/active_record_shards/tasks.rb:3:in `<top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `require'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `block in require'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:236:in `load_dependency'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:251:in `require'
/Users/staugaard/code/zendesk/zendesk_inbox/lib/tasks/db.rake:1:in `<top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `block in load'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:236:in `load_dependency'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/activesupport-3.2.17/lib/active_support/dependencies.rb:245:in `load'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/railties-3.2.17/lib/rails/engine.rb:425:in `block in load_tasks'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/railties-3.2.17/lib/rails/engine.rb:425:in `each'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/railties-3.2.17/lib/rails/engine.rb:425:in `load_tasks'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/railties-3.2.17/lib/rails/application.rb:145:in `load_tasks'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/railties-3.2.17/lib/rails/railtie/configurable.rb:30:in `method_missing'
/Users/staugaard/code/zendesk/zendesk_inbox/Rakefile:7:in `<top (required)>'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/rake_module.rb:28:in `load'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/rake_module.rb:28:in `load_rakefile'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:683:in `raw_load_rakefile'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:94:in `block in load_rakefile'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:93:in `load_rakefile'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:77:in `block in run'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:176:in `standard_exception_handling'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/lib/rake/application.rb:75:in `run'
/opt/boxen/rbenv/versions/2.1.1/lib/ruby/gems/2.1.0/gems/rake-10.3.1/bin/rake:33:in `<top (required)>'
/opt/boxen/rbenv/versions/2.1.1/bin/rake:23:in `load'
/opt/boxen/rbenv/versions/2.1.1/bin/rake:23:in `<main>'

"where" doesn't work when on_shard (Rails 3.2)

ActiveRecordShards tries to read from not sharded database when calling

ActiveRecord::Base.on_shard(1){ Project.where(:id => 1) }

Steps to reproduce:

create a new rails 3.2.14 app
add active_record_shards to Gemfile
create database.yml

development:
  adapter: mysql2
  encoding: utf8
  database: mango
  pool: 5
  host: 127.0.0.1
  username: root
  password:
  slave:
    database: mango_slave
  shards:
    1:
      database: mango_shard0
      slave:
        database: mango_shard0_slave
    2:
      database: mango_shard1
      slave:
        database: mango_shard1_slave

$ bundle exec rails g model Project name:string

class CreateProjects < ActiveRecord::Migration
  shard :all

  def change
    create_table :projects do |t|
      t.string :name

      t.timestamps
    end
  end
end

$ bundle exec rake db:migrate
$ bundle exec rails c

1.9.3-p429 :008 > ActiveRecord::Base.on_shard(1){ Project.connection.current_database }
   (2.5ms)  SELECT DATABASE() as db
 => "mango_shard0"
1.9.3-p429 :006 > ActiveRecord::Base.on_shard(1){ Project.all }
  Project Load (0.3ms)  SELECT `projects`.* FROM `projects`
 => []
1.9.3-p429 :001 > ActiveRecord::Base.on_shard(1){ Project.where(:id => 1) }
  Project Load (0.3ms)  SELECT `projects`.* FROM `projects` WHERE `projects`.`id` = 1
ActiveRecord::StatementInvalid: Mysql2::Error: Table 'mango.projects' doesn't exist: SELECT `projects`.* FROM `projects`  WHERE `projects`.`id` = 1

Am I missing something obvious here?

Status of the v4.0.0beta versions

What is the state of the v4.0.0betaX versions ?

We upgraded to the v4 version a year ago when it seemed like it would be the way to to go for Rails5 support.

It's been over a year now since v4 has had any new release, while the v3 has had regular updates and even landed Rails5 support. There doesn't seem to be a difference in features, so is v4 still relevant ?

active_record_shards not supporting rails 7.0.1 or 7.1

When i used not_sharded in model then it is proving
NameError: undefined local variable or method `not_sharded'

Bug: Additional connection pool is made when primary DB has no slave

When the DB is configured without a slave in particular environment the connection name for non-sharded DB should be resolved as "primary" but not Rails.env.

The condition in this return doesn't cover the use case when slave is not configured for the environment:
https://github.com/zendesk/active_record_shards/blob/master/lib/active_record_shards/shard_selection.rb#L54

How to specify for a model to always shard to a particular database

My use case is that we have 10 sharded (exact copies) databases, and then 1 unique one. So most of our models map to the sharded dbs, then my unique db has a couple models. I want to always have an active record query on the unique db model go to that shard without having to specify

on_shard('unique_db') { Execute my active record query}

It sounded like I could use not_sharded for this. However if I set my unique db as my default one in database.yml development section, and then my shards are the 10 copied dbs, when rails server loads, I think it tries to query the schema for each of my normal models against the unique_db now and fails (since those tables don't exist in the unique_db). Any clue if this is possible with this gem?

I have a stack overflow question here to see if I can do it myself.
http://stackoverflow.com/questions/40314874/how-to-wrap-every-rails-activerecord-query-using-an-around-action

Thanks!

Flaky test

From time to time I’ve seen the Rails 4.2 tests fail with a few (1-2) examples failing with NoMethodError: undefined method `mysql2_connection' for ActiveRecord::Base:Class. Re-running the test suite usually makes the test pass, and it makes me feel bad for it…

Example:

Database rake tasks::db:create#test_0001_creates the database and all shards:
NoMethodError: undefined method `mysql2_connection' for ActiveRecord::Base:Class
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/activerecord-4.2.9/lib/active_record/dynamic_matchers.rb:26:in `method_missing'
    /home/travis/build/zendesk/active_record_shards/lib/active_record_shards/tasks.rb:94:in `root_connection'
    /home/travis/build/zendesk/active_record_shards/lib/active_record_shards/tasks.rb:40:in `block (3 levels) in <top (required)>'
    /home/travis/build/zendesk/active_record_shards/lib/active_record_shards/tasks.rb:31:in `each'
    /home/travis/build/zendesk/active_record_shards/lib/active_record_shards/tasks.rb:31:in `block (2 levels) in <top (required)>'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `block in execute'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `each'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:250:in `execute'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:194:in `block in invoke_with_call_chain'
    /home/travis/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:187:in `invoke_with_call_chain'
    /home/travis/build/zendesk/active_record_shards/gemfiles/vendor/bundle/ruby/2.4.0/gems/rake-12.0.0/lib/rake/task.rb:180:in `invoke'
    /home/travis/build/zendesk/active_record_shards/test/helper.rb:75:in `rake'
    /home/travis/build/zendesk/active_record_shards/test/tasks_test.rb:42:in `block (3 levels) in <top (required)>'

cc @pschambacher @grosser

rake db:test:purge not working

I have rake db:test:purge failing in my application.

@lukkry could you please help me fix it?

Looking at the code in https://github.com/rails/rails/blob/v4.0.5/activerecord/lib/active_record/tasks/mysql_database_tasks.rb#L44-L47, a connection should be established in line 45, but executing line 46 raises a ConnectionNotEstablished:

» bin/rake db:test:purge
rake aborted!
ActiveRecord::ConnectionNotEstablished: ActiveRecord::ConnectionNotEstablished
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/connection_adapters/abstract/connection_pool.rb:546:in `retrieve_connection'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/connection_handling.rb:79:in `retrieve_connection'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/connection_handling.rb:53:in `connection'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/tasks/mysql_database_tasks.rb:8:in `connection'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/tasks/mysql_database_tasks.rb:46:in `purge'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/tasks/database_tasks.rb:137:in `purge'
/Users/bquorning/Code/hc/vendor/bundle/ruby/2.1.0/gems/activerecord-4.0.5/lib/active_record/railties/databases.rake:364:in `block (3 levels) in <top (required)>'
Tasks: TOP => db:test:purge
(See full trace by running task with --trace)

Make all models not_sharded by default

I am trying to make all my models not sharded by default with.

But it results in a weird behavior. on_shard call doesn't establish connection to shard because connection_specification_name calls is_sharded? on ActiveRecord::Base which is false.

The following code results in AR::ConnectionNotEstablished:

ActiveRecord::Base.not_sharded
class Visitor < ActiveRecord::Base
  self.sharded = true
end

ActiveRecord::Base.on_shard(1) do
 Visitor.first
end

While the following would work:

Visitor.on_shard(1) do
 Visitor.first
end

Does `active_record_shards` support `rails 5.2.1`?

Does active_record_shards support rails 5.2.1?

All shard names must be integers

my first reaction was 'what??'

trying to evaluate your gem as a replacement for octopus which does not work with rails 5.2

is it possible to use named shards without forking the gem?

Is it possible to set shard(:none) automatically for all migrations?

Hi there,

I'm only using the slave config on my database.yml. I don't want to do anything related with shards for now. I'm wondering if there is a way to avoid the raise exception that happens if we don't add that shard(:none) on each migration file. I was thinking to MonkeyPatch the ActualMigrationExtension and get rid of that. Is there other solution for this?

Thank you

cc: @pschambacher, @bquorning, @grosser

Dynamic shards (aka for each account)

Current gem configuration allows to specify a limit number of shards which (as "Usage" section suggest) should be used as buckets for each account (or account-like entity).

In my case I want each account to have a separated database. That will allow to save the disk space in the sharded tables by not including the account_id column in indexes and table itself.
In other words, I want to use the Account#id as a shard_id.
There is no benefit in storing several accounts under the same shard.

Currently this is not possible due to static configurations of the shards that allows to define a limited and static set of shards.

Is there a way to define dynamic shards? Are you interested in the PR?

zendesk / active_record_shards Goto Github PK

active_record_shards's People

Contributors

Stargazers

Watchers

Forkers

active_record_shards's Issues

Issue 1, queries sent to the replica have missing conditions

Issue 2, queries sent to the wrong server

Local setup

Recommend Projects

Recommend Topics

Recommend Org