databasecleaner / database_cleaner-active_record Goto Github PK
View Code? Open in Web Editor NEWStrategies for cleaning databases using ActiveRecord. Can be used to ensure a clean state for testing.
License: MIT License
Strategies for cleaning databases using ActiveRecord. Can be used to ensure a clean state for testing.
License: MIT License
Current edge rails (future Rails 7.1) produces an error from cleaning in :truncation
or :deletion
strategies, due to ActiveRecord::SchemaMigration.table_name
apparently being removed.
eg
Failure/Error: DatabaseCleaner.clean_with(:truncation)
NoMethodError:
undefined method `table_name' for ActiveRecord::SchemaMigration:Class
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:73:in `migration_table_name'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/truncation.rb:271:in `migration_storage_names'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/generic/truncation.rb:14:in `initialize'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/truncation.rb:238:in `initialize'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/base.rb:128:in `new'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/base.rb:128:in `create_strategy'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/base.rb:75:in `clean_with'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/configuration.rb:93:in `block in clean_with'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/configuration.rb:93:in `each'
# /Users/jrochkind/.gem/ruby/3.0.4/gems/database_cleaner-1.99.0/lib/database_cleaner/configuration.rb:93:in `clean_with'```
It think possibly this this is the commit where it moved, but still wrapping my head around it: rails/rails@436277d. Hm, it looks like maybe instead of being a class method, SchemaMigration now has to be initialized as an instance with a particular connection to get a table name... this seems unclear how to patch in DC.
It looks like similar things have happened in the past that effected database_cleaner, eg DatabaseCleaner/database_cleaner#476
I run my CI on edge rails to get early notice of anything that will break it... or in this case, early notice of something that broke database_cleaner, preventing me from running my CI on rails edge at present!
Recently released 0.7.1 seems to break on JRuby (both 1.8 and 1.9 modes). Here's travis-ci.org build link.
$ bundle exec rake
183/home/vagrant/.rvm/rubies/jruby-1.6.5.1/bin/jruby -S rspec ./spec/irc_client_spec.rb ./spec/github_spec.rb ./spec/travis/github_api_spec.rb ./spec/travis/config_spec.rb ./spec/travis/renderer_spec.rb ./spec/travis/mailer/build_spec.rb ./spec/travis/mailer/helper/build_spec.rb ./spec/travis/notifications/secure_config_spec.rb ./spec/travis/notifications/handler/pusher_spec.rb ./spec/travis/notifications/handler/campfire_spec.rb ./spec/travis/notifications/handler/archive_spec.rb ./spec/travis/notifications/handler/email_spec.rb ./spec/travis/notifications/handler/worker_spec.rb ./spec/travis/notifications/handler/irc_spec.rb ./spec/travis/notifications/handler/webhook_spec.rb ./spec/travis/notifications/handler/worker/payload_spec.rb ./spec/travis/model/job_spec.rb ./spec/travis/model/user_spec.rb ./spec/travis/model/repository_spec.rb ./spec/travis/model/ssl_key_spec.rb ./spec/travis/model/request_spec.rb ./spec/travis/model/service_hook_spec.rb ./spec/travis/model/build_spec.rb ./spec/travis/model/worker_spec.rb ./spec/travis/model/token_spec.rb ./spec/travis/model/artifact/log_spec.rb ./spec/travis/model/build/matrix_spec.rb ./spec/travis/model/build/states_spec.rb ./spec/travis/model/build/notifications_spec.rb ./spec/travis/model/build/denormalize_spec.rb ./spec/travis/model/request/states_spec.rb ./spec/travis/model/request/branches_spec.rb ./spec/travis/model/request/payload/github_spec.rb ./spec/travis/model/job/tagging_spec.rb ./spec/travis/model/job/cleanup_spec.rb ./spec/travis/model/job/queue_spec.rb ./spec/travis/model/job/test/states_spec.rb ./spec/travis/model/job/configure/states_spec.rb ./spec/travis/model/worker/states_spec.rb ./spec/json/worker/job_test_spec.rb ./spec/json/worker/job_configure_spec.rb ./spec/json/webhook/build_finished_spec.rb ./spec/json/pusher/job_finished_spec.rb ./spec/json/pusher/job_created_spec.rb ./spec/json/pusher/job_log_spec.rb ./spec/json/pusher/build_started_spec.rb ./spec/json/pusher/worker_spec.rb ./spec/json/pusher/job_started_spec.rb ./spec/json/pusher/build_finished_spec.rb
184TypeError: superclass mismatch for class PostgreSQLAdapter
185 ConnectionAdapters at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/active_record/truncation.rb:80
186 ActiveRecord at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/active_record/truncation.rb:7
187 (root) at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/active_record/truncation.rb:6
188 require at org/jruby/RubyKernel.java:1038
189 require at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:240
190 load_dependency at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:223
191 new_constants_in at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:640
192 new_constants_in at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:639
193 load_dependency at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:223
194 require at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/activesupport-3.1.3/lib/active_support/dependencies.rb:240
195 orm_strategy at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/active_record/truncation.rb:98
196 create_strategy at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/base.rb:34
197 strategy= at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/base.rb:48
198 strategy= at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/configuration.rb:42
199 each at org/jruby/RubyArray.java:1612
200 strategy= at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/database_cleaner-0.7.1/lib/database_cleaner/configuration.rb:42
201 (root) at /home/vagrant/builds/travis-ci/travis-core/spec/support/active_record.rb:18
202 require at org/jruby/RubyKernel.java:1038
203 (root) at /home/vagrant/builds/travis-ci/travis-core/spec/support/active_record.rb:2
204 load at org/jruby/RubyKernel.java:1063
205 load_spec_files at /home/vagrant/builds/travis-ci/travis-core/spec/travis/renderer_spec.rb:698
206 collect at org/jruby/RubyArray.java:2318
207 load_spec_files at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/rspec-core-2.8.0/lib/rspec/core/configuration.rb:698
208 run at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/rspec-core-2.8.0/lib/rspec/core/command_line.rb:22
209 run_in_process at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/rspec-core-2.8.0/lib/rspec/core/runner.rb:80
210 run at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/rspec-core-2.8.0/lib/rspec/core/runner.rb:69
211 autorun at /home/vagrant/.rvm/gems/jruby-1.6.5.1/gems/rspec-core-2.8.0/lib/rspec/core/runner.rb:10
212 call at org/jruby/RubyProc.java:270
213 call at org/jruby/RubyProc.java:224
Hi. I've been having trouble with making database cleaner work when trying to delete data from different psql schemas, but in the same database.
database.yml:
test: &test
adapter: postgresql
database: test_project
host: localhost
port: 6432
user: postgres
test_non_public:
<<: *test
schema_search_path: non_public
What I did in spec_helper:
config.before(:suite) do
DatabaseCleaner.clean_with :deletion
DatabaseCleaner[:active_record, connection: :test_non_public].clean_with :deletion
end
From what I understand it happens because of this code:
def lookup_from_connection_pool
if ::ActiveRecord::Base.respond_to?(:descendants)
database_name = connection_hash["database"] || connection_hash[:database]
models = ::ActiveRecord::Base.descendants
models.detect { |m| m.connection_pool.spec.config[:database] == database_name }
end
end
This method is invoked from connection_class, and because both connection "test" and "test_non_public" are configured to connect to the same database it ends up returning a model with an incorrect connection. This could be solved (I think) by adding one more condition, so that besides checking for the database_name it also checks for the schema_search_path.
While writing this issue I ended up editing the code as follows and it worked:
def lookup_from_connection_pool
if ::ActiveRecord::Base.respond_to?(:descendants)
database_name = connection_hash["database"] || connection_hash[:database]
schema_search_path = connection_hash["schema_search_path"] || connection_hash[:schema_search_path]
models = ::ActiveRecord::Base.descendants
models.detect do |m|
m.connection_pool.spec.config[:database] == database_name &&
m.connection_pool.spec.config[:schema_search_path] == schema_search_path
end
end
end
Does it look like a good fix? If not how should I approach this problem so that I can solve this issue?
Thanks
database: postgresql 11
Rails 6
rspc-rails ~>4.0
database_cleaner-active_record ~>1.8
spec_helper.rb
require 'database_cleaner-active_record'
DatabaseCleaner[:active_record, connection: :test]
tables_to_be_excluded = %w[public.spatial_ref_sys public.foo]
config.before(:suite) do
load "#{Rails.root}/db/seeds.rb"
DatabaseCleaner.strategy = :truncation, { except: tables_to_be_excluded }
DatabaseCleaner.clean_with(:truncation, { except: tables_to_be_excluded })
puts 'Cleaned up database and feed seeds'
# this step will wipe all data in all tables
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
Hey
We have been using the multiverse gem for a long time to manage multiple databases, but with an upgrade to Rails 6, we are moving to use their built-in setup.
Previously, we had the following:
DatabaseCleaner[:active_record].strategy = :deletion
DatabaseCleaner[:active_record, { connection: :provider_data_test }].strategy = :deletion
With the following in the Model's
establish_connection :"provider_data_#{Rails.env}"
The new way of doing it is by having the following in database.yml:
test:
primary:
<<: *default
provider_data:
<<: *provider_data_default
And this in the models:
connects_to database: { writing: :provider_data, reading: :provider_data }
How should Database cleaner be setup for this? It's clearing the primary db, but it's not clearing the secondary dbs. I have tried this for example:
DatabaseCleaner[:active_record, { connection: :provider_data }].strategy = :deletion
I had noticed that my specs are getting very slow, and started investigating strategies. Apparently the more you truncate, the longer it takes. This behavior seems very strange:
irb(main):009:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t=> 1.023764
irb(main):010:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 0.793042
irb(main):011:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 0.829454
irb(main):012:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 0.870453
irb(main):013:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 0.940325
irb(main):014:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 1.03202
irb(main):015:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 2.176829
irb(main):016:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 2.466992
irb(main):017:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 2.341618
irb(main):018:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 2.4591
irb(main):019:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 3.078432
irb(main):020:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 3.056646
irb(main):021:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 3.314674
irb(main):022:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 3.514017
irb(main):023:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 3.822936
irb(main):024:0> t = Time.now; 10.times{ DatabaseCleaner.clean_with :truncation }; Time.now - t
=> 4.109673
For large databases, this is way faster than disabling/enabling all the triggers.
set session_replication_role='replica';
Hi, so im at wits end trying to get this to work. Right now were connecting to MS SQL Server (2014) VM using the Activerecord gem for SQL Server: gem 'activerecord-sqlserver-adapter'
I can't seem to get DB Cleaner to work with this. It's worked great in the past using capybara/selenium webdriver.
I know you have to set it to use truncation strategy, but even putting:
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.clean_with(:truncation)
DatabaseCleaner.start
at the beginning of a "it" rspec test, it complains about invalid object name instantly once it gets to that test:
Failure/Error: DatabaseCleaner.clean_with(:truncation)
ActiveRecord::StatementInvalid:
TinyTds::Error: Invalid object name 'user_table'.: DELETE FROM [user_table];
(user_table is just an example here)
I ensured it's connecting to the right database because when I take out Database cleaner it works just fine and I can see the database being populated correctly. So I know my database.yml is set up correctly.
Im not really sure what to try here, or if it's just an incompatibility with SQL Server perhaps? I can't seem to find many examples of people using DB Cleaner and SQL Server.
With SQLite3, DatabaseCleaner tries to truncate views.
initialize`: SQLite3::SQLException: cannot modify <table> because it is a view: DELETE FROM <table>
Issue appears to be here.
def database_cleaner_view_cache
@views ||= select_values("select table_name from information_schema.views where table_schema = '#{current_database}'") rescue []
end
Running select table_name from information_schema.views
in SQLite3 returns
no such table: information_schema.views: select table_name from information_schema.views
Looks like this was fixed for other adapters here.
I am using a standard :transaction
DatabaseCleaner block with rspec:
config.around(:each) do |example|
DatabaseCleaner[:active_record, connection: :test].cleaning do
example.run
end
end
When trying to profile the queries in the test suite, I noticed there was a large number of COMMIT statements: one for each test, and equivalent to the ROLLBACK query count. After digging, it seems the strategy runs a blank transaction in the DatabaseCleaner#begin block: https://github.com/DatabaseCleaner/database_cleaner/blob/cda982c9b4a8f3f116ed7e39766eb1447e0197aa/lib/database_cleaner/active_record/transaction.rb#L12
From what I can tell with git's blame/log, this was added in 1af146f and then modified as a fix for DatabaseCleaner/database_cleaner#200. These seem to indicate that this empty transaction only needs to be run once, perhaps at the start of the suite, or perhaps with a flag indicating it's been run once and doesn't need to run again.
It's clearly not the end of the world, but I am currently on a spec optimisation push and this extra BEGIN/COMMIT per test is having a not-insignificant (but admittedly not major) impact on test time: My metrics indicate it's adding about 15 seconds of time to a suite with 6200ish tests.
Happy to help and create a pull request if there's consensus that I haven't misunderstood the purpose of that statement!
We have an application that needs to support multiple database engines, so our Gemfile
contains multiple JDBC adapters as so:
gem 'activerecord-jdbcpostgresql-adapter', '~>1.3.7'
gem 'activerecord-jdbcmssql-adapter', '~>1.3.7'
gem 'activerecord-jdbcmysql-adapter', '~>1.3.7'
The app works as we expect, and each setup with a different database works well, however we've just introduced the mysql
adapter, and have now found that the database cleaner is failing to truncate
the database.
On my machine, I have a postgres
database running. If I open up a rails console, and attempt to clean the database with :truncation
I get the following error
TypeError: superclass mismatch for class Mysql2Adapter
We haven't changed any code in our app, and if I comment out the mysql JDBC adapter from the Gemfile
and bundle install again, I am once again able to clean the database.
This is the stack trace that I get with the error
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/active_record/truncation.rb:209:in `ConnectionAdapters'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/active_record/truncation.rb:176:in `ActiveRecord'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/active_record/truncation.rb:175:in `(root)'
from org/jruby/RubyKernel.java:1065:in `require'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/activesupport-4.1.0/lib/active_support/dependencies.rb:247:in `require'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/activesupport-4.1.0/lib/active_support/dependencies.rb:232:in `load_dependency'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/activesupport-4.1.0/lib/active_support/dependencies.rb:247:in `require'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/base.rb:1:in `(root)'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/base.rb:108:in `orm_strategy'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/base.rb:34:in `create_strategy'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/base.rb:38:in `clean_with'
from org/jruby/RubyArray.java:1613:in `each'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/configuration.rb:85:in `clean_with'
from /Users/adrianh/.rvm/gems/jruby-1.7.12/gems/database_cleaner-1.0.1/lib/database_cleaner/configuration.rb:85:in `clean_with'
It's very similar (exact same error message) to other issues I've found here, however the line numbers are different so I'm assuming the issue is something different, given that the others have also been resolved and this is still appearing.
We're using Rails 3.2.11, Cucumber, and Database Cleaner with transaction cleaning. The basic problem we're seeing is that ActiveRecord is returning objects from previous tests because the query cache is not cleared. Here's an example:
Test 1:
User.create(:email => '[email protected]')
Test 2:
# The Rails log indicates that this hits the ActiveRecord
# query cache, returning the User created in Test 1.
User.find_by_email('[email protected]')
We're currently getting around this by using a Cucumber After
hook, like so:
After do |scenario|
ActiveRecord::Base.connection.clear_query_cache
end
My expectation is that either Database Cleaner or Rails should handle clearing the query cache at the end of each scenario/rollback.
This may not be a Database Cleaner issue (I think it depends on if Rails is responsible for clearing the query cache on rollback, or if calling code is).
We use the PostGIS extension and the rgeo AR gem. We use truncation for our js
tests as well as our before(:suite)
strategy.
The problem is that the PostGIS extension creates a table spatial_ref_sys
that is owned by the user who installs the extension (namely, a super user).
Running bundle exec rspec
fails with the following error:
/Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `async_exec': PG::InsufficientPrivilege: ERROR: must be owner of relation spatial_ref_sys (ActiveRecord::StatementInvalid)
: ALTER TABLE "spatial_ref_sys" ENABLE TRIGGER USER;ALTER TABLE "filtered_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "processed_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "raw_tracked_points" ENABLE TRIGGER USER;ALTER TABLE "resource_intersections" ENABLE TRIGGER USER;ALTER TABLE "schema_migrations" ENABLE TRIGGER USER;ALTER TABLE "users" ENABLE TRIGGER USER;ALTER TABLE "feature_types" ENABLE TRIGGER USER;ALTER TABLE "features" ENABLE TRIGGER USER
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `block in execute'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:473:in `block in log'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activesupport-4.2.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/abstract_adapter.rb:467:in `log'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:154:in `execute'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:23:in `rescue in ensure in disable_referential_integrity'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:20:in `ensure in disable_referential_integrity'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/activerecord-4.2.1/lib/active_record/connection_adapters/postgresql/referential_integrity.rb:20:in `disable_referential_integrity'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/active_record/truncation.rb:235:in `clean'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/base.rb:40:in `clean_with'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `block in clean_with'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `each'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/database_cleaner-1.4.1/lib/database_cleaner/configuration.rb:91:in `clean_with'
from /Users/<user>/Development/<company>/<project>/spec/support/database_cleaner.rb:3:in `block (2 levels) in <top (required)>'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/example.rb:333:in `instance_exec'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/example.rb:333:in `instance_exec'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/hooks.rb:357:in `run'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `block in run_hooks_with'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `each'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1559:in `run_hooks_with'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/configuration.rb:1525:in `with_suite_hooks'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:109:in `block in run_specs'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/reporter.rb:62:in `report'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:108:in `run_specs'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:86:in `run'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:70:in `run'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/lib/rspec/core/runner.rb:38:in `invoke'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/gems/rspec-core-3.2.3/exe/rspec:4:in `<top (required)>'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/rspec:23:in `load'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/rspec:23:in `<main>'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/ruby_executable_hooks:15:in `eval'
from /Users/<user>/.rvm/gems/ruby-2.2.0@<project>/bin/ruby_executable_hooks:15:in `<main>'
The error comes from the call to disable_referential_integrity
because it calls all tables, ignoring any tables that might have been exclude
d via the truncation
options.
Our database_cleaner.rb
file:
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation, {except: %w[spatial_ref_sys]})
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation, {except: %w[spatial_ref_sys]}
end
config.before(:each) do
DatabaseCleaner.start
end
config.append_after(:each) do
DatabaseCleaner.clean
end
end
One workaround is to execute the tests as a db super user. Another would be to change the ownership of the spatial_ref_system
table. Neither of these are very desirable.
Is there a way to skip tables in the disable_referential_integrity
call?
I created a script to demonstrate this behavior:
#!/usr/bin/env ruby
begin
require "bundler/inline"
rescue LoadError => e
$stderr.puts "Bundler version 1.10 or later is required. Please update your bundler"
raise e
end
gemfile(true) do
source "https://rubygems.org"
gem "activerecord", "4.2.7.1"
gem "mysql2", "0.4.5"
gem "database_cleaner", "1.5.3"
gem "rspec", "3.5.0"
end
require "active_record"
require "rspec"
require "rspec/autorun"
require "database_cleaner"
ActiveRecord::Base.establish_connection(
adapter: "mysql2",
username: "root",
password: "password",
host: "localhost",
database: "ft_test"
)
ActiveRecord::Base.connection.recreate_database "ft_test"
ActiveRecord::Schema.define do
create_table :widgets, force: true do |t|
t.string "description", limit: 255
end
add_index "widgets", ["description"], name: "widgets_description_fts", type: :fulltext
end
class Widget < ActiveRecord::Base
def self.text_search(query)
where("MATCH(description) AGAINST(:query)", query: query)
end
def self.like_search(query)
where("description like ?", "%#{query}%")
end
end
RSpec.configure do |config|
config.default_formatter = "doc"
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
end
RSpec.describe Widget, type: :model do
describe ".text_search" do
after(:all) { Widget.delete_all }
it "fails when cleaning with start/clean" do
DatabaseCleaner.start
text_search_test
DatabaseCleaner.clean
end
it "fails when cleaning with block" do
DatabaseCleaner.cleaning do
text_search_test
end
end
it "succeeds with manual cleaning (fails at random, unless it is first or the only test run)" do
Widget.delete_all
text_search_test
Widget.delete_all
end
end
describe ".like_search" do
it "succeeds when cleaning with start/clean" do
DatabaseCleaner.start
like_search_test
DatabaseCleaner.clean
end
it "succeeds when cleaning with block" do
DatabaseCleaner.cleaning do
like_search_test
end
end
it "succeeds with manual cleaning" do
Widget.delete_all
like_search_test
Widget.delete_all
end
end
private
def create_widgets
Widget.create()
match = Widget.create(description: "The quick brown fox jumps over the lazy dog")
Widget.create()
match
end
def text_search_test
match = create_widgets
expect(Widget.text_search("fox")).to contain_exactly match
end
def like_search_test
match = create_widgets
expect(Widget.like_search("fox")).to contain_exactly match
end
end
Hi,
I am using the hanami framework and the rspec lib to test my code and PG database. When I trying to test one action I get following error:
PG::InFailedSqlTransaction: ERROR: current transaction is aborted, commands ignored until end of transaction block
My action has been wrapped in repository transaction:
def call(params, dependencies)
begin
dependencies.repository.transaction do
create_user(params, dependencies)
assign_tags(params, dependencies)
assign_notes(params, dependencies)
end
rescue Hanami::Model::Error, Domain::Errors::Exception => error
raise Domain::Errors::CreateUserFailed, error.message
end
end
Here you can see part of my test which are failing:
it 'should raise exception if params contains duplicated email' do
expect{service.call(duplicated_email)}
.to raise_exception(Domain::Errors::CreateUserFailed)
.with_message('Duplicated values found (id or
email)')
end
it 'should raise exception if params contains duplicated note id' do
expect{service.call(duplicated_note_id)}
.to raise_exception(Domain::Errors::CreateUserFailed)
end
First test pass but the second one raises exception I mentioned earlier. I know this is caused that previous test raised exception in transaction block. I tried to configure DB cleaner to fix this but without success.
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with :deletion
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
I'm in the process of upgrading a rails app from Rails 3.2 LTS to Rails 4.0. Under Rails 3.2 database_cleaner was operating fine, but under 4.0, some feature specs are failing with errors of the form:
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column "increment_by" does not exist
LINE 1: ...foos_id_seq"', (SELECT COALESCE(MAX("id")+(SELECT increment_...
: SELECT setval('"foos_id_seq"', (SELECT COALESCE(MAX("id")+(SELECT increment_by FROM "foos_id_seq"), (SELECT min_value FROM "foos_id_seq")) FROM "foos"), false)
I've recreated the test
db and tried both the :deletion
and :truncation
strategies along with all their various options, but to no avail. Can anyone shed any light on why it might be breaking?
ruby: 2.3.8
rails: 4.0.13
database_cleaner: 1.99.0
PostgreSQL: 13.4
I've used DatabaseCleaner for several years (thanks!) without issues. But my current setup seems to not work properly.
Basic setup:
On noteworthy thing about the app here is that it's using multiple databases: One main database and several tenant databases. In the test environment, this currently means two databases in total (main + test tenant).
Here's the test setup:
TEST_TENANT = 'the_tenant'
TEST_TENANT_API_KEY = SecureRandom.uuid.gsub('-', '')
RSpec.configure do |config|
config.mock_with :rspec
config.expect_with :rspec
config.raise_errors_for_deprecations!
# config.order = 'random'
config.before(:suite) do
ApiApp.establish_connection # ApiApp is connected to the main DB
primary_cleaner.clean_with(:truncation)
tenant = ApiApp.where(identifier: TEST_TENANT).first_or_create!(
name: 'The tenant',
api_key: TEST_TENANT_API_KEY
)
# force full recreate of tenant DB
config = ActiveRecord::Base.configurations[tenant.identifier]
ActiveRecord::Tasks::DatabaseTasks.drop(config) rescue nil
ActiveRecord::Tasks::DatabaseTasks.create(config)
ActiveRecord::Tasks::DatabaseTasks.load_schema(config, :sql, 'db/structure.sql')
# seed it
AppSetter.with(TEST_TENANT) do
Dir["#{Dir.pwd}/spec/seeds/*.rb"].sort.each { |f| load f }
end
end
config.before(:each) do |example|
AppSetter.set(TEST_TENANT)
puts ['[RSpec] start before', "number of patients: #{Patient.count}", Patient.connection.instance_variable_get(:@config)[:database]].inspect
# primary_cleaner.start
tenant_cleaner.start
puts ['[RSpec] end before', "number of patients: #{Patient.count}", Patient.connection.instance_variable_get(:@config)[:database]].inspect
end
config.append_after(:each) do
puts ['[RSpec] start append_after', "number of patients: #{Patient.count}", Patient.connection.instance_variable_get(:@config)[:database]].inspect
# primary_cleaner.clean
tenant_cleaner.clean
puts ['[RSpec] end append_after', "number of patients: #{Patient.count}", Patient.connection.instance_variable_get(:@config)[:database]].inspect
puts '-' * 100
end
private
def primary_cleaner
@primary_cleaner ||= cleaner_for(connection: ENV['RACK_ENV'].to_sym)
end
def tenant_cleaner
@tenant_cleaner ||= cleaner_for(connection: TEST_TENANT.to_sym)
end
def cleaner_for(options)
DatabaseCleaner[:active_record, options].tap { |cleaner| cleaner.strategy = :transaction }
end
end
Now when I have 2 examples in RSpec which both use the same basic setup with a test patient (Patient.create!(email: '[email protected]', ...)
), the uniqueness constraint on email fails because apparently the record exists already. The debugging code in the output (from the puts
statements above) confirms this:
(1) ["[RSpec] start before", "number of patients: 0", "api_test_the_tenant"]
["[DatabaseCleaner] before start", "open transactions: 0", "api_test_the_tenant"]
(2) ["[DatabaseCleaner] after start", "open transactions: 1", "api_test_the_tenant"]
["[RSpec] end before", "number of patients: 0", "api_test_the_tenant"]
(3) ["[RSpec] start append_after", "number of patients: 1", "api_test_the_tenant"]
["[DatabaseCleaner] before clean", "open transactions: 1", "api_test_the_tenant"]
(4) ["[DatabaseCleaner] after clean", "open transactions: 0", "api_test_the_tenant"]
(5) ["[RSpec] end append_after", "number of patients: 1", "api_test_the_tenant"]
----------------------------------------------------------------------------------------------------
(6) ["[RSpec] start before", "number of patients: 1", "api_test_the_tenant"]
["[DatabaseCleaner] before start", "open transactions: 0", "api_test_the_tenant"]
["[DatabaseCleaner] after start", "open transactions: 1", "api_test_the_tenant"]
["[RSpec] end before", "number of patients: 1", "api_test_the_tenant"]
["[RSpec] start append_after", "number of patients: 1", "api_test_the_tenant"]
["[DatabaseCleaner] before clean", "open transactions: 1", "api_test_the_tenant"]
["[DatabaseCleaner] after clean", "open transactions: 0", "api_test_the_tenant"]
["[RSpec] end append_after", "number of patients: 1", "api_test_the_tenant"]
----------------------------------------------------------------------------------------------------
(I've numbered some lines to make it easier to follow. The [DatabaseCleaner]
output comes from some puts
statements I've put in the corresponding strategy in https://github.com/DatabaseCleaner/database_cleaner/blob/master/lib/database_cleaner/active_record/transaction.rb.)
(1) Before the first example (in the first before
block), there are 0 patients in the tenant database => expected.
(2) tenant_cleaner.start
has caused an open transaction in the tenant database => expected.
(3) Running the example has created a patient in the tenant database => expected.
(4) tenant_cleaner.clean
rolls back the transaction => expected.
(5) There is still 1 patient in the tenant database => this is NOT expected.
(6) The before
block of the next example confirms that 1 patient has remained in the tenant database => this is NOT expected.
Commenting in/out the cleaning of the primary database doesn't change anything (I wouldn't have expected it to either, but I tried it nonetheless).
Does anyone have any insights as to what's going wrong here?
Thanks a lot in advance.
DatabaseCleaner treats CockroachDB as PostgreSQL (which makes sense because CockroachDB emulated PostgreSQL to some extent). It uses the RESTART IDENTITY
clause in the TRUNCATE TABLE
query, which isn't supported by CockroachDB. There should probably be separate handling for CockroachDB that doesn't use that clause, and perhaps runs a separate query to reset the auto-increment counter.
It's sometimes necessary to add pg_catalog
explicitly to the search path (if you don't want it added implicitly at the beginning of the search path). It then gets picked up by current_schemas(false)
which causes DatabaseCleaner to attempt truncation of system tables. I can work around it with a monkey patch but I don't think there is ever a reason to want to truncate system tables -- could these be always excluded?
AR has adapters for a lot more than Sqlite, MySQL, and PostgreSQL. Can we test our code that works against those, too? What does the AR repo do about this?
Hi,
I can find on README.md
that we can use some options
DatabaseCleaner[:active_record].strategy = :truncation, only: ["users"]
This syntax is not ruby valid.
How do you specify options for your strategy ?
Regards,
We switched from truncation to deletion for a project of ours because for some reason truncation was incredibly slow on some clients (apparently this has been observed before).
Unfortunately, on some machines this breaks the tests because mysql is complaining about a duplicate index. So to me this suggests that the indices are not reliably reset on all clients.
The behaviour seems particularly erratic:
However, the behaviour differs only across machines, it's consistent on one specific machine.
I suppose it must have something to do with specific mysql configuration options, but I'm at a loss as to what it could be in particular.
I realise this is not a precise problem description but maybe somebody around here has a clue?
Edit:
On a second look, it doesn't even appear to be consistent on a single machine. At least it works perfectly fine in the Arch Linux VM today. That's even weirder.
Edit2:
Works on Arch Linux with MySQL, but not mariadb, it seems.
I'm getting an ODBC::Error Invalid cursor state during the database_cleaner "clean" step.
Stack trace provided below.
ODBC::Error: 24000 (0) [FreeTDS][SQL Server]Invalid cursor state: SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = 'BASE TABLE' ORDER BY TABLE_NAME (ActiveRecord::StatementInvalid)
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:389:in run' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:389:in
block in raw_connection_run'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver_adapter.rb:502:in with_sqlserver_error_handling' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:384:in
raw_connection_run'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:376:in _raw_select' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:371:in
block in raw_select'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:280:in block in log' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activesupport-3.2.2/lib/active_support/notifications/instrumenter.rb:20:in
instrument'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract_adapter.rb:275:in log' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:371:in
raw_select'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/database_statements.rb:9:in select_rows' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-3.2.2/lib/active_record/connection_adapters/abstract/database_statements.rb:38:in
select_values'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver/schema_statements.rb:11:in tables' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/active_record/truncation.rb:25:in
database_cleaner_table_cache'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/active_record/truncation.rb:243:in tables_to_truncate' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/active_record/truncation.rb:235:in
block in clean'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/activerecord-sqlserver-adapter-3.2.9/lib/active_record/connection_adapters/sqlserver_adapter.rb:267:in disable_referential_integrity' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/active_record/truncation.rb:231:in
clean'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/base.rb:77:in clean' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/configuration.rb:79:in
block in clean'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/configuration.rb:79:in each' /usr/local/rvm/gems/ruby-1.9.2-p320/gems/database_cleaner-0.9.1/lib/database_cleaner/configuration.rb:79:in
clean'
/usr/local/rvm/gems/ruby-1.9.2-p320/gems/cucumber-rails-1.3.0/lib/cucumber/rails/hooks/database_cleaner.rb:9:in `After'
Dependabot just upgraded database_cleaner-active_record
to 2.0 and now all tests fail.
I used to have DatabaseCleaner.allow_remote_database_url = true
in my code.
Failure/Error: require File.expand_path('../config/environment', __dir__)
DatabaseCleaner::Safeguard::Error::RemoteDatabaseUrl:
ENV['DATABASE_URL'] is set to a remote URL. Please refer to https://github.com/DatabaseCleaner/database_cleaner#safeguards
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:45:in `run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `block in run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `each'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/safeguard.rb:104:in `run'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaner.rb:31:in `initialize'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `new'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:41:in `add_cleaner'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `block in []'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `fetch'
# /usr/local/bundle/gems/database_cleaner-core-2.0.0/lib/database_cleaner/cleaners.rb:12:in `[]'
# /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner/active_record.rb:7:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# /usr/local/bundle/gems/database_cleaner-active_record-2.0.0/lib/database_cleaner-active_record.rb:1:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/bundler-2.1.4/lib/bundler.rb:174:in `require'
# ./config/application.rb:22:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:53:in `require_relative'
# ./config/environment.rb:4:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# ./spec/rails_helper.rb:6:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `require'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:23:in `block in require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/loaded_features_index.rb:92:in `register'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:22:in `require_with_bootsnap_lfi'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:31:in `require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `block in require'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:324:in `require'
# ./spec/uploaders/invoice_uploader_spec.rb:3:in `<main>'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
# /usr/local/bundle/gems/bootsnap-1.6.0/lib/bootsnap/load_path_cache/core_ext/kernel_require.rb:59:in `load'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `block in load'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:291:in `load_dependency'
# /usr/local/bundle/gems/activesupport-6.0.3.4/lib/active_support/dependencies.rb:318:in `load'
I also tried to set DatabaseCleaner.url_allowlist = ['postgres://postgres@postgres']
, but this did not help.
The SelectiveTruncation strategy (where database_cleaner looks at "table_rows" in the information_schema) is always applied with a combination of ActiveRecord and Mysql2 adapter. Unfortunately this strategy is unreliable because table_rows is not guaranteed to be accurate. I'm using MariaDB 10.0.16 and I currently have a cucumber javascript scenario where this happens fairly predictably (can't share that in it current state, sorry).
The MariaDB documentation indicates that this information is not reliable with InnoDB tables: https://mariadb.com/kb/en/mariadb/information-schema-tables-table/
The same thing is documented for MySQL 5.6: http://dev.mysql.com/doc/refman/5.6/en/tables-table.html
Looking at information_schema for that particular test database I do notice that AUTO_INCREMENT > 1 for the table that has data and TABLE_ROWS = 0. However, I also see an AUTO_INCREMENT > 1 for tables that (no longer) have data in them.
Perhaps it might be enough to use ... AND (table_rows > 0 OR auto_increment > 1) to clean only those tables that still have or at least had data in them. Otherwise it might be best to just turn this selectivetrunction off, or at least provide an option for that.
Hi,
we upgraded from 1.7.0 to 1.8.2 and now getting the following error when running specs
An error occurred while loading spec_helper.
Failure/Error: require 'database_cleaner/active_record'
ArgumentError:
Delegation needs a target. Supply an options hash with a :to key as the last argument (e.g. delegate :hello, to: :greeter).
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/core_ext/module/delegation.rb:154:in `delegate'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-1.8.2/lib/database_cleaner/base.rb:56:in `<class:Base>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-1.8.2/lib/database_cleaner/base.rb:8:in `<module:DatabaseCleaner>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-1.8.2/lib/database_cleaner/base.rb:7:in `<top (required)>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `block in require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-1.8.2/lib/database_cleaner/configuration.rb:1:in `<top (required)>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `block in require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-1.8.2/lib/database_cleaner.rb:3:in `<top (required)>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `block in require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/database_cleaner-active_record-1.8.0/lib/database_cleaner/active_record.rb:2:in `<top (required)>'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `block in require'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:240:in `load_dependency'
# /Users/kairamuenke/.gem/ruby/2.5.7/gems/activesupport-4.2.11.1/lib/active_support/dependencies.rb:274:in `require'
# ./spec/spec_helper.rb:24:in `<top (required)>'
When both MySQL and Postgres ActiveRecord models are used, and when testing MySQL-backed code, Truncation strategy includes BOTH ::DatabaseCleaner::ActiveRecord::MysqlAdapter and ::DatabaseCleaner::ActiveRecord::PostgreSQLAdapter into ::DatabaseCleaner::ActiveRecord::AbstractAdapter, which, in turn, gets included into ActiveRecord::AbstractAdapter.
Since PostgreSQLAdapter gets included last, even when ActiveRecord::Base.connection is a MySQL connection, DatabaseCleaner attempts to execute Postgres-specific truncations, resulting in errors:
NameError: undefined local variable or method `postgresql_version' for #<ActiveRecord::ConnectionAdapters::Mysql2Adapter:0x0000000692feb8>
occurred at /usr/local/rvm/gems/ruby-1.9.2-p290/gems/database_cleaner-1.1.1/lib/database_cleaner/active_record/truncation.rb:123:in `db_version'
Hello - thanks for putting together database_cleaner
! I've been using the gem for a while and I know maintenance takes effort, so appreciate those who contribute to this project.
I'm using the following setup -
Like many multi-tenant applications, my app utilizes postgres schemas which store parrallel, independent copies of my database tables.
So if I have two tenants (earth
, mars
) and two tables (users
, companies
) I would have the following tables in my DB -
schema_migrations
ar_internal_metadata
public.users
public.companies
earth.users
earth.companies
mars.users
mars.companies
When using DatabaseCleaner.strategy = :truncation
, it looks for a list of tables in my DB to truncate (see here).
When I run that query against my local test database -
SELECT schemaname || '.' || tablename
FROM pg_tables
WHERE
tablename !~ '_prt_' AND
tablename <> 'schema_migrations' AND tablename <> 'ar_internal_metadata'
AND schemaname = ANY (current_schemas(false))
;
?column?
-------------------------------------------
public.users
public.companies
(2 rows)
As you can see, it only returns the public schemas, because of the schemaname =
criteria. This effectively doesn't wipe my non-public schema tables, causing run-over between tests.
I understand that there are options to use a transaction instead, or even the only: [..]
parameter to list all my tables. But it seems like this should work as is without having to use one of those as a workaround (and if my DB is quite large, a whitelist can get messy and inefficient across thousands of tests).
Would it make sense to update the query logic to something like
SELECT schemaname || '.' || tablename
FROM pg_tables
WHERE
tablename !~ '_prt_' AND
tablename <> 'schema_migrations' AND tablename <> 'ar_internal_metadata'
AND schemaname NOT IN ('pg_catalog', 'information_schema')
;
Thanks!
Related to: https://github.com/DatabaseCleaner/database_cleaner/issues/225
I'm building an app using Rails 5.0.0.rc1 and I ran into the fact that the ar_internal_metadata
table is being cleaned with the :truncation
strategy. This is similar to DatabaseCleaner/database_cleaner#317.
When this happens and ActiveRecord::Migration.maintain_test_schema!
is being used, tests/specs will fail if there are pending migrations and database_cleaner
has had a chance to run and clear the metadata table. The error looks like this:
$ bundle exec rspec
rails aborted!
ActiveRecord::NoEnvironmentInSchemaError:
Environment data not found in the schema. To resolve this issue, run:
bin/rails db:environment:set RAILS_ENV=test
/Users/mike/dev/my_app/bin/rails:9:in `require'
/Users/mike/dev/my_app/bin/rails:9:in `<top (required)>'
/Users/mike/dev/my_app/bin/spring:13:in `require'
/Users/mike/dev/my_app/bin/spring:13:in `<top (required)>'
bin/rails:3:in `load'
bin/rails:3:in `<main>'
Tasks: TOP => db:test:load => db:test:purge => db:check_protected_environments
(See full trace by running task with --trace)
Coverage report generated for RSpec to /Users/mike/dev/my_app/coverage. 29 / 547 LOC (5.3%) covered.
/usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:572:in `check_pending!': (ActiveRecord::PendingMigrationError)
Migrations are pending. To resolve this issue, run:
bin/rails db:migrate RAILS_ENV=test
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:585:in `load_schema_if_pending!'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:591:in `block in maintain_test_schema!'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:822:in `suppress_messages'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:596:in `method_missing'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/activerecord-5.0.0.rc1/lib/active_record/migration.rb:591:in `maintain_test_schema!'
from /Users/mike/dev/my_app/spec/rails_helper.rb:32:in `<top (required)>'
from /Users/mike/dev/my_app/spec/controllers/widgets_controller_spec.rb:2:in `require'
from /Users/mike/dev/my_app/spec/controllers/widgets_controller_spec.rb:2:in `<top (required)>'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/configuration.rb:1333:in `load'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/configuration.rb:1333:in `block in load_spec_files'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/configuration.rb:1331:in `each'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/configuration.rb:1331:in `load_spec_files'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/runner.rb:106:in `setup'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/runner.rb:92:in `run'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/runner.rb:78:in `run'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/lib/rspec/core/runner.rb:45:in `invoke'
from /usr/local/var/rbenv/versions/2.3.1/lib/ruby/gems/2.3.0/gems/rspec-core-3.5.0.beta3/exe/rspec:4:in `<top (required)>'
from /usr/local/var/rbenv/versions/2.3.1/bin/rspec:23:in `load'
from /usr/local/var/rbenv/versions/2.3.1/bin/rspec:23:in `<main>'
As with was the case with DatabaseCleaner/database_cleaner#317, adding the table to the except
list is a workaround:
DatabaseCleaner.clean_with :truncation, except: %w(public.ar_internal_metadata)
I am currently trying to run a rspec test with Capybara and Database cleaner and I repetitively receive this threading error with Mysql.
Gemfile.lock
rspec (3.3.0)
capybara (2.4.4)
capybara-webkit (1.4.1)
database_cleaner (1.5.1)
Failure/Error: Unable to find matching line from backtrace
ActiveRecord::StatementInvalid:
Mysql2::Error: This connection is in use by: #<Thread:0x007f9fa58b9f88 sleep>: BEGIN
# /Users/reidcooper/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:303:in `query'
# /Users/reidcooper/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1.11/lib/active_record/connection_adapters/abstract_mysql_adapter.rb:303:in `block in execute'
spec_helper.rb
config.use_transactional_fixtures = false
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each, :js => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each, :feature => true) do
DatabaseCleaner.strategy = :truncation
end
config.before(:each) do
Warden.test_mode!
DatabaseCleaner.start
end
config.after(:each) do
Spree::User.auditing_enabled = false # if we don't disable auditing we run into a StackLevelTooDeep
DatabaseCleaner.clean
Warden.test_reset!
end
config.around(:each) do |example|
DatabaseCleaner.strategy = :truncation
DatabaseCleaner.cleaning do
example.run
end
end
The Spec File
it "should let me navigate to the registration page", :js => true do
visit '/products/tshirt'
save_and_open_screenshot
expect(page.status_code).to eq(200)
expect(page).to have_content "CONTENT"
end
Now the webpage screenshot shows up correct, however, I feel like its something to do with the javascript causing a threading issue.
Is it possible with deletion
strategy to reset ids sequence (Postgres) ? Cannot use truncation strategy, because it is really slow for us. (truncation
- 8 sec, deletion
0.3 sec)
group :test do
gem 'database_cleaner-active_record'
end
from the docs
I have a remote SQL Server and a Postgres database. Truncate commands are executed in the PostgreSqlAdapter syntax even for SQL Server models and therefore data is not being truncated for SQL Server. The truncate commands is in the format truncate table "public"."Table1";
which is obviously not going to work for SQL Server. I don't know what to do. This is my configuration.
The same configuration was working fine in a previous project where I used multiple MariaDB databases but here the adapter is different.
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.add_cleaner(:active_record, SQL_SERVER)
DatabaseCleaner.clean_with(:truncation)
DatabaseCleaner.strategy = :transaction
end
config.before(:each) do
DatabaseCleaner.start
end
config.after(:each) do
DatabaseCleaner.clean
end
end
We have some code where we reload an instance and then call update_attributes.
instance.reload.update_attributes error: error, last_sync_failure: Time.now
After the update from 1.2.0 to 1.6.1 this suddenly fails because it's trying to do an insert instead of an update in the database (making a database restraint fail).
The really funny thing is
instance.reload.reload.update_attributes error: error, last_sync_failure: Time.now
works just fine. As does
instance.tap(&:restore_attributes).update_attributes error: error, last_sync_failure: Time.now
No time right now to create a small testrepo, but figured I'd mention it anyway.
No other gems or code where updated.
I have a method like this:
def make_foo(bar, params)
foo = nil
self.transaction do
foo = self.create!(name: bar) rescue nil
end
unless foo
self.transaction do
foo = Something.find_by(name: bar).update(params)
end
end
foo
end
This method relies on a unique index on name to raise PG:Error, in which it will update the object instead. This is all fine until I use transaction in database_cleaner and it complains about
I tried using truncation for the test, and it somewhat works, but I'm just wondering if this is something that can be supported (sounds like nested transaction?)? Essentially I'd like to get the transaction to restart within the test, if that's possible.
Hi, This is probably not the repository for this issue, but putting it here incase someone else stumbles across it. Feel free to close it.
We are using DatabaseCleaner with Postgres 9.6.16 in a small Event Source application. Recently after adding a few more specs we noticed the database was consistently becoming deadlocked in our CI environment, across multiple EC2 instances each downloading a fresh Postgres container. I'm unable to replicate this locally.
Inspecting Postgres running queries shows two, one against each database (an "events" and "projections" database)
pid | age | usename | query
-----+-----------------+----------+-------------------------------------------------------------------------------
67 | 00:22:19.032665 | postgres | SELECT NULL AS "nil" FROM "some_table" LIMIT 1
66 | 00:22:19.056778 | postgres | DELETE FROM "aggregates"
(2 rows)
These hang, as far as I can tell, indefinitely. There's a bunch of locks for the DELETE FROM
query but all are granted
``` | f postgres=# SELECT * FROM pg_locks; locktype | database | relation | page | tuple | virtualxid | transactionid | classid | objid | objsubid | virtualtransaction | pid | mode | granted | f astpath ---------------+----------+----------+------+-------+------------+---------------+---------+-------+----------+--------------------+-----+---------------------+---------+-- -------- relation | 12407 | 11695 | | | | | | | | 5/19 | 639 | AccessShareLock | t | t virtualxid | | | | | 5/19 | | | | | 5/19 | 639 | ExclusiveLock | t | t relation | 12407 | 11797 | | | | | | | | 4/139 | 613 | AccessShareLock | t | t virtualxid | | | | | 4/139 | | | | | 4/139 | 613 | ExclusiveLock | t | t virtualxid | | | | | 3/1939 | | | | | 3/1939 | 67 | ExclusiveLock | t | t relation | 16384 | 123305 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 2 | 1 | 3/1939 | 67 | ExclusiveLock | t | f transactionid | | | | | | 40493 | | | | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123294 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 7 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123283 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123312 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123302 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123306 | | | | | | | | 3/1939 | 67 | ShareLock | t | f relation | 16384 | 123306 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123311 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123285 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 10 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123320 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 12 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123314 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 9 | 1 | 3/1939 | 67 | ExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 11 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123323 | | | | | | | | 3/1939 | 67 | ShareLock | t | f advisory | 16384 | | | | | | 0 | 8 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123274 | | | | | | | | 3/1939 | 67 | AccessShareLock | t | f relation | 16384 | 123274 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123290 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123318 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 1 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123301 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123296 | | | | | | | | 3/1939 | 67 | ShareLock | t | f relation | 16384 | 123296 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123315 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123319 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123291 | | | | | | | | 3/1939 | 67 | ShareLock | t | f relation | 16384 | 123291 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f object | 16384 | | | | | | 2615 | 2200 | 0 | 3/1939 | 67 | AccessShareLock | t | f advisory | 16384 | | | | | | 0 | 4 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123313 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f transactionid | | | | | | 40487 | | | | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123316 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 3 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123282 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123309 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123299 | | | | | | | | 3/1939 | 67 | ShareLock | t | f transactionid | | | | | | 40488 | | | | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123317 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123325 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123289 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123304 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f relation | 16384 | 123276 | | | | | | | | 3/1939 | 67 | AccessShareLock | t | f relation | 16384 | 123276 | | | | | | | | 3/1939 | 67 | ShareLock | t | f relation | 16384 | 123276 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 5 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123287 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f transactionid | | | | | | 40496 | | | | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123288 | | | | | | | | 3/1939 | 67 | AccessExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 6 | 1 | 3/1939 | 67 | ExclusiveLock | t | f advisory | 16384 | | | | | | 0 | 13 | 1 | 3/1939 | 67 | ExclusiveLock | t | f relation | 16384 | 123280 | | | | | | | | 3/1939 | 67 | ShareLock | t | f ```
Switching the cleaning from transactional
to truncation
resolves the issue.
I've just spent an afternoon debugging why some of my tables are truncated despite being in except
option and finally found out that in case of postgres they are just silently truncated because they reference other tables that should be legitimately truncated.
def truncate_tables(table_names)
return if table_names.nil? || table_names.empty?
execute("TRUNCATE TABLE #{table_names.map{|name| quote_table_name(name)}.join(', ')} RESTART IDENTITY CASCADE;")
end
I personally feel that this is not very intuitive because tables are truncated silently and it takes a while to get to the root cause of the problem if you have one, especially when you are working on complex application with several dozens of tables with complex structure and references.
I think that user should be able to decide how he wants to truncate the tables especially that there is a RESTRICT option when truncating with postgres and normally this is the default option. I would be super happy if I would be able to explicitly set that I want to use this option maybe with another option restrict: true
so I know excatly when I come across this reference problem. If it's not a complex change I'll do it happily myself :)
If there is more than one connection in the connection pool DatabaseCleaner will only begin the transaction on one of the connections.
https://github.com/rails/rails/blob/master/activerecord/lib/active_record/fixtures.rb#L979
Any ActiveRecord call afterwards will use the next connection in the pool which does not have a transaction.
When DatabaseCleaner closes the transaction, it will only do so on the next active connection. Thus leaving a connection with an active transaction in the connection pool which can deadlock when doing a deletion cleanup or any table modifications.
DatabaseCleaner should start and end transactions the same way ActiveRecord does by collecting all available connections and starting transactions on all of them.
start: https://github.com/rails/rails/blob/06a9e18e646845ded0f0a46a14c1eb6b3a14e55a/activerecord/lib/active_record/fixtures.rb#L949
end: https://github.com/rails/rails/blob/06a9e18e646845ded0f0a46a14c1eb6b3a14e55a/activerecord/lib/active_record/fixtures.rb#L967
Is there a specific reason that truncate is the only option that allows cascading?
Why is there no simple DELETE CASCADE
statement (which is a lot safer as far as not deleting records that aren't supposed to be)?
Is there really no changelog for this gem?
This is very obvious error I think.
I have this code.
describe "the signin process", type: :feature do
it "signs me in" do
@admin_user = FactoryBot.create(:admin_user)
visit '/users/sign_in'
within("#form-login") do
fill_in 'user_email', with: @admin_user.email
fill_in 'user_password', with: @admin_user.password
end
click_button 'Sign in'
users = User.all()
puts users[3] #Debug message to confirm new user is added to db
expect(page).to have_content 'Welcome'
end
end
And this is the active_database_cleaner configration
RSpec.configure do |config|
config.use_transactional_fixtures = false
config.before(:suite) do
# DatabaseCleaner.clean_with(:truncation)
end
config.before(:each) do
DatabaseCleaner.strategy = :transaction
DatabaseCleaner.start #if I comment this line it works
end
config.append_after(:each) do
DatabaseCleaner.clean
end
end
But click_button doesn't login and it remains or redirected back to login form
if I change the :transaction to :truncation it works.
any idea?
I am testing upgrading my app with rails main branch (7.1-alpha)
and I am getting an error with this gem
DEPRECATION WARNING: #tables currently returns both tables and views. This behavior is deprecated and will be changed with Rails 5.1 to only return tables. Use #data_sources instead.
This is being thrown when i run DatabaseCleaner.clean_with(:truncation)
After some searching, i found 2 PRs that fixed this and were merged onto master, PR1 PR2, but it seems like these fixes got lost somewhere.
I used git bisect to find which commit broke and it may have been this one, b83ed7d , but i do admit that i narrowed my search to what commit broke it on the database_cleaner gem, not on database_cleaner-active_record gem, so i am not sure
I am working on an older app and I want to install version 1.7.0 to avoid the deprecation warning related to DatabaseCleaner.connections
which is introduced in 1.8.0. A gem we use, cucumber-rails
is on an older version which triggers this warning, and upgrading won't be simple (requires upgrading Ruby).
My question is, why are versions of database_cleaner-active_record before 1.8.0 no longer available on Rubygems? Thanks.
In the very simple case you can change it to use truncation and ignore certain tables. Anything beyond this is not currently possible.
I create a default Admin user in my seed, this needs to exist all the time, but my cucumber tests create and delete users all over the place and it needs to be clear at the start of each scenario, expect for the Admin user..
It'd be nice to be able to set specific rules for tables..
random pseudo code rambling:
DatabaseCleaner.configure do |config|
config.strategy = :truncate
config.strategy[:users] = :deletion, "WHERE NOT `type` = 'Admin'"
config.strategy[:categories] = :preserve
end
Hi,
i am trying to use the DatabaesCleaner gem in several Rails' tests (Minitest) where i can't use Rails' built-in use_transactional_tests
feature.
In the following test, i disabled use_transactional_tests
as they do not work well in combination with Shrine
gem (see Testing with Shrine) and i am trying to use DatabaseCleaner
with the :deletion
strategy instead.
# in Gemfile
gem "rails", "~> 6.0.3", ">= 6.0.3.2"
group :test do
gem "database_cleaner-active_record", "~> 1.8"
end
# test/models/file_upload_test.rb
require 'test_helper'
class FileUploadTest < ActiveSupport::TestCase
# turn of transactional_tests and clean up with DatabaseCleaner
self.use_transactional_tests = false
DatabaseCleaner.strategy = :deletion
setup do
DatabaseCleaner.start
# copy file fixture and create file object
file = File.open(Rails.root.join("test/fixtures/files/image.png"))
copypath = File.join(File.dirname(file), "#{SecureRandom.hex(16)}#{File.extname(file.path)}")
FileUtils.cp(file.path, copypath)
@file = File.open(copypath)
end
teardown do
# delete copied file fixture if exists
File.delete(@file.path) if File.exist?(@file.path)
DatabaseCleaner.clean
end
test "attaches file to upload" do
@upload = FileUpload.create(file: @file)
assert_equal :store, @upload.reload.file.storage_key
end
end
Running the test i am receiving an ArgumentError: wrong number of arguments (given 3, expected 1)
(see stacktrace below). When i remove DatabaseCleaner
from the Gemfile
and from the test setup there is no error.
Error:
FileUploadTest#test_attaches_file_to_upload:
ArgumentError: wrong number of arguments (given 3, expected 1)
/usr/local/bundle/gems/database_cleaner-active_record-1.8.0/lib/database_cleaner/active_record/truncation.rb:152:in `truncate_tables'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/tasks/database_tasks.rb:221:in `block in truncate_tables'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/connection_handling.rb:251:in `swap_connection_handler'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/connection_handling.rb:159:in `with_handler'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/connection_handling.rb:117:in `connected_to'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/tasks/database_tasks.rb:213:in `truncate_tables'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/tasks/database_tasks.rb:359:in `reconstruct_from_schema'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/test_databases.rb:16:in `block in create_and_load_schema'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/test_databases.rb:14:in `each'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/test_databases.rb:14:in `create_and_load_schema'
/usr/local/bundle/gems/activerecord-6.0.3.2/lib/active_record/test_databases.rb:8:in `block in <module:TestDatabases>'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:63:in `block in after_fork'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:62:in `each'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:62:in `after_fork'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:79:in `block (2 levels) in start'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:75:in `fork'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:75:in `block in start'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:74:in `times'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:74:in `each'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:74:in `map'
/usr/local/bundle/gems/activesupport-6.0.3.2/lib/active_support/testing/parallelization.rb:74:in `start'
/usr/local/bundle/gems/minitest-5.14.1/lib/minitest.rb:138:in `run'
/usr/local/bundle/gems/minitest-5.14.1/lib/minitest.rb:68:in `block in autorun'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application.rb:175:in `fork'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application.rb:175:in `serve'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application.rb:145:in `block in run'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application.rb:139:in `loop'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application.rb:139:in `run'
/usr/local/bundle/gems/spring-2.1.0/lib/spring/application/boot.rb:19:in `<top (required)>'
/usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
/usr/local/lib/ruby/site_ruby/2.7.0/rubygems/core_ext/kernel_require.rb:72:in `require'
-e:1:in `<main>'
Regards
I am migrating from Rails 4.0 to 4.2. Everything worked fine with Rails 4.0. The issue I am having is with rails 4.2 (something might have changed)
Anyway, I suspect database cleaner is the problem in my tests. When I run test independently, all works well. But when I run my full suite of test, I am getting
ActiveRecord::NoDatabaseError: ActiveRecord::NoDatabaseError: No such file or directory @ dir_s_mkdir - /mnt/hgfs/zumobi/feeder/tmp/microsites/f213d554-bc6d-43fa-885e-f89f27cbf23a
The issue from what I can tell is that I have a normal set of active record models that talks to a postgres db by default. But I also have a small separate set of sqlite to generate sqlite3 db using data stored in postgres and those model use as well active record.
Those special model to use activerecord setup their own establish_connection with
Mymodel.establish_connection(connection_details)
But seems like DatabaseCleaner is not liking this. Iit seems to me that after it run the sqlite3 db test against those special models, the connection is remaining against that sqlite db (which is gone as the end of the tests concerning sqlite3 models). Then when other model get tested that uses Postgres, It still trying to access the sqlite 3 db
Any idea what I can do to fix this? I can try to make a repro model but would need more time
With the recent change on #58 you started using RESTRICT But it breaks our tests right now.
You should only use RESTRICT if the KEEP_TABLES option is defined. Otherwise, CASCADE is the only option to truncate related tables without getting errors.
here is the error:
ActiveRecord::StatementInvalid:
PG::FeatureNotSupported: ERROR: cannot truncate a table referenced in a foreign key constraint
DETAIL: Table "teams" references "events".
HINT: Truncate table "teams" at the same time, or use TRUNCATE ... CASCADE.
After upgrading Rails from 6.0.3.4 to 6.1.1 I got issues caused by changes in ActiveRecord.
Below is the code:
DatabaseCleaner[:active_record, connection: :test]
RSpec.configure do |config|
config.before(:suite) do
DatabaseCleaner.clean_with(:truncation, pre_count: true)
end
...
end
and error stacktrace:
An error occurred in a `before(:suite)` hook.
Failure/Error: DatabaseCleaner.clean_with(:truncation, pre_count: true)
NoMethodError:
undefined method `spec' for #<ActiveRecord::ConnectionAdapters::ConnectionPool:0x000055ae96b423d8>
Did you mean? inspect
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:91:in `block in lookup_from_connection_pool'
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:91:in `each'
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:91:in `detect'
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:91:in `lookup_from_connection_pool'
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/base.rb:63:in `connection_class'
.../gems/database_cleaner-1.8.5/adapters/database_cleaner-active_record/lib/database_cleaner/active_record/truncation.rb:237:in `clean'
.../gems/database_cleaner-1.8.5/lib/database_cleaner/base.rb:61:in `clean_with'
.../gems/database_cleaner-1.8.5/lib/database_cleaner/configuration.rb:92:in `block in clean_with'
.../gems/database_cleaner-1.8.5/lib/database_cleaner/configuration.rb:92:in `each'
.../gems/database_cleaner-1.8.5/lib/database_cleaner/configuration.rb:92:in `clean_with'
# ./spec/support/database_cleaner.rb:8:in `block (2 levels) in <top (required)>'
I want to see if we can refactor to do what we need to do without modifying AR in place.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.