db-migrate / pg Goto Github PK
View Code? Open in Web Editor NEWA postgresql driver for db-migrate.
License: Other
A postgresql driver for db-migrate.
License: Other
The Drop DB command doesn't work if you're trying to drop the database which you're connected to. You can get around this by creating another environment that connects to the template1 or postgres database, and then drop the one you want.
I can't think of an intelligent way to bake this behaviour into the driver so we don't need to use a workaround, any ideas? If you have an implementation idea I'm happy to take a stab at implementing it.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Using the latest postgres docker image gives the below error
This looks like a problem with the docker image, but would be very easy to accomodate
Essentially postgres returns { server_version: '10.3 (Debian 10.3-1.pgdg90+1)' }
and a little version = version.split(' ')[0]
after index.js:72
fixes it. Or one could downgrade to postgres:9.6
, which is what I've done.
This may be a "Won't Fix" but hopefully it'll help anyone else facing the same error
[ERROR] TypeError: Invalid Version: 10.3 (Debian 10.3-1.pgdg90+1).0
at new SemVer (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:305:11)
at compare (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:578:10)
at Function.gte (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:627:10)
at Object.<anonymous> (/Users/aaronlang/not-safe-not-fair/node_modules/db-migrate-pg/index.js:176:42)
at tryCatcher (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:512:31)
at Promise._settlePromise (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:569:18)
at Promise._settlePromise0 (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:614:10)
at Promise._settlePromises (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:693:18)
at Async._drainQueue (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:133:16)
at Async._drainQueues (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:143:10)
at Immediate.Async.drainQueues [as _onImmediate] (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:17:14)
at runCallback (timers.js:756:18)
at tryOnImmediate (timers.js:717:5)
at processImmediate [as _immediateCallback] (timers.js:697:5)
Running a migration against postgres v12 or v13[1] fails with the error:
error: type "string" does not exist
that's coming from this bit of code in index.js:
// we case run_on to string, for dbs that may have bigger precision
// than miliseconds
_getKV: function (table, key) {
var sql =
`SELECT ${this.escapeDDL('key')},
${this.escapeDDL('value')},
${this.escapeDDL('run_on')}::STRING // <------ THIS should be ::TEXT
FROM ` +
this._escapeDDL +
table +
this._escapeDDL +
` WHERE ${this.escapeDDL('key')} = $1`;
return this.allAsync(sql, [key]).then(([row]) => row);
},
string
isn't one of the postgres data types, but text
is.
[1] Probably other versions too, those are just the ones I tested.
Refers to db-migrate/node-db-migrate#338
Currently the default behavior of the used postgres driver itself is to use the current username as database name if no database was configured. This most probably results in a database does not exists error.
Default to use the database postgres
which always exists.
I am getting the following error in version 1.3.0:
Error: No such driver found, please try to install it via npm install db-migrate-pg or npm install -g db-migrate-pg
at Object.exports.connect (node_modules/db-migrate/lib/driver/index.js:77:11)
at Object.exports.connect (node_modules/db-migrate/connect.js:15:10)
at /home/vinicius/Luizalabs/Projects/fat-copy/faturamento-api/node_modules/db-migrate/lib/commands/up.js:15:13
at tryCatcher (node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromiseCtx (node_modules/bluebird/js/release/promise.js:641:10)
at _drainQueueStep (node_modules/bluebird/js/release/async.js:97:12)
at _drainQueue (node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:461:21)
Until version 1.2.2 it was working perfectly, but after new versions were published the lib is showing an error.
Any solution?
The error is not on my side (as @wzrdtales said in issue 74), since until yesterday in version 1.2.2 everything was working perfectly.
After the publication of the new versions (yesterday, May 4th) the library is no longer working.
My node version is v12.22.12
Error description:
db-migrate/node-db-migrate#323
Solution:
Unset length when auto_increment is set.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
https://github.com/db-migrate/pg/blob/master/index.js#L564-L566
arguments[index] = arguments[index].map(function (value) {
return typeof value === 'string' ? value : JSON.stringify(value);
});
What is this function doing? db-migrate
#insert doesn't do anything with de-stringifying json
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Hi and thank you for building this library.
When running db-migrate with the following config
{
"sql-file": true,
"dev": {
"driver": "pg",
"user": {
"ENV": "POSTGRES_USER"
},
"password": {
"ENV": "POSTGRES_PASSWORD"
},
"host": {
"ENV": "POSTGRES_HOST"
},
"database": "main",
"port": "5432",
"ssl": {
"sslcert": { "ENV": "POSTGRES_CA" },
"sslkey": { "ENV": "POSTGRES_KEY" },
"sslmode": "disable"
}
}
}
we get
[ERROR] ReferenceError: fs is not defined
at exports.connect (/usr/src/app/node_modules/db-migrate-pg/index.js:705:46)
at connect (/usr/src/app/node_modules/db-migrate/lib/driver/index.js:93:12)
...
This is due to fs not being initialised but used here: https://github.com/db-migrate/pg/blob/master/index.js#L704
It also looks like db-migrate-pg is expecting ssl config to be file instead of variables. This is not the case on our use case as we store sslcert and sslkey in secrets as a value instead of a file.
Thanks!
There can be a problem related to the way the server reports it's version info (depending opt how it was compiled) which is causing the server check to fail, like so
yarn run v1.4.0
$ db-migrate up -m ./node_modules/byte-shared/migrations -e ${NODE_ENV:-development} && db-migrate up -e ${NODE_ENV:-development}
[ERROR] TypeError: Invalid Version: 10.2 (Ubuntu 10.2-1.pgdg14.04+1).0
at new SemVer (/app/node_modules/semver/semver.js:305:11)
at compare (/app/node_modules/semver/semver.js:578:10)
at Function.gte (/app/node_modules/semver/semver.js:627:10)
at Object.<anonymous> (/app/node_modules/db-migrate-pg/index.js:176:42)
This is related to #27 and also #31 but both the accepted and proposed solutions are incorrect/incomplete. For example, the show server_version
command can return 10.2 (Ubuntu 10.2-1.pgdg14.04+1)
. A better approach is to use show server_version_num
which returns a “machine readable” version number, as per https://www.postgresql.org/docs/10/static/runtime-config-preset.html#GUC-SERVER-VERSION-NUM and then manipulate this number to get the semver version. Something like this:
const major = Math.floor(server_version_num / 10000)
const minor = Math.floor((server_version_num - major * 10000) / 100)
const patch = Math.floor(server_version_num - major * 10000 - minor * 100)
const version = `${major}.${minor}.${patch}` // 10.0.2
Let me know if you agree/are happy with this approach and I'll go ahead and prepare a PR.
The latest version of this package appears to be 1.2.2, but the CHANGELOG.md only has release notes up to 0.5.1.
What has changed since then?
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The schema schema: 'xxx'
is not reflected when used with ENV
option inside the database.json
file.
{
"dev": {
"ENV": "POSTGRES_CONNECTION_STRING",
"schema": "my_schema"
}
}
Here is the db-migrate/pg log:
[INFO] require: db-migrate-pg
[INFO] connecting
[INFO] connected
[SQL] show server_version_num
[SQL] SHOW search_path
[SQL] SET search_path TO "$user","public"
[SQL] SELECT table_name FROM information_schema.tables WHERE table_name = 'migrations' AND table_schema = 'public'
[INFO] creating table: migrations
[SQL] CREATE TABLE IF NOT EXISTS "migrations" ("id" SERIAL PRIMARY KEY NOT NULL, "name" VARCHAR (255) NOT NULL, "run_on" TIMESTAMP NOT NULL) << schema not reflected
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I have followed the instruction in the documentation, created two files that will create a pets table and a owners table. I specified a schema name in the database.json file. However when the migration finishes, I found out that the new tables created are in the public schema!
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
When upgrading to latest version, which upgrades pg driver to 8, it breaks ssl with self signed certs
Related to: brianc/node-postgres#2009
Driver major version updates should result in major version bump to prevent breaking scenarios.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I put my database migrations under ./data/migrations directory. However, when I ran db-migrate create --config data/config.json -e pg --migration-dir data/migrations users
it created migration file under ./migrations directory. Is this behavior expected?
Hello,
I'm running an application that uses [email protected]
, which has [email protected]
as a dependency.
Our compilation process for local devleopment runs two migrations: one set of base migrations to build our PostgreSQL schema, and another set of migrations under the scope dev
to add some seed data to the database.
Before today, our migrations ran fine. However, we're now seeing the following message in our logs:
[WARN] The driver you are using does not support the new state management. Please raise an issue in the repository of your driver maintainer
Our data now fails to seed because the dev
-scoped migration is running our base migrations a second time. Because the migrations already ran in the past, the second run fails and never gets to the seed migrations further down the line.
We did not update any packages. Is there something I am missing? I can provide additional data if requested.
Thanks!
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
When using a schema: 'mySchema'
configuration in the database.json, I can run db-migrate and it will create a migrations table on the
publicschema. Running the migration again leads to a second
migrationstable inside the
mySchema` schema.
I would guess that this might require a better configuration possibility for the migrations schema which is currently only possible via the --table my_migrations
command line argument. I would like to configure a like like myOtherSchema.my_migrations
, because I don't want to mix real/customer data with technical data (migrations).
Sidenote: is it possible to configure the migration table via the database.json? I tried, but failed.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I am getting the following error in version 1.3.0:
Error: No such driver found, please try to install it via npm install db-migrate-pg or npm install -g db-migrate-pg
at Object.exports.connect (node_modules/db-migrate/lib/driver/index.js:77:11)
at Object.exports.connect (node_modules/db-migrate/connect.js:15:10)
at /home/vinicius/Luizalabs/Projects/fat-copy/faturamento-api/node_modules/db-migrate/lib/commands/up.js:15:13
at tryCatcher (node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromiseCtx (node_modules/bluebird/js/release/promise.js:641:10)
at _drainQueueStep (node_modules/bluebird/js/release/async.js:97:12)
at _drainQueue (node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:461:21)
Until version 1.2.2 it was working perfectly, but after new versions were published the lib is showing an error.
Any solution?
my node version: v12.22.12
Does anyone know an easy way to get around this problem. I can't drop the DB since it's in use:
node node_modules/db-migrate/bin/db-migrate db:drop postgres
node node_modules/db-migrate/bin/db-migrate db:create postgres
Error: Failed to create database! error: database "postgres" already exists and is running in a postgres docker container.
The following runs fine in postgres, but when part of a migration in a .sql file, I get the error "error: function update_trigger_status() does not exist":
CREATE FUNCTION update_trigger_status()
RETURNS TRIGGER AS $$
BEGIN
NEW.last_active_flip := CASE WHEN NEW.active <> OLD.active AND NEW.active = 't' THEN now()
ELSE OLD.last_active_flip END;
NEW.last_inactive_flip := CASE WHEN NEW.active <> OLD.active AND NEW.active = 'f' THEN now()
ELSE OLD.last_inactive_flip END;
RETURN NEW;
END;
$$ language 'plpgsql';
node-db-migrate version: 0.10.0-beta.23
db-migrate-pg version: 0.2.5
tried other delimiters, didn't work. The plpgsql extension is installed.
As the migration scripts used to run in a single transaction , when trying to run the db-migrate:up for pg, getting the below error.
ALTER TYPE ... ADD cannot run inside a transaction block
Query that i am trying to run in script.
ALTER TYPE document_type ADD VALUE 'new value' AFTER 'existing value';
package version i am using in project .
"pg": "^8.5.1",
"db-migrate": "^0.11.11",
"db-migrate-pg": "^1.2.2",
Is there any workaround to skip the transaction for a particular or any suggestions ? It would be very helpful
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Refers to db-migrate/node-db-migrate#339
showDatabase
should reply with an array of database name strings.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I'm creating my first table through db-migrate (YEAH) and added a foreign key to one of the columns:
exports.up = function(db, callback) {
db.createTable('code_values', {
columns: {
id: {
type: "int",
primaryKey: true,
notNull: true,
autoIncrement: true
},
category_id: {
type: "int",
notNull: true,
foreignKey: {
name: "code_values_code_category_id_fk",
table: "code_categories",
mapping: {
category_id: "id"
}
}
}
}
},
callback);
};
I get the following error when I run the migration:
[ERROR] TypeError: Cannot read property 'onDelete' of undefined at Object.addForeignKey (/test/node_modules/db-migrate-pg/index.js:542:13) at /test/node_modules/db-migrate-base/index.js:124:21 at Object.recurseCallbackArray (/test/node_modules/db-migrate-base/index.js:104:37) at Object.<anonymous> (/test/node_modules/db-migrate-base/index.js:243:21) From previous event: at Object.createTable (/test/node_modules/db-migrate-base/index.js:241:6) at /usr/local/lib/node_modules/db-migrate/lib/driver/shadow.js:34:50 at runCallback (timers.js:763:18) at tryOnImmediate (timers.js:734:5) at processImmediate (timers.js:716:5)
I expected the table to be created without the explicit rules added to the foreign key. As for the foreign key, I expected it to default to NO ACTION for ON DELETE and ON UPDATE.
I was able to get things working when adding the following rules to the foreign key:
rules: { onDelete: 'RESTRICT', onUpdate: 'RESTRICT' }
Looking at db-migrate-pg's index.js, I believe the problem is that the function addForeignKey mandates a value for rules; and subsequently when rules.onDelete is called, the variable is undefined.
I'm willing to check this out further and work on a fix, but would love feedback in advance, especially since this would be my first time contributing to an open-source project. Please let me know your thoughts and understand, I'd like to isolate my first potential commit to this specific item (updated documentation included of course) and look globally in the future.
db-migrate version: 0.11.1
db-migrate-pg version: 0.4.0
Additional information:
- Node version: 9.9.0
- Platform: Mac
- PostgreSQL 10.3
- Postgres 2.1.5
---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/62575565-addforeignkey-error-cannot-read-property-ondelete-of-undefined?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github).
db-migrate-pg
will create the migrations
table in the correct schema, however, because db-migrate-pg
does not implement addMigrationRecord
, the default sql
method is invoked, and db-migrate
will attempt to insert the migration record into a different, possibly wrong, possible non-existing table.
Expected Behavior:
[SQL] INSERT INTO "public"."migrations" ("name", "run_on") VALUES ($1, $2) [ [ 'init/20190117194708-initialization',
2019-01-17T20:25:17.622Z ] ]
Actual:
[SQL] INSERT INTO "migrations" ("name", "run_on") VALUES ($1, $2) [ [ 'init/20190117194708-initialization',
2019-01-17T20:25:17.622Z ] ]
[ERROR] 20190117194708-initialization { error: relation "migrations" does not exist
According to PG docs multiple hosts can be supplied in connection string, however db-migrate is not supporting it and parsing it all wrong.
for example:
If I supply DATABSE_URL as postgres://user:@localhost:6432,localhost:5432/db_name
it is parsing as this:
{
driver: 'postgres',
user: 'user',
password: '',
database: ':6432,localhost/db_name',
host: 'localhost',
port: '5432'
}
However, using psql
I am able to log in with same connection string.
Scope Configuration does not seem to work at all with PostgreSQL.
ERROR: syntax error at or near "`" at character 20. STATEMENT: SET search_path TO `databaseName`
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
upstream issue brianc/node-postgres#1000
postgres update required
I'd like to propose replacing vows
test runner with something that's more modern and actively supported. The motivation is that on trying to run the test suite with [email protected]
to validate moving from TravisCI to GH actions, I get the following error:
$ npm run test
> [email protected] test
> node node_modules/.bin/vows
node[10176]: ../src/node_contextify.cc:710:static void node::contextify::ContextifyScript::New(const FunctionCallbackInfo<v8::Value> &): Assertion `args[1]->IsString()' failed.
1: 0x104d131dc node::Abort() [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
2: 0x104d1303c node::AppendExceptionLine(node::Environment*, v8::Local<v8::Value>, v8::Local<v8::Message>, node::ErrorHandlingMode) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
3: 0x104d091e4 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
4: 0x104e8ed2c v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
5: 0x104e8e50c v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
6: 0x104e8e080 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
7: 0x1054576ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
8: 0x1053e8470 Builtins_JSBuiltinsConstructStub [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
9: 0x1054e6550 Builtins_ConstructHandler [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
...
41: 0x1053e902c Builtins_JSEntryTrampoline [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
42: 0x1053e8cc4 Builtins_JSEntry [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
43: 0x104f1731c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
44: 0x104f16a74 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
45: 0x104e4db40 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
46: 0x104ce5fec node::ExecuteBootstrapper(node::Environment*, char const*, std::__1::vector<v8::Local<v8::String>, std::__1::allocator<v8::Local<v8::String> > >*, std::__1::vector<v8::Local<v8::Value>, std::__1::allocator<v8::Local<v8::Value> > >*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
47: 0x104ce7088 node::StartExecution(node::Environment*, char const*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
48: 0x104ce6d94 node::StartExecution(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
49: 0x104c6c318 node::LoadEnvironment(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
50: 0x104d48d3c node::NodeMainInstance::Run(int*, node::Environment*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
51: 0x104d48a58 node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
52: 0x104ce9040 node::Start(int, char**) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
53: 0x1a2a07e50 start [/usr/lib/dyld]
[1] 10155 abort npm run test
On upgrading to [email protected]
, it fixes the above error, but starts failing for other reasons. While I could probably work through and get vows up and running again, I think the time would be better spent just moving this library to a test runner that's actually supported and more widely used in the community. I know that the db-migrate
uses @hapi/lab
so that'd be an option, but that's still somewhat obscure within the broader community, so I'd suggest mocha or jest which are orders of magnitude more widely used and known. However, I'm happy to try for whatever suite @wzrdtales might choose.
Hi there,
I saw some advice on https://devcenter.heroku.com/articles/release-phase#design-considerations that suggests grabbing an "advisory lock" on the database before running a migration.
Basically, try SELECT pg_try_advisory_lock(migration);
at the start of the transaction.
t
then nothing currently has the lock, called 'migration', so proceed.I think that would be a one-line change in this module. Good idea?
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I want to have db-migrate authenticate by socket / peer auth, but this doesn't seem possible without hacking an env var.
const dbMigrateConfig = {
default: 'local',
local: {
driver: 'pg',
database: 'mydatabase',
host: '/var/run/postgres',
native: true
}
};
'#ENOENT /var/run/postgres'
process.env['DBTEST'] = 'postgres:///${mydatabase}?host=/var/run/postgresql&native=true';
const dbMigrateConfig = {
default: 'local',
local: {
ENV: "DBTEST"
}
};
success
.
Any chance the normal connection settings could allow connection via a socket?
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Related to #32; when PG produces a version like one below, the regex matcher matches on the wrong string, causing the following error to occur:
[ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Invalid Version: 0.18.04
at module.exports (/home/bttf/wyl-today/node_modules/db-migrate/lib/commands/helper/assert.js:9:14)
at /home/bttf/wyl-today/node_modules/db-migrate/lib/commands/up.js:29:16
at new SemVer (/home/bttf/wyl-today/node_modules/semver/semver.js:323:11)
at compare (/home/bttf/wyl-today/node_modules/semver/semver.js:614:10)
at Function.gte (/home/bttf/wyl-today/node_modules/semver/semver.js:663:10)
at Object.<anonymous> (/home/bttf/wyl-today/node_modules/db-migrate-pg/index.js:171:43)
at tryCatcher (/home/bttf/wyl-today/node_modules/bluebird/js/release/util.js:16:23)
at Promise._settlePromiseFromHandler (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:547:31)
at Promise._settlePromise (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:604:18)
at Promise._settlePromise0 (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:649:10)
at Promise._settlePromises (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:729:18)
at _drainQueueStep (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:93:12)
at _drainQueue (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:86:9)
at Async._drainQueues (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:102:5)
at Immediate.Async.drainQueues [as _onImmediate] (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:15:14)
at processImmediate (internal/timers.js:456:21)
Version output
pgdb=# select version() as version;
version
----------------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 10.12 (Ubuntu 10.12-0ubuntu0.18.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, 64-bit
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
delete this
Review this old PR, possibly migrate it and restore the credits to @zinigor
db-migrate/node-db-migrate#141
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
db-migrate 0.11.5
db-migrate-pg 0.5.0
knex 0.16.5
express 4.16.4
When run migration like
const knex = require('../config/knex');
'use strict';
var dbm;
var type;
var seed;
/**
* We receive the dbmigrate dependency from dbmigrate initially.
* This enables us to not have to rely on NODE_PATH.
*/
exports.setup = function(options, seedLink) {
dbm = options.dbmigrate;
type = dbm.dataType;
seed = seedLink;
};
exports.up = async (db) => {
const offers = await knex.select().from('offers');
console.log(offers);
};
exports.down = async (db) => {
// ...
};
exports._meta = {
"version": 1
};
Get in console:
$ - npm run db-migrate up
> [email protected] db-migrate /Users/asapozhnikov/Projects/darly/soosty-be
> db-migrate --config config/database.json "up"
[INFO] Processed migration 20190416095203-some migration name
[INFO] Done
and it hangs on.
All code inside migration executes well and does what it should.
It hangs on even if there any other migrations after this one.
I have second migration, which works well if I run it without first one(which doesn't work well), but when I run both migrations - they do all what they should do, but in the end it hangs on after Done.
When using the following up function to generate a table with a combined primary key, one of which references another table:
exports.up = function (db) {
return db.createTable('repositories', {
user_id: {
type: 'string',
length: 64,
primaryKey: true,
notNull: true,
foreignKey: {
name: 'repositories_users_id_fk',
table: 'users',
rules: {
onDelete: 'CASCADE',
onUpdate: 'RESTRICT'
},
mapping: 'id'
}
},
id: {
type: 'string',
length: 128,
primaryKey: true,
notNull: true
},
country_code_evaluated: {
type: 'boolean',
notNull: true,
defaultValue: false
},
country_code: {
type: 'string',
length: 3,
notNull: false
},
repository_evaluated: {
type: 'boolean',
notNull: true,
defaultValue: false
}
});
};
The following SQL Code is generated when executing db-migrate up
(I checked via --dry-run
and -v
):
CREATE TABLE "repositories" ("user_id" VARCHAR (64) NOT NULL, "id" VARCHAR (128) NOT NULL, "country_code_evaluated" BOOLEAN NOT NULL DEFAULT false, "country_code" VARCHAR (3) , "repository_evaluated" BOOLEAN NOT NULL DEFAULT false, PRIMARY KEY ("user_id", "id")) ;
ALTER TABLE "repositories" ADD CONSTRAINT "repositories_users_id_fk" FOREIGN KEY ([ '"user_id"' ]) REFERENCES "users" ([ '"id"' ]) ON DELETE CASCADE ON UPDATE RESTRICT;
Which throws the assertion [ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: syntax error at or near "["
when executed.
My environment
Node: v12.1.0
NPM: 6.9.0
db-migrate 0.11.5
db-migrate-pg 0.5.0
Hi, there is currently no way to create a column with the type timestamp with time zone
.
When I look into the code I found that datetime
is mapped to timestamp
. So no matter if I define a timestamp
or a datetime
I will have a timezone
.
In the MySQL documentation for DATETIME
you can see that this type of column takes care of local time zone.
Is there something wrong to map datetime
to timestamp with time zone
?
I tried to update the mapping locally and all work fine for me. Do you see any issue doing this ?
If not I can submit a PR with these changes.
Thanks.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
it seems to only operate on
leaving out
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
The library errors as follows:
users_db_1 | 2023-01-26 17:47:59.294 UTC [76] ERROR: permission denied for schema public at character 28
users_db_1 | 2023-01-26 17:47:59.294 UTC [76] STATEMENT: CREATE TABLE IF NOT EXISTS "migrations_state" ("key" VARCHAR PRIMARY KEY NOT NULL UNIQUE, "value" TEXT NOT NULL, "run_on" T
IMESTAMP NOT NULL)
turnstile_1 | [ERROR] unhandledRejection
turnstile_1 | [ERROR] error: permission denied for schema public
turnstile_1 | at Parser.parseErrorMessage (/opt/app/node_modules/pg-protocol/dist/parser.js:287:98)
turnstile_1 | at Parser.handlePacket (/opt/app/node_modules/pg-protocol/dist/parser.js:126:29)
turnstile_1 | at Parser.parse (/opt/app/node_modules/pg-protocol/dist/parser.js:39:38)
turnstile_1 | at Socket.stream.on (/opt/app/node_modules/pg-protocol/dist/index.js:11:42)
turnstile_1 | at emitOne (events.js:116:13)
turnstile_1 | at Socket.emit (events.js:211:7)
turnstile_1 | at addChunk (_stream_readable.js:263:12)
turnstile_1 | at readableAddChunk (_stream_readable.js:250:11)
turnstile_1 | at Socket.Readable.push (_stream_readable.js:208:10)
turnstile_1 | at TCP.onread (net.js:601:20)
turnstile_1 | npm ERR! code ELIFECYCLE
turnstile_1 | npm ERR! errno 1
turnstile_1 | npm ERR! [email protected] migrate: `db-migrate "up"`
turnstile_1 | npm ERR! Exit status 1
turnstile_1 | npm ERR!
turnstile_1 | npm ERR! Failed at the [email protected] migrate script.
turnstile_1 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
turnstile_1 |
turnstile_1 | npm ERR! A complete log of this run can be found in:
turnstile_1 | npm ERR! /root/.npm/_logs/2023-01-26T17_47_59_320Z-debug.log
This error comes from an update in postgres 15 which requires updates to the public schema to be done by a user with particular explicitly-set permissions. Article / postgres documentation.
Stepping back to Postgres 14 fixes this issue but isn't a great long term solution.
There should be an option to specify the schema through env. variables, same as with DATABASE_URL
, e.g. DATABASE_SCHEMA
. Now there is no way to use DATABASE_URL
for postgres without also creating a database.json (config) file.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
I am using the new managed Postgres service in Azure. My application doesn't have a problem connecting to actual PostgreSQL databases. However, a migrate up against the managed database consistently results in a fatal exception that occur after the migration completes.
Since the migrations run successfully, the problem seems to be related to how the connection is getting terminated.
Error: read ECONNRESET
at exports._errnoException (util.js:1050:11)
at TLSWrap.onread (net.js:581:26)
The application uses the latest version of db-migrate for node and the most recent pg driver.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
database.json
{
"dev": {
"driver": "pg",
"ssl": "false",
"user":"postgres",
"password": {"ENV": "DB_PASS_LOCAL"},
"host": {"ENV": "DB_HOST_LOCAL"},
"database": {"ENV": "DB_NAME_LOCAL"},
"port": {"ENV": "DB_PORT_LOCAL"},
"schema": {"ENV": "DB_SCHEMA_NAME_LOCAL"}
},
"sql-file" : true
}
Not sure why I get this, when I've specified false
for ssl
when I run db-migrate up
:
The server does not support SSL connections
I have installed db-migrate
and db-migrate-pg
driver all locally and run it with script hook in package.json
"scripts": {
"migrate": "node node_modules/db-migrate/bin/db-migrate"
In my database.json
file I have described environments for dev
and test
as shown below
{
"dev": {
"driver": "pg",
"user": {"ENV": "PGUSER"},
"password": {"ENV": "PGPASSWORD"},
"host": {"ENV": "PGHOST"},
"database": {"ENV": "PGDATABASE"}
},
"test": {
"driver": "pg",
"user": "myusername",
"password": "mypwd",
"host": "localhost",
"database": "test"
}
Also am using dotenv and that is where the envs for dev are picked up.
I can run migrations on the dev database successfully yarn run migrate up
But when I change the environment with the --env option or -e option with test, it runs the migrations on the dev environment which of course will return no migrations to run
>yarn run migrate up --env test
yarn run v0.27.5
$ node node_modules/db-migrate/bin/db-migrate "up"
[INFO] No migrations to run
[INFO] Done
Done in 0.90s.
I dont know what the issue is?
Hello,
I'm trying to deploy on Heroku but it complains that SSL is not enabled. I tried activate "native: true"
in my database.json file :
{
"prod": {
"ENV": "DATABASE_URL",
"native": true
},
"dev": {
"ENV": "DATABASE_URL"
}
}
I had no luck. By slightly changing the code in index.js
:
exports.connect = function(config, intern, callback) {
internals = intern;
log = intern.mod.log;
type = intern.mod.type;
console.log("config", config); // Added a console.log
if (config.native) { pg = pg.native; }
var db = config.db || new pg.Client(config);
callback(null, new PgDriver(db, config.schema, intern));
};
We can see that config doesn't contain a native
property, so it never enters the condition.
It seems as though the "Server version check" with semver fails with my server which is Debian 10.3.1
Just by looking into the file where the error occurs (index -> line 172) I would say that the extra parsing of the os version does not take into account the fact that the os being 2-digit-long breaks the check, since, the sub-sub-version is not appended correctly and the 'semver.gte()' check fails.
Am I right, has anyone encountered thi issue, and is there a fix?
Let's assume that database is not available. When connect is called, then null error is returned.
exports.connect = function(config, intern, callback) {
internals = intern;
log = intern.mod.log;
type = intern.mod.type;
if (config.native) { pg = pg.native; }
var db = config.db || new pg.Client(config);
callback(null, new PgDriver(db, config.schema, intern));
};
After that init is called, inside which this code is executed.
init: function(connection, schema, intern) {
this._escapeString = '\'';
this._super(intern);
this.internals = intern;
this.connection = connection;
this.schema = schema || "public";
this.connection.connect();
},
The last string this.connection.connect();
in case of an connection refused error, will just hide it. Connect function can accept callback(err) as parameter, which, in my opinion, should be processed, and given back to the user.
Otherwise even in case of a connection error user gets a driver and will try to execute some queries, which will execute 'this.connection.query.apply(this.connection, params);' and cause an uncaught exception.
--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/30367396-uncaught-exception-if-not-possible-to-connect-to-databse?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github). } else if (config.ssl?.sslmode) {
^
SyntaxError: Unexpected token '.'
Unlike mysql's auto_increment
constraint, postgresql has 3 different types for autoincrement field smallserial
, serial
, bigserial
.
For example,
db.createTable('user', {
id: { type: type.BIG_INTEGER, primaryKey: true, notNull: true, autoIncrement: true }
})
I tried this code but it creates serial
type which is unexpected behavior. From quick look of source code, I think it would be same for smallserial
type as well. It could be easily fixed like below.
createColumnConstraint: function(spec, options, tableName, columnName) {
var constraint = [],
cb;
if (spec.primaryKey && options.emitPrimaryKey) {
if (spec.autoIncrement) {
switch (spec.type) {
case type.SMALL_INTEGER: constraint.push('SMALLSERIAL');
case type.BIG_INTEGER: constraint.push('BIGSERIAL');
default: constraint.push('SERIAL');
}
}
constraint.push('PRIMARY KEY');
}
...
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Additionally, I am unclear in general about how Postgres version support is intended to be handled for this package (see patches below)
First off, these are the test suite failure messages:
createTable has column metadata for the event table
✗ that has raw column
»
actual expected
CURRENT_TIMESTAMPnow()
// macros.js:14
✗ that has special CURRENT_TIMESTAMP column
»
actual expected
CURRENT_TIMESTAMPnow()
// macros.js:14
I have a patch that addresses these test cases. However the Travis configuration appears to only test one version, so the conditionals in this patch would not be covered. I am not familiar enough with Travis to know if a setup with multiple Postgres versions is possible.
I used a quick and dirty docker-compose setup to test multiple versions locally (branch here) and found that a handful of tests also fail on Postgres 8.
The motivation for all this was that I want to be able to use Postgres 10 declarative partitioning in migrations. I have a PoC here (with some very sad test hacks because I can't determine the version before the code runs). But it seems there are some issues (such as this ticket) that require more insight from the core team before version-specific features can be implemented in a straightforward way.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.