db-migrate / pg Goto Github PK

View Code? Open in Web Editor NEW

66.0 66.0 51.0 503 KB

A postgresql driver for db-migrate.

License: Other

JavaScript 100.00%

pg's People

Contributors

Stargazers

Watchers

Forkers

salsita vertiman johnydays t-ashula bluehalo worldspawn socialtables yocontra bill-kitsune pragmawolf jlamendo ianp vukicevic ianliu shekhar1435 eletype jaspax mschnee sonya devotis mmalecki ilgiz-badamshin surajdevesan usealloy aniket-gojek wlaurance andyburke-forks griffinschneider jmeirelesfood sduluganti blakearnold drakmail prichmp ivarconr dallen66 arslbbt oleksm felzek seaber-io greenstand gharibyan lgtm-migrator jessicaschissato roggervalf cereala

pg's Issues

Not able to add enum value through Migration script

As the migration scripts used to run in a single transaction , when trying to run the db-migrate:up for pg, getting the below error.
ALTER TYPE ... ADD cannot run inside a transaction block

Query that i am trying to run in script.

ALTER TYPE document_type ADD VALUE 'new value' AFTER 'existing value';
package version i am using in project .
"pg": "^8.5.1",
"db-migrate": "^0.11.11",
"db-migrate-pg": "^1.2.2",

Is there any workaround to skip the transaction for a particular or any suggestions ? It would be very helpful

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

createTable errors out when you specify length and autoincrement in postgres

Error description:
db-migrate/node-db-migrate#323

Solution:

Unset length when auto_increment is set.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

addForeignKey Error -->Cannot read property 'onDelete' of undefined

I'm submitting a...

Bug report
Feature request
Question

Current behavior

I'm creating my first table through db-migrate (YEAH) and added a foreign key to one of the columns:

exports.up = function(db, callback) {
db.createTable('code_values', {
columns: {
id: {
type: "int",
primaryKey: true,
notNull: true,
autoIncrement: true
},
category_id: {
type: "int",
notNull: true,
foreignKey: {
name: "code_values_code_category_id_fk",
table: "code_categories",
mapping: {
category_id: "id"
}
}
}
}
},
callback);
};

I get the following error when I run the migration:

[ERROR] TypeError: Cannot read property 'onDelete' of undefined at Object.addForeignKey (/test/node_modules/db-migrate-pg/index.js:542:13) at /test/node_modules/db-migrate-base/index.js:124:21 at Object.recurseCallbackArray (/test/node_modules/db-migrate-base/index.js:104:37) at Object.<anonymous> (/test/node_modules/db-migrate-base/index.js:243:21) From previous event: at Object.createTable (/test/node_modules/db-migrate-base/index.js:241:6) at /usr/local/lib/node_modules/db-migrate/lib/driver/shadow.js:34:50 at runCallback (timers.js:763:18) at tryOnImmediate (timers.js:734:5) at processImmediate (timers.js:716:5)

Expected behavior

I expected the table to be created without the explicit rules added to the foreign key. As for the foreign key, I expected it to default to NO ACTION for ON DELETE and ON UPDATE.

What is the motivation / use case for changing the behavior?

I was able to get things working when adding the following rules to the foreign key:

rules: { onDelete: 'RESTRICT', onUpdate: 'RESTRICT' }

Looking at db-migrate-pg's index.js, I believe the problem is that the function addForeignKey mandates a value for rules; and subsequently when rules.onDelete is called, the variable is undefined.

I'm willing to check this out further and work on a fix, but would love feedback in advance, especially since this would be my first time contributing to an open-source project. Please let me know your thoughts and understand, I'd like to isolate my first potential commit to this specific item (updated documentation included of course) and look globally in the future.

Environment


db-migrate version: 0.11.1
db-migrate-pg version: 0.4.0

Additional information:
- Node version: 9.9.0
- Platform:  Mac
- PostgreSQL 10.3
- Postgres 2.1.5



---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/62575565-addforeignkey-error-cannot-read-property-ondelete-of-undefined?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github).

Test suite fails on Postgres 10+

Additionally, I am unclear in general about how Postgres version support is intended to be handled for this package (see patches below)

First off, these are the test suite failure messages:

    createTable has column metadata for the event table
      ✗ that has raw column
        »
        actual expected

        CURRENT_TIMESTAMPnow()
         // macros.js:14

      ✗ that has special CURRENT_TIMESTAMP column
        »
        actual expected

        CURRENT_TIMESTAMPnow()
         // macros.js:14

I have a patch that addresses these test cases. However the Travis configuration appears to only test one version, so the conditionals in this patch would not be covered. I am not familiar enough with Travis to know if a setup with multiple Postgres versions is possible.

I used a quick and dirty docker-compose setup to test multiple versions locally (branch here) and found that a handful of tests also fail on Postgres 8.

The motivation for all this was that I want to be able to use Postgres 10 declarative partitioning in migrations. I have a PoC here (with some very sad test hacks because I can't determine the version before the code runs). But it seems there are some issues (such as this ticket) that require more insight from the core team before version-specific features can be implemented in a straightforward way.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Create migration under custom directories

I put my database migrations under ./data/migrations directory. However, when I ran db-migrate create --config data/config.json -e pg --migration-dir data/migrations users it created migration file under ./migrations directory. Is this behavior expected?

The server does not support SSL connections

database.json

{
  "dev": {
    "driver": "pg",
    "ssl": "false",
    "user":"postgres",
    "password": {"ENV": "DB_PASS_LOCAL"},
    "host": {"ENV": "DB_HOST_LOCAL"},
    "database": {"ENV": "DB_NAME_LOCAL"},
    "port": {"ENV": "DB_PORT_LOCAL"},
    "schema": {"ENV": "DB_SCHEMA_NAME_LOCAL"}
  },
  "sql-file" : true
}

Not sure why I get this, when I've specified false for ssl when I run db-migrate up:

The server does not support SSL connections

Drop Db doesn't (can't) work if connected to the db you want to drop

The Drop DB command doesn't work if you're trying to drop the database which you're connected to. You can get around this by creating another environment that connects to the template1 or postgres database, and then drop the one you want.

I can't think of an intelligent way to bake this behaviour into the driver so we don't need to use a workaround, any ideas? If you have an implementation idea I'm happy to take a stab at implementing it.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

BUG: Support for Postgres 15

The library errors as follows:

users_db_1   | 2023-01-26 17:47:59.294 UTC [76] ERROR:  permission denied for schema public at character 28                                                                              
users_db_1   | 2023-01-26 17:47:59.294 UTC [76] STATEMENT:  CREATE TABLE IF NOT EXISTS "migrations_state" ("key" VARCHAR  PRIMARY KEY NOT NULL UNIQUE, "value" TEXT  NOT NULL, "run_on" T
IMESTAMP  NOT NULL)                                                                                                                                                                      
turnstile_1  | [ERROR] unhandledRejection                                                                                                                                                
turnstile_1  | [ERROR] error: permission denied for schema public                                                                                                                        
turnstile_1  |     at Parser.parseErrorMessage (/opt/app/node_modules/pg-protocol/dist/parser.js:287:98)                                                                                 
turnstile_1  |     at Parser.handlePacket (/opt/app/node_modules/pg-protocol/dist/parser.js:126:29)                                                                                      
turnstile_1  |     at Parser.parse (/opt/app/node_modules/pg-protocol/dist/parser.js:39:38)                                                                                              
turnstile_1  |     at Socket.stream.on (/opt/app/node_modules/pg-protocol/dist/index.js:11:42)                                                                                           
turnstile_1  |     at emitOne (events.js:116:13)                                                                                                                                         
turnstile_1  |     at Socket.emit (events.js:211:7)                                                                                                                                      
turnstile_1  |     at addChunk (_stream_readable.js:263:12)                                                                                                                              
turnstile_1  |     at readableAddChunk (_stream_readable.js:250:11)                                                                                                                      
turnstile_1  |     at Socket.Readable.push (_stream_readable.js:208:10)                                                                                                                  
turnstile_1  |     at TCP.onread (net.js:601:20)                                                                                                                                         
turnstile_1  | npm ERR! code ELIFECYCLE                                                                                                                                                  
turnstile_1  | npm ERR! errno 1                                                                                                                                                          
turnstile_1  | npm ERR! [email protected] migrate: `db-migrate "up"`                                                                                                                       
turnstile_1  | npm ERR! Exit status 1                                                                                                                                                    
turnstile_1  | npm ERR!                                                                                                                                                                  
turnstile_1  | npm ERR! Failed at the [email protected] migrate script.                                                                                                                    
turnstile_1  | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.                                                                        
turnstile_1  |                                                                                                                                                                           
turnstile_1  | npm ERR! A complete log of this run can be found in:                                                                                                                      
turnstile_1  | npm ERR!     /root/.npm/_logs/2023-01-26T17_47_59_320Z-debug.log

This error comes from an update in postgres 15 which requires updates to the public schema to be done by a user with particular explicitly-set permissions. Article / postgres documentation.

Stepping back to Postgres 14 fixes this issue but isn't a great long term solution.

Migration in a Postgresql database does not migrate in the specific schema.

I have followed the instruction in the documentation, created two files that will create a pets table and a owners table. I specified a schema name in the database.json file. However when the migration finishes, I found out that the new tables created are in the public schema!

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

BUG: Schema option is not reflected

The schema schema: 'xxx' is not reflected when used with ENV option inside the database.json file.

{
  "dev": {
    "ENV": "POSTGRES_CONNECTION_STRING",
    "schema": "my_schema"
  }
}

Here is the db-migrate/pg log:

[INFO] require: db-migrate-pg
[INFO] connecting
[INFO] connected
[SQL] show server_version_num
[SQL] SHOW search_path
[SQL] SET search_path TO "$user","public"
[SQL] SELECT table_name FROM information_schema.tables WHERE table_name = 'migrations' AND table_schema = 'public'
[INFO] creating table: migrations
[SQL] CREATE TABLE IF NOT EXISTS "migrations" ("id"   SERIAL PRIMARY KEY NOT NULL, "name" VARCHAR (255) NOT NULL, "run_on" TIMESTAMP  NOT NULL) << schema not reflected

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Missing driver function addMigrationRecord causes migration failure because of missing schema

db-migrate-pg will create the migrations table in the correct schema, however, because db-migrate-pg does not implement addMigrationRecord, the default sql method is invoked, and db-migrate will attempt to insert the migration record into a different, possibly wrong, possible non-existing table.

Expected Behavior:

[SQL] INSERT INTO "public"."migrations" ("name", "run_on") VALUES ($1, $2) [ [ 'init/20190117194708-initialization',
    2019-01-17T20:25:17.622Z ] ]

Actual:

[SQL] INSERT INTO "migrations" ("name", "run_on") VALUES ($1, $2) [ [ 'init/20190117194708-initialization',
    2019-01-17T20:25:17.622Z ] ]
[ERROR] 20190117194708-initialization { error: relation "migrations" does not exist

error: type "string" does not exist

Running a migration against postgres v12 or v13[1] fails with the error:
error: type "string" does not exist
that's coming from this bit of code in index.js:

  // we case run_on to string, for dbs that may have bigger precision
  // than miliseconds
  _getKV: function (table, key) {
    var sql =
      `SELECT ${this.escapeDDL('key')},
        ${this.escapeDDL('value')},
        ${this.escapeDDL('run_on')}::STRING // <------ THIS should be ::TEXT
        FROM ` +
      this._escapeDDL +
      table +
      this._escapeDDL +
      ` WHERE ${this.escapeDDL('key')} = $1`;
    return this.allAsync(sql, [key]).then(([row]) => row);
  },

string isn't one of the postgres data types, but text is.

[1] Probably other versions too, those are just the ones I tested.

ssl configs are not working

Hi and thank you for building this library.

When running db-migrate with the following config

{
  "sql-file": true,
  "dev": {
    "driver": "pg",
    "user": {
      "ENV": "POSTGRES_USER"
    },
    "password": {
      "ENV": "POSTGRES_PASSWORD"
    },
    "host": {
      "ENV": "POSTGRES_HOST"
    },
    "database": "main",
    "port": "5432",
    "ssl": {
      "sslcert": { "ENV": "POSTGRES_CA" },
      "sslkey": { "ENV": "POSTGRES_KEY" },
      "sslmode": "disable"
    }
  }
}

we get

[ERROR] ReferenceError: fs is not defined
    at exports.connect (/usr/src/app/node_modules/db-migrate-pg/index.js:705:46)
    at connect (/usr/src/app/node_modules/db-migrate/lib/driver/index.js:93:12)
...

This is due to fs not being initialised but used here: https://github.com/db-migrate/pg/blob/master/index.js#L704

It also looks like db-migrate-pg is expecting ssl config to be file instead of variables. This is not the case on our use case as we store sslcert and sslkey in secrets as a value instead of a file.

Thanks!

No such driver found

I am getting the following error in version 1.3.0:

     Error: No such driver found, please try to install it via npm install db-migrate-pg or npm install -g db-migrate-pg
      at Object.exports.connect (node_modules/db-migrate/lib/driver/index.js:77:11)
      at Object.exports.connect (node_modules/db-migrate/connect.js:15:10)
      at /home/vinicius/Luizalabs/Projects/fat-copy/faturamento-api/node_modules/db-migrate/lib/commands/up.js:15:13
      at tryCatcher (node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromiseCtx (node_modules/bluebird/js/release/promise.js:641:10)
      at _drainQueueStep (node_modules/bluebird/js/release/async.js:97:12)
      at _drainQueue (node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:461:21)

Until version 1.2.2 it was working perfectly, but after new versions were published the lib is showing an error.

Any solution?

my node version: v12.22.12

broken version 1.2.3

  } else if (config.ssl?.sslmode) {
                        ^

SyntaxError: Unexpected token '.'

Migration hangs on after Done if it includes `select` queries.

db-migrate 0.11.5
db-migrate-pg 0.5.0
knex 0.16.5
express 4.16.4

When run migration like

const knex = require('../config/knex');

'use strict';

var dbm;
var type;
var seed;

/**
  * We receive the dbmigrate dependency from dbmigrate initially.
  * This enables us to not have to rely on NODE_PATH.
  */
exports.setup = function(options, seedLink) {
  dbm = options.dbmigrate;
  type = dbm.dataType;
  seed = seedLink;
};

exports.up = async (db) => {
  const offers = await knex.select().from('offers');
  console.log(offers);
};

exports.down = async (db) => {
  // ...
};

exports._meta = {
  "version": 1
};

Get in console:

$ - npm run db-migrate up

> [email protected] db-migrate /Users/asapozhnikov/Projects/darly/soosty-be
> db-migrate --config config/database.json "up"

[INFO] Processed migration 20190416095203-some migration name
[INFO] Done

and it hangs on.
All code inside migration executes well and does what it should.
It hangs on even if there any other migrations after this one.

I have second migration, which works well if I run it without first one(which doesn't work well), but when I run both migrations - they do all what they should do, but in the end it hangs on after Done.

Connect to postgres db on empty database configuration

Refers to db-migrate/node-db-migrate#338

Current behavior

Currently the default behavior of the used postgres driver itself is to use the current username as database name if no database was configured. This most probably results in a database does not exists error.

Proposed solution

Default to use the database postgres which always exists.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/29956196-connect-to-postgres-db-on-empty-database-configuration?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github).

[ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: self signed certificate

When upgrading to latest version, which upgrades pg driver to 8, it breaks ssl with self signed certs

Related to: brianc/node-postgres#2009

Driver major version updates should result in major version bump to prevent breaking scenarios.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Can't db.insert()

https://github.com/db-migrate/pg/blob/master/index.js#L564-L566

    arguments[index] = arguments[index].map(function (value) {
      return typeof value === 'string' ? value : JSON.stringify(value);
    });

What is this function doing? db-migrate#insert doesn't do anything with de-stringifying json

https://github.com/db-migrate/db-migrate-base/blob/7af3f80bad7fa5c64509a4435cce68891403c0d1/index.js#L339

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

function not created when run from sql file

The following runs fine in postgres, but when part of a migration in a .sql file, I get the error "error: function update_trigger_status() does not exist":

CREATE FUNCTION update_trigger_status()
RETURNS TRIGGER AS $$
BEGIN
  NEW.last_active_flip := CASE WHEN NEW.active <> OLD.active AND NEW.active = 't' THEN now()
                                 ELSE OLD.last_active_flip END;
  NEW.last_inactive_flip := CASE WHEN NEW.active <> OLD.active AND NEW.active = 'f' THEN now()
                                 ELSE OLD.last_inactive_flip END;
  RETURN NEW;
END;
$$ language 'plpgsql';

node-db-migrate version: 0.10.0-beta.23
db-migrate-pg version: 0.2.5

tried other delimiters, didn't work. The plpgsql extension is installed.

Uncaught exception if not possible to connect to databse

Let's assume that database is not available. When connect is called, then null error is returned.

exports.connect = function(config, intern, callback) {

    internals = intern;

    log = intern.mod.log;
    type = intern.mod.type;

    if (config.native) { pg = pg.native; }
    var db = config.db || new pg.Client(config);
    callback(null, new PgDriver(db, config.schema, intern));
};

After that init is called, inside which this code is executed.

init: function(connection, schema, intern) {
        this._escapeString = '\'';
        this._super(intern);
        this.internals = intern;
        this.connection = connection;
        this.schema = schema || "public";
        this.connection.connect();
    },

The last string this.connection.connect(); in case of an connection refused error, will just hide it. Connect function can accept callback(err) as parameter, which, in my opinion, should be processed, and given back to the user.

Otherwise even in case of a connection error user gets a driver and will try to execute some queries, which will execute 'this.connection.query.apply(this.connection, params);' and cause an uncaught exception.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/30367396-uncaught-exception-if-not-possible-to-connect-to-databse?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F14258523&utm_medium=issues&utm_source=github).

supporting multiple host in connection string does not work.

According to PG docs multiple hosts can be supplied in connection string, however db-migrate is not supporting it and parsing it all wrong.

for example:
If I supply DATABSE_URL as postgres://user:@localhost:6432,localhost:5432/db_name
it is parsing as this:

{
    driver: 'postgres',
    user: 'user',
    password: '',
    database: ':6432,localhost/db_name',
    host: 'localhost',
    port: '5432'
}

However, using psql I am able to log in with same connection string.

Lib broken in node v12

I am getting the following error in version 1.3.0:

     Error: No such driver found, please try to install it via npm install db-migrate-pg or npm install -g db-migrate-pg
      at Object.exports.connect (node_modules/db-migrate/lib/driver/index.js:77:11)
      at Object.exports.connect (node_modules/db-migrate/connect.js:15:10)
      at /home/vinicius/Luizalabs/Projects/fat-copy/faturamento-api/node_modules/db-migrate/lib/commands/up.js:15:13
      at tryCatcher (node_modules/bluebird/js/release/util.js:16:23)
      at Promise._settlePromiseFromHandler (node_modules/bluebird/js/release/promise.js:547:31)
      at Promise._settlePromise (node_modules/bluebird/js/release/promise.js:604:18)
      at Promise._settlePromiseCtx (node_modules/bluebird/js/release/promise.js:641:10)
      at _drainQueueStep (node_modules/bluebird/js/release/async.js:97:12)
      at _drainQueue (node_modules/bluebird/js/release/async.js:86:9)
      at Async._drainQueues (node_modules/bluebird/js/release/async.js:102:5)
      at Immediate.Async.drainQueues [as _onImmediate] (node_modules/bluebird/js/release/async.js:15:14)
      at processImmediate (internal/timers.js:461:21)

Until version 1.2.2 it was working perfectly, but after new versions were published the lib is showing an error.

Any solution?

The error is not on my side (as @wzrdtales said in issue 74), since until yesterday in version 1.2.2 everything was working perfectly.
After the publication of the new versions (yesterday, May 4th) the library is no longer working.

My node version is v12.22.12

changeColumn doesn't take into account many column spec options

it seems to only operate on

unique
notNull
defaultValue
type

leaving out

length
primaryKey
autoIncrement
foreignKey

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Node 6 compatibility

upstream issue brianc/node-postgres#1000

postgres update required

Feature Request - env. variable for setting migrations table schema DATABASE_SCHEMA

There should be an option to specify the schema through env. variables, same as with DATABASE_URL, e.g. DATABASE_SCHEMA. Now there is no way to use DATABASE_URL for postgres without also creating a database.json (config) file.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Scope Configuration doesn't work with Postgres

Scope Configuration does not seem to work at all with PostgreSQL.

If "database" is specified in the sub configuration (config.json), it generates an invalid statement:

ERROR: syntax error at or near "`" at character 20. STATEMENT: SET search_path TO `databaseName`

If a "database" is not specified, the process freezes.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Datetime should be mapped to timesamp "with time zone" instead of "without time zone"

Hi, there is currently no way to create a column with the type timestamp with time zone.

When I look into the code I found that datetime is mapped to timestamp. So no matter if I define a timestamp or a datetime I will have a timezone.

In the MySQL documentation for DATETIME you can see that this type of column takes care of local time zone.

Is there something wrong to map datetime to timestamp with time zone ?

I tried to update the mapping locally and all work fine for me. Do you see any issue doing this ?

If not I can submit a PR with these changes.

Thanks.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Native driver option does not work

Hello,
I'm trying to deploy on Heroku but it complains that SSL is not enabled. I tried activate "native: true" in my database.json file :

{
    "prod": {
        "ENV": "DATABASE_URL",
        "native": true
    },
    "dev": {
        "ENV": "DATABASE_URL"
    }
}

I had no luck. By slightly changing the code in index.js :

exports.connect = function(config, intern, callback) {

    internals = intern;

    log = intern.mod.log;
    type = intern.mod.type;
    console.log("config", config); // Added a console.log
    if (config.native) { pg = pg.native; }
    var db = config.db || new pg.Client(config);
    callback(null, new PgDriver(db, config.schema, intern));
};

We can see that config doesn't contain a native property, so it never enters the condition.

Changelog is outdated

The latest version of this package appears to be 1.2.2, but the CHANGELOG.md only has release notes up to 0.5.1.

What has changed since then?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Replace vows with supported test suite

I'd like to propose replacing vows test runner with something that's more modern and actively supported. The motivation is that on trying to run the test suite with [email protected] to validate moving from TravisCI to GH actions, I get the following error:

$ npm run test

> [email protected] test
> node node_modules/.bin/vows

node[10176]: ../src/node_contextify.cc:710:static void node::contextify::ContextifyScript::New(const FunctionCallbackInfo<v8::Value> &): Assertion `args[1]->IsString()' failed.
 1: 0x104d131dc node::Abort() [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 2: 0x104d1303c node::AppendExceptionLine(node::Environment*, v8::Local<v8::Value>, v8::Local<v8::Message>, node::ErrorHandlingMode) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 3: 0x104d091e4 node::contextify::ContextifyScript::New(v8::FunctionCallbackInfo<v8::Value> const&) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 4: 0x104e8ed2c v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 5: 0x104e8e50c v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<true>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 6: 0x104e8e080 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 7: 0x1054576ec Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 8: 0x1053e8470 Builtins_JSBuiltinsConstructStub [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
 9: 0x1054e6550 Builtins_ConstructHandler [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
...
41: 0x1053e902c Builtins_JSEntryTrampoline [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
42: 0x1053e8cc4 Builtins_JSEntry [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
43: 0x104f1731c v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
44: 0x104f16a74 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, int, v8::internal::Handle<v8::internal::Object>*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
45: 0x104e4db40 v8::Function::Call(v8::Local<v8::Context>, v8::Local<v8::Value>, int, v8::Local<v8::Value>*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
46: 0x104ce5fec node::ExecuteBootstrapper(node::Environment*, char const*, std::__1::vector<v8::Local<v8::String>, std::__1::allocator<v8::Local<v8::String> > >*, std::__1::vector<v8::Local<v8::Value>, std::__1::allocator<v8::Local<v8::Value> > >*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
47: 0x104ce7088 node::StartExecution(node::Environment*, char const*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
48: 0x104ce6d94 node::StartExecution(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
49: 0x104c6c318 node::LoadEnvironment(node::Environment*, std::__1::function<v8::MaybeLocal<v8::Value> (node::StartExecutionCallbackInfo const&)>) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
50: 0x104d48d3c node::NodeMainInstance::Run(int*, node::Environment*) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
51: 0x104d48a58 node::NodeMainInstance::Run() [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
52: 0x104ce9040 node::Start(int, char**) [/opt/homebrew/Cellar/node@16/16.19.0/bin/node]
53: 0x1a2a07e50 start [/usr/lib/dyld]
[1]    10155 abort      npm run test

On upgrading to [email protected], it fixes the above error, but starts failing for other reasons. While I could probably work through and get vows up and running again, I think the time would be better spent just moving this library to a test runner that's actually supported and more widely used in the community. I know that the db-migrate uses @hapi/lab so that'd be an option, but that's still somewhat obscure within the broader community, so I'd suggest mocha or jest which are orders of magnitude more widely used and known. However, I'm happy to try for whatever suite @wzrdtales might choose.

Invalid SQL Syntax when generating Table with Foreign Key

When using the following up function to generate a table with a combined primary key, one of which references another table:

exports.up = function (db) {
    return db.createTable('repositories', {
        user_id: {
            type: 'string',
            length: 64,
            primaryKey: true,
            notNull: true,
            foreignKey: {
                name: 'repositories_users_id_fk',
                table: 'users',
                rules: {
                    onDelete: 'CASCADE',
                    onUpdate: 'RESTRICT'
                },
                mapping: 'id'
            }
        },
        id: {
            type: 'string',
            length: 128,
            primaryKey: true,
            notNull: true
        },
        country_code_evaluated: {
            type: 'boolean',
            notNull: true,
            defaultValue: false
        },
        country_code: {
            type: 'string',
            length: 3,
            notNull: false
        },
        repository_evaluated: {
            type: 'boolean',
            notNull: true,
            defaultValue: false
        }
    });
};

The following SQL Code is generated when executing db-migrate up (I checked via --dry-run and -v):

CREATE TABLE  "repositories" ("user_id" VARCHAR (64) NOT NULL, "id" VARCHAR (128) NOT NULL, "country_code_evaluated" BOOLEAN  NOT NULL DEFAULT false, "country_code" VARCHAR (3) , "repository_evaluated" BOOLEAN  NOT NULL DEFAULT false, PRIMARY KEY ("user_id", "id")) ;
ALTER TABLE "repositories" ADD CONSTRAINT "repositories_users_id_fk" FOREIGN KEY ([ '"user_id"' ]) REFERENCES "users" ([ '"id"' ]) ON DELETE CASCADE ON UPDATE RESTRICT;

Which throws the assertion [ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: syntax error at or near "[" when executed.

My environment
Node: v12.1.0
NPM: 6.9.0
db-migrate 0.11.5
db-migrate-pg 0.5.0

Duplicated 'migrations' table when using schemas

When using a schema: 'mySchema' configuration in the database.json, I can run db-migrate and it will create a migrations table on the publicschema. Running the migration again leads to a secondmigrationstable inside themySchema` schema.

I would guess that this might require a better configuration possibility for the migrations schema which is currently only possible via the --table my_migrations command line argument. I would like to configure a like like myOtherSchema.my_migrations, because I don't want to mix real/customer data with technical data (migrations).

Sidenote: is it possible to configure the migration table via the database.json? I tried, but failed.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Unable to use Peer auth

I want to have db-migrate authenticate by socket / peer auth, but this doesn't seem possible without hacking an env var.

const dbMigrateConfig = {
        default: 'local',
        local: {
              driver: 'pg',
              database: 'mydatabase',
              host: '/var/run/postgres',
              native: true
        }
};

'#ENOENT /var/run/postgres'

process.env['DBTEST'] = 'postgres:///${mydatabase}?host=/var/run/postgresql&native=true';
const dbMigrateConfig = {
        default: 'local',
        local: {
                ENV: "DBTEST"
        }
};

success.

Any chance the normal connection settings could allow connection via a socket?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Support smallserial and bigserial data type.

Unlike mysql's auto_increment constraint, postgresql has 3 different types for autoincrement field smallserial, serial, bigserial.

For example,

db.createTable('user', {
    id: { type: type.BIG_INTEGER, primaryKey: true, notNull: true, autoIncrement: true }
})

I tried this code but it creates serial type which is unexpected behavior. From quick look of source code, I think it would be same for smallserial type as well. It could be easily fixed like below.

    createColumnConstraint: function(spec, options, tableName, columnName) {
        var constraint = [],
            cb;

        if (spec.primaryKey && options.emitPrimaryKey) {
            if (spec.autoIncrement) {
                switch (spec.type) {
                    case type.SMALL_INTEGER: constraint.push('SMALLSERIAL');
                    case type.BIG_INTEGER: constraint.push('BIGSERIAL');
                    default: constraint.push('SERIAL');
                }
            }
            constraint.push('PRIMARY KEY');
        }
     ...

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

table inheritance

Review this old PR, possibly migrate it and restore the credits to @zinigor

db-migrate/node-db-migrate#141

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Fails to assert pg version correctly

Related to #32; when PG produces a version like one below, the regex matcher matches on the wrong string, causing the following error to occur:

[ERROR] AssertionError [ERR_ASSERTION]: ifError got unwanted exception: Invalid Version: 0.18.04
    at module.exports (/home/bttf/wyl-today/node_modules/db-migrate/lib/commands/helper/assert.js:9:14)
    at /home/bttf/wyl-today/node_modules/db-migrate/lib/commands/up.js:29:16
    at new SemVer (/home/bttf/wyl-today/node_modules/semver/semver.js:323:11)
    at compare (/home/bttf/wyl-today/node_modules/semver/semver.js:614:10)
    at Function.gte (/home/bttf/wyl-today/node_modules/semver/semver.js:663:10)
    at Object.<anonymous> (/home/bttf/wyl-today/node_modules/db-migrate-pg/index.js:171:43)
    at tryCatcher (/home/bttf/wyl-today/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/home/bttf/wyl-today/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/bttf/wyl-today/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (internal/timers.js:456:21)

Version output

pgdb=# select  version() as version;
                                                                version
----------------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 10.12 (Ubuntu 10.12-0ubuntu0.18.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, 64-bit

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Failed to drop database! error: cannot drop the currently open database

Does anyone know an easy way to get around this problem. I can't drop the DB since it's in use:

node node_modules/db-migrate/bin/db-migrate db:drop postgres
node node_modules/db-migrate/bin/db-migrate db:create postgres

Error: Failed to create database! error: database "postgres" already exists and is running in a postgres docker container.

Grab an advisory lock for migration as per Heroku recommendations

Hi there,

I saw some advice on https://devcenter.heroku.com/articles/release-phase#design-considerations that suggests grabbing an "advisory lock" on the database before running a migration.

Basically, try SELECT pg_try_advisory_lock(migration); at the start of the transaction.

It if returns t then nothing currently has the lock, called 'migration', so proceed.
If not, fail the migration.
The lock will be release when the transaction commits or rolls back.

I think that would be a one-line change in this module. Good idea?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Invalid Version: 10.3 (Debian 10.3-1.pgdg90+1).0

Using the latest postgres docker image gives the below error

This looks like a problem with the docker image, but would be very easy to accomodate

Essentially postgres returns { server_version: '10.3 (Debian 10.3-1.pgdg90+1)' } and a little version = version.split(' ')[0] after index.js:72 fixes it. Or one could downgrade to postgres:9.6, which is what I've done.

This may be a "Won't Fix" but hopefully it'll help anyone else facing the same error

[ERROR] TypeError: Invalid Version: 10.3 (Debian 10.3-1.pgdg90+1).0
    at new SemVer (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:305:11)
    at compare (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:578:10)
    at Function.gte (/Users/aaronlang/not-safe-not-fair/node_modules/semver/semver.js:627:10)
    at Object.<anonymous> (/Users/aaronlang/not-safe-not-fair/node_modules/db-migrate-pg/index.js:176:42)
    at tryCatcher (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/Users/aaronlang/not-safe-not-fair/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:756:18)
    at tryOnImmediate (timers.js:717:5)
    at processImmediate [as _immediateCallback] (timers.js:697:5)

"The driver you are using does not support the new state management."

Hello,

I'm running an application that uses [email protected], which has [email protected] as a dependency.

Our compilation process for local devleopment runs two migrations: one set of base migrations to build our PostgreSQL schema, and another set of migrations under the scope dev to add some seed data to the database.

Before today, our migrations ran fine. However, we're now seeing the following message in our logs:

[WARN] The driver you are using does not support the new state management. Please raise an issue in the repository of your driver maintainer

Our data now fails to seed because the dev-scoped migration is running our base migrations a second time. Because the migrations already ran in the past, the second run fails and never gets to the seed migrations further down the line.

We did not update any packages. Is there something I am missing? I can provide additional data if requested.

Thanks!

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Migration fails when running on Heroku

There can be a problem related to the way the server reports it's version info (depending opt how it was compiled) which is causing the server check to fail, like so

yarn run v1.4.0
$ db-migrate up -m ./node_modules/byte-shared/migrations -e ${NODE_ENV:-development} && db-migrate up -e ${NODE_ENV:-development}
[ERROR] TypeError: Invalid Version: 10.2 (Ubuntu 10.2-1.pgdg14.04+1).0
    at new SemVer (/app/node_modules/semver/semver.js:305:11)
    at compare (/app/node_modules/semver/semver.js:578:10)
    at Function.gte (/app/node_modules/semver/semver.js:627:10)
    at Object.<anonymous> (/app/node_modules/db-migrate-pg/index.js:176:42)

This is related to #27 and also #31 but both the accepted and proposed solutions are incorrect/incomplete. For example, the show server_version command can return 10.2 (Ubuntu 10.2-1.pgdg14.04+1). A better approach is to use show server_version_num which returns a “machine readable” version number, as per https://www.postgresql.org/docs/10/static/runtime-config-preset.html#GUC-SERVER-VERSION-NUM and then manipulate this number to get the semver version. Something like this:

const major = Math.floor(server_version_num / 10000)
const minor = Math.floor((server_version_num - major * 10000) / 100)
const patch = Math.floor(server_version_num - major * 10000 - minor * 100)
const version = `${major}.${minor}.${patch}` // 10.0.2

Let me know if you agree/are happy with this approach and I'll go ahead and prepare a PR.

Add showDatabase method

Refers to db-migrate/node-db-migrate#339

showDatabase should reply with an array of database name strings.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

DELETE

delete this

ECONNRESET in Azure Postgres

I am using the new managed Postgres service in Azure. My application doesn't have a problem connecting to actual PostgreSQL databases. However, a migrate up against the managed database consistently results in a fatal exception that occur after the migration completes.

Since the migrations run successfully, the problem seems to be related to how the connection is getting terminated.

Error: read ECONNRESET
    at exports._errnoException (util.js:1050:11)
    at TLSWrap.onread (net.js:581:26)

The application uses the latest version of db-migrate for node and the most recent pg driver.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Migrations not done for test environment with --env or -e option Win10

I have installed db-migrate and db-migrate-pg driver all locally and run it with script hook in package.json

"scripts": {
    "migrate": "node node_modules/db-migrate/bin/db-migrate"

In my database.json file I have described environments for dev and test as shown below

{
  "dev": {
    "driver": "pg",
    "user": {"ENV": "PGUSER"},
    "password": {"ENV": "PGPASSWORD"},
    "host": {"ENV": "PGHOST"},
    "database": {"ENV": "PGDATABASE"}
  },
  "test": {
    "driver": "pg",
    "user": "myusername",
    "password": "mypwd",
    "host": "localhost",
    "database": "test"
  }

Also am using dotenv and that is where the envs for dev are picked up.

I can run migrations on the dev database successfully yarn run migrate up

But when I change the environment with the --env option or -e option with test, it runs the migrations on the dev environment which of course will return no migrations to run

>yarn run migrate up --env test
yarn run v0.27.5
$ node node_modules/db-migrate/bin/db-migrate "up"
[INFO] No migrations to run
[INFO] Done
Done in 0.90s.

I dont know what the issue is?

Connector fails on server version 10.3

It seems as though the "Server version check" with semver fails with my server which is Debian 10.3.1
Just by looking into the file where the error occurs (index -> line 172) I would say that the extra parsing of the os version does not take into account the fact that the os being 2-digit-long breaks the check, since, the sub-sub-version is not appended correctly and the 'semver.gte()' check fails.

Am I right, has anyone encountered thi issue, and is there a fix?

db-migrate / pg Goto Github PK

pg's People

Contributors

Stargazers

Watchers

Forkers

pg's Issues

I'm submitting a...

Current behavior

Expected behavior

What is the motivation / use case for changing the behavior?

Environment

Current behavior

Proposed solution

Recommend Projects

Recommend Topics

Recommend Org