Code Monkey home page Code Monkey logo

pg-diff's Introduction

pg-diff's People

Contributors

michaelsogos avatar michaelsogos4g avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

pg-diff's Issues

Feature Request: Compare comments

Hello,

The comments in our database are mission critical. Tables, schema, columns, constraints, and views all have comments that I need to compare and create a diff for. Can this functionality be added to pg-diff?

The goal is to add 'COMMENT ON' lines in the generated patch file that will add, set to null, and/or update comments on targetClient so that it matches up with the comments on sourceClient.

Thank you for your time

Adding constrant is not idempotent

In the schema compare, if idempotent = true, when pg-diff creates the query to add a constraint, it creates this command:

 ALTER TABLE IF EXISTS "public". "Account_tax_regulation" ADD CONSTRAINT "uk_4j7hbk7rp1tmvj37ae7gdfl7" UNIQUE (name);

however, this only verifies that the table exists and not that the constraint exists. so when the script is executed, if the constraint already exists, it fails.

to be able to check existence, there is no "ADD COSTRAINT IF EXISTS" in postgress, therefore it is necessary to resort to the search query.

in order to do this, I will give you an example:

DO $$ begin
      IF NOT EXISTS (SELECT constraint_name
                                           FROM information_schema.constraint_column_usage
                                           WHERE table_schema = 'public' 
                                               AND table_name = 'account_tax_regulation' 
                                               AND constraint_name = 'uk_4j7hbk7rp1tmvj37ae7gdfl7') THEN
                       ALTER TABLE IF EXISTS "public". "Account_tax_regulation" ADD CONSTRAINT 
                                            "uk_4j7hbk7rp1tmvj37ae7gdfl7" UNIQUE (name);
      END IF;
END $$;

the commands DO $$ and END $$; are necessary in order to run IF STATEMENT in a script

IMPORTANT NOTE:
in case you have to look for a foreign key you have to look for it in the external table as a contraint not in the one in which the foreign Key is missing
eg:

DO $$ begin
               IF NOT EXISTS (SELECT constraint_name
                                         FROM information_schema.constraint_column_usage
                                       WHERE table_schema = 'public' 
                                           AND table_name = 'base_partner' 
                                           AND constraint_name = 'fk_qlqlr0w9e99cy3fu0y9339rnq') THEN
                         ALTER TABLE IF EXISTS "public". "Sale_sale_order" ADD CONSTRAINT 
                                       "fk_qlqlr0w9e99cy3fu0y9339rnq" FOREIGN KEY (client_partner) 
                                        REFERENCES base_partner (id);
       END IF;
END $$;

base_partner in this case is the external reference table of the foreign key and the SELECT must be made on that table

best regards
Mattia

Functions are always shown as different even when not updated

PostgreSQL Version: 12.2 (I upgraded both sides to make sure that the issue wasn't mismatched versions)

Example:

The diff gives me this function:

CREATE OR REPLACE FUNCTION public.get_browser_from_user_agent(user_agent text)
 RETURNS text
 LANGUAGE sql
 PARALLEL SAFE
AS $function$
	 SELECT CASE
        WHEN user_agent LIKE '%Firefox/%' THEN 'Firefox'
        WHEN user_agent LIKE '%Chrome/%' OR user_agent LIKE '%CriOS%' THEN 'Chrome'
        WHEN user_agent LIKE '%MSIE %' THEN 'IE'
        WHEN user_agent LIKE '%MSIE+%' THEN 'IE'
        WHEN user_agent LIKE '%Trident%' THEN 'IE'
        WHEN user_agent LIKE '%iPhone%' THEN 'iPhone Safari'
        WHEN user_agent LIKE '%iPad%' THEN 'iPad Safari'
        WHEN user_agent LIKE '%Opera%' THEN 'Opera'
        WHEN user_agent LIKE '%BlackBerry%' AND user_agent LIKE '%Version/%' THEN 'BlackBerry WebKit'
        WHEN user_agent LIKE '%BlackBerry%' THEN 'BlackBerry'
        WHEN user_agent LIKE '%Android%' THEN 'Android'
        WHEN user_agent LIKE '%Safari%' THEN 'Safari'
        WHEN user_agent LIKE '%bot%' THEN 'Bot'
        WHEN user_agent LIKE '%http://%' THEN 'Bot'
        WHEN user_agent LIKE '%www.%' THEN 'Bot'
        WHEN user_agent LIKE '%Wget%' THEN 'Bot'
        WHEN user_agent LIKE '%curl%' THEN 'Bot'
        WHEN user_agent LIKE '%urllib%' THEN 'Bot'
        ELSE 'Unknown'
      END
$function$
;
ALTER FUNCTION "public"."get_browser_from_user_agent"(text) OWNER TO ----------;

After I run it, it still shows up in the diff exactly as if it was never ran.

Postgresql 9.4 and below throws error due to use of regrole

The regrole type appears to have been introduced in PG 9.5+

In retrieveSchema.js this query return `SELECT nspname, nspowner::regrole::name as owner FROM pg_namespace WHERE nspname IN ('${schemas.join("','")}') fails on PG <= 9.4

I note there is a TODO to not use regrole casting. Maybe this would fix the issue?

Feature request: generate config file

It would be really great if there was a command line option to generate a dummy config file. Or better still take 2 connection strings and populate the config file too

SSL options

So I'm having a problem with this. While migrating data from Azure postgres to AWS RDS, I've discovered both services now require SSL enabled. I dont seem to have a way to tell the script to do that and it returns complaining about the lack of SSL in our connection.

I've worked around it by temporarily disabling SSL on both sides (Easy on Azure, and in typical fashion a little trickier in AWS [create a custom parameter group, set rds.ssl_enable [or whatever its called again] to 0 , then apply it, then reboot]) but its a pretty shady way of doing things. Thankfully I've been able to re-enable afterwards to correct the temporary lowering of security, but its the kind of move that'd give most security consultants an aneurism.

Any chance of letting us set those settings in the config? I figure this'll be a fairly common problem.

Disabling comments

Hi!
Is it possible to disable lines with comments of functions, indexes, constraints, etc.
Is it possible to specify this in the config file?

Data Compare - Arrays

Bit of background here... I'm using Sequelize to create the models and in there I have one column which is declared as a string array.

@column(DataType.ARRAY({type: DataType.STRING}))
attributes: string[]

This generates a table where the column is defined as...

attributes character varying(255)[]

When I store data in that column, it takes the format...

'{ATTR_1, ATTR_2}'

Inside the __generateSqlFormattedValue function, it tries to handle arrays by doing a value.join as if the data was genuinely an array as opposed to a comma separated value enclosed in curly braces as above. And thus, join is not a function leading to...

TypeError: value.join is not a function
at Object.__generateSqlFormattedValue (C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\sqlScriptGenerator.js:631:35) at Object.generateDeleteTableRecordScript (C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\sqlScriptGenerator.js:593:54)
at C:\Users\rossc\AppData\Local\Yarn\Data\global\node_modules\pg-diff-api\src\api\CompareApi.js:1361:30

Not sure whether somehow my data is stored incorrectly by Postgres or whether its a bug in the code.

Would appreciate your thoughts.

Options to remove IF EXISTS (and similar) statements from generated scripts

It would be nice to have an option to remove IF EXISTS (and similar) statements from the generated migrations scripts. These statements are dangerous in the context of schema migrations, as they can hide serious issues.

For instance, ALTER TABLE IF EXISTS ... would silently fail if the table does not exist, and one could think the migration did correctly apply, while it failed because of a major issue (a missing table !!). Much better to have the script fail with a clear error message saying the table is missing.

Thanks for the otherwise awesome tool !

Question: Rollback to a certain version on a same system

Hello Michaelsogos.

I started using pg-diff, and it works like a charm.

Could you just tell me if it's possible to revert to a certain version on a source system.
Target system aside for now, let's consider the following case.
I am developing my data model, adding some functions and saving it to migration table - V1. Then I add some other db stuff and save it again - V2. Next I start working on some new feature and create V3. What if I realize that this feature is not needed or I just didn't implement it correctly, is there some convenient way to revert back to V2 or V1?
To summarize, I would like to track my development progress on the source system and revert to a certain version if needed. For example: Compare V2 and V3 and rollback all changes made during V2 >> V3 migration

Best regards,
Ivan

create type not implemented?

Hi!
I think the generated types are ignored for source and target comparison. I tested with two databases where there are several compound types in the source code, but they didn't make it into the resulting script. CREATE TYPE ...

Data Compare does not work when tables are not in public schema.

  • The data compare functionality breaks when we compare tables which are not in public schema.
  • This is due to the function "__collectTableRecords" in retrieveRecords.js,
  • The query : "SELECT MD5(ROW(${config.keyFields.join(',')})::text) AS "rowHash", * FROM ${table}" checks only in public schema by default and "Relation not found" error is generated. Hence schema has to be prefixed to the table in this query.
  • possible query for this section is : SELECT MD5(ROW(${config.keyFields.join(',')})::text) AS "rowHash", * FROM ${config.schema||'public'}.${table}

Triggers not implemented?

I think triggers are ignored for the source/target comparison. I tested with two databases where the source has a few new tables with triggers on it, but there are no CREATE TRIGGER ... statements in the resulting script.

Redshift compatibility

aws redshift uses a flavour of postgres and a couple of funcoitns aren't exactly the same.
for instance:
show server_version; actually is SELECT version();
because of this error the comparison fails instantly as I'm guessing that's pretty early in the process.
please would you consider allowing us to specify redshift and alter some sys queries?

` Author: [object Object]
Version: 2.2.0
PostgreSQL: 9.6+
License: MIT
Description: PostgreSQL schema and data comparing tool

CONFIGURED OPTIONS
Script Author: your-name-or-nickname-or-anything-else
Output Directory: /home/zeubs/Downloads/pg-diff/sqlscripts
Schema Namespaces: will be retrieve dynamically from database
Data Compare: DISABLED

[|------------------] 10% - Connecting to source database ...
error: must be superuser to examine "server_version"
at Parser.parseErrorMessage (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:287:98)
at Parser.handlePacket (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:126:29)
at Parser.parse (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/parser.js:39:38)
at Socket.stream.on (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-protocol/dist/index.js:11:42)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
`

Please provide sample diff output in documentation

From the documentation, it's not quite clear if this would provide the functionality I want. Could you add a "Hello World" type of example output? It could be based on just one or maybe two tables with only a few rows. Then show the output from running pg-diff on this small example.

error: there are multiple default operator classes for data type text

[||||||-------------] 33% - Collecting PRIVILEGES for table "public"."schedulegeneratorschedulegeneratorscollection_heal[||||||-------------] 33% - Collecting PRIVILEGES for table "public"."agroinsuranceaccident"
error: there are multiple default operator classes for data type text
error: there are multiple default operator classes for data type text
at Connection.parseE (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:614:13)
at Connection.parseMessage (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:413:19)
at Socket. (C:\Users\bkvariani\AppData\Roaming\npm\node_modules\pg-diff-cli\node_modules\pg\lib\connection.js:129:22)
at Socket.emit (events.js:198:13)
at addChunk (_stream_readable.js:288:12)
at readableAddChunk (_stream_readable.js:269:11)
at Socket.Readable.push (_stream_readable.js:224:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)

image

Errors are printed to stdout instead of stderr

Running this:

pg-diff -c generate migration > /dev/null

results in no output. Even though there is no file there.

$ pg-diff -c generate migration
                              _   _    __    __                  _   _
  _ __     __ _            __| | (_)  / _|  / _|           ___  | | (_)
 | '_ \   / _` |  _____   / _` | | | | |_  | |_   _____   / __| | | | |
 | |_) | | (_| | |_____| | (_| | | | |  _| |  _| |_____| | (__  | | | |
 | .__/   \__, |          \__,_| |_| |_|   |_|            \___| |_| |_|
 |_|      |___/

     Author: Michael Sogos <[email protected]> (https://github.com/michaelsogos)
    Version: 2.1.0
 PostgreSQL: 9.6+
    License: MIT
Description: PostgreSQL schema and data comparing tool


Error: Cannot find module '/project-dir/pg-diff-config.json'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Function.LoadConfig (/usr/local/lib/node_modules/pg-diff-cli/src/ConfigHandler.js:11:20)
    at Run (/usr/local/lib/node_modules/pg-diff-cli/main.js:68:32)
    at Object.<anonymous> (/usr/local/lib/node_modules/pg-diff-cli/main.js:15:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
Please create the configuration file "pg-diff-config.json" in the same folder where you run pg-diff!

Major bug: Line Ending causes program to fail

This issue happens on macOS after yarn add when running yarn pg-diff -c development migration :

env: node\r: No such file or directory
error Command failed with exit code 127.

It can be fixed when I go to node_modules/.bin/pg-diff and I manually change the line ending (I use vscode but there are scripts for that) from CRLF to LF.

It is mandatory to do this, otherwise I cannot use your program.

migration option historyTableSchema

hello, great tool!
With version 2.4.1 and PostgreSQL: 9.6+ (15.3) if you set migrationOptions.historyTableSchema to a schema different from 'public':

  • table historyTableSchema.historyTableName is not created into historyTableSchema schema (even if schema exists) but into public schema
  • when you save a patch (-s parameter)
    • if table historyTableSchema.historyTableName exists patch is corretly save into it
    • table public.historyTableName is created (but remains empty because patch is correctly saved into historyTableSchema.historyTableName)

Pg-diff doesn't detect dropped tables

In my development database i dropped some tables that also exist on the "production" database. Then when running "pg-diff -c development patch_name" it gave empty file. I guess it doesn't detect deleted tables.

Is there a possibility to have this kind of feature?

error: "st_extent" is an aggregate function

Version: 1.2.8
PG Version: 11.2
Plugins: PostGIS is installed

When comparing two databases and PostGIS is installed, this seems to blow up with the following error:

error: "st_extent" is an aggregate function
    at Connection.parseE (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:602:11)
    at Connection.parseMessage (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:399:19)
    at Socket.<anonymous> (/Users/ashish/.nvm/versions/node/v10.15.1/lib/node_modules/pg-diff-cli/node_modules/pg/lib/connection.js:121:22)
    at Socket.emit (events.js:189:13)
    at addChunk (_stream_readable.js:284:12)
    at readableAddChunk (_stream_readable.js:265:11)
    at Socket.Readable.push (_stream_readable.js:220:10)
    at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)

Missing semicolon at the end of sequence

VERSION
pg-diff-cli: 1.2.3

Repro steps
Created new sequence using postgres:
CREATE SEQUENCE voucher_code_seq START 1

Run:
pg-diff -c development sequence_name_script
pg-diff -m development

Observed behavior
Throws error when executing migration for the next line after the statement for creating sequence
"error syntax near alter"

Desired behavior
sequence

column a.attidentity not found error 25% into connecting to database

After I finished creating my own pg-diff-config.json file, I called

'pg-diff -c development initial-script'

which began connecting to databases, however, at 25% I get this error:

error: column a.attidentity does not exist

attidentity seems to be a postgres column, not any of the columns that are in the tables in my two databases. Any ideas how to fix the error? I've already double checked to make sure that the username I'm inputting has admin rights to access the database.

Data compare doesn't detect updates of NULL values

Hi !

I'm comparing two schema, where the newer schema has new columns with data.

The data compare feature correctly creates the INSERT statement (populating the new columns with the values), but does not include the new columns in UPDATE statements.

Example :

-- SCHEMA A
CREATE TABLE public.test (
	id serial NOT NULL,
	"name" varchar NULL,
	CONSTRAINT test_pk PRIMARY KEY (id)
);
INSERT INTO test VALUES (1, 'john');
-- SCHEMA B
CREATE TABLE public.test (
	id serial NOT NULL,
	"name" varchar NULL,
	"surname" varchar NULL,
	CONSTRAINT test_pk PRIMARY KEY (id)
);
INSERT INTO test VALUES (1, 'johnny', 'wayne');
INSERT INTO test VALUES (2, 'alfred', 'hitchcock');

Using config

{
  "test": {
    "targetClient": {
      ...
      "database": "testa",
      ...
    },
    "sourceClient": {
      ...
      "database": "testb",
      ...
    },
    "compareOptions": {
      ...
      "dataCompare": {
        "enable": true,
        "tables": [
          {"tableSchema":"public", "tableName":"test", "tableKeyFields":["id"]}
        ]
      }
    }
  }
}

I get the script

-- ...
ALTER TABLE IF EXISTS "public"."test" ADD COLUMN IF NOT EXISTS "surname" varchar NULL  ;
-- ...
INSERT INTO "public"."test" ("id", "name", "surname")  VALUES (2, 'alfred', 'hitchcock');
UPDATE "public"."test" SET "name" = 'johnny' WHERE "id" = 1;
-- ...

See that "john" is correctly updated to "johnny", but the "wayne" is not populated at all, while "hitchcock" is correctly populated for the newly inserted value.

int[] field data compare - TypeError: value.replace is not a function

Hi @michaelsogos,
thank you for this awesome tool! Sadly, I'm experiencing a TypeError with data-compare as soon as a table contains a filled integer[] field. If data-compare is disabled or the field contains null, everything works fine. I'm looking forward for any help. Thank you!

This is how it looks like:

$ pg-diff -c development initial-script
                              _   _    __    __                  _   _
  _ __     __ _            __| | (_)  / _|  / _|           ___  | | (_)
 | '_ \   / _` |  _____   / _` | | | | |_  | |_   _____   / __| | | | |
 | |_) | | (_| | |_____| | (_| | | | |  _| |  _| |_____| | (__  | | | |
 | .__/   \__, |          \__,_| |_| |_|   |_|            \___| |_| |_|
 |_|      |___/

     Author: [object Object]
    Version: 2.0.4
 PostgreSQL: 9.6+
    License: MIT
Description: PostgreSQL schema and data comparing tool


CONFIGURED OPTIONS
         Script Author: pg-diff-cli
      Output Directory: /data/pg-diff/sqlscripts
     Schema Namespaces: public
          Data Compare: ENABLED

[||||||||||||||-----] 75% - SEQUENCE objects have been compared
TypeError: value.replace is not a function
    at Object.__generateSqlFormattedValue (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/sqlScriptGenerator.js:598:22)
    at Object.generateInsertTableRecordScript (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/sqlScriptGenerator.js:545:26)
    at /usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1180:10
    at Array.forEach (<anonymous>)
    at Function.compareTableRecords (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1140:37)
    at Function.compareTablesRecords (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:1018:31)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Function.compare (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:63:9)
    at async PgDiff.compare (/usr/local/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/index.js:36:10)
    at async Run (/usr/local/lib/node_modules/pg-diff-cli/main.js:79:26)

Steps to reproduce:

  • Set up two postgres databases version 11.9
  • Create table in sourceClient database using attached diff_test.sql, second db remains empty
  • Enable data comparison and run pg-diff -c development initial-script

pg-diff-config.json:

...
            "dataCompare": {
                "enable": true,
                "tables": [
                    {
                        "tableName": "diff_test",
                        "tableSchema": "public",
                        "tableKeyFields": ["id"]
                    }
                ]
            }
...

diff_test.sql

create table public.diff_test (
    id   serial not null
        constraint diff_test_pk
            primary key,
    nums integer[]
);
INSERT INTO public.diff_test (id, nums) VALUES (1, '{1,2}');

Pg-diff data-compare changes for NULL values doesn't work

VERSION

pg-diff-cli: 1.2.3

In my development database i had created a table with some columns set as nullable, added data and everything worked as it should.

Later i decided to change some of the fields which had a default value of NULL to their appropriate data type, but as soon as i run "pg-diff -c development patch_name" it gave empty patch file. No changes were detected.

For fields that were not NULL from the same table but got altered, the "pg-diff -c development patch_name" worked properly.

Data compare option in pg-diff-config.json is enabled and set properly.

Column Does not Exist

On the face of it, this seems similar to #24. The last status output was "AGGREGATE objects have been compared. at 75%

At this point though, as it's not telling me the table so I've not yet understood which property or function it has an issue with (there's quite a few referencing nodeid).
#24 says this is fixed in 2.0.2, I'm using...

Version: 2.3.1
PostgreSQL: 9.6+

Full error message is...

error: column "nodeid" does not exist
at Parser.parseErrorMessage (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:287:98)
at Parser.handlePacket (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:126:29)
at Parser.parse (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\parser.js:39:38)
at Socket. (node_modules\pg-diff-cli\node_modules\pg-protocol\dist\index.js:11:42)
at Socket.emit (events.js:315:20)
at addChunk (_stream_readable.js:295:12)
at readableAddChunk (_stream_readable.js:271:9)
at Socket.Readable.push (_stream_readable.js:212:10)
at TCP.onStreamRead (internal/stream_base_commons.js:186:23)
column "nodeid" does not exist

Anyone know what to look for or how to find out which table its referring to?

Thank you.

compareOptions.schemaCompare.roles is required but not caught in config load

Replication Steps:

  1. Create a pg-diff-config with a compareOptions.schemaCompare =
"schemaCompare": {
                "namespaces": ["public"],				
                "idempotentScript": true, 
                "dropMissingTable": false, 
                "dropMissingView": false, 
                "dropMissingFunction": false 
},

Expected: Fails with an error saying roles is required (or a null check making them not required)
Actual: TypeError: Cannot read property 'length' of undefined

Error :No such file or directory.

I've followed the steps in https://michaelsogos.github.io/pg-diff/ but get the following error:

$ pg-diff -h
Error: : No such file or directory
$ pg-diff -c development initial-script
Error: : No such file or directory

Executed from pg-diff-master directory

I've installed node.js successfully. Node version v6.17.1. I don't believe this is an issue with Node.js because I can run run node -v, npm -v successfully and write a message back to the terminal window successfully.

node is set in PATH - /usr/bin/node

How do I resolve this issue?

compare content of two schemas with different names

Is it possible to compare the DB1.schemaA with DB2.schemaB?
In other words if two schemas on two different databases have different names but are supposed to contains the same tables, etc. is it possible to perform the comparison? How to configure the mapping in this scenario?

TypeError: value.replace is not a function

$ pg-diff -c development initial-migration
                              _   _    __    __                  _   _ 
  _ __     __ _            __| | (_)  / _|  / _|           ___  | | (_)
 | '_ \   / _` |  _____   / _` | | | | |_  | |_   _____   / __| | | | |
 | |_) | | (_| | |_____| | (_| | | | |  _| |  _| |_____| | (__  | | | |
 | .__/   \__, |          \__,_| |_| |_|   |_|            \___| |_| |_|
 |_|      |___/                                                        

     Author: [object Object]
    Version: 1.2.8
 PostgreSQL: 9.6+
    License: MIT
Description: PostgreSQL schema and data comparing tool


CONFIGURED OPTIONS
         Script Author: @G
      Output Directory: .../ops/migrations/sqlscripts
     Schema Namespaces: public
     Idempotent Script: ENABLED
          Data Compare: ENABLED


Connected to PostgreSQL 11.4 (Debian 11.4-1.pgdg90+1) on [192.168.99.100:5432/dev] ✓
Connected to PostgreSQL 11.4 (Debian 11.4-1.pgdg90+1) on [192.168.99.100:6543/pro] ✓

Collect SOURCE database objects
[|||||||||||||||||||] 100% - Database objects collected!

Collect TARGET database objects
[|||||||||||||||||||] 100% - Database objects collected!

Compare SOURCE with TARGET database objects
[|||||||||||||||||||] 100% - Database objects compared!

Collect SOURCE tables records
[|||||||||||||||||||] 100% - Table records collected!

Collect TARGET tables records
[|||||||||||||||||||] 100% - Table records collected!

Compare SOURCE with TARGET database table records
[-------------------] 0% - Comparing table "public"."entity" records

TypeError: value.replace is not a function
TypeError: value.replace is not a function
    at Object.__generateSqlFormattedValue (.../ops/migrations/node_modules/pg-diff-cli/src/sqlScriptGenerator.js:353:34)
    at Object.generateInsertTableRecordScript (.../ops/migrations/node_modules/pg-diff-cli/src/sqlScriptGenerator.js:316:35)
    at sourceTableRecords.records.rows.forEach (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:100:45)
    at Array.forEach (<anonymous>)
    at Object.__compareTableRecords (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:69:41)
    at Object.compareTablesRecords (.../ops/migrations/node_modules/pg-diff-cli/src/compareRecords.js:44:22)
    at __runComparison (.../ops/migrations/node_modules/pg-diff-cli/main.js:306:49)
    at process._tickCallback (internal/process/next_tick.js:68:7)

error: column p.prokind does not exist

error: column p.prokind does not exist

Sorry, don't have much here, and a little debugging shows that the issue is actually in the pg-diff-api node module but it's through this so I figured I'd make the issue here. I ended up fixing it by removing the AND p.prokind = 'f'

Aggregate Functions not supported

I have a database with an aggregate function, can you either support Aggregate functions or remove them from being analyzed in the diff by using AND proisagg is false in CatalogApi.js getFunctions

Running on Jenkins Linux "Execute Shell Script"

Getting this error when running "Execute Shell" command with pg-diff:

TypeError: process.stdout.clearLine is not a function
at EventEmitter. (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:74:21)
at EventEmitter.emit (events.js:200:13)
at Function.compare (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/api/CompareApi.js:19:16)
at PgDiff.compare (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/node_modules/pg-diff-api/src/index.js:36:27)
at Run (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:78:39)
at Object. (/home/ubuntu/.nvm/versions/node/v12.16.1/lib/node_modules/pg-diff-cli/main.js:14:1)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:643:32)
at Function.Module._load (internal/modules/cjs/loader.js:556:12)

Feature request: some options and defaults for file handling

Some feature requests:

  • A -f flag to provide the config file location.
  • A -o flag for the output file location.
  • Make the 'outputDirectory' field optional
  • By default put the output in the working directory instead of relative to the config file.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.