sparknetworks / pgpm Goto Github PK
View Code? Open in Web Editor NEWPackage manager for Postgres 9.4+
License: MIT License
Package manager for Postgres 9.4+
License: MIT License
format of the set of db connection strings will understand one more attribute: unique_name (could be the same as dbname):
[
{
"host": "HOST_NAME",
"dbname": "DBNAME",
"port": PORT,
"environment": "ENVIRONMENT_NAME",
"unique_name": "UNIQUE NAME ALIAS"
}
]
make pgpm also send emails if it is set in general config file.
Configuration for SMTP:
"email": {
"type": "SMTP",
"credentials": {
"username": "user",
"password": "pass"
},
"host": "smtp.example.com",
"port": 465,
"from": "[email protected]",
"to": "[email protected]",
"subject": "Subject",
"body": "Body"
}
add --vcs-ref
command to pgpm deploy
call.
This will store vcs reference in deployment_events table
Track when was the last deployment, what files were updated
There should be pgpm install
command that must be run before pgpm deploy
can be used.
Installation procedure will:
_pgpm
schema in the db unless it was created already.tracked_schemas
that will include info from config.json
(e.g. name, current version, type, dependencies)It's enough to use session_user instead of current_user as default for this column.
I could prove that by hacking "ALTER TABLE ... ALTER COLUMN ... SET DEFAULT session_user" - and it worked.
Consider the following:
flehmann=# create function pgpm_test() returns text as $$select current_user::text$$ security definer language sql;
CREATE FUNCTION
flehmann=# select pgpm_test();
--> flehmann
flehmann=# alter function pgpm_test() owner to postgres;
ALTER FUNCTION
flehmann=# select pgpm_test();
--> postgres
flehmann=# create function pgpm_test1() returns text as $$select session_user::text$$ security definer language sql;
CREATE FUNCTION
flehmann=# select pgpm_test1();
--> flehmann
flehmann=# alter function pgpm_test1() owner to postgres;
ALTER FUNCTION
flehmann=# select pgpm_test1();
--> flehmann
Add set <connection_set_name>
argument to the call of pgpm that will get a set of connection strings to deploy to with command
db_set_name to be defined in config file that can be attached with --global-config <global_config_file_path>
argument
if --log-file flag is not given logs still should be written somewhere
After we deployed one new schema with "-o " option, but connecting as one personal (super-) user, only the schema is owned by the , but all the objects, in this case types and functions, are owned by the one personal (super-) user.
This is not what we expect and want. Please make the owner-role owner of all objects.
The connecting (super-) user should still appear in the "_pgpm" deployment logs, but not in the real structures.
Note, that the person, who executes a pgpm deployment will probably have to be superuser, but may not have the credentials of the owner-role. As far as i know, there is no "switch user" in postgres and if there was, we wouldn't want to use it here, because we want to identify the individual user in the deployment logs.
Currently the whole schema is dropped if run in unsafe mode. Change it to dropping only specific objects of schema.
For example, functions, types, views, but not tables
Currently code is crumpled in a single file. Ok for prototyping but we need to refactor it to reflect demands for scale and future development.
add to config.json the following attributes:
functions: {path: [PATH], strategy: [LIST OF FILES TO BE DEPLOYED RELATIVE TO PATH]}
types: {path: [PATH], strategy: [LIST OF FILES TO BE DEPLOYED RELATIVE TO PATH]}
old functions_path
and types_path
attributes will be ignored
The option "-u" seems to be broken, because without it i can install. The error message is:
...
2015-08-17 12:38:15,579 - pgpm.deploy - INFO - scripts/functions/_find_schema.sql
Traceback (most recent call last):
File "/usr/local/bin/pgpm", line 11, in
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/pgpm/deploy.py", line 783, in main
install_manager(arguments)
File "/usr/local/lib/python2.7/dist-packages/pgpm/deploy.py", line 364, in install_manager
cur.execute(GRANT_DEFAULT_USAGE_PRIVILEGES.format(_variables.PGPM_SCHEMA_NAME, ', '.join(user_roles)))
psycopg2.ProgrammingError: schema "_pgpm" does not exist
Dependency entry in config.json
file will be parsed. Concept is somewhat similar to https://docs.npmjs.com/files/package.json#dependencies
_
sign (.
is not allowed character for schema names)1_2_3
or 01_02_03
(latter format is for formatting purposes)x
notation like 01_02_XX
or 01_02_xx
or 1_2_X
or 1_2_x
>01_02_03
or <2
x
for any latest version of packagepgpm
in order to satisfy dependancyExample:
{ "dependencies" :
{ "foo" : "1_2_3"
, "bar" : ">=01_02_03"
, "baz" : ">01_02_XX"
, "boo" : "1_2_x"
, "qux" : "<2"
, "thr" : "x"
}
}
when deploying a set of functions, one of them failed, because its return_type didn't exist - a common case. It would be nice to see in the error message (stdout and/or log) a hint, which (or which first) of the "CREATE FUNCTION" scripts had the error.
rename --user
flag to --usage
flag and use --user
flag as a user that connects to DB when used set
option.
At the same time if user is not specified, os user must be used for connection attempt and warning message output
Add functions to pgpm that will start/stop auditing changes to specified tables
Sometimes fo rone feature i have to do ddl-changes in 2 (or more) schemas/repos. So my ddl-scripts naturally are named the same (in my case edarling_cid_tracking/ddl/acs_162_flehmann.sql and edarling_dwh_data/ddl/acs_162_flehmann.sql)
When deploying the second, I get "ddl/acs_162_flehmann.sql is not executed for schema edarling_dwh_data as it has already been executed before."
You would probably just add a column "package" to _pgpm.table_evolutions_log and add the package as parameter to _pgpm._is_table_ddl_executed(p_file_name text)
pgpm is not accepting version that contain characters.
Example:
"version" : "11_07_00", -> Works correctly
"version" : "11_v_00", -> Throws an error
Track deployment events with installation events table
{
"issue-tracker": {
"type": "JIRA",
"url": "URL_TO_API",
"username": "USERNAME",
"password": "PASSWORD",
}
}
pgpm deploy <connection_string> -f file.sql - works wherever "file.sql" is in repo
pgpm deploy <connection_string> -f actual_relative_path_to_file/file.sql - doesn't work
add support for two parameters:
--force-table-cascade
will (when dropping schema objects) drop dependent table objects (like columns).a
has column c
with type t
. When dropping type t
with cascade column c
will be dropped as well.--force-inter-schema-cascade
will (when dropping schema objects) drop dependent objects in other schema (not table related like columns, for those you will need to use --force-table-cascade
as well.When deploying by file in existing schema, allow adding new types
E.G.
functions_path : ["path_1", "path_2"]
allow mentioning users and owners of schema in config file. Using it through -o and -u commands will override config file with a warning
For some reason by default all newly created functions have by default execute permission for public role. So we need to revoke it.
This should be supported
pgpm deploy <connection_string> --add-config file_name
it will add aditional config file on top of config.json
and will override config.json
's settings
add --issue-ref
command to pgpm deploy
call.
This will store issue reference in deployment_events table
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.