yandex-qatools / postgresql-embedded Goto Github PK

View Code? Open in Web Editor NEW

493.0 493.0 90.0 311 KB

Embedded PostgreSQL Server

License: Other

Java 94.86% Batchfile 5.14%

postgresql-embedded's People

Contributors

Stargazers

Watchers

Forkers

linearregression grundic filippobosi smecsia jeetah tteats shtratos rbraley skaliber zemudkram sarkkine filius pigalon danpeter yrashk fdemilde vsuslov bigloser guanzhou-zhao blove319 talmdal cstroe novator007 janzyka wiltonlazary melezov leafknode cuulee codelutin hellblazer ntatsumi unkarjedy shisholik onukristo geen-berezovsky mathieu-pousse linshehai alexeylevin cthadeu jaspercloud oilid julianladisch archenroot bokzor vaisala-oss nielsdoucet matthewgillett rm3l wahid-anwar-m4 zsyphon sumelio badthinking dspasojevic footaku fwbrasil reeson2003 jlemay mukteshkrmishra miwroblewski emiguelt-fork attila-kiss-it marshallpierce drivera73 whoopinc tool-recommender-bot cass-green indyaah vorburger chiinlam diegosilva13 talbot nequissimus hjohn pravin-raha lars-n geofrey-s mdomasevicius gtsarenkov azukovskij juaby dizhaung s11110101111 frankfanslc xueqianing francescomuia bulksecuritygeneratorprojectv2 ddling1216 aschoerk compsuite-team

postgresql-embedded's Issues

clean-up of embedded process folder fails & incorrect console/log messages from "stopInternal"

I have created a maven-plugin that wraps around the postgresql-embedded component, using it to successfully start an embedded postgres process during the pre-integration phase of the maven lifecycle and stop/shutdown the process during the post-integration-phase of the lifecycle. I got the idea after posting this stackoverflow question and got a great response by user tunaki.

Doing my testing on Mac OSX (El Capitan/10.11.5) with maven Apache Maven 3.3.9 and java 8.

And, from my pom file:

    <dependency>
        <groupId>ru.yandex.qatools.embed</groupId>
        <artifactId>postgresql-embedded</artifactId>
        <version>1.16</version>
    </dependency>

I believe most of the functionality is working properly, except for the shutdown process. The code is managed by a PostgresEmbeddedService class. Here is what my start() method looks like:

public void start() throws Exception {
    DownloadConfigBuilder downloadConfigBuilder = new DownloadConfigBuilder();
    downloadConfigBuilder.defaultsForCommand(Command.Postgres);
    if (StringUtils.isNotEmpty(proxyHost) && proxyPort != null) {
        downloadConfigBuilder.proxyFactory(new HttpProxyFactory(proxyHost, proxyPort));
    }

    IRuntimeConfig runtimeConfig = new RuntimeConfigBuilder().defaults(Command.Postgres)
        .artifactStore(new ArtifactStoreBuilder().defaults(Command.Postgres).download(downloadConfigBuilder))
        .build();
    final PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getInstance(runtimeConfig);
    Version dbVersion = Version.valueOf(this.version);

    final PostgresConfig configDb = new PostgresConfig(dbVersion, new Net(host, port), new Storage(dbName), 
        new Timeout(timeout), new Credentials(username, password));
    PostgresExecutable exec = runtime.prepare(configDb);
    postgresProcess = exec.start();
    this.mavenLog.debug("Embedded postgres process Started with PID: " + postgresProcess.getProcessId());
    started = true;
}

And, here is my stop() method:

public void stop() throws Exception {
    this.mavenLog.info("Attempting to Stop the Embedded Postgres process");
    if (!stopped) {
        if (postgresProcess != null) {
        this.mavenLog.debug("stopping Postgres process with PID: " + postgresProcess.getProcessId());
            started = false;
        postgresProcess.stop();
        postgresProcess = null;
        this.mavenLog.info("Stop of Embedded Postgres process Succeeded!");
        stopped = true;
            } else {
            this.mavenLog.info("Postgresql process does Not exist - Nothing to Stop!");
            }
        }
}

Note that postgresProcess is an instance member of the class. It is instantiated in start()and held in memory as long as the maven process continues until stop() is called in post-integration-test.

Here is the problem I am seeing. It seems that the process.stop() is kind of working, but outputs some confusing console/log output. Here is the excerpt from my maven console (in debug mode - first line is my log statement from my PostgresEmbeddedService class):

[INFO] Attempting to Stop the Embedded Postgres process
[DEBUG] stopping Postgres process with PID: 1043
Oct 27, 2016 7:07:43 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
INFO: trying to stop postgresql
[DEBUG] Detected pid: 1057
[INFO] start AbstractPostgresConfig{storage=Storage{dbDir=/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/db-content-a4949f8d-32a3-48b8-9d34-90d0c9cce4ff, dbName='blahblah', isTmpDir=true}, network=Net{host='localhost', port=5432}, timeout=Timeout{startupTimeout=30000}, credentials=Credentials{username='myUser', password='myPassword'}, args=[stop], additionalInitDbParams=[]}
[DEBUG] waiting for server to shut down....
[DEBUG]  done
server stopped

Oct 27, 2016 7:07:44 PM ru.yandex.qatools.embed.postgresql.PostgresProcess sendStopToPostgresqlInstance
INFO: Cleaning up after the embedded process (removing /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549)...
[DEBUG] could delete /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549
Oct 27, 2016 7:07:45 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql with command, try next
[DEBUG] Detected pid: 1058
[INFO] stopOrDestroyProcess: process hasn't exited 

[INFO] execSuccess: false [kill, -2, 1043]
Oct 27, 2016 7:07:46 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql, try next
[DEBUG] Detected pid: 1059
[INFO] stopOrDestroyProcess: process hasn't exited 

[INFO] execSuccess: false [kill, 1043]
Oct 27, 2016 7:07:46 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql, try next
Oct 27, 2016 7:07:46 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql the second time, try one last thing
Oct 27, 2016 7:07:46 PM ru.yandex.qatools.embed.postgresql.PostgresProcess deleteTempFiles
WARNING: Could not delete temp db dir: /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/db-content-a4949f8d-32a3-48b8-9d34-90d0c9cce4ff
[WARNING] Could not delete pid file: /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/pgsql/bin/postgres.pid
[INFO] Stop of Embedded Postgres process Succeeded!

That last line is log statement from my class PostgresEmbeddedService.
It seems like the actual server process is indeed being stopped. When I acutally look for that process with PID 1043:

$ ps -1043
  PID TTY           TIME CMD

Also, please note this output at the very end of my maven run:

INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:47 min
[INFO] Finished at: 2016-10-27T19:07:46-04:00
[INFO] Final Memory: 90M/848M
[INFO] ------------------------------------------------------------------------
[WARNING] Could not delete pid file: /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/pgsql/bin/pg_ctl.pid
[WARNING] Could not delete pid file: /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/pgsql/bin/createdb.pid
[WARNING] Could not delete pid file: /var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549/pgsql/bin/initdb.pid
Exception in thread "Thread-7" java.lang.NoClassDefFoundError: de/flapdoodle/embed/process/extract/ExtractedFileSets
    at de.flapdoodle.embed.process.store.ArtifactStore.removeFileSet(ArtifactStore.java:90)
    at ru.yandex.qatools.embed.postgresql.ext.PostgresArtifactStore.removeFileSet(PostgresArtifactStore.java:65)
    at de.flapdoodle.embed.process.runtime.Executable.stop(Executable.java:77)
    at de.flapdoodle.embed.process.runtime.Executable$JobKiller.run(Executable.java:90)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: de.flapdoodle.embed.process.extract.ExtractedFileSets
    at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
    at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
    ... 5 more

Further, I also notice that the maven temp file (/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-9e977da4-fb54-414e-b5ca-566936bc4549) _has actually_ been cleaned up/removed:

$ pwd
/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T 
$ ls -l postgres*
ls: postgres*: No such file or directory

So, is this mainly an issue with logging? Is there some thread-related timing issue going on here? Is this stop() process working properly with only some bad logging messages?

It seems like the issue is logging/console output only, but I found the messages confusing.

Thanks in advance for any help/feedback.

--Aram

Hardcoding of download URL needs to be grained on architecture level

Latest download for 9.6.2 is 9.6.2-2 for Windows x64, but 9.6.2-1 for others.

Postgres does not start on OS X 10.11

I'm using version 1.15 and I get the following error when I try to start embedded Postgres:

16/09/10 00:01:38 INFO Executable: start AbstractPostgresConfig{storage=Storage{dbDir=/var/folders/ds/qdbhlt795_s_ljp96lk0tr580000gp/T/postgresql-embed-609858de-b9f5-4992-918b-14fa031c77a7/db-content-a601dfa2-2edc-4cc6-b039-80dc4a5b5eee, dbName='teszt', isTmpDir=true}, network=Net{host='127.0.1.1', port=37700}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='local', password='local'}, args=[], additionalInitDbParams=[]}
Sep 10, 2016 12:01:59 AM ru.yandex.qatools.embed.postgresql.PostgresProcess onAfterProcessStart
SEVERE: Failed to read PID file (File '/var/folders/ds/qdbhlt795_s_ljp96lk0tr580000gp/T/postgresql-embed-609858de-b9f5-4992-918b-14fa031c77a7/db-content-a601dfa2-2edc-4cc6-b039-80dc4a5b5eee/postmaster.pid' does not exist)

I am able to start it from the terminal after I extract the binaries.

Using Command Psql to import a dump

Hi,
there is the need of a feature to import a dump file into the created database by using the Psql Command .
I would suggest something like in the PostgresProcess.java

java
    /**
     * import into database from file
     */
    public void importFromFile(File file) {
        if (file.exists()) {
            runCmd(getConfig(), Psql, "", new HashSet<>(singletonList("import into " + getConfig().storage().dbName() + " failed")),
                    1000,
                    new String[]{
                            "-U", getConfig().credentials().username(),
                            "-d", getConfig().storage().dbName(),
                            "-h", getConfig().net().host(),
                            "-f", file.getAbsolutePath()
                    }
            );
        }
    }

Thank you

Problem with encoding on WIndows 10

Hello

We are using this library to write tests and on OS X it works like a charm. But when we try to execute tests on WIn10 x64 we are getting following errors:

ERROR: character with byte sequence 0xc5 0xbc in encoding "UTF8" has no equivalent in encoding "WIN12

I am suspecting that somehow default encoding for Win is different than default encoding for OS X and that is causing this issue, but I can't find a working approach to fix it,

ru.yandex.qatools.embed.postgresql.TestPostgresStarter.java fails

Hi Team,

I tried to run ru.yandex.qatools.embed.postgresql.TestPostgresStarter.java from my IDE.

Postgresql server fails to start, I have attached the stack trace for reference. There is a SEVERE message in stack trace about some PID file
OS: Windows 7 64 bit OS
JDK: Oracle 8 JDK
IDE: IntelliJ IDEA

Stacktrace:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Extract C:\Users\u6024002\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
Extract C:\Users\u6024002\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

May 15, 2015 5:32:44 PM ru.yandex.qatools.embed.postgresql.PostgresProcess onAfterProcessStart
SEVERE: Failed to read PID file (File 'C:\Users\u6024002\AppData\Local\Temp\embedpostgres-db-d9e8a898-599e-4070-811a-4e1fd9caec82\postmaster.pid' does not exist)
Extract C:\Users\u6024002\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:207)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
    at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
    at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
    at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
    at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
    at org.postgresql.Driver.makeConnection(Driver.java:410)
    at org.postgresql.Driver.connect(Driver.java:280)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:270)
    at ru.yandex.qatools.embed.postgresql.TestPostgresStarter.setUp(TestPostgresStarter.java:42)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.net.ConnectException: Connection refused: connect
    at java.net.DualStackPlainSocketImpl.connect0(Native Method)
    at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:79)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at java.net.Socket.connect(Socket.java:538)
    at org.postgresql.core.PGStream.<init>(PGStream.java:60)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:101)
    ... 38 more


java.lang.NullPointerException
    at ru.yandex.qatools.embed.postgresql.TestPostgresStarter.tearDown(TestPostgresStarter.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)


java.lang.NullPointerException
    at ru.yandex.qatools.embed.postgresql.TestPostgresStarter.tearDown(TestPostgresStarter.java:47)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Extract C:\Users\u6024002\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

Proxy support

I'm currently trying to get postgresql-embedded to work, but unfortunately our corporate proxy is blocking the download.
Would be nice if something like
mvn clean install -DargLine="-Dhttp.proxyHost=www.proxy.com -Dhttp.proxyPort=8000"
would be supported.
Is there a way to set the proxy within the code as a work around for now?

Never be able to stop PostgresProcess properly

I always got the following warning:

sDec 23, 2016 2:29:59 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
INFO: trying to stop postgresql
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess sendStopToPostgresqlInstance
INFO: Cleaning up after the embedded process (removing /tmp/postgresql-embed-65eef1e1-5328-48dd-bef4-950c6dbe0984)...
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql with command, try next
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql, try next
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql, try next
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
WARNING: could not stop postgresql the second time, try one last thing
Dec 23, 2016 2:30:00 PM ru.yandex.qatools.embed.postgresql.PostgresProcess deleteTempFiles
WARNING: Could not delete temp db dir: /tmp/postgresql-embed-65eef1e1-5328-48dd-bef4-950c6dbe0984/db-content-8a126cef-37fa-4003-84e7-70bc90134def

I know that if I want to delete the /tmp/postgresql-embed..., I need to sudo previledge. I'm not sure if that's the reason lead the stop problem. If so, how should I change the tmp dir to where I have general privilege to delete? If not, what's the possible reason that make the process not be able to stop?

Debug message pg_ctl: too many command-line arguments (first is "start")

I'm not sure if this is actually an issue or not, but it does look a bit worrying. When PostgresProcess.stop() is called, I see the following in the log:

DEBUG r.y.q.e.p.e.LogWatchStreamProcessor - pg_ctl: too many command-line arguments (first is "start")

DEBUG r.y.q.e.p.e.LogWatchStreamProcessor - Try "pg_ctl --help" for more information.

Here is a simple snippet that could reproduce it for me:

public class PostgresEmbeddedTester {
    private static final Logger LOG = LoggerFactory.getLogger(PostgresEmbeddedTester.class);

    public static void main(String[] args) throws Exception {
        LOG.info("Starting postgres");
        PostgresProcess process = PostgresStarter.getDefaultInstance().prepare(new PostgresConfig(Version.Main.PRODUCTION,
                "test-db")).start();

        LOG.info("Stopping postgres");
        process.stop();
    }
}

(With logback-classic in the classpath with default config, which has DEBUG enabled).

This is using postgres-embedded version 1.14 on a 64-bit Ubuntu Mate 15.10.

process does not come up on ubuntu machine

Hi Team,

I tried running the embedded postgresql from my machine. It works fine on local ubuntu machine but on remote CI setup, I always get the following logs as it doesn't work.

Extract /home/teamcity/.embedpostgresql/postgresql-9.4.1-1-linux-x64-binaries.tar.gz START
Extract /home/teamcity/.embedpostgresql/postgresql-9.4.1-1-linux-x64-binaries.tar.gz START
[postgres error]LOG:  database system was shut down at 2015-05-19 05:57:27 UTC
[postgres error] LOG:  database system is ready to accept connections
[postgres error] LOG:  autovacuum launcher started
[postgres error] ERROR [2015-05-19 05:57:29,259] ru.yandex.qatools.embed.postgresql.PostgresProcess: Failed to read PID file (File '/tmp/embedpostgres-db-c9be464c-8cf5-435e-8f9a-76fb642e8232/postmaster.pid' does not exist)
Extract /home/teamcity/.embedpostgresql/postgresql-9.4.1-1-linux-x64-binaries.tar.gz START
FATAL:  could not open relation mapping file "global/pg_filenode.map": No such file or directory
[postgres error] FATAL:  could not open relation mapping file "global/pg_filenode.map": No such file or directory
[postgres error] JDBC URL is jdbc:postgresql://127.0.0.1:57039/test?user=test&password=test
FATAL:  could not open relation mapping file "global/pg_filenode.map": No such file or directory
[postgres error] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 14.986 sec <<< FAILURE!
Running com.......Test
[05:57:31][com.......Test] testJdbi...
[05:57:31][testJdbi...] org.postgresql.util.PSQLException: FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
[05:57:31]
[testJdbi...] org.postgresql.util.PSQLException: FATAL: could not open relation mapping file "global/pg_filenode.map": No such file or directory
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:420)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:195)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:127)
    at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
    at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
    at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:41)
    at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
    at org.postgresql.Driver.makeConnection(Driver.java:414)
    at org.postgresql.Driver.connect(Driver.java:282)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:270)
    at com....testJdbi...setUp(Jdbi..Test.java:70)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)
[05:57:31][testJdbi...] java.lang.NullPointerException
[05:57:31]
[testJdbi...] java.lang.NullPointerException
    at com....Jdbi..Test.tearDown(Jdbi..Test.java:85)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:53)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:123)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:104)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:164)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:110)
    at org.apache.maven.surefire.booter.SurefireStarter.invokeProvider(SurefireStarter.java:175)
    at org.apache.maven.surefire.booter.SurefireStarter.runSuitesInProcessWhenForked(SurefireStarter.java:107)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:68)

Regards,
Shashank

Mount database in RAM

It is possible to mount database in RAM?

Speedup unit tests by moving MySql data to memory

Embedded Postgres doesn't shut down properly

I've used the postgresql-embedded library to run up a Postgres database for my application when running in Integration Test or Development mode. This starts up perfectly well by using the example from the Readme, but shutting down doesn't work correctly. Either by calling

PostgresProcess.stop()

or else

Executable.stop()

If I call PostgresProcess.stop() then I get an exception thrown, whereas if I call Executable.stop() then I don't get any exception but in both cases the postgres process is left running.

The exception that I get is:

java.lang.NullPointerException: executable is NULL
    at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet.<init>(ImmutableExtractedFileSet.java:42)
    at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder.build(ImmutableExtractedFileSet.java:106)
    at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:84)
    at ru.yandex.qatools.embed.postgresql.ext.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:69)
    at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:56)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.runCmd(PostgresProcess.java:91)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.shutdownPostgres(PostgresProcess.java:79)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.sendStopToPostgresqlInstance(PostgresProcess.java:75)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:57)

The instance with different versions of the server does not work

TestD.groovy :

import ru.yandex.qatools.embed.postgresql.EmbeddedPostgres
import ru.yandex.qatools.embed.postgresql.distribution.Version

class TestD {
	static void main(String[] args) {
		def postgres = new EmbeddedPostgres(Version.Main.V9_6)
		postgres.start()
		println postgres.connectionUrl
		postgres.stop()

		def postgres1 = new EmbeddedPostgres(Version.Main.V9_5)
		postgres1.start()
		println postgres1.connectionUrl
		postgres1.stop()
	}
}

output:

Connected to the target VM, address: '127.0.0.1:61257', transport: 'socket'
:run TestD
Extract /Users/victor.alenkov/.embedpostgresql/postgresql-9.6.2-1-osx-binaries.zip START
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /Users/victor.alenkov/.embedpostgresql/postgresql-9.6.2-1-osx-binaries.zip DONE
Optional[jdbc:postgresql://localhost:61271/postgres?user=postgres&password=postgres]

Optional[jdbc:postgresql://localhost:61279/postgres?user=postgres&password=postgres]

Disconnected from the target VM, address: '127.0.0.1:61257', transport: 'socket'
Could not delete pid file: /var/folders/w5/bxtb9q7n635fxz03k0c4_4240000gn/T/postgresql-embed-db74018f-34c8-43a7-a126-6d89a731141f/pgsql/bin/createdb.pid
Could not delete pid file: /var/folders/w5/bxtb9q7n635fxz03k0c4_4240000gn/T/postgresql-embed-db74018f-34c8-43a7-a126-6d89a731141f/pgsql/bin/initdb.pid
could not delete /var/folders/w5/bxtb9q7n635fxz03k0c4_4240000gn/T/postgresql-embed-db74018f-34c8-43a7-a126-6d89a731141f/pgsql/bin/pg_ctl.pid. Will try to delete it again when program exits.
Exception in thread "Thread-27" java.lang.IllegalStateException: Shutdown in progress
	at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66)
	at java.lang.Runtime.addShutdownHook(Runtime.java:211)
	at de.flapdoodle.embed.process.io.file.FileCleaner.forceDeleteOnExit(FileCleaner.java:52)
	at de.flapdoodle.embed.process.io.file.Files.forceDelete(Files.java:125)
	at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:173)
	at de.flapdoodle.embed.process.runtime.AbstractProcess$JobKiller.run(AbstractProcess.java:243)
	at java.lang.Thread.run(Thread.java:748)

BUILD SUCCESSFUL in 12s
4 actionable tasks: 2 executed, 2 up-to-date
20:26:39: External task execution finished 'run TestD'.

EmbeddedPostgres#stop does not stop the database

The function EmbeddedPostgres#stop only gets current process.

Problem brining postgres up on windows 7 using version 1.15

There is a problem with the args which are used in the ProcessBuilder which causes postgres not to launch properly on windows with no errors or any output indicating an issue. Subsequently createdb.exe fails its 3 attempts. This problem occurs using Command.PgCtl as well as Postgres. I tried this on two windows 7 systems (32 and 64 bit) both fail with this same problem. On 'nix systems it works fine.

The issue is the "-o "-p " "-h 127.0.0.1"" is not properly inserted into the args of RuntimeBuilder. The args need to be added separately

args.add("-o");
args.add("-p -h 127.0.0.1");

Here is a snippet of my workaround:

private void start() {
        IRuntimeConfig config = new RuntimeConfigBuilder().defaults(Command.PgCtl)
                 // rewrite args using ICommandLinePostProcessor
                .commandLinePostProcessor(new CommandLinePostProcessor(net.port()))
                .build();
}

private static class CommandLinePostProcessor implements ICommandLinePostProcessor {
    private int port;
    public CommandLinePostProcessor(int port) {
        this.port = port;
    }

    @Override
    public List<String> process(Distribution distribution, List<String> args) {
        String command = args.get(0);
        if (!command.contains("pg_ctl")) {
            return args;
        }
        String datadir = args.get(3);
        args.clear();
        args.add(command);
        args.add(format("-o"));
        // this is the issue, rewriting fixes the problem
        args.add(format("-p %s -h 127.0.0.1", port));
        args.add("-D");
        args.add(datadir);
        args.add("-w");
        stopArgs = new ArrayList<String>(args);
        args.add("start");
        return args;
    }
}

Extract file set once

I was quite surprised to see multiple entries of Extract <tar.gz file> START in my log output and thought the entire distribution was extracted multiple times, indicating some bug. But after some inspection, it seems like only one particular files is extracted each time.

However, de.flapdoodle.embed.process.extract.AbstractExtractor#extract allows to extract multiple files at once.

It would be an optimization to only extract the full file set once, rather than extracting each one individually when they're needed (especially since, as far I know, tar archives can only be read sequentially, you can't just seek to the position of the file you want and only read that). I understand that sometimes some binaries are not needed, but even just to start an instance, 3 binaries are extracted and used (and because stop is called in a shutdown hook, usually 4 binaries are used in a typical case)..

I looked at the code a bit, and it would seem it's quite possible to create a FileSet based on ru.yandex.qatools.embed.postgresql.Command and extract them during prepare(), but the code changes are a bit too deep for me to grasp fully unfortunately, so I can't provide a PR at this time.

When using the pg_ctl command pg does not shutdown

When using pg_ctl instead of postgres to start the db the pg process is not shutdown properly. It attempts to kill a non-existent pid and therefore pg continues to run after the process dies.

To workaround the issue I used ProcessBuilder to run "/path/to/pg_ctl -D </path/to/datadir> -w stop" in a shutdown hook.

Postgresql embedded tries to extract twice

I am trying to run a unit test on windows that uses postgresql-embedded version 1.3 .
I use the following code:

PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getDefaultInstance();
        final PostgresConfig config = new PostgresConfig(PRODUCTION, new AbstractPostgresConfig.Net(
                "localhost", 30000
        ), new AbstractPostgresConfig.Storage("test"), new AbstractPostgresConfig.Timeout(),
                new AbstractPostgresConfig.Credentials("user", "password"));
        PostgresExecutable exec = runtime.prepare(config);
        process = exec.start();
        String url = String.format("jdbc:postgresql://%s:%s/%s?user=%s&password=%s",
                config.net().host(),
                config.net().port(),
                config.storage().dbName(),
                config.credentials().username(),
                config.credentials().password()
        );
        conn = DriverManager.getConnection(url);

It fails with the following:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Extract C:\Users\ihazan\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
Extract C:\Users\ihazan\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
[postgres error]FATAL:  lock file "postmaster.pid" already exists
[postgres error] HINT:  Is another postgres (PID 9364) running in data directory "C:/Users/ihazan/AppData/Local/Temp/embedpostgres-db-7ffc9f37-858b-4db4-bb5a-c5c5bedecf22"?
[postgres error] 
Extract C:\Users\ihazan\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

While troubleshooting, i realized that for some reason postgres is being extracted twice instead of once. I confirmed that by putting a breakpoint in the PostgresArtifactStore::extract function and it hits the breakpoint twice. Isn't that a problem. I am also seeing many postgres folders being created in the same run in the temp folder.
The breakpoint is hit once in the call stack of runtime.prepare(config) and another time in the call stack of exec.start().

Thx a lot for any help.

PostgreSQL instance can not start due to missing postgresql.conf file (Windows 10)

I have a strange problem with starting postgresql-embedded under Win10. It does not start and when I debug and captur exact command being called I got:

C:\Users\Tomek\AppData\Local\Temp\postgresql-embed-a0a6c49c-5653-48c9-a992-98fd43979b88\pgsql\bin\postgres.exe 
-p 64582 -h localhost 
-D C:\Users\Tomek\AppData\Local\Temp\postgresql-embed-a0a6c49c-5653-48c9-a992-98fd43979b88\db-content-1b058745-4905-4d72-81c1-6b35a43da9ce

And when I tried to execute manually it I got error:

postgres cannot access the server configuration file 
"C:/Users/Tomek/AppData/Local/Temp/postgresql-embed-a0a6c49c-5653-48c9-a992-98fd43979b88/db-content-1b058745-4905-4d72-81c1-6b35a43da9ce/postgresql.conf": 
No such file or directory

I have checked this path and no such file is there, I even tried running Windows with "Safe mode" to make sure that no other application inteferes, but the problem persists.

So my questions are:

Does postgresql-embedded create such file and uses it somehow? I wasn't able to locate related code.
Maybe it's a problem with / returned in error message instead of \, but I have checked both Powershell and Cmder and both return same stuff so I doubt that is related with my missing conf file problem.

Thank you in advance for help.

Privileges on windows

Hi, trying to run this on windows, get the following error:

"Execution of PostgreSQL by a user with administrative permissions is not permitted.
The server must be started under an unprivileged user ID to prevent possible system security compromises. See the documentation for more information on how to properly start the server."

Is there some configuration to specify another user to start as, or to disable this in postgres?

Any plans to support version 9.6.x of postgres?

AbstractPGProcess.<init> throws NullPointerException on Windows because PackagePaths.getFileSet returns incorrect pattern

I have tested it with postgresql-embedded-2.0 on Windows 10 x64, which downloads the following ZIP file.
http://get.enterprisedb.com/postgresql/postgresql-9.5.5-1-windows-x64-binaries.zip

On Windows method ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet() creates incorrect pattern ^.*pgsql\\bin\\postgres.exe$

at de.flapdoodle.embed.process.config.store.FileSet$Builder.addEntry(FileSet.java:119)
at ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet(PackagePaths.java:51)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:46)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Then method de.flapdoodle.embed.process.extract.AbstractExtractor.extract() extracts a org.apache.commons.compress.archivers.zip.ZipFile$Entry which ZipArchiveEntry.name field correctly equals to pgsql/bin/postgres.exe

at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:71)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

But that is a mismatch because of which the de.flapdoodle.embed.process.store.PostgresFilesToExtract$1.type() method returns FileType.Library instead of the expected FileType.Executable

at de.flapdoodle.embed.process.store.PostgresFilesToExtract$1.type(PostgresFilesToExtract.java:105)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:75)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Thus, the de.flapdoodle.embed.process.extract.AbstractExtractor.extract() method has created an instance of de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder with the _executable field still being null, because of which the ImmutableExtractedFileSet instance constructor throws NullPointerException

at java.lang.NullPointerException.<init>(NullPointerException.java:70)
at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet.<init>(ImmutableExtractedFileSet.java:45)
at de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet$Builder.build(ImmutableExtractedFileSet.java:122)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:92)
at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:76)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

The NullPointerException is caught by the de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet() method, which then returns de.flapdoodle.embed.process.store.EmptyFileSet instead of the expected de.flapdoodle.embed.process.extract.ImmutableExtractedFileSet

at de.flapdoodle.embed.process.store.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:80)
at de.flapdoodle.embed.process.store.CachedPostgresArtifactStore.extractFileSet(CachedPostgresArtifactStore.java:60)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:57)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:129)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

That is how the EmbeddedPostgres.start() method creates an instance of ru.yandex.qatools.embed.postgresql.PostgresExecutable with field de.flapdoodle.embed.process.runtime.Executable.executable that contains the unexpected EmptyFileSet instead of the expected ImmutableExtractedFileSet, then it attempts to execute it.

at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

Because of that the ru.yandex.qatools.embed.postgresql.AbstractPGProcess instance constructor runs the EmptyFileSet.executable() method which always returns null instead of a java.io.File value, which is actually always a bad coding style.

at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:72)
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init>(AbstractPGProcess.java:19)
at ru.yandex.qatools.embed.postgresql.PostgresProcess.<init>(PostgresProcess.java:67)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:101)
- locked <0x29fb> (a ru.yandex.qatools.embed.postgresql.PostgresExecutable)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

The received null value is then passed to the de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile() method which was not designed to accept it, so it throws the originally mentioned NullPointerException which is never caught anywhere since then.

at java.lang.NullPointerException.<init>(NullPointerException.java:60)
at de.flapdoodle.embed.process.runtime.AbstractProcess.pidFile(AbstractProcess.java:125)
at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:72)
at ru.yandex.qatools.embed.postgresql.AbstractPGProcess.<init>(AbstractPGProcess.java:19)
at ru.yandex.qatools.embed.postgresql.PostgresProcess.<init>(PostgresProcess.java:67)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:25)
at ru.yandex.qatools.embed.postgresql.PostgresExecutable.start(PostgresExecutable.java:13)
at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:101)
- locked <0x29fb> (a ru.yandex.qatools.embed.postgresql.PostgresExecutable)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:130)
at ru.yandex.qatools.embed.postgresql.EmbeddedPostgres.start(EmbeddedPostgres.java:95)

So I suggest that you fix the incorrect executable zip file entry pattern generation in your method ru.yandex.qatools.embed.postgresql.PackagePaths.getFileSet() and probably consider replacing or fixing the bad code style de.flapdoodle.embed.process library which has methods that return null values and passes it to its own methods while they are not designed for accepting null values.

FATAL: ShmemIndex entry size is wrong for data structure "XLOG Ctl"

Trying to run it on Win8 x64 and it fails (
At first there was installed another instanse of PostgreSql - after deleting the same error:

FATAL:  ShmemIndex entry size is wrong for data structure "XLOG Ctl": expected 273304, actual 4209304

Full log :

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/denis.veselovsky/.m2/repository/org/slf4j/slf4j-log4j12/1.7.2/slf4j-log4j12-1.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/denis.veselovsky/.m2/repository/org/slf4j/slf4j-simple/1.7.7/slf4j-simple-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Extract C:\Users\denis.veselovsky\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
Extract C:\Users\denis.veselovsky\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
log4j:WARN No appenders could be found for logger (de.flapdoodle.embed.process.runtime.ProcessControl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
[postgres error]LOG:  database system was shut down at 2015-06-08 13:23:21 EEST
[postgres error] LOG:  database system is ready to accept connections
[postgres error] LOG:  autovacuum launcher started
[postgres error] Jun 08, 2015 1:23:23 PM ru.yandex.qatools.embed.postgresql.PostgresProcess onAfterProcessStart
SEVERE: Failed to read PID file (File 'C:\Users\DENISV~1.ROK\AppData\Local\Temp\embedpostgres-db-e02fdf75-af19-4ac6-950f-bc5637cf0770\postmaster.pid' does not exist)
Extract C:\Users\denis.veselovsky\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START
FATAL:  ShmemIndex entry size is wrong for data structure "XLOG Ctl": expected 273304, actual 4209304
[postgres error] FATAL:  ShmemIndex entry size is wrong for data structure "XLOG Ctl": expected 273304, actual 4209304
[postgres error] FATAL:  ShmemIndex entry size is wrong for data structure "XLOG Ctl": expected 273304, actual 4209304
[postgres error] 
org.postgresql.util.PSQLException: FATAL: ShmemIndex entry size is wrong for data structure "XLOG Ctl": expected 273304, actual 4209304
    at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:420)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:195)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
    at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:127)
    at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
    at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
    at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:41)
    at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
    at org.postgresql.Driver.makeConnection(Driver.java:414)
    at org.postgresql.Driver.connect(Driver.java:282)
    at java.sql.DriverManager.getConnection(DriverManager.java:664)
    at java.sql.DriverManager.getConnection(DriverManager.java:270)
    at ProxyTest.PostgreSqlTest.test1(PostgreSqlTest.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runners.Suite.runChild(Suite.java:127)
    at org.junit.runners.Suite.runChild(Suite.java:26)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

Jun 08, 2015 1:23:23 PM ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
INFO: trying to stop postgresql
Extract C:\Users\denis.veselovsky\.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

LOG:  could not open temporary statistics file "pg_stat/lobal.tmp": No such file or directory
[postgres error] Exception in thread "Thread-6" java.lang.IllegalStateException: Shutdown in progress
    at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66)
    at java.lang.Runtime.addShutdownHook(Runtime.java:211)
    at de.flapdoodle.embed.process.io.file.FileCleaner.forceDeleteOnExit(FileCleaner.java:52)
    at de.flapdoodle.embed.process.io.file.Files.forceDelete(Files.java:122)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.deleteTempFiles(PostgresProcess.java:143)
    at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:74)
    at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:170)
    at de.flapdoodle.embed.process.runtime.AbstractProcess$JobKiller.run(AbstractProcess.java:243)
    at java.lang.Thread.run(Thread.java:745)

PostgresProcess.stopInternal raises false alarm

        if (!waitUntilProcessHasStopped(5000)) {
            logger.severe("Postgres has not been stopped within 10s! Something's wrong!");
        }

will always issue a severe log entry, when the process actually has been successfully terminated, as waitUntilProcessHasStopped returns false when the process has stopped.

add postgis support

If it's possible can you create a version of postgres with postgis installed?

Problem with finding postgres.exe in cached dir under Windows

Hi!

Method extractFileSet() can't match exe file name with pattern as pattern slash is "/" while in filename it's "\".
So, the process can't start. Evidently, there's no such problem in *nix systems.

Best regards,
artpass

Capturing failures

Can you add a section to the documentation outlining how to check if a file succeeded?

when i restart program...

hello...

I am interested in this project.

So I tested a few things.

I've been using to embed by javafx is good.

However, if you quit the program, and run again , the stored data disappear .

How can I keep the data stored in the db when the program is restarted ?

there is not options...

PostgresStarter... PostgresConfig... and so on...

thanks...

Make it Java 9 compatible

Please make it work with Java 9 too.

Thank you.

java.util.NoSuchElementException

Hi Team,

I tried to use postgresql-embedded library to unit test my dao, however I was getting java.util.NoSuchElementException when I tried to start embedded postgresql db server.
I had cloned this git repository and ran ru.yandex.qatools.embed.postgresql.TestPostgresStarter.java, still it was resulting in java.util.NoSuchElementException. My environment details as as given below:

OS: Windows 64 bit
JDK: Oracle Java 8
Postgresql version: All version

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Extract C:\Users\u6024002.embedpostgresql\postgresql-9.4.1-1-windows-x64-binaries.zip START

java.util.NoSuchElementException
at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:713)
at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:734)
at java.util.Collections$3.nextElement(Collections.java:5216)
at de.flapdoodle.embed.process.extract.ZipExtractor$ZipArchiveWrapper.getNextEntry(ZipExtractor.java:58)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:68)
at ru.yandex.qatools.embed.postgresql.ext.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:69)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:56)
at ru.yandex.qatools.embed.postgresql.TestPostgresStarter.setUp(TestPostgresStarter.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

java.lang.NullPointerException
at ru.yandex.qatools.embed.postgresql.TestPostgresStarter.tearDown(TestPostgresStarter.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Process finished with exit code -1

Missing msvcr120.dll in windows x64 archive

Running on Windows 7 64bit, during startup I got:

XII 03, 2015 10:51:38 DOP. ru.yandex.qatools.embed.postgresql.PostgresProcess onAfterProcessStart
SEVERE: Failed to read PID file (File 'C:\Users\beranekm\AppData\Local\Temp\postgresql-embed-c4f7a4b4-b9cb-4cc1-a622-2de8305deff2\db-content-e5264bb1-3489-4280-982d-8e892a4580db\postmaster.pid' does not exist)
Exception in thread "main" java.lang.NullPointerException
    at Main.main(Main.java:23)
Failed to extract file set: java.lang.NullPointerException
XII 03, 2015 10:51:39 DOP. ru.yandex.qatools.embed.postgresql.PostgresProcess stopInternal
INFO: trying to stop postgresql

after investigation, i find out, that binaries need msvcr12.dll which is not in archive (postgresql-9.4.4-1-windows-x64-binaries.zip)

the strange is, that this library is in c:\Windows\SysWOW64\msvcr120.dll

how can I solve this problem, what are you recommend?

[question] Can I specify directory for Unix Socket listening?

Related to my previous question. As I plan to do some hpc benchmarks with extension I also would like to disable TCP stack, or additionally enable Unix Sockets directory(ies) to prevent TCP overhead.

Is it possible?

Thanks.

Multiple "Extract" entries in log

I see multiple "Extract" entries in log. Does it mean there is something incorrectly configured and postgres is extracted multiple times?

Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz START
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz DONE
Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz START
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz DONE
12:27:22.383 [main] INFO  d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-81f37bfb-0f67-4fd8-a60b-50a1cc3187e0/db-content-9575c295-3933-4cbe-a551-8d9c7a14ef74, dbName='test', isTmpDir=true}, network=Net{host='localhost', port=40759}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='pass'}, args=[], additionalInitDbParams=[]}
Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz START
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/denis/.embedpostgresql/postgresql-9.5.5-1-linux-x64-binaries.tar.gz DONE
12:27:23.173 [main] INFO  d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-81f37bfb-0f67-4fd8-a60b-50a1cc3187e0/db-content-9575c295-3933-4cbe-a551-8d9c7a14ef74, dbName='test', isTmpDir=true}, network=Net{host='localhost', port=40759}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='pass'}, args=[test], additionalInitDbParams=[]}
12:27:26.174 [main] INFO  d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-81f37bfb-0f67-4fd8-a60b-50a1cc3187e0/db-content-9575c295-3933-4cbe-a551-8d9c7a14ef74, dbName='test', isTmpDir=true}, network=Net{host='localhost', port=40759}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='pass'}, args=[], additionalInitDbParams=[]}

slow startup

currently the startup time is about 50 seconds which is too long.
Most time is spend for file coppies.

System:
-Windows
-ssd
-i7

i have implemented a low level optimisation for write files in de.flapdoodle.embed.process.io.file.Files
Although this is alreday faster (20 secounds) it is still too slow.

Maybe there are other ways to improve the performance e.g.
-only replace the postgress data folder
-filter not required files like docus or tools
etc

static final ThreadPoolExecutor syncThread;
static {
	syncThread = new ThreadPoolExecutor(20,20,1,TimeUnit.MINUTES,new ArrayBlockingQueue<Runnable>(20));
	syncThread.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
}
public static void write(InputStream in, File output) throws IOException {
	ByteArrayOutputStream out = new ByteArrayOutputStream(65536*10);
	ByteStreams.copy(in,out);
	FileChannel c = FileChannel.open(output.toPath(), StandardOpenOption.WRITE, StandardOpenOption.CREATE,StandardOpenOption.READ);
	final byte[] array = out.toByteArray();
	MappedByteBuffer b = c.map(FileChannel.MapMode.READ_WRITE, 0, array.length);
	b.put(array);
	syncThread.submit(()->{
			b.force();
			try {
				c.close();
			} catch (IOException e) {
				throw new RuntimeException(e);
			}
	});
}

Too many logs in the output

Way too many logs are shown in the output for two operations: prepare and start.
The first "Extract ..." log appears during the prepare phase and two additional during the start phase.

The code that outputs the logs:

PostgresStarter<PostgresExecutable, PostgresProcess> runtime =  PostgresStarter.getDefaultInstance();
final PostgresConfig config = new PostgresConfig( V9_4,
    new AbstractPostgresConfig.Net( HOST, PORT ),
    new AbstractPostgresConfig.Storage( DB_NAME ),
    new AbstractPostgresConfig.Timeout(),
    new AbstractPostgresConfig.Credentials( USERNAME, PASSWORD ) );
config.getAdditionalInitDbParams()
      .addAll( asList( "-E", "SQL_ASCII", "--locale=C", "--lc-collate=C", "--lc-ctype=C" ) );

PostgresExecutable exec = runtime.prepare( config );
process = exec.start();

The logs:

Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz START
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz DONE
Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz START
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz DONE
2017-02-25_00:09:03.783 INFO d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-269d8860-7423-4065-94f1-1c142a0e002d/db-content-2e25256c-0f8b-4bf7-8e68-e8ea611e2578, dbName='db', isTmpDir=true}, network=Net{host='localhost', port=54542}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='password'}, args=[], additionalInitDbParams=[-E, SQL_ASCII, --locale=C, --lc-collate=C, --lc-ctype=C]}
Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz START
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /home/mnowotnik/.embedpostgresql/postgresql-9.4.10-1-linux-x64-binaries.tar.gz DONE
2017-02-25_00:09:04.456 INFO d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-269d8860-7423-4065-94f1-1c142a0e002d/db-content-2e25256c-0f8b-4bf7-8e68-e8ea611e2578, dbName='db', isTmpDir=true}, network=Net{host='localhost', port=54542}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='password'}, args=[db], additionalInitDbParams=[]}
2017-02-25_00:09:07.457 INFO d.f.embed.process.runtime.Executable - start AbstractPostgresConfig{storage=Storage{dbDir=/tmp/postgresql-embed-269d8860-7423-4065-94f1-1c142a0e002d/db-content-2e25256c-0f8b-4bf7-8e68-e8ea611e2578, dbName='db', isTmpDir=true}, network=Net{host='localhost', port=54542}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{username='user', password='password'}, args=[], additionalInitDbParams=[-E, SQL_ASCII, --locale=C, --lc-collate=C, --lc-ctype=C]}

Fails to start when run as user without HOME directory

starting embedded postgres (Version: 9.4.1208) with a user without home directory like this:
(the important thing it to set .artifactStorePath(new SubdirTempDir())
as it failed trying to use the new UserHome() with PostgresStarter.getDefaultInstance())

val pgConfig: PostgresConfig = new PostgresConfig(
      Version.V9_5_0,
      pgHost, Network.getFreeServerPort, "test")
    pgConfig.getAdditionalInitDbParams.addAll(util.Arrays.asList("--auth=trust", "--auth-host=trust", "--auth-local=trust"))
    val logWatch: LogWatchStreamProcessor = new LogWatchStreamProcessor("started", new util.HashSet[String](singletonList("failed")), new LoggingOutputStreamProcessor(Logger.getLogger("postgres"), Level.ALL))
    val runTimeConfig = new RuntimeConfigBuilder()
    .defaults(Command.Postgres)
    .artifactStore(new ArtifactStoreBuilder()
      .defaults(Command.Postgres)
      .download(
        new DownloadConfigBuilder()
          .defaultsForCommand(Command.Postgres)
          .artifactStorePath(new SubdirTempDir()).build()
      )
    ).processOutput(new ProcessOutput(logWatch, logWatch, logWatch))
    .build()

    PostgresStarter.getInstance(runTimeConfig).prepare(pgConfig).start()

It still wants to create $HOME/.embedpostgresql when calling initdb here: https://github.com/yandex-qatools/postgresql-embedded/blob/master/src/main/java/ru/yandex/qatools/embed/postgresql/PostgresProcess.java#L70 which does not honor the given IRuntimeConfig. Why does ot need to create its own in this method?

Gitlab build: postmaster.pid does not exist

Hi,
everything works fine when i start the database on my local client. But when i used the CI feature from Gitlab i get the following error:
error.txt

Any suggestions?

Binaries extraction occurring repeatedly in a single execution

In short, I created a simple code with the following snippet using postgresql-embedded:

PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getDefaultInstance();
final PostgresConfig config = PostgresConfig.defaultWithDbName("test");
PostgresExecutable exec = runtime.prepare(config);
PostgresProcess process = exec.start();
[...]
process.stop();

When running that code, the following message appears 4 times:

Extract /home/user/.embedpostgresql/postgresql-9.4.4-1-linux-binaries.tar.gz DONE

I've noticed that every time the method PostgresProcess.runCmd() is called internally, these two methods are called:

de.flapdoodle.embed.process.runtime.Starter.prepare(CONFIG)
de.flapdoodle.embed.process.extract.AbstractExtractor.extract(IDownloadConfig, File, FilesToExtract)

This happens when:

PostgresExecutable exec = runtime.prepare(config) is executed
ru.yandex.qatools.embed.postgresql.PostgresProcess.onBeforeProcess(IRuntimeConfig) calls initdb
ru.yandex.qatools.embed.postgresql.PostgresProcess.onAfterProcessStart(ProcessControl, IRuntimeConfig) calls createdb
ru.yandex.qatools.embed.postgresql.PostgresProcess.sendStopToPostgresqlInstance() calls pg_ctl ... stop

Once PostgreSQL binaries were extracted to a temporary directory, why do they need to be extracted again each time PostgresProcess.runCmd() is invoked?

Attached is the entire class code:
PostgresEmbedded.pdf

Port not passed through when using a custom config

When I'm configuring the embedded postgres to run with a Custom configuration the port option does not seem to passed through to the underlying Postgres process, I see the following output when running

$ ps ax 
...
4571   ??  S      0:00.02 /private/var/folders/z3/f3xjjsr12ks22t16mtm29q_w0000gn/T/postgresql-embed-eed63e85-74b2-4511-a804-37f36aab3e02/pgsql/bin/postgres -D /var/folders/z3/f3xjjsr12ks22t16mtm29q_w0000gn/T/postgresql-embed-eed63e85-74b2-4511-a804-37f36aab3e02/db-content-f67979d1-0e6c-4b2b-94b4-15234acbf760 -h 127.0.0.1

So only the host and data directory are passed to the postgres process

This is with the configuration as the following, and supposably port 4711, I also checked and the database is actually running on the default port 5432.

final PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getDefaultInstance();
final AbstractPostgresConfig.Net network = new AbstractPostgresConfig.Net("127.0.0.1", 4711);
final AbstractPostgresConfig.Storage storage = new AbstractPostgresConfig.Storage("db");
final AbstractPostgresConfig.Credentials credentials = new AbstractPostgresConfig.Credentials("user", "password");
final AbstractPostgresConfig.Timeout timeout = new AbstractPostgresConfig.Timeout();
final PostgresConfig config = new PostgresConfig(V9_4, network, storage, timeout, credentials);
PostgresExecutable exec = runtime.prepare(config);
process = exec.start();

 System.out.println("Running Postgres...");
Thread.sleep(10000000);

This was tested on MacOSX with version 1.13

Problem with running postgres on Windows with low preveligies

Hello.
I saw issue:
#28
I have same problem.
I tryed to configure it myself using pg_ctl java API from your library but it does not work.
I have checked out your project and have started a test with pg_ctl localy.
That show me same error as i have in my own project.

I attach a file of your test console log.

I have tryed to start it manually and it works fine.

embedded_postgres.txt

Passing InitDB Arguments

Hi,
I appreciate your work a lot.

I'm trying to re-import a dump, but it fails because 'value too long for column...'.
The reason is that we have german umlauts in the dumped data.
To make this work there should be a possibility to pass some arguments to the InitDB process.
Like new String[]{"-E", "'UTF-8'", "--lc-collate='de_DE.UTF-8'", "--lc-ctype=locale='de_DE.UTF-8'"}
Do you think it will be ever possible to pass some arguments to the InitDb process?
Something like in : https://gist.github.com/dev-mobile/9383ee242c26bafac2b5

Thank you

Failed to stop postgres by pg_ctl!

I'm using this code to start Embedded Postgresql

@Component
class EmbeddedPostqreSql {

    private EmbeddedPostgres postgres
    private DataSource dataSource

    @Value('${spring.datasource.dbname}')
    private String dbname

    @Value('${spring.datasource.username}')
    private String username

    @Value('${spring.datasource.password}')
    private String password

    @Bean
    DataSource dataSource() {
        if (null == postgres) {
            startDatabase()
        }

        if (null == dataSource) {
            final PostgresConfig config = postgres.getConfig().get()
            final String jdbcUrl = String.format("jdbc:postgresql://%s:%s/%s",
                    config.net().host(),
                    config.net().port(),
                    config.storage().dbName()
            )

            dataSource = new DriverManagerDataSource(jdbcUrl, username, password)
        }

        return dataSource
    }

    synchronized void startDatabase() {
        if (null == postgres) {
            try {
                System.out.println("Starting Embedded PostqreSQL")
                Path embeddedPostgresCachedRuntimeConfig = Paths.get(System.getProperty("java.io.tmpdir") + "/cached.embedded.postgres")
                if (!Files.exists(embeddedPostgresCachedRuntimeConfig)) {
                    Files.createDirectory(embeddedPostgresCachedRuntimeConfig)
                }

                postgres = new EmbeddedPostgres()
                postgres.start(cachedRuntimeConfig(embeddedPostgresCachedRuntimeConfig),
                        "localhost", 5432,
                        dbname, username, password,
                        Collections.emptyList())

                System.out.println("PostqreSQL started")
            } catch (IOException e) {
                e.printStackTrace()
            }
        }
    }

    synchronized void stopDatabase() {
        dataSource.connection.close()
        postgres.stop()
    }
}

But getting these Exceptions after completing the tests:

2017-05-10 10:47:22.003 INFO 59145 --- [ Thread-2] r.y.q.embed.postgresql.PostgresProcess : trying to stop postgresql
2017-05-10 10:47:22.324 WARN 59145 --- [ Thread-2] r.y.q.embed.postgresql.PostgresProcess : Failed to stop postgres by pg_ctl!

java.lang.RuntimeException: Failed to initialize the executable
at ru.yandex.qatools.embed.postgresql.PostgresStarter.newExecutable(PostgresStarter.java:70) ~[postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresStarter.newExecutable(PostgresStarter.java:24) ~[postgresql-embedded-2.1.jar:na]
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:59) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:49) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.runCmd(PostgresProcess.java:123) [postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.runCmd(PostgresProcess.java:81) [postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.shutdownPostgres(PostgresProcess.java:136) [postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.sendStopToPostgresqlInstance(PostgresProcess.java:176) [postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:149) [postgresql-embedded-2.1.jar:na]
at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:170) [de.flapdoodle.embed.process-2.0.1.jar:na]
at de.flapdoodle.embed.process.runtime.Executable.stop(Executable.java:73) [de.flapdoodle.embed.process-2.0.1.jar:na]
at de.flapdoodle.embed.process.runtime.Executable$JobKiller.run(Executable.java:90) [de.flapdoodle.embed.process-2.0.1.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_60]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_60]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_60]
at java.lang.reflect.Constructor.newInstance(Constructor.java:422) ~[na:1.8.0_60]
at ru.yandex.qatools.embed.postgresql.PostgresStarter.newExecutable(PostgresStarter.java:68) ~[postgresql-embedded-2.1.jar:na]
... 12 common frames omitted
Caused by: java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) ~[na:1.8.0_60]
at java.lang.Runtime.addShutdownHook(Runtime.java:211) ~[na:1.8.0_60]
at de.flapdoodle.embed.process.runtime.ProcessControl.addShutdownHook(ProcessControl.java:246) ~[de.flapdoodle.embed.process-2.0.1.jar:na]
at de.flapdoodle.embed.process.runtime.Executable.(Executable.java:58) [de.flapdoodle.embed.process-2.0.1.jar:na]
at ru.yandex.qatools.embed.postgresql.AbstractPGExecutable.(AbstractPGExecutable.java:13) ~[postgresql-embedded-2.1.jar:na]
at ru.yandex.qatools.embed.postgresql.PgCtlExecutable.(PgCtlExecutable.java:18) ~[postgresql-embedded-2.1.jar:na]
... 17 common frames omitted

2017-05-10 10:47:22.343 INFO 59145 --- [ Thread-2] d.f.e.process.runtime.ProcessControl : execSuccess: true [kill, 59159]

EDIT: Using latest Version 2.1 on MacOS

How can I solve the Shutdown Problem?

Connection error on Windows during AppVeyor build

Hi,

I've been using postgresql-embedded for tests in JDBDT (http://github.com/edrdo/jdbdt) without any problems on MacOS and Linux. I was just trying to enable AppVeyor continuous integration for Windows builds when I encounted some errors.

The way I use postgresql-embedded is simple (full code here
in https://github.com/edrdo/jdbdt/blob/master/src/test/java/org/jdbdt/postgresql/PostgreSQLSuite.java)

On setup I have:

  postgres = new EmbeddedPostgres(Version.V9_6_2);
  Path cachePath = FileSystems.getDefault().getPath(System.getProperty("user.home") + "/.embedpostgresql/");
  String url = postgres.start(EmbeddedPostgres.cachedRuntimeConfig(cachePath));

During a AppVeyor build - check https://ci.appveyor.com/project/edrdo/jdbdt/build/3 - the JDBC driver is unable to connect to the embedded Postgres database.

   org.postgresql.util.PSQLException: Connection to localhost:1047 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
at org.postgresql.Driver.makeConnection(Driver.java:450)
at org.postgresql.Driver.connect(Driver.java:252)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)

Maybe this is an Appveyor issue with restricted port use? In that case how can I configure the port for the "cached runtime config" . More generally, the point of the cached config is to avoid downloading the PostgreSQL binaries, but I gather that there is no way to set the URL parameters (name, pass, port, etc). I looked up the postgresql-embedded source code but I couldn't determine how to set up these together with a cached config, probably I'm missing something ...

Problem initializing the DB on Windows 10 x64

My team has successfully started using this library.
However, when I work on Windows 10 x64, I found that DB is not getting initialized.

After debugging for root cause
Found that following 2 lines need to be reversed or better solution would be to --pgdata while specifying the Data directory.

InitDbProcess - Line 69, 70

        ret.add(config.storage().dbDir().getAbsolutePath());
        ret.addAll(config.getAdditionalInitDbParams());

Since the DB Storage argument is coming before the additional parameters like -E, Windows command comes back throwing exception saying

initdb: too many command-line arguments (first is "-E")

If I remove the Additional Init DB Params that I want to set [Locale, Encoding, Collate, ...], then it works absolutely fine.

Also let me know if any workaround is possible for the same.

Postgres 9.4.1 Binaries on OS X don't unpack

Using the Embedded PostgreSQL server on Linux works perfectly fine. Using it on OS X doesn't work though. I get the error:

Caused by: java.util.NoSuchElementException: null
at java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:713)
at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:734)
at java.util.Collections$3.nextElement(Collections.java:5275)
at de.flapdoodle.embed.process.extract.ZipExtractor$ZipArchiveWrapper.getNextEntry(ZipExtractor.java:58)
at de.flapdoodle.embed.process.extract.AbstractExtractor.extract(AbstractExtractor.java:68)
at ru.yandex.qatools.embed.postgresql.ext.PostgresArtifactStore.extractFileSet(PostgresArtifactStore.java:69)
at de.flapdoodle.embed.process.runtime.Starter.prepare(Starter.java:56)

When running from Java code. However, the Zip file that was downloaded seems to be OK, and I can unpack it using both the zip and jar tools.

$ jar -xf ../postgresql-9.4.1-1-osx-binaries.zip
$ ls
pgsql
$

postgres does not start on win7 x64

Extract embedpostgresql\postgresql-9.5.0-1-windows-x64-binaries.zip DONE 10:28:46.497 [main] DEBUG d.f.e.process.runtime.ProcessControl - Detected pid: 4648 10:28:52.940 [main] INFO d.f.embed.process.runtime.Executable - start ru.yandex.qatools.embed.postgresql.config.PostgresConfig@282003e1 10:28:52.949 [main] DEBUG d.f.e.process.runtime.ProcessControl - Detected pid: 11520 Bře 15, 2016 10:29:12 DOP. ru.yandex.qatools.embed.postgresql.PostgresProcess onAfterProcessStart SEVERE: Failed to read PID file (File 'C:\catalog\temp\postmaster.pid' does not exist) 10:29:13.665 [main] DEBUG d.f.e.process.runtime.ProcessControl - Detected pid: 10792

Postgres embedded service does not start on win7 x64

im using version 1.10

this is how im starting postgres

PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getDefaultInstance(); config = new PostgresConfig(PRODUCTION, new AbstractPostgresConfig.Net( "localhost", findFreePort() ), new AbstractPostgresConfig.Storage("test", "temp"), new AbstractPostgresConfig.Timeout(), new AbstractPostgresConfig.Credentials("user", "password")); PostgresExecutable exec = runtime.prepare(config); process = exec.start();

[question] Can I load extension into Embedded PostgreSQL?

I am interested in doing PostgreSQL laying on RAM disk with timescaleDB extension some benchmarks against Chronicle Map (off heap key value store) for timeseries data.

Can I load custom extensions into embedded db engine?

Postgres cannot be downloaded from other than default place

We created our own packages and put them into our own Nexus repository. Unfortunately when embedded postgres starts it uses this nexus settings for the main (postgres) command but for the rest the defaults are used. This means for example that for "InitDb" command it tries to download the package again from the default location. Even though the package has already been downloaded from nexus. The config we have:

new RuntimeConfigBuilder().defaults(command)
            .artifactStore(new ArtifactStoreBuilder()
                .defaults(Command.Postgres)
                .download(new DownloadConfigBuilder()
                    .defaultsForCommand(command)
                    .downloadPath(postgrePackage.getNexusUrl() + "/service/local/repositories/" + postgrePackage.getRepository() + "/content/")
                    .packageResolver(packageResolver)
                    .build())
                .build())
            .build();

progressListener cannot be overridden

By default, the stdout gets bombarded with >= 2 times with

Extract ...

of all the files that get extracted. Overriding runtimeConfig.artifactStore.download.progressListener doesn't work because AbstractExtractor gets extracted a few times during a test and some of the code-paths revert back the postgresql-embedded's default download config settings. In reality, it makes the tool hard to use b/c the "extract" output saturates more useful information that the tester would otherwise miss. See PostgresProcess.runCmd.

Failure to delete temp files when running in Spring Context

I have wired up a PostgresService to be used in our Spring context-based tests. It is Lifecycle aware (it implements Lifecycle) and is started through start() method and stopped through a stop method which calls

process.stop();

When the test completes and the context is shutting down, I get an IllegalStateException, and the temporary files created at startup are not deleted. It seems to occur when a shutdown hook is added to clean up the files:

2016-02-26 15:09:00,992 [Thread-6] INFO de.flapdoodle.embed.process.runtime.ProcessControl - execSuccess: true [kill, -2, 69506] 2016-02-26 15:09:01,157 [Thread-6] WARN de.flapdoodle.embed.process.io.file.Files - could not delete /var/folders/22/1f6xflcj60qgv_27xwtyxr1m0000gn/T/postgresql-embed-4f4bcf51-7ba0-401d-8a1d-ecf5f55bb1ba/db-content-0ca454b3-56c3-4b75-88ba-d5ad911f7fd9. Will try to delete it again when program exits. 2016-02-26 15:09:01,159 [Thread-47] INFO com.anzcro.o3.util.db.PostgresService - STOPPED EMBEDDED POSTGRES 2016-02-26 15:09:01,159 [Thread-47] INFO org.springframework.beans.factory.support.DefaultListableBeanFactory - Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@26aa12dd: defining beans [propertyPlaceholderConfigurerBusinessBase,postgresService,...]; root of factory hierarchy Exception in thread "Thread-6" java.lang.IllegalStateException: Shutdown in progress at java.lang.ApplicationShutdownHooks.add(ApplicationShutdownHooks.java:66) at java.lang.Runtime.addShutdownHook(Runtime.java:211) at de.flapdoodle.embed.process.io.file.FileCleaner.forceDeleteOnExit(FileCleaner.java:52) at de.flapdoodle.embed.process.io.file.Files.forceDelete(Files.java:125) at ru.yandex.qatools.embed.postgresql.PostgresProcess.deleteTempFiles(PostgresProcess.java:157) at ru.yandex.qatools.embed.postgresql.PostgresProcess.stopInternal(PostgresProcess.java:117) at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:170) at de.flapdoodle.embed.process.runtime.AbstractProcess$JobKiller.run(AbstractProcess.java:243) at java.lang.Thread.run(Thread.java:745)

It looks like it's failing to register the shutdown hook that deletes the files.

yandex-qatools / postgresql-embedded Goto Github PK

postgresql-embedded's People

Contributors

Stargazers

Watchers

Forkers

postgresql-embedded's Issues

Recommend Projects

Recommend Topics

Recommend Org