mariadb4j / mariadb4j Goto Github PK
View Code? Open in Web Editor NEWMariaDB Embedded in Java JAR
License: Apache License 2.0
MariaDB Embedded in Java JAR
License: Apache License 2.0
While working on fusesource/leveldbjni#90, it occurred to me that a kina similar use case would make sense for MariaDB4j as well.. The mariaDB4j-core DB class should be be run and control a mysqld found on the system, e.g. in PATH, or at a location set in the DBConfigurationBuilder, if none is found on classpath to unpack.
I have been successfully using MariaDB4j until today. All of a sudden I am getting the following error:
"ERROR 1045 (28000): Access denied for user 'Grant'@'localhost' (using password: NO)"
And this exception a few lines below:
nested exception is ch.vorburger.exec.ManagedProcessException: An error occurred while running a command: create database if not exists 'test'
I'm calling the database as a Spring bean as follows:
@Bean
public void testDatabase()throws ManagedProcessException{
// Start database
DBConfigurationBuilder configBuilder = DBConfigurationBuilder.newBuilder();
configBuilder.setPort( 3310 );
configBuilder.setDataDir("/work/db");
db = DB.newEmbeddedDB( configBuilder.build() );
db.start();
db.createDB( "test" );
}
I have deleted the database files (/work/db) in case there was corruption but didn't help. I am running this on a Windows machine.
Has anyone seen this before and know what to do to correct?
First of all thank you for providing mariaDB4j. It works very well and it is a very good solution for my needs.
I have a bunch of users working on the same server. Let's say user1 test's his DAO against mariaDB4j. A folder named 'mariaDB' will be created in the temp folder of the server. After the tests are done, user2 wants to test his database tests too, but errors are occurring which are due to the directory which has already been created by user 1.
Is there a way to allow parallel execution of tests against the mariaDB database?
Doing a mvn release:prepare now suddenly fails with this error:
[INFO] --- spring-boot-maven-plugin:1.3.6.RELEASE:repackage (default) @ mariaDB4j-app ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] mariaDB4j-pom ...................................... SUCCESS [ 3.141 s]
[INFO] mariaDB4j-core ..................................... SUCCESS [ 13.064 s]
[INFO] mariaDB4j (all-in-one artifact) .................... SUCCESS [02:04 min]
[INFO] mariaDB4j-app ...................................... SUCCESS [ 20.745 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:42 min
[INFO] Finished at: 2016-07-25T02:14:10+02:00
[INFO] Final Memory: 53M/925M
[INFO] ------------------------------------------------------------------------
[INFO] Checking in modified POMs...
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git add -- pom.xml mariaDB4j-core/pom.xml mariaDB4j/pom.xml mariaDB4j-app/pom.xml
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git rev-parse --show-toplevel
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git status --porcelain .
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[WARNING] Ignoring unrecognized line: ?? mariaDB4j-app/pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? mariaDB4j-core/pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? mariaDB4j/pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? pom.xml.releaseBackup
[WARNING] Ignoring unrecognized line: ?? release.properties
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git commit --verbose -F /tmp/maven-scm-241871119.commit pom.xml mariaDB4j-core/pom.xml mariaDB4j/pom.xml mariaDB4j-app/pom.xml
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git symbolic-ref HEAD
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[INFO] Executing: /bin/sh -c cd /home/vorburger/dev/MariaDB4j/git/MariaDB4j && git push [email protected]:vorburger/MariaDB4j.git/mariaDB4j-pom refs/heads/master:refs/heads/master
[INFO] Working directory: /home/vorburger/dev/MariaDB4j/git/MariaDB4j
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] mariaDB4j-pom ...................................... FAILURE [02:54 min]
[INFO] mariaDB4j-core ..................................... SKIPPED
[INFO] mariaDB4j (all-in-one artifact) .................... SKIPPED
[INFO] mariaDB4j-app ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:55 min
[INFO] Finished at: 2016-07-25T02:14:12+02:00
[INFO] Final Memory: 19M/287M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-release-plugin:2.5.3:prepare (default-cli) on project mariaDB4j-pom: Unable to commit files
[ERROR] Provider message:
[ERROR] The git-push command failed.
[ERROR] Command output:
[ERROR] fatal: remote error:
[ERROR] vorburger/MariaDB4j.git/mariaDB4j-pom is not a valid repository name
[ERROR] Email [email protected] for help
We can see that the remote URL ([email protected]:vorburger/MariaDB4j.git/mariaDB4j-pom) is wrong - it should be without the artifactId (just [email protected]:vorburger/MariaDB4j.git), of course.
Given that there is a 2.5.0 tag from the previous release I did on May 5th, I'm struggling to understand what could have broken this. Unless I had done that tag manually because it already wasn't working then? I forgot.
I'd guess vorburger@97344bf broke it, but I did that JUST before the 2.5.0, so ... I don't think so. Anyway that weird 3.0-r1585899 is gone with the wind I think, and not coming back (no idea where it came from!), so that's moot.
Wonderinf if f8e6b64 which moved the into a non-SNAPSHOT pom.xml could have caused it to get confused like that? I'll try to move it, and we'll see.
Is it possible to deploy MariaDB4j in Maven Central to avoid build the Project everytime we want to use it? Thank you.
Please look at section 4 of the Apache v2 License, at apache.org. I believe that this binary is not eligible for the Apache v2 license as it redistributes incompatibly licenses binaries, specifically what appears to be a mysql binary. I believe you must include a NOTICE file explaining any of these 3p dependencies with incompatible licenses.
MariaDB versions in each platform ?
Change logs ?
the latest version of MariaDB used does not support fulltext research 'ngram' function.
I have this error Caused by: java.sql.SQLException: Function 'ngram' is not defined when i try to execute this command :CREATE FULLTEXT index LABEL_IDX on ACTOR (label) WITH PARSER ngram;
i used the latest version (2.2.1) of the library.
I think the problem came from MariaDB version.
Can you fix this please?
As a part of getting a error logs, i added "--log-error=error.log" location as part of my configuration for DB Startup. This is meant to redirect stderr to error.log file. Due to this Parent Process fails with the error "Caused by: ch.vorburger.exec.ManagedProcessException: Database does not seem to have started up correctly? Magic string not seen in 30000ms: mysqld: ready for connections". My guess is that parent process which is looking for string "mysqld: ready for connections" does not find as it looking for it in console, but now this is getting written to error.log.
I am wondering if anybody else has faced this issue and if there is a known workaround.
MariaDB supports JSON type starting with 10.0.16.
But I can't create JSON field here.
Do you planning to support it ?
If I try to build the latest version from source then I get a build error:
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Error configuring: org.springframework.boot:spring-boot-maven-plugin. Reason: Unable to retrieve component configurator for plugin configuration
[INFO] ------------------------------------------------------------------------
[INFO] Trace
org.apache.maven.lifecycle.LifecycleExecutionException: Error configuring: org.springframework.boot:spring-boot-maven-plugin. Reason: Unable to retrieve component configurator for plugin configuration
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:723)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.forkProjectLifecycle(DefaultLifecycleExecutor.java:1205)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.forkLifecycle(DefaultLifecycleExecutor.java:1033)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:643)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:569)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:539)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:284)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.PluginConfigurationException: Error configuring: org.springframework.boot:spring-boot-maven-plugin. Reason: Unable to retrieve component configurator for plugin configuration
at org.apache.maven.plugin.DefaultPluginManager.populatePluginFields(DefaultPluginManager.java:1368)
at org.apache.maven.plugin.DefaultPluginManager.getConfiguredMojo(DefaultPluginManager.java:724)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:468)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
... 21 more
Caused by: org.codehaus.plexus.component.repository.exception.ComponentLookupException: Unable to lookup component 'org.codehaus.plexus.component.configurator.ComponentConfigurator', it could not be created
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:335)
at org.apache.maven.plugin.DefaultPluginManager.populatePluginFields(DefaultPluginManager.java:1351)
... 24 more
Caused by: org.codehaus.plexus.component.factory.ComponentInstantiationException: Could not instanciate component: role: 'org.codehaus.plexus.component.configurator.ComponentConfigurator', implementation: 'org.codehaus.plexus.component.configurator.BasicComponentConfigurator', role hint: 'basic'
at org.codehaus.plexus.component.factory.java.JavaComponentFactory.makeException(JavaComponentFactory.java:77)
at org.codehaus.plexus.component.factory.java.JavaComponentFactory.newInstance(JavaComponentFactory.java:62)
at org.codehaus.plexus.DefaultPlexusContainer.createComponentInstance(DefaultPlexusContainer.java:1464)
at org.codehaus.plexus.component.manager.AbstractComponentManager.createComponentInstance(AbstractComponentManager.java:93)
at org.codehaus.plexus.component.manager.ClassicSingletonComponentManager.getComponent(ClassicSingletonComponentManager.java:92)
at org.codehaus.plexus.DefaultPlexusContainer.lookup(DefaultPlexusContainer.java:331)
... 25 more
Caused by: java.lang.NoClassDefFoundError: org/codehaus/plexus/classworlds/realm/ClassRealm
at java.lang.Class.getDeclaredConstructors0(Native Method)
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2493)
at java.lang.Class.getConstructor0(Class.java:2803)
at java.lang.Class.newInstance(Class.java:345)
at org.codehaus.plexus.component.factory.java.JavaComponentFactory.newInstance(JavaComponentFactory.java:44)
... 29 more
Caused by: java.lang.ClassNotFoundException: org.codehaus.plexus.classworlds.realm.ClassRealm
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at org.codehaus.classworlds.RealmClassLoader.loadClassDirect(RealmClassLoader.java:195)
at org.codehaus.classworlds.DefaultClassRealm.loadClass(DefaultClassRealm.java:255)
at org.codehaus.classworlds.DefaultClassRealm.loadClass(DefaultClassRealm.java:274)
at org.codehaus.classworlds.RealmClassLoader.loadClass(RealmClassLoader.java:214)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 34 more
Thanks for the excellent project. I've been having issues where the DB process crashes (too much load, linux OOM killer etc..) and the parent java process has no way of knowing it. I do see a ManagedProcess class, which is private to DB. It would be great if
The meanwhile quite mature and battle tested ch.vorburger.exec helpers (with the tests here) would actually make for a nice new project living separately from MariaDB4j (which would then depend on it, of course) ....
This is a draft documentation for its README: Launching external processes from Java using the raw java.lang.ProcessBuilder API directly can be a little cumbersome. The [Apache Commons Exec library][1] makes it a little easier. The [ch.vorburger.exec library][2] further extends upon Commons Exec to make it truly convenient:
ManagedProcess proc = new ManagedProcessBuilder("path-to-your-executable-binary")
.addArgument("arg1")
.addArgument("arg2")
.setWorkingDirectory(new File("/tmp"))
.setDestroyOnShutdown(true)
.setConsoleBufferMaxLines(7000)
.build();
proc.start();
int status = proc.waitForExit();
int status = proc.waitForExitMaxMsOrDestroy(3000);
String output = proc.getConsole();
proc.startAndWaitForConsoleMessageMaxMs("started!", 7000);
// use service offered by external process...
proc.destroy();
Please comment / upvote if you would find this interesting & useful?
Hi, when running MariaDB4j on a alpine container, we met error:
./mysqld: /usr/lib/libstdc++.so.6: no version information available (required by ./mysqld)
If we must depend gnu libstdc++, could we build a alpine version (based on musl libc) mysqld?
I stoped the service , then data miss ? How? success?
Currently cleanupOnExit deletes the baseDir if it thinks it is a temporary directory, which is determined with isTemporaryDirectory.
isTemoraryDirectory uses directory.startsWith(SystemUtils.JAVA_IO_TMPDIR) to determine whether it is a "temporary directory". I think there might be two different definitions of temporary directory used here though. cleanupOnExit should delete the baseDir if it is a directory which is temporary, as in around only for a short period of time. SystemUtils.JAVA_IO_TMPDIR will tell you if it is a directory for storing temporary files, not if it's a directory that can be removed without issue. The best example of this is '/tmp/', which is what we had it set to.
In fact, cleanupOnExit should not delete any directories it has not created itself.
One of my co-workers got this error when starting our Spring Boot project that uses MariaDB4j. This was on Mac OSX
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib Referenced from: /private/var/folders/w6/_r897gfx27b3bqj9_zvckvth0000gn/T/MariaDB4j/base/bin/my_print_defaults Reason: image not found
He doesn't use homebrew but uses macports which I think has things installed in different directories. Though it could happen for anyone with libs installed in different directories
The fix was to run
sudo mkdir -p /usr/local/opt/openssl
sudo ln -s /opt/local/lib /usr/local/opt/openssl/.
Not sure if this is a bug worth fixing, or just need to add some documentation tips?
https://github.com/vorburger/MariaDB4j/issues/9 mentioned Embedded Process Util (https://github.com/flapdoodle-oss/de.flapdoodle.embed.process), used by https://github.com/flapdoodle-oss/de.flapdoodle.embed.mongo and others.
I did not that, and it looks quite interesting as a foundation. As it would be essentially a rewrite, it's not something I have time for say a night next week, but raising this issue to further evaluate... some time, maybe.
https://mariadb.com/kb/en/mariadb/running-mysqld-as-root/
Is it addressed by 8da9aa6? I'm using docker. I'm about to look into the viability of just having a mysql container and talking to localhost. Already looked into running as a non-root user. No dice.
Update: I think I've figured out how to do a non-root run.
How can I get a more recent version of MariaDB for OSX when MariaDB4J starts up? Looks like it's getting 5.5.34. Any way to get something in the 10.1 series?
Hi there,
I am trying to somehow replace the H2 with Mariadb4j for tests in spring boot apps, but i'm having problems with that.
The H2 db startsup very nice out of the box if only available on classpath. It would be a great boost for popularity of this project, if it was prepared for spring boot. A lot of people are complaining about H2 incompatibility.
Trying to use this on Windows. I fail quickly with a ManagedProcessException with cause:
"mysql_install_db was not found, neither in bin/ nor in scripts/ under c:\temp\maria"
I set the base to something simple so it wasn't looking in my program files folder. It appears that the file does exist with an "exe" extension. Is this a bug or is there something else I should be doing?
On windows this class is working, but not on linux:
@SpringBootApplication
public class BootMariaScriptApplication implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(BootMariaScriptApplication.class, args);
}
public void run(String... args) throws Exception {
DB db = DB.newEmbeddedDB(9306);
db.start();
db.createDB("test");
db.run(sqlscript, "user", "password", "test");
}
22:03:14.728 [main] ERROR o.s.boot.SpringApplication - Application startup failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:675) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:690) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:321) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:957) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:946) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
at be.florentbo.maria.BootMariaScriptApplication.main(BootMariaScriptApplication.java:17) [maria-0.0.1-SNAPSHOT.jar!/:0.0.1-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_60]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_60]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_60]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_60]
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53) [maria-0.0.1-SNAPSHOT.jar!/:0.0.1-SNAPSHOT]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
Caused by: ch.vorburger.exec.ManagedProcessException: An error occurred while installing the database
at ch.vorburger.mariadb4j.DB.install(DB.java:117) ~[mariaDB4j-2.1.3.jar!/:na]
at ch.vorburger.mariadb4j.DB.newEmbeddedDB(DB.java:68) ~[mariaDB4j-2.1.3.jar!/:na]
at ch.vorburger.mariadb4j.DB.newEmbeddedDB(DB.java:83) ~[mariaDB4j-2.1.3.jar!/:na]
at be.florentbo.maria.BootMariaScriptApplication.run(BootMariaScriptApplication.java:38) [maria-0.0.1-SNAPSHOT.jar!/:0.0.1-SNAPSHOT]
at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:672) [spring-boot-1.2.5.RELEASE.jar!/:1.2.5.RELEASE]
... 11 common frames omitted
Caused by: ch.vorburger.exec.ManagedProcessException: Program /tmp/MariaDB4j/base/bin/mysql_install_db --datadir=/tmp/MariaDB4j/data/9306 --basedir=/tmp/MariaDB4j/base --no-defaults --force --skip-name-resolve (in working directory /tmp/MariaDB4j/base) failed, exitValue=1, last 100 lines of console:
/tmp/MariaDB4j/base/bin/my_print_defaults: 1: /tmp/MariaDB4j/base/bin/my_print_defaults: Syntax error: word unexpected (expecting ")")
Installing MariaDB/MySQL system tables in '/tmp/MariaDB4j/data/9306' ...
/tmp/MariaDB4j/base/bin/mysqld: 1: /tmp/MariaDB4j/base/bin/mysqld: Syntax error: ")" unexpected
Installation of system tables failed! Examine the logs in
/tmp/MariaDB4j/data/9306 for more information.
The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:
shell> /scripts/mysql_install_db --defaults-file=~/.my.cnf
You can also try to start the mysqld daemon with:
shell> /tmp/MariaDB4j/base/bin/mysqld --skip-grant --general-log &
and use the command line tool /tmp/MariaDB4j/base/bin/mysql
to connect to the mysql database and look at the grant tables:
shell> /tmp/MariaDB4j/base/bin/mysql -u root mysql
mysql> show tables
Try 'mysqld --help' if you have problems with paths. Using
--general-log gives you a log in /tmp/MariaDB4j/data/9306 that may be helpful.
The latest information about mysql_install_db is available at
http://kb.askmonty.org/v/installing-system-tables-mysql_install_db.
MariaDB is hosted on launchpad; You can find the latest source and
email lists at http://launchpad.net/maria
Please check all of the above before mailing us! And remember, if
you do mail us, you should use the /tmp/MariaDB4j/base/scripts/mysqlbug script!
at ch.vorburger.exec.ManagedProcess.checkResult(ManagedProcess.java:279) ~[mariaDB4j-2.1.3.jar!/:na]
at ch.vorburger.exec.ManagedProcess.waitForExitMaxMsWithoutLog(ManagedProcess.java:397) ~[mariaDB4j-2.1.3.jar!/:na]
at ch.vorburger.exec.ManagedProcess.waitForExit(ManagedProcess.java:367) ~[mariaDB4j-2.1.3.jar!/:na]
at ch.vorburger.mariadb4j.DB.install(DB.java:114) ~[mariaDB4j-2.1.3.jar!/:na]
... 15 common frames omitted
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:377) ~[commons-exec-1.1.jar!/:1.1]
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:46) ~[commons-exec-1.1.jar!/:1.1]
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:188) ~[commons-exec-1.1.jar!/:1.1]
22:03:14.729 [main] INFO o.s.b.c.e.AnnotationConfigEmbeddedWebApplicationContext - Closing org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext@134caa4: startup date [Fri Aug 28 22:03:10 CEST 2015]; root of context hierarchy
22:03:14.733 [main] INFO o.s.j.e.a.AnnotationMBeanExporter - Unregistering JMX-exposed beans on shutdown
Run mvn install on MariaDB4j on Fedora 24 causes:
[main] INFO ch.vorburger.mariadb4j.DB - mysqld executable: /tmp/MariaDB4j/base/bin/mysqld.exe
Tests run: 3, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.289 sec <<< FAILURE!
simulatedStartOSX(ch.vorburger.mariadb4j.StartSimulatedForAllPlatformsTest) Time elapsed: 0.07 sec <<< ERROR!
java.lang.RuntimeException: Error unpacking embedded DB
at ch.vorburger.mariadb4j.DB.unpackEmbeddedDb(DB.java:307)
at ch.vorburger.mariadb4j.StartSimulatedForAllPlatformsTest.checkPlatformStart(StartSimulatedForAllPlatformsTest.java:63)
at ch.vorburger.mariadb4j.StartSimulatedForAllPlatformsTest.simulatedStartOSX(StartSimulatedForAllPlatformsTest.java:53)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.io.FileNotFoundException: /tmp/MariaDB4j/base/bin/mysqld (Text file busy)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at org.apache.commons.io.FileUtils.openOutputStream(FileUtils.java:360)
at org.apache.commons.io.FileUtils.openOutputStream(FileUtils.java:319)
at org.apache.commons.io.FileUtils.copyToFile(FileUtils.java:1552)
at org.apache.commons.io.FileUtils.copyInputStreamToFile(FileUtils.java:1528)
at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1478)
at ch.vorburger.mariadb4j.Util.extractFromClasspathToFile(Util.java:133)
at ch.vorburger.mariadb4j.DB.unpackEmbeddedDb(DB.java:299)
... 31 more
I would like to secure the DB, does MariaDB4j needs the --skip-grant-tables argument? Can it be removed safely?
The docs show db.getConnection() as the correct way to get a connection (go figure) but that method is no longer part of the API; it would be nice to see an end-to-end example of a client project using mariadb4j so users don't have to work out the process every time they need to crank up mysql in a project.
When using setDataDir() if the path contains spaces then newEmbeddedDB throws an exception. Looks like command line arguments are not being escaped. This could be an attack vector for a malicious user who has control over the database file name.
Exception in thread "main" ch.vorburger.exec.ManagedProcessException: An error occurred while installing the database
at ch.vorburger.mariadb4j.DB.install(DB.java:117)
at ch.vorburger.mariadb4j.DB.newEmbeddedDB(DB.java:68)
...
Caused by: ch.vorburger.exec.ManagedProcessException: Program [/var/folders/m7/vwmfxlgd5655j5ysw0qbtyg00000gn/T/MariaDB4j/base/bin/mysql_install_db, "--datadir=/Users/john/my database", --basedir=/private/var/folders/m7/vwmfxlgd5655j5ysw0qbtyg00000gn/T/MariaDB4j/base, --no-defaults, --force, --skip-name-resolve] (in working directory /var/folders/m7/vwmfxlgd5655j5ysw0qbtyg00000gn/T/MariaDB4j/base) failed, exitValue=1, last 100 lines of console:
Installing MariaDB/MySQL system tables in './data' ...
/private/var/folders/m7/vwmfxlgd5655j5ysw0qbtyg00000gn/T/MariaDB4j/base/bin/mysqld: Too many arguments (first extra is '"--datadir=/Users/john/my').
150831 16:41:53 [ERROR] Aborting
While investigating and trying to reproduce https://github.com/vorburger/MariaDB4j/issues/10, and (so far..) ONLY time I managed to reproduce https://github.com/vorburger/MariaDB4j/issues/10, I've hit and investigated an issue which isn't directly related to https://github.com/vorburger/MariaDB4j/issues/10 per se - but an equally bad problem in it's own right:
In case of a failure to start the DB, from a main() method, the log shows this, and the JVM got "stuck" and never quit:
669 [main] INFO ch.vorburger.mariadb4j.DB - Starting up the database... 669 [main] INFO ch.vorburger.mariadb4j.DB - mysqld executable: /tmp/MariaDB4j/base/45063/bin/mysqld 669 [main] INFO ch.vorburger.exec.ManagedProcess - Starting Program /tmp/MariaDB4j/base/45063/bin/mysqld --no-defaults --console --skip-grant-tables --max_allowed_packet=64M --basedir=/tmp/MariaDB4j/base/45063 --datadir=/tmp/MariaDB4j/data/45063 --port=45063 --socket=/tmp/MariaDB4j/mysql.45063.sock (in working directory /tmp/MariaDB4j/base/45063) 681 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: The InnoDB memory heap is disabled 681 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Mutexes and rw_locks use GCC atomic builtins 681 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Compressed tables use zlib 1.2.3 681 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Using Linux native AIO 686 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Initializing buffer pool, size = 128.0M 691 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Completed initialization of buffer pool 694 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: The first specified data file ./ibdata1 did not exist: 694 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: a new database to be created! 694 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:32 InnoDB: Setting file ./ibdata1 size to 10 MB 694 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait... 731 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:33 InnoDB: Log file ./ib_logfile0 did not exist: new to be created 731 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile0 size to 5 MB 731 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait... 769 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140723 23:11:33 InnoDB: Log file ./ib_logfile1 did not exist: new to be created 769 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile1 size to 5 MB 769 [Thread-6] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait... 771 [main] INFO ch.vorburger.exec.ManagedProcess - Thread is now going to wait for ""mysqld: ready for connections."" to appear in Console output of process Program /tmp/MariaDB4j/base/45063/bin/mysqld --no-defaults --console --skip-grant-tables --max_allowed_packet=64M --basedir=/tmp/MariaDB4j/base/45063 --datadir=/tmp/MariaDB4j/data/45063 --port=45063 --socket=/tmp/MariaDB4j/mysql.45063.sock (in working directory /tmp/MariaDB4j/base/45063) for max. 10000ms 10838 [main] WARN ch.vorburger.exec.ManagedProcess - Timed out waiting for ""mysqld: ready for connections."" after 10000ms (returning false) Exception in thread "main" ch.vorburger.exec.ManagedProcessException: Database does not seem to have started up correctly? Magic string not seen: mysqld: ready for connections. at ch.vorburger.mariadb4j.DB.start(DB.java:141) at ch.vorburger.mariadb4j.tests.StartManyTimes.main(StartManyTimes.java:17)
It looked like a Deadlock, but isn't one in the technical sense, here's the relevant jstack:
"Thread-5" daemon prio=10 tid=0x00007ff06c00d000 nid=0x278e runnable [0x00007ff09e56c000] java.lang.Thread.State: RUNNABLE at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:272) at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) at java.io.BufferedInputStream.read(BufferedInputStream.java:334) - locked <0x00000007aef7d110> (a java.lang.UNIXProcess$ProcessPipeInputStream) at java.io.FilterInputStream.read(FilterInputStream.java:107) at org.apache.commons.exec.StreamPumper.run(StreamPumper.java:105) at java.lang.Thread.run(Thread.java:744)
Locked ownable synchronizers: - None
"Thread-4" prio=10 tid=0x00007ff0c4247000 nid=0x278c in Object.wait() [0x00007ff09e6a6000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000007aef7ae78> (a java.lang.UNIXProcess) at java.lang.Object.wait(Object.java:503) at java.lang.UNIXProcess.waitFor(UNIXProcess.java:210) - locked <0x00000007aef7ae78> (a java.lang.UNIXProcess) at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:347) at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:46) at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:188)
Locked ownable synchronizers: - None
"process reaper" daemon prio=10 tid=0x00007ff06c00a000 nid=0x275f runnable [0x00007ff09e5a5000] java.lang.Thread.State: RUNNABLE at java.lang.UNIXProcess.waitForProcessExit(Native Method) at java.lang.UNIXProcess.access$200(UNIXProcess.java:54) at java.lang.UNIXProcess$3.run(UNIXProcess.java:174) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)
Locked ownable synchronizers: - <0x00000007aea8c370> (a java.util.concurrent.ThreadPoolExecutor$Worker)
The problem is that Thread-4 is NOT a daemon, so on JVM shutdown it just waits, and waits, and waits...
https://issues.apache.org/jira/browse/EXEC-69 looks like the culprit behind this!!
I'll fix this in MariaDB4j by using the DaemonExecutor workaround attached to EXEC-69 until they fix it upstream in Apache Commons Exec.
the default should be utf8
collation_server=utf8_unicode_ci
character_set_server=utf8
Hey guys,
first of all thanks for this awesome piece of software. It helps us a lot in our current project.
There is only one problem:
Shutdown Hook Deletion Thread for Temporary DB ../tmp/devdb 2017-01-04 19:49:42 INFO [c.v.m.DB] - cleanupOnExit() ShutdownHook now stopping database
Exec Default Executor 2017-01-04 19:49:44 INFO [c.v.e.ManagedProcess] - Program [/tmp/MariaDB4j/base/bin/mysqld, --no-defaults, --console, --skip-grant-tables, --max_allowed_packet=64M, --basedir=/tmp/MariaDB4j/base, --datadir=/home/vaadin/repos/tmp/devdb, --port=4223, --socket=/tmp/MariaDB4j.4223.sock] (in working directory /tmp/MariaDB4j/base) just exited, with value 0
Shutdown Hook Deletion Thread for Temporary DB ../tmp/devdb 2017-01-04 19:49:44 INFO [c.v.e.ManagedProcess] - Successfully destroyed Program [/tmp/MariaDB4j/base/bin/mysqld, --no-defaults, --console, --skip-grant-tables, --max_allowed_packet=64M, --basedir=/tmp/MariaDB4j/base, --datadir=/home/vaadin/repos/tmp/devdb, --port=4223, --socket=/tmp/MariaDB4j.4223.sock] (in working directory /tmp/MariaDB4j/base)
Shutdown Hook Deletion Thread for Temporary DB ../tmp/devdb 2017-01-04 19:49:44 INFO [c.v.m.DB] - Database stopped.
Shutdown Hook Deletion Thread for Temporary DB ../tmp/devdb 2017-01-04 19:49:44 INFO [c.v.m.DB] - cleanupOnExit() ShutdownHook quietly deleting temporary DB base directory: /tmp/MariaDB4j/base
Exception in thread "Shutdown Hook Deletion Thread for Temporary DB ../tmp/devdb" java.lang.NoClassDefFoundError: org/apache/commons/io/FilenameUtils
at org.apache.commons.io.FileUtils.isSymlink(FileUtils.java:2924)
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1534)
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270)
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653)
at org.apache.commons.io.FileUtils.deleteQuietly(FileUtils.java:1566)
at ch.vorburger.mariadb4j.DB$1.run(DB.java:373)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.FilenameUtils
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
at org.eclipse.jetty.webapp.WebAppClassLoader.loadClass(WebAppClassLoader.java:560)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 6 more
That happens if we start our web application via the maven jetty plugin and stop it via Ctrl-C.
It does not hurt us much but it's annoying. I already tried to set the stopWait in the plugin's configuration but that did not help.
Has someone else the same problem?
Cheers,
Paul
The lib logs a lot of messages as error messages. I think this messages should be more considered as info messages.
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:12 [Note] /tmp/MariaDB4j/base/bin/mysqld: Normal shutdown
[error] c.v.e.ManagedProcess - mysqld:
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:12 InnoDB: Starting shutdown...
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: The InnoDB memory heap is disabled
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Mutexes and rw_locks use GCC atomic builtins
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Compressed tables use zlib 1.2.3
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Using Linux native AIO
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Initializing buffer pool, size = 128.0M
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Completed initialization of buffer pool
[error] c.v.e.ManagedProcess - mysqld: InnoDB: The first specified data file ./ibdata1 did not exist:
[error] c.v.e.ManagedProcess - mysqld: InnoDB: a new database to be created!
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Setting file ./ibdata1 size to 10 MB
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile0 size to 5 MB
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile1 size to 5 MB
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Doublewrite buffer not found: creating new
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Doublewrite buffer created
[error] c.v.e.ManagedProcess - mysqld: InnoDB: 127 rollback segment(s) active.
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Creating foreign key constraint system tables
[error] c.v.e.ManagedProcess - mysqld: InnoDB: Foreign key constraint system tables created
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:14 InnoDB: Waiting for the background threads to start
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:15 Percona XtraDB (http://www.percona.com) 5.5.33a-MariaDB-31.1 started; log sequence number 0
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:15 [Note] Plugin 'FEEDBACK' is disabled.
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:15 [Note] Server socket created on IP: '0.0.0.0'.
[error] c.v.e.ManagedProcess - mysqld: 150730 9:41:15 [Note] /tmp/MariaDB4j/base/bin/mysqld: ready for connections.
[error] c.v.e.ManagedProcess - mysqld: Version: '5.5.33a-MariaDB' socket: '/tmp/MariaDB4j.3333.sock' port: 3333 MariaDB Server
Hi,
When trying to execute this -
Connection connection = DriverManager.getConnection("jdbc:mysql://localhost/myDb", "root", "");
I'm getting error -
Access denied for user 'root'@'localhost' (using password: NO)
Here is my full code
static class MySqlRule extends ExternalResource {
DB db = null;
MySqlRule() {
DBConfigurationBuilder configBuilder = DBConfigurationBuilder.newBuilder();
configBuilder.setPort(3306); // OR, default: setPort(0); => autom. detect free port
try {
db = DB.newEmbeddedDB(configBuilder.build());
db.start();
db.source("db/1_create-database.sql");
db.source("db/2_create-tables.sql", "root", "", "myDb");
db.source("db/3_seed-data.sql", "root", "", "myDb");
} catch (ManagedProcessException e) {
e.printStackTrace();
}
}
@Override
protected void after() {
try {
db.stop();
} catch (ManagedProcessException e) {
e.printStackTrace();
}
}
}
mysqldump and mysqlimport should be available so we could make portable backups.
I noticed that the latest mariadb4j is failing on jitpack https://jitpack.io/com/github/vorburger/MariaDB4j/mariaDB4j-2.2.0/build.log
Maybe it could be fixed so it can be used, it seems like a small checkstyle error.
I've just noticed that @osimola fork has more interesting stuff than what he's submitted as PRs so far... ;-) In particular commit osimola@b629843 refers to something I did have in mind, but never got to:
@osimola how about instead of just removing the binaries, how about having them in a separate JAR / Maven artifact? So we'd could split up the current one into two JARs - the "code" and the "binaries". Maybe a third pom Maven artifact pulling them both, for simple use as-is-now. Plus probably a little bit of work to make the name of where on the classpath the binaries are (package name, platform dependant & version) - they're currently completely hard-coded.
Shoult if you have time & interest & would like to do that?
Many thanks vorburger, very interesting idea, I am comfortable with MySQL so this solutions suits me.
Is MariaDB4j stable enough for production? I need a similar tool/solution for all-in-one application. There won't much access on the DB, but I need the data to be safe, and I would not mind a decent performance.
The cleanupOnExit()
ShutdownHook logs warnings if the directory, it tries to delete, does not exists anymore. Currently I get warnings in my log, also if the directories doesn't exists.
[warn] c.v.m.DB - cleanupOnExit() ShutdownHook: An error occurred while deleting a directory
java.io.IOException: Unable to delete directory /tmp/MariaDB4j/data/3333.
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1541) ~[commons-io-2.4.jar:2.4]
at ch.vorburger.mariadb4j.DB$1.run(DB.java:336) ~[mariaDB4j-2.1.3.jar:na]
[warn] c.v.m.DB - cleanupOnExit() ShutdownHook: An error occurred while deleting a directory
java.io.IOException: Unable to delete directory /tmp/MariaDB4j/data/3333.
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1541) ~[commons-io-2.4.jar:2.4]
at ch.vorburger.mariadb4j.DB$1.run(DB.java:336) ~[mariaDB4j-2.1.3.jar:na]
[warn] c.v.m.DB - cleanupOnExit() ShutdownHook: An error occurred while deleting a directory
java.io.IOException: Unable to delete file: /tmp/MariaDB4j/base/share/norwegian/errmsg.sys
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at ch.vorburger.mariadb4j.DB$1.run(DB.java:340) ~[mariaDB4j-2.1.3.jar:na]
[warn] c.v.m.DB - cleanupOnExit() ShutdownHook: An error occurred while deleting a directory
java.io.IOException: Unable to delete directory /tmp/MariaDB4j/base/share.
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1541) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at ch.vorburger.mariadb4j.DB$1.run(DB.java:340) ~[mariaDB4j-2.1.3.jar:na]
[warn] c.v.m.DB - cleanupOnExit() ShutdownHook: An error occurred while deleting a directory
java.io.IOException: Unable to delete file: /tmp/MariaDB4j/base/share/swedish/errmsg.sys
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4]
at ch.vorburger.mariadb4j.DB$1.run(DB.java:340) ~[mariaDB4j-2.1.3.jar:na]
using a base dir with a space, such as
--datadir=/path/path with space/more-path
causes the install to fail. The path params should be properly quoted in order to handle this.
We try to use this package for running unit tests against a "real" local MySQL database. This also works very good so far.
The only problem is, that it doesn't seem to be able to start and stop the database sequentially.
It would be nice to have a method, like "isRunning" to check, if the database is still running. Another option would be to have an additonal stopAndWait
I use MariaDB4j to test against a MySQL compatible database inside a Play Framework 2 application. If I run the tests from the play console all works as expected. But if I run the tests inside Bamboo or IntelliJ IDEA then the MariaDB process hangs sporadically. I've tried to give more memory or use the newest MariaDB version but this doesn't help.
In IntelliJ IDEA the process exits with exit code 137 and no database process is alive. In Bamboo I must stop the build by hand and then the I must kill the MariaDB process. I run the tests on Linux machines. The processes have no high memory or CPU consumption.
This is the log from a typical MariaDB start:
10:44:37.859 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: The InnoDB memory heap is disabled
10:44:37.861 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Mutexes and rw_locks use GCC atomic builtins
10:44:37.861 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Compressed tables use zlib 1.2.3
10:44:37.861 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Using Linux native AIO
10:44:37.861 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Initializing buffer pool, size = 128.0M
10:44:37.864 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Completed initialization of buffer pool
10:44:37.866 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: The first specified data file ./ibdata1 did not exist:
10:44:37.866 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: a new database to be created!
10:44:37.866 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Setting file ./ibdata1 size to 10 MB
10:44:37.866 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
10:44:37.868 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Log file ./ib_logfile0 did not exist: new to be created
10:44:37.868 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile0 size to 5 MB
10:44:37.868 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
10:44:37.869 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Log file ./ib_logfile1 did not exist: new to be created
10:44:37.869 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Setting log file ./ib_logfile1 size to 5 MB
10:44:37.869 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Database physically writes the file full: wait...
10:44:37.870 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Doublewrite buffer not found: creating new
10:44:37.875 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Doublewrite buffer created
10:44:37.879 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: 127 rollback segment(s) active.
10:44:37.880 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Creating foreign key constraint system tables
10:44:37.882 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: InnoDB: Foreign key constraint system tables created
10:44:37.882 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:37 InnoDB: Waiting for the background threads to start
10:44:38.890 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:38 Percona XtraDB (http://www.percona.com) 5.5.37-MariaDB-34.0 started; log sequence number 0
10:44:38.891 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:38 [Note] Plugin 'FEEDBACK' is disabled.
10:44:38.891 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:38 [Note] Server socket created on IP: '0.0.0.0'.
10:44:38.895 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: 140514 10:44:38 [Note] /tmp/mariaDB4j_8f233556-fd08-47e6-816f-69d3863d8fb0/3333/bin/mysqld: ready for connections.
10:44:38.895 [Thread-5] ERROR ch.vorburger.exec.ManagedProcess - mysqld: Version: '5.5.37-MariaDB' socket: '/tmp/mariaDB4j_8f233556-fd08-47e6-816f-69d3863d8fb0.mariadb.sock' port: 3333 MariaDB Server
If the error occurs, then the log stops always with one of the above log entries. It never stops with the same log entry. If the Bamboo build hangs I can connect to the database process and view the database structure. If I kill the database process then the Bamboo build resumes with failed test results. There exists no socket file If the build process hangs but the database is accessible over the port 3333.
Any idea what can solve this issue?
Please upgrade the support to 10.x also for Linux. Thx.
I had had some fun re. coming up with something better than Apache Commons Exec in https://github.com/vorburger/MariaDB4j/tree/master/src/main/java/ch/vorburger/exec (and it works).
I've stumbled over https://github.com/zeroturnaround/zt-exec which looks similar, probably better - may be one day I could switch over to that.
This looks neat http://www.javadoc.io .. should add this to README.
There is even a Badge for it, seen e.g. on https://github.com/awaitility/awaitility
When using dbInstance.source(..) method with an empty string as password a prompt is shown asking for the password.
db.source("sql/schema.sql", "root", "", "test_db");
I think the method should either validate input and throw an error when an empty string is supplied or treat it the same as a null value.
How to listen binlog ?
I have problem with MariaDB4j . When I run it as Junit test , I get this error :
Exception in thread "Shutdown Hook Deletion Thread for Temporary DB C:\Users\petroski\AppData\Local\Temp\MariaDB4j\data\53163" java.lang.NoSuchMethodError: org.apache.commons.io.FileUtils.deleteQuietly(Ljava/io/File;)Z
at ch.vorburger.mariadb4j.DB$1.run(DB.java:369)
Any help with this issue ?
Thank you
Linux native dependency includes two bundled libraries libjemalloc and libeaio.
If we change BaseLib from default these libraries would not be found and in order to fix that LibDir should be set relative to BaseLib. To reproduce - run it on linux with basedir set to something other then SystemUtils.JAVA_IO_TMPDIR + "/MariaDB4j/base"
configBuilder = DBConfigurationBuilder.newBuilder();
configBuilder.setBaseDir(dbFolder.getAbsolutePath());
configBuilder.setLibDir(new File(dbFolder, "libs").getAbsolutePath()); // bundled libs are not found if LibDir is not set to this relative path
It is due to code in DBConfigurationBuilder constructor
this.baseDir = SystemUtils.JAVA_IO_TMPDIR + "/MariaDB4j/base";
this.libDir = this.baseDir + "/libs";
If you leave libDir as null on initialization and change implementation of getLibDir I would this it would do the trick
public String getLibDir() {
if (this.libDir == null)
return this.baseDir+"/libs"
else
return this.libDir;
}
Hi.
I noticed the comment "Code review comments most welcome; I'm not 100% sure the thread concurrency time is right; is there a chance a console message may be "missed" here, and we block forever?" in ManagedProcess.java.
There is: If the line is written after you take the copy of console log in getRecentLines but before you initialize the CheckingConsoleOutputStream, it will be missed. A better way to do this would be initializing CheckingConsoleOutputStream first: this way the message line may be read twice but never missed.
Additionally, I think the lines
stdouts.removeOutputStream(checkingConsoleOutputStream);
stderrs.removeOutputStream(checkingConsoleOutputStream);
should be in a finally block so that they will be executed even if Thread.sleep() on the line before is interrupted.
You could also skip the whole sleep and poll loop and use a latch from java.util.concurrent instead.
And finally, CircularFifoBuffer is unsynchronized by default, access to it should be synchronized.
Hi,
I'm having a problem because my windows user is "Daniel SMITH" and the ConfigurationBuilder don't recognize it:
ERROR:
ch.vorburger.exec.ManagedProcessException: Asked to wait for "mysqld.exe: ready for connections." from Program [C:\Users\DANIEL1\AppData\Local\Temp\MariaDB4j\base\bin\mysqld.exe, --no-defaults, --console, --skip-grant-tables, --max_allowed_packet=64M, --basedir="C:\Users\Daniel SMITH\AppData\Local\Temp\MariaDB4j\base", --datadir="C:\Users\Daniel SMITH\AppData\Local\Temp\MariaDB4j\data\3307", --port=3307] (in working directory C:\Users\DANIEL1\AppData\Local\Temp\MariaDB4j\base), but it already exited! (without that message in console), last 100 lines of console:
...........
C:\Users\DANIEL~1\AppData\Local\Temp\MariaDB4j\base\bin\mysqld.exe: Too many arguments (first extra is 'SMITH\AppData\Local\Temp\MariaDB4j\base').
170119 16:56:13 [ERROR] Aborting
If I configure the ConfigurationBuilder with baseDir, example configurationBuilder.setBaseDir("C:/mariadb");
I have this ERROR:
ch.vorburger.exec.ManagedProcessException: Program [C:\mariadb\bin\mysqld.exe, --no-defaults, --console, --skip-grant-tables, --max_allowed_packet=64M, --basedir=C:\mariadb, --datadir=C:\mariadb, --port=3307] (in working directory C:\mariadb) failed, exitValue=1, last 100 lines of console:
............
170119 18:07:41 [Note] InnoDB: Completed initialization of buffer pool
170119 18:07:41 [ERROR] InnoDB: .\ibdata1 can't be opened in read-write mode
170119 18:07:41 [ERROR] InnoDB: The system tablespace must be writable!
170119 18:07:41 [ERROR] Plugin 'InnoDB' init function returned error.
170119 18:07:41 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
Thanks
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.