nosan / embedded-cassandra Goto Github PK

At the moment, it is not possible to choose to not delete the working directory when stopping the embedded Cassandra. In my use case (drop in replacement for a real Cassandra), I still need persistence of the stored data, even when using the embedded Cassandra.

As far as I have seen, there has been an option to configure that behavior in the past. Would it be possible to reimplement that?

Hi :)

first of all thanks for your work :)
I tried to integrate it in a spring project and in some tests it worked fine, but in some cases it throws a NoSuchMethodException.
im not sure how to reproduce it but it looks like spring is looking for a default constructor in EmbeddedCassandraConfiguration$EmbeddedClusterFactoryBean which is missing.
i am using Spring boot 2.1.0 M2/Spring 5.1.0.RC2

any chances you know how to fix this? Maybe its really just a missing constructor?
thank you in advance :)

Here's the stacktrace:

Caused by: java.lang.NoSuchMethodException: com.github.nosan.embedded.cassandra.spring.EmbeddedCassandraConfiguration$EmbeddedClusterFactoryBean.()
at java.lang.Class.getConstructor0(Class.java:3082) ~[na:1.8.0_181]
at java.lang.Class.getDeclaredConstructor(Class.java:2178) ~[na:1.8.0_181]
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:78) ~[spring-beans-5.1.0.RC2.jar:5.1.0.RC2]
... 66 common frames omitted

Hi! First of all, thanks for such a valuable project!

I need to set some environment variables to specify how much memory Cassandra should use (MAX_HEAP_SIZE and HEAP_NEWSIZE), but it seems that is no API for specifying them.

Would you consider this feature?

I need to add some jar on class path of Casandra before Cassandra starts. How to do that?

Hi,

is it possible to run EmbeddedCassandra with random ports?
We use different test tags and it looks like gradle runs each tag in parallel, even though parallel build is disabled, so we get the following error:
java.net.BindException: Address already in use: JVM_Bind
because the port is already used

I saw i can set the ports in v2 and tried to set it to 0, but then spring doesn't know the (random) ports.

Is it possible to randomize the ports somehow? And if not, do you plan to support this in future versions?

Thanks in advance

Using the TestCassandra there's no way to register a type-codec before the session is built and returned.

Currently the only way is a workaround:

        CqlSession session = (CqlSession) cassandra.getConnection().get();
        CqlSession customizedSession = CqlSession.builder()
                .withConfigLoader(session.getContext().getConfigLoader())
                .addTypeCodecs(new MyTypeCodec())
                .build();
        session.close();

I'm working in an embedded application which will handle high volume of data and I'm think to use cassandra for persistence layer.

Is this project production tier or is it just usable for unit test?

Hi...

I am using

  val cassandra = new Cassandra()

not sure how to set a port

Regards

Hey there,

For one of the tests, I have to use embedded-cassandra of yours as I have done the migration of the project from JDK 8 to 11. I have put the annotation @EmbbededCassandra on one of the test class but I am getting the following exception. Can you give me some more insights what I have to do? I am using the test dependency only which u have mentioned. I am using Spring boot 2.x.x and JDK 11.

Caused by: java.io.IOException: Cassandra Process is not alive. Please see logs for more details. Last (5) lines: [0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/tmp/embedded-cassandra-b1948e73-cdf5-4284-8e55-35f6b958d600/apache-cassandra-3.11.3/bin/../logs/gc.log instead. intx ThreadPriorityPolicy=42 is outside the allowed range [ 0 ... 1 ] Improperly specified VM option 'ThreadPriorityPolicy=42' Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. at com.github.nosan.embedded.cassandra.local.DefaultCassandraProcess.throwException(DefaultCassandraProcess.java:366) at com.github.nosan.embedded.cassandra.local.DefaultCassandraProcess.lambda$await$1(DefaultCassandraProcess.java:272) at com.github.nosan.embedded.cassandra.local.WaitUtils.await(WaitUtils.java:48) at com.github.nosan.embedded.cassandra.local.DefaultCassandraProcess.await(DefaultCassandraProcess.java:270) at com.github.nosan.embedded.cassandra.local.DefaultCassandraProcess.start(DefaultCassandraProcess.java:172) at com.github.nosan.embedded.cassandra.local.LocalCassandra.startInternal(LocalCassandra.java:242) at com.github.nosan.embedded.cassandra.local.LocalCassandra.lambda$start$0(LocalCassandra.java:147)

Hi,
in

the default Proxy is NO_PROXY, for example when using the simple
Artifact.ofVersion("XXX").

Sometimes, the same code must be run with or without a proxy. In such cases, It's nice that HttpURLConnection automatically picks up the -DproxyHost and -DproxyPort.

Maybe you could leave the proxy to null and in the connect method do
if (proxy != null) {
url.openConnection(proxy);
} else {
url.openConnection();
}
?

Cheers,
Jon

see gh-32 for more details.

Currently CqlExecutionListener and EmbeddedCassandraContextCustomizer use
ApplicationContext.getBeanProvider(...) which was added in Spring 5.1.0. It would be nice to have backward compatibility with Spring 5.0.X.

A single test case is working fine, the same test case is throwing the following error when running with multiple test cases.
Tried different stable cassandra versions but facing the same issue.

Caused by: com.github.nosan.embedded.cassandra.CassandraException: Unable to await: DefaultCassandra{name='cassandra-0', version='4.0-beta4'} at com.github.nosan.embedded.cassandra.DefaultCassandra.await(DefaultCassandra.java:257) at com.github.nosan.embedded.cassandra.DefaultCassandra.start(DefaultCassandra.java:97) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1914) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1856) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1784) ... 148 common frames omitted Caused by: java.io.IOException: 'UnixCassandraDatabase{process=java.lang.UNIXProcess@397fac68}' is not alive. Please see logs for more details.

Using embedded-cassandra-spring-boot-starter 4.x.x with yaml:

#Cassandra config file.
cassandra.embedded.config-file=classpath:cu-cassandra.yaml
#Config properties, that should be merged with properties from cassandra.yaml.
cassandra.embedded.config-properties.native_transport_port=9042
#Cassandra environment variables.
cassandra.embedded.environment-variables.JAVA_HOME=${JAVA8_HOME}
#Logger name, that consumes Cassandra STDOUT and STDERR outputs.
cassandra.embedded.logger=Cassandra
#Cassandra instance name.
cassandra.embedded.name=cassandra-0
#Sets if the created Cassandra should have a shutdown hook registered.
cassandra.embedded.register-shutdown-hook=true
#Startup timeout.
cassandra.embedded.startup-timeout=2m
#Cassandra native Java Virtual Machine (JVM) system parameters.
cassandra.embedded.system-properties.[cassandra.jmx.local.port]=7199
#Cassandra working directory.
cassandra.embedded.working-directory=target/cassandra-latest-version
#Additional resources, that should be copied into the working directory.
cassandra.embedded.working-directory-resources.[conf/cassandra.yaml]=classpath:cu-cassandra.yaml

Hi

Would it be possible to add a functionality which allows user to check that if Embedded Cassandra is already running then don't start a new version.

My test cases are organised into specs and the runner runs specs in parallel. Due to parallelism, each spec starts its version of Embedded Cassandra which overwhelms the machine. I can't order the execution of tests. So it will help me greatly if only one instance is stared and others just use it.

thanks

Correct me if I wrong but this project seems not allows to start Cassandra instance as embedded service. By "embedded" I assume that it should be started within same JVM from which it was started. In fact it use external java.lang.Process which is more like Java wrapper for Cassandra process.
If it so then probably better to mention this in project readme for not to confuse people.

could be, that i have misunderstood something, but I am getting an error, if I use embedded-cassandra together with datastax-driver 4.5.1 and cassandra 4.0-alpha4.
as soon as I do start the test, I do get an error, that the keyspace configured in my application.conf (outside of the test-directory) is not existing. this is true, because on starting the embedded-cassandra, no keyspace is available. Now I have removed this setting (session-keyspace) from my application.conf and create the keyspace via a CqlDataset. This seems to work fine, but I would like to use the keyspace in the application.conf to be able to make the keyspace configurable.
There any way, to start the driver for the initial CQL-Loading without those settings, and only later use this config in a second phase, when I do insert and select data?
my test setup (using kotlin) and junit5 is looking something like:

companion object {
    @JvmField
    var cqlSessionCassandraConnectionFactory = CqlSessionCassandraConnectionFactory()

    val logger = LoggerFactory.getLogger(RepositoryTest::class.java)

    @JvmField
    @RegisterExtension
    val cassandraExtension = CassandraExtension()
        .withCassandraFactory(EmbeddedCassandraFactory().apply {
            setArtifact(Artifact.ofVersion("4.0-alpha4"))
        })
        .withCqlDataSet(CqlDataSet.ofClasspaths("test.cql"))

    @BeforeAll
    @JvmStatic
    internal fun beforeAll() {
        // add keyspace to cqlSession to avoid using keyspace in queries
        cqlSessionCassandraConnectionFactory.sessionBuilderCustomizers +=
            Consumer { it.withKeyspace("kcd") }
    }
}

great project by the way. i do like it.

Hi,
I'm using @EmbeddedCassandra and I want to start only one cassandra for a bunch of classes. If the context is cached between two classes (when they have the exact same context configuration), then I get what I want.

However, it's hard to make sure that all classes have the same context (for example, using @MockBean forces a new context creation for each test class). I think that using @ContextHierarchy is the proper solution here, with a parent context holding the @EmbeddedCassandra beans and child context holding the @MockBeans getting recreated. Do you have a working example of using @ContextHierarchy to use a single cassandra instance accross multiple classes ? You could also add it to the docs.

Thanks,
Jon

Hi,

I have a scala project that I build under java 11 and my integration tests fail due to embedded-cassandra's lack of support. Do you have any plans for making it possible to run embedded-cassandra under java11?

Currently, Apache Cassandra does not support JDK > 8 versions.

I am using Junit4 ClassRule and I am seeing this issue:

[2020-03-06 18:07:24] [main] INFO  c.g.n.e.cassandra.EmbeddedCassandra - EmbeddedCassandra[name='cassandra-1', version='3.11.6'] has been started and ready for connections!
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.881 sec <<< FAILURE!
mypackageandclass  Time elapsed: 4.881 sec  <<< ERROR!
com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (no host was tried)
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:84)
        at com.datastax.driver.core.exceptions.NoHostAvailableException.copy(NoHostAvailableException.java:37)
        at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
        at com.datastax.driver.core.DefaultResultSetFuture.getUninterruptibly(DefaultResultSetFuture.java:245)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:68)
        at com.datastax.driver.core.AbstractSession.execute(AbstractSession.java:43)
        at com.github.nosan.embedded.cassandra.api.connection.ClusterCassandraConnection.execute(ClusterCassandraConnection.java:52)
        at com.github.nosan.embedded.cassandra.api.connection.ClusterCassandraConnection.execute(ClusterCassandraConnection.java:33)
        at java.util.ArrayList.forEach(ArrayList.java:1257)
        at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1082)
        at com.github.nosan.embedded.cassandra.junit4.test.CassandraRule.before(CassandraRule.java:219)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)

My rule looks like this:

    public static final CassandraRule TEST_CASSANDRA_RULE = new CassandraRule(cassandraFactory -> {
        cassandraFactory.setArtifact(Artifact.ofVersion(Version.of("3.11.6")));
        cassandraFactory.getJvmOptions().add("-Xmx1g");
        cassandraFactory.getJvmOptions().add("-Xms1g");
        cassandraFactory.setPort(9042);
    }).withCqlDataSet(CqlDataSet.ofClasspaths("persistence/ddl.cql"));

I am on 3.11.6 and 3.0.2 version of this library.

Sometimes 1 min is not enough to download an archive (e.g. slow speed, VPN, etc.) in concurrency execution mode, tests could be failed due to the small wait time.

Hi
I have found out, that the usage of OpenJDK is not working for embedded-cassandra-junit5-test:3.0.3 under Windows 10. However it is working perfectly on Linux.

It is perfectly reproducible, as soon as one is selecting the Oracle JDK via the

EmbeddedCassandraBuilder()
withArtifact(Artifact.ofVersion("4.0-alpha4"))
.withJavaHome(Paths.get("C:\\javasoft\\oracle\\Java\\jre1.8.0_251"))

the Junit 5 Test it running. If one select any OpenJDK ( no mather what Version between 8 and 13 ) it is not working. Instead it shows the following error when used

• with OpenJDK 13:
  -> Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or newer). Java 13 is not supported.
• with OpenJDK 11:
  ->Cassandra 4.0 requires either Java 8 (update 151 or newer) or Java 11 (or newer). Java 11.0.6 is not supported.

Thank you, Regards Markus

Hi,
In the docs, it says

4.4. Run with Java 13
There are several ways to use Java 13:
Use Apache-Cassandra 4.0-alpha3 version

But a few days ago (2020-04-24) 4.0-alpha4 was published and 4.0-alpha3 removed.
Maybe adding version discovery functionality would be good ? What do you think ? The simplest idea would be a "LATEST", but more complex versions are also possible. This would prevent things from breaking when new versions are released (at the cost of making the tests not very reproducible)

Cheers,
Jon

Hi, it's me again

it looks like now it fails when starting with Spring Boot 2.0.4/Spring 5.0.8
i guess Spring needs a Default Constructor (Maybe annotated with Autowired Annotation?)
not sure what i can do here

here's the stack trace:

Caused by: java.lang.IllegalArgumentException: No visible constructors in class com.github.nosan.embedded.cassandra.spring.EmbeddedCassandraConfiguration$EmbeddedClusterFactoryBean
at org.springframework.cglib.proxy.Enhancer.filterConstructors(Enhancer.java:666) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.cglib.proxy.Enhancer.generateClass(Enhancer.java:567) ~[spring-core-5.0.8.RELEASE.jar:5.0.8.RELEASE]

Currently, ClusterFactory and CqlSessionFactory don't provide an easy way to configure SSL and Authentication. Folks have to extends these classes and override:

protected Cluster buildCluster(Cluster.Builder builder) {
  	return builder.build();
  }

or

	protected CqlSession buildCqlSession(CqlSessionBuilder sessionBuilder) {
		return sessionBuilder.build();
	}

	protected DriverConfigLoader buildDriverConfigLoader(ProgrammaticDriverConfigLoaderBuilder driverBuilder) {
		return driverBuilder.build();
	}

it is a bit awkward and would be great to provide a way to configure these options via setters

I'm not seeing a way to override the URL for the embedded Cassandra download link when using the EmbeddedCassandra annotation in unit tests. I'm behind a proxy and would like to place the binary in my company's internal storage so I don't need to mess with a proxy in my CI/CD pipeline.

Is there a simple way to do this? I'm using SpringBoot 2 and JUnit 5.

Hi, thanks for the great work. I'm using the @EmbeddedCassandra (v 1.1.1 of the lib) annotation in my integration test, and it is working perfectly when running locally at my Mac. However, when trying to run the test as part of the build process on Bamboo it complains about being run as root, and Cassandra won't start.

Running Cassandra as root user or group is not recommended - please start Cassandra using a different system user.
If you really want to force running Cassandra as root, use -R command line option.

Is there any common solution to get past this? By configuration, or even being able to supply the -R option in some way.

I have tried using Cassandra 4.0-beta4 with embedded-cassandra 4.0.1 on Windows 10 which fails giving the following error:

com.kobil.ssms.ast.storage.lib.CassandraStorageTest *** ABORTED ***
  com.github.nosan.embedded.cassandra.CassandraException: Unable to await: DefaultCassandra{name='cassandra-0', version='4.0-beta4'}
  at com.github.nosan.embedded.cassandra.DefaultCassandra.await(DefaultCassandra.java:257)
  at com.github.nosan.embedded.cassandra.DefaultCassandra.start(DefaultCassandra.java:97)
  at com.kobil.ssms.ast.storage.lib.CassandraStorageTest.beforeAll(CassandraStorageTest.scala:32)
  at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:212)
  at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
  at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
  at com.kobil.ssms.ast.storage.lib.CassandraStorageTest.run(CassandraStorageTest.scala:25)
  at org.scalatest.Suite.callExecuteOnSuite$1(Suite.scala:1175)
  at org.scalatest.Suite.$anonfun$runNestedSuites$1(Suite.scala:1222)
  at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
  ...
  Cause: java.io.IOException: 'WindowsCassandraDatabase{process=Process[pid=9552, exitValue=1]}' is not alive. Please see logs for more details.

Errors:

C:\Users\JOHANN~1.LEU\AppData\Local\Temp\apache-cassandra-4.0-beta4-8590206010260606935\bin\cassandra.ps1 : The term 
'C:\Users\JOHANN~1.LEU\AppData\Local\Temp\apache-cassandra-4.0-beta4-8590206010260606935\bin\cassandra.ps1' is not recognized as the name of a cmdlet, function, script file, or operable

This is most definitely caused by the cassandra.ps1 script not being part of the 4.0-beta4 package.

I suppose this is not under your control, but the incompatibility should be at least documented. At the moment, the documentation states that embedded-cassandra supports Windows, MacOS and Linux (Ubuntu) AND Cassandra versions 2.x-4.x. I hope for a solution for that problem in the future, but for now I understand that it is impossible to fix.

Hi,

why is this not exposed and only some ports are?

It would be nice to have this exposed in Cassandra interface because when I am integrating this with other tooling which is talking to multiple Cassandra instances, I want to be able to distinguish one port / instance from the other.