qbicsoftware / core-utils-lib Goto Github PK

This input data:

`[qeana10@openBIS1605test fastq_pass]$ tree

QMFPJ005AE
-- QMFPJ005AE.fastq.gz
QMFPJ007AU
-- QMFPJ007AU.fastq.gz

...produces this error:

Executing NanoporeParser to validate Directory
Specified directory could not be validated
#/children/0/children/0/name: string [fastq_pass] does not match pattern fastq_fail
#/children/0/children/0/children/0: required key [children] not found
#/children/0/children/0/children/0/name: string [.fastq] does not match pattern Q\w{4}\d{3}[A-X][A-X0-9].*
#/children/0/children/0/children/0/file_type: string [fastq] does not match pattern fastq.gz
#/children/0/children/0/children/0: required key [children] not found
#/children/0/children/0/children/0/name: string [.fastq] does not match pattern Q\w{4}\d{3}[A-X][A-X0-9].*
#/children/0/children/0/children/0/file_type: string [fastq] does not match pattern fastq.gz
#/children/0/children/0/name: string [fastq_pass] does not match pattern fast5_pass
...

After that, the fastq_pass folder is tried to match against other folders and files in the schema, so it might look a bit confusing.
Seems to me, that the fastq_pass rule does not work, because the child rule does not work. The child rule does not work, since the barcode pattern is being searched in the file ending, not in the file name. The file type pattern check fails, too! Since this works in the non-pooling case, I assume different code is used here.

All QBiC tools should have a --quiet flag that would simply disable output to console. This could be done using log4j's API.

#39
Expected behaviour

Tests are run and show failure or not dependent on return values.

Observed behaviour

Tests fail because the initialization of a testing interface fails:

java.lang.IllegalStateException: Could not initialize plugin: interface org.mockito.plugins.MockMaker
...
Caused by: java.lang.IllegalStateException: Failed to load interface org.mockito.plugins.MockMaker implementation declared in sun.misc.CompoundEnumeration@1554909b
	at org.mockito.internal.configuration.plugins.PluginInitializer.loadImpl(PluginInitializer.java:55)
	at org.mockito.internal.configuration.plugins.PluginLoader.loadPlugin(PluginLoader.java:63)
	at org.mockito.internal.configuration.plugins.PluginLoader.loadPlugin(PluginLoader.java:48)
	at org.mockito.internal.configuration.plugins.PluginRegistry.<init>(PluginRegistry.java:17)
	at org.mockito.internal.configuration.plugins.Plugins.<clinit>(Plugins.java:14)
	at org.mockito.junit.MockitoJUnit.rule(MockitoJUnit.java:35)
	at life.qbic.cli.ToolExecutorTest.<init>(ToolExecutorTest.java:51)

Steps to reproduce

Run mvn test or have a look at Github actions.

Currently service URLs must be known to the application developer and hardcoded. We built a service registry with Consul, that provides a service registry for service discovery.

It would be nice to have a Service Factory for applications, that automatically detects available service types from the registry and provides them to the application.

I don't have the time right now to keep working on this one, but this how I found this problem:

• All unit tests passed locally using IntelliJ and OpenJDK 8.
• Travis' java compiler (OpenJDK 8) complained about not finding JavaFX symbols.
• I spent a few minutes on Google, tried a couple of things but I could not get to compile JavaFX code using Ubuntu Trusty images in Travis (OpenJDK 8).
• I changed to Oracle's JDK 8 in Travis, now the code compiled, but the jUnit ToolExecutorJavaFXTest.testWithEmptyToolProperties errored.
• I enabled debug in Maven and still got no more information.
• After inspecting the erroring jUnit, I realized that I left that jUnit there by mistake.
• Since I spent so much time figuring this one out, I thought that maybe it was worthy of writing an issue.

After that, in my frustration, I decided to simply add the @Ignore annotation.

I suggest to work on this one after this issue has been closed.

Expected Behaviour

The nanopore parser warns and does not validate data in case metadata fields are empty.

Observed Behaviour

The nanopore parser fails while trying to parse empty metadata fields, without throwing an informative warning

Steps to reproduce

Possible reasons

There are already methods that test if some of the metadata is empty (see finalizeMetadata()), but the aforementioned code runs earlier, leading to an error.

We are expanding our Java-based toolset, so it might make sense to use a generic way to read configuration files.

In order for this to be generic enough, default configuration files should be named after the tool they are related to. For instance, a tool named nexus-listener-service should have its default configuration file under ~/.nexus-listener-service.config.

To make things simple, we can make all config files to be yml files and implement this as part of the ToolExecutor mini framework. The framework would read configuration files upon tool start-up, use the annotated ...Command class to map each entry in the configuration file (if any) to its corresponding picocli parameter to create an instance of an AbstractCommand. Parameters given via the command line would of course override values found in the configuration file.

We introduced OSGI functionality with #38 to make the core-utils lib compatible with liferay 7.
However to allow for an independent backward compatible deployment it would make sense to have distinct osgi and non-osgi profiles in the pom.xml. (see https://github.com/qbicsoftware/offer-manager-2-portlet/blob/master/pom.xml as an example)
This would allow us to only set the dependencies necessary for the seperate usages.

Is your feature request related to a problem? Please describe.
Currently the NanoporeParser returns the everit ValidationException if the provided filetree doesn't follow the predefined schema.
This leads to the Issue that the applications using the NanoporeParser have to include the everit package as a dependency to interpret the exception.
Additionally, it returns different uncatched exceptions(IOException/ParserException/FileNotFoundException) dependent on why the parsing fails.

Describe the solution you'd like
Replace the ValidationException with the custom DatasetValidationException akin to the BioionformaticAnalysisParser and the MaxQuantParser.
The different Exceptions which could be generated during the parsing of a filetree, should be catched akin to the BioionformaticAnalysisParser and the MaxQuantParser and a custom DataParserException should be thrown for easier debugging.