ihtsdo / delta-generator-tool Goto Github PK

Hi,

Upon testing the tool, we have found and issue with how path separators are used in the resulting archive.

Generating a delta RF2 using Windows will register the individual file paths like this:

/SnomedCT_InternationalRF2_PRODUCTION_20211130T120000Z\Delta\Terminology\sct2_Concept_Delta_INT_20211130.txt

Whereas the official SNOMED CT release archive uses the standardized format:

/SnomedCT_InternationalRF2_PRODUCTION_20210731T120000Z/Delta/Terminology/sct2_Concept_Delta_INT_20210731.txt

This inconsistency most likely comes from using File.separator and ZipEntry, both being system dependant when it comes to path separators.

According to the official ZIP file format specification:

4.4.17.1 The name of the file, with optional relative path.
The path stored MUST NOT contain a drive or
device letter, or a leading slash. All slashes
MUST be forward slashes '/' as opposed to
backwards slashes '' for compatibility with Amiga
and UNIX file systems etc. If input came from standard
input, there is no file name field.

It might seem minor, but in case someone uses this tool to create a delta RF2 archive, which is then eventually fed into another tool expecting UNIX style zip entries, file processing could fail.

Some useful stackoverflow discussions:

• Zip file is created with windows path separator
• File separators of Path name of ZipEntry?

Snippet to list path of zip entries:

public static void main(String[] args) throws IOException {
	try (FileSystem zipfs = FileSystems.newFileSystem(Paths.get("C:\\path\\to\\SnomedCT_InternationalRF2_PRODUCTION_20210731T120000Z.zip"), null)) {
		for (final Path path : zipfs.getRootDirectories()) {
			Files.walkFileTree(path, new SimpleFileVisitor<Path>() {
				@Override 
				public FileVisitResult visitFile(final Path path, final BasicFileAttributes attrs) throws IOException {
					System.out.println(path.toAbsolutePath().toString());
					return FileVisitResult.CONTINUE;
				}
			});
		}
	}
}

Regards,

Gábor

Hi,

I've just installed the newest version of Java and when I try to execute DeltaGeneratorTool I get the following error.

Any help would be most appreciated.

Regards,

C:\Prueba delta>java -jar DeltaGeneratorTool.jar 20210731 SnomedCT_InternationalRF2_PRODUCTION_20220228T120000Z.zip
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.UnsupportedClassVersionError: org/snomed/otf/delta/DeltaGeneratorTool has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$100(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.launcher.LauncherHelper.checkAndLoadMain(Unknown Source)

Background
The current implementation of the delta generation tool creates archives that can contain multiple states of each component. This is like a filtered full file.

For example, between 20210731 and 20220131 the International Edition has 848 relationships that changed more than once (because of internal monthly releases). There are also 438 axioms that have more than one published state between these dates.

Problem
Tools that process RF2 delta files don't expect to receive multiple states of each component. This reduces the usefulness of this tool. Arguably tools that process RF2 deltas should not have to process multiple states for each component. For example if you try to load one of these multi-state delta files into Snowstorm the import will fail silently resulting in multiple concurrent states of components in the same commit.

Enhancement
It would be great to give implementors the choice of creating a delta with only the latest state of each component. That would make this tool really useful for creating deltas that tools expect and can process quickly.