Why do you need the classes extracted?

from capsule.

• When my (war, in this case) project contains classes of it's own that are not part of a library
• In the case the current project needs to override/patch any classes from a dependent jar. This is unfortunately a very common use-case working with libraries which for one reason or another cannot be changed upstream.

from capsule.

Also META-INF is not unpacked. There are a few resource store in the manifest that I need at runtime.

What's the strategy behind this selective exclusion? Is there anything else that is filtered out that might be of use? Or is the strategy that fat capsule shouldn't use the unpack option?

from capsule.

The strategy is in this method:

    private static boolean shouldExtractFile(String fileName) {
        if (fileName.equals(Capsule.class.getName().replace('.', '/') + ".class")
                || (fileName.startsWith(Capsule.class.getName().replace('.', '/') + "$") && fileName.endsWith(".class")))
            return false;
        if (fileName.endsWith(".class"))
            return false;
        if (fileName.startsWith("capsule/"))
            return false;
        final String dir = getDirectory(fileName);
        if (dir != null && dir.startsWith("META-INF"))
            return false;
        return true;
    }

So class files and META-INF aren't extracted, but that's because the capsule jar itself is on the classpath so any simple file in it can be directly accessed even if not extracted. Why is that an issue?

from capsule.

I was under the impression the entire archive/capsule was unpacked and run completely individually from the archive itself. If that's the case then all resources needs to be extracted, classes in particular.

Somehow, I'm starting to think that's not its intended use, rather something more in line with, run the application in the archive, but extract jar-files to share jar-cache between different capsules .. or something along those lines. Not sure that makes a lot of sence for fat jars?

from capsule.

Ideally, we'd only want to extract those files that can't be accessed by the classpath if embedded in the capsule jar. But why does it make a difference? If the classpath is set correctly, the app should be oblivious to whether files are extracted or not. If it isn't transparent, please explain why, so that the policy can be changed.

from capsule.

Due to how I perceived the workings of the cache directory the classpath isn't setup correctly I suspect. Or rather, I haven't changed it at all. I use a few of those SomeClass.getClass().getResource("/META_INF/manifest.mf") statements to pick up classpath resources but ended up with a NPE and got a bit stumped.

Wouldn't SomeClass.getClass().getResource("/META_INF/manifest.mf") return a path outside of the capsule archive if SomeClass was a in a jar-file that was extracted to the cache directory?

from capsule.

I think your mistake is doing getResource("/META_INF/manifest.mf") rather than getResource("META_INF/manifest.mf"). Can you try it without the leading '/' (which, I believe, is wrong in any case)?

from capsule.

That works if I manually copy the META-INF directory file to the extract location.

It seems to me either all resources or none needs to be extracted to have this working.

from capsule.

Something is wrong, then. What is the classpath your application sees (System.getProperty("java.class.path"))? Isn't the capsule jar there?

from capsule.

java -jar D:\...\foo\target\foo_0.1-SNAPSHOT.war

The classpath prints as

D:\...\foo\target\foo_0.1-SNAPSHOT.war
C:\...\capsule\apps\foo_0.1-SNAPSHOT
C:\...\capsule\apps\foo_0.1-SNAPSHOT\someDepency.jar
C:\...\capsule\apps\foo_0.1-SNAPSHOT\javax.servlet-api-3.1.0.jar
C:\...\capsule\apps\foo_0.1-SNAPSHOT\javax.servlet.jsp-2.3.2.jar
etc.

The classpath thus points to the war-file and all extracted jar-files.

from capsule.

Then how come getResource doesn't find the files? Is it run by a different classloader?

from capsule.

I just struck me, neither of the libraries in WEB-INF/lib/*.jar are printed as being part of the classpath. Seeing as I try to achieve a runnable war, I have all the actual libs in WEB-INF/lib with only capsule and jetty in the root of the capsule archive.

from capsule.

Because they shouldn't be. The servlet container should add them to its classloader.

from capsule.

Although, you could add "WEB-INF/lib" to the App-Class-Path attribute, but I don't think that's what you want. WEB-INF should strictly be the responsibility of the servlet container (be it Jetty or an external Tomcat or something).

from capsule.

That is true. Indeed they are found since the application is run properly. Only the controller I use within a library in WEB-INF/lib that uses this getResource fails. There's a chance that's got to do with the embedded jetty configuration too.

from capsule.

Yeah, I'd check to make sure that controller is run with a correct class loader. Do this:

System.out.println(Arrays.toString(((URLClassLoader)MyController.class.getClassLoader()).getURLs()));

from capsule.

Yes that does indeed contain WEB-INF/classes and every single library in WEB-INF/lib.

One jar-file in WEB-INF/lib performs the MyController.class.getResources("META-INF/manifest.mf") which should return the manifest file from the jar-file itself, but instead the manifest from the capsule archive root is retrieved.

from capsule.

getResources("META-INF/manifest.mf") should return (an enumeration of) all JAR manifests found on the classpath. You can never assume there's only one, or know in which order they'll be returned. This is true regardless of what Capsule does.

from capsule.

getResourceAsStream I mean, but yeah you're probably right. It's just that these conditions never occur when dropping the war in an application server.

That leaves either a classpath setting or the jetty configuration to blame I suppose. Perhaps the latter is plausible seeing as /WEB-INF/classes/ is available to the classloaded but classes therein doesn't take precedence over classes in jar-files (which otherwise is the case when dropping the war-file directly in e.g., tomcat or jetty).

I will look into the jetty configuration for additional clues. So far you've been a true champion and I really appreciate your flawless effort!

from capsule.

It's just that these conditions never occur when dropping the war in an application server.

It's not that they never occur, but that the Java Servlet specification recommends (though, does not require) that a web module's class loader look in the local class loader before delegating to its parent. That getResourceAsStream assumes that's the behavior. Indeed, if the class in question is loaded by Jetty (as it should be), you should check the Jetty classloader configuration. But first you must make sure that Jetty's classloader is the one that's loaded the class calling getResourceAsStream.

from capsule.

Yeah! I'll play around with it and post the results for posterity when I get the chance. I've got the feeling the classloader configuration is the key to both problems, overriding classes and reading a jars' own manifest

from capsule.

If any changes are required from Capsule, if you let me know about them in the next couple of days, I'll be able to get them into v0.8.0 due next week.

from capsule.

With some experimentation (brace yourself) there are some interesting discrepencies between the various containers.

Legend:

Directories

$p = The project base directory
$c = The archive in the capsule cache directory
$j = The archive in the Jetty webapps directory
$t = The archive in the Tomcat webapps directory
$tw = The Tomcat work directory
$m2 = The Capsule library as located in in MAVEN_HOME

Run command

mvn jetty:run   Running the project using the eclipse maven-jetty-plugin
capsule     Running the capsule using java -jar artifact.war
jetty-capsule   Dropping the capsule war in jetty
jetty-war       Dropping the regular non-capsule war in jetty
tomcat-war      Dropping the regular non-capsule war in jetty
tomcat-capsule  Dropping the capsule war in jetty

Test results

First the code is given followed by the toString() result of that for each respective container.

this.getClass().getResource("/META-INF/MANIFEST.MF");
======================================================================
mvn jetty:run   jar:file:/$m2/META-INF/MANIFEST.MF
capsule         jar:file:/$c/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-capsule   jar:file:/$j/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-war       jar:file:/$j/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
tomcat-war      file:/$tw/loader/META-INF/MANIFEST.MF
tomcat-capsule  file:/$tw/loader/META-INF/MANIFEST.MF

this.getClass().getResourceAsStream("/META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
capsule         sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-capsule   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-war       sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
tomcat-war      java.io.ByteArrayInputStream
tomcat-capsule  java.io.ByteArrayInputStream

servletContext.getResource("/META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   file:$p/target/tmp/indexsvc-gui-2_2_3-SNAPSHOT_war1/META-INF/MANIFEST.MF
capsule         null
jetty-capsule   file:$j/webapp/META-INF/MANIFEST.MF
jetty-war       file:$j/webapp/META-INF/MANIFEST.MF
tomcat-war      jndi:/localhost/foo-0.1-SNAPSHOT/META-INF/MANIFEST.MF
tomcat-capsule  jndi:/localhost/foo-0.1-SNAPSHOT/META-INF/MANIFEST.MF

servletContext.getResourceAsStream("/META-INF/MANIFEST.MF)
======================================================================
mvn jetty:run   java.io.FileInputStream
capsule         null
jetty-capsule   java.io.FileInputStream
jetty-war       java.io.FileInputStream
tomcat-war      java.io.ByteArrayInputStream
tomcat-capsule  java.io.ByteArrayInputStream

servletContext.getResourceAsStream("META-INF/MANIFEST.MF)
======================================================================
mvn jetty:run   null
capsule         null
jetty-capsule   null
jetty-war       null
tomcat-war      java.io.ByteArrayInputStream
tomcat-capsule  java.io.ByteArrayInputStream

Thread.currentThread().getContextClassLoader().getResource("/META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   jar:file:$m2/META-INF/MANIFEST.MF
capsule         jar:file:$c/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-capsule   jar:file:$j/webapp/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-war       jar:file:$j/webapp/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
tomcat-war      null
tomcat-capsule  null

Thread.currentThread().getContextClassLoader().getResource("META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   jar:file:$m2/META-INF/MANIFEST.MF
capsule         jar:file:$c/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-capsule   jar:file:$j/webapp/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
jetty-war       jar:file:$j/webapp/WEB-INF/lib/aopalliance-1.0.jar!/META-INF/MANIFEST.MF
tomcat-war      file:$tw/loader/META-INF/MANIFEST.MF
tomcat-capsule  file:$tw/loader/META-INF/MANIFEST.MF

Thread.currentThread().getContextClassLoader().getResourceAsStream("/META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
capsule         sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-capsule   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-war       sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
tomcat-war      null
tomcat-capsule  null

Thread.currentThread().getContextClassLoader().getResourceAsStream("META-INF/MANIFEST.MF")
======================================================================
mvn jetty:run   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
capsule         sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-capsule   sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
jetty-war       sun.net.www.protocol.jar.JarURLConnection$JarURLInputStream
tomcat-war      java.io.ByteArrayInputStream
tomcat-capsule  java.io.ByteArrayInputStream

Summary. Interoperability is not a trivial thing. Probably best at this point would be to migrate away from reading the manifest file runtime into using e.g., classpath properties; though that requires some testing on it's own.

from capsule.

I don't think I'm seeing what you're seeing. First, the type of the input stream doesn't matter. What matters is the content. Different containers are free to give you different implementations, but that won't affect your results. So other than servletContext (which I'm not sure how you're getting in each of the cases), it seems like you're getting the same result, i.e. the same manifest, in all cases. It doesn't matter if it's still in the JAR or extracted. It's still the same file. If so, what's the problem? Am I missing something?

from capsule.

It's not the same file in most cases (sorry the output doesn't clearly show that yet). My goal is to find the approach that works best in all containers to avoid limiting the use of the built artifacts. Finding that sweet spot tells me whether a change is necessary or not - but it feels doable at this point.

The second point is the class overriding mechanism (where I suggested unpacking the class-files into the capsule cache directory) but I've not had the chance to properly test how to best approach that yet. So currently there's no action, just trying to document the journey :)

from capsule.

Your conclusion is correct, though. In Java you just can't get "your own JAR"'s manifest unless you have full control (or full knowledge) of the classloader you're using.

BTW, why do you need to read the manifest at all?

from capsule.

The manifest was just the most conveniant way to access some properties not otherwisely available. With the assistance of a few maven plugins that's all been moved to a properties file located in the classpath. That means we can move that part behind us!

To the real problem - I have the capsule structure as described by the very first post. That is the project classes put in WEB-INF/classes.

When running the capsule I start an embedded jetty with the war-path ${capsule.dir} - which contains everything but the class-files as they're never extracted. What are the options around this? It appears to be either prevent unzipping the archive or also unzip the classes in the achive. What do you think?

from capsule.

Why is the war-path capsule.dir rather than capsule.jar? Isn't the capsule itself a WAR? Let Jetty unpack whatever it wants on its own (BTW, I wouldn't mind making an exception so that Capsule would extract classes in WEB-INF/classes, I just don't fully understand the situation).

from capsule.

BTW, the way I'd do it is make the "executable WAR" capsule Extract-Capsule : false, have it declare Jetty as a maven dependency, and let the container (be it the "embedded" Jetty or a Tomcat instance) extract the WAR if it wants to. But even if you'd rather embed Jetty in the capsule (thus necessitating extraction), I think the WAR passed to Jetty should be the capsule JAR, rather than the extracted cache directory, or would that not work?

from capsule.

Using capsule.jar without extracing it is an option. In the main class I could tell jetty to run the capsule.jar directly (or least I'm pretty sure that is possible).

In a sense I use capsule as a way to run an exploded jar for quicker startup. Now I could let jetty extract the archive instead if I wished to although I'm not sure what that means yet in terms of cleanup when running a new version. So the options as I see them

1. In the main class, invoke jetty and tell it to run capsule.jar¹⁾ directly, extracting nothing.
2. In the main class, invoke jetty, tell it to extract capsule.jar and run the exploded archive
3. In the main class, invoke jetty to run the exploded archive capsule.dir as extracted by capsule

It's option 3 the caused me to raise this issue since the archive classes never were extracted. I'm fine with either option but started off with option 3, which I've come to understand now was not the intended use of capsule.dir.

^{1) Where the capsule.jar variable points to the file path/to/cache/dir/myCapsule.war}

from capsule.

Here's what I'd like you to try. In the manifest, either put:

Application: org.eclipse.jetty:jetty-runner:9.2.2.v20140723
Args: --path /myapp $CAPSULE_JAR

Or, embed jetty-runner in the capsule's root and:

Application-Class: org.eclipse.jetty.runner.Runner
Args: --path /myapp $CAPSULE_JAR

That should be it. There's not need for any custom jetty class.

(See here for more detailed docs)

from capsule.

That's a cool utility! I need some custom jetty configuration so whether that's done by a java class or a context.xml isn't that big of a change. Though it seems the advantage of embedding the jetty-runner is fewer jars to worry about.

All in all, embedding jetty or the jetty-runner into the capsule war means the capsule must be extracted (or all jetty classes needs to be unpacked). This is fine because jetty can run the original capsule war as is.

Now, since capsule alreadys extract almost everything it would be nice if jetty could run the extracted files directly, per the original request. I propose an option extractAll (set to false by default) for this purpose. How does that sound?

from capsule.

If you don't want to unpack the capsule, you have two options: declare the dependency rather than embed it (which would have the added advantage of sharing the same Jetty jars with all similar capsules), or explode the Jetty jars into the capsule (as you'd do for a fat jar).

I'm not dismissing the idea of extractAll altogether, but I'm not too fond of it. It makes Capsule's implementation (which files it extracts and when) part of the public API. The only reason I've made capsule.dir public was for those occasions where it is absolutely required to get to the files (say, from startup scripts), but perhaps that was a bad decision. In any case, whether the capsule is extracted once, twice or not at all is an implementation detail that the user shouldn't care about. So unless there's a more compelling reasons for extractAll (it'a absolutely necessary for something, or it improves performance by a lot), I'd rather we not do it.

from capsule.

I agree it's not a huge performance boost and I respect your ideals when it comes to the purpose of the capsule. After all should the future require it, withdrawing an option from the public API is a nasty business.

So with the purpose of the cache dir communicated clearly and a workaround possible I'm fine with closing this issue. Again, thanks for your dedication and insight into this topic.

from capsule.

But are we good on creating "executable WARs"? You've got a working solution?

from capsule.

Yeah it works, but most shortcoming are due to the assembly plugin. In the end I've gone for a capsule that extracts it's jar-files to be able to locate jetty. Jetty is then used to run the original capsule archive which in all essence is a regular war-file (apart from some extra files/libs in the archive root). If I find time for it perhaps I'll add a demo project illustrating the concept.

from capsule.

Great! If you have the time, you should also try declaring the Jetty dependency in the manifest rather than embedding it (you can still embed all other dependencies).

from capsule.