Code Monkey home page Code Monkey logo

org.openntf.nsfodp's Introduction

NSF ODP Tooling

This project contains tooling for dealing with NSF on-disk-project representations in Maven and Eclipse.

There are three main components: a Maven plugin, a set of Domino OSGi plugins, and a set of Eclipse plugins. In tandem, they provide several features:

ODP Compiler

The ODP compiler allows the use of a Domino server or local Notes installation to compile an on-disk project into a full NSF without the need of Domino Designer. This compilation supports classic design elements as well as XPages, and allows for using OSGi plugins to resolve classes and XPages components.

To use this for remote compilation, install the Domino plugins on an otherwise-clean Domino server - this is important to allow the plugins to be loaded and unloaded dynamically without interfering with existing plugins.

ODP Exporter

The ODP exporter allows the use of a Domino server to export an NSF into a Designer-compatible ODP format.

Eclipse Tooling

The Eclipse plugins provide the Eclipse IDE with basic knowledge of the ODP and autocompletion capabilities for XPages and Custom Controls.

Currently, autocompletion knows about the stock components and Extension Library that ship with 10.0.1 as well as any Custom Controls inside the same project.

Additionally, it adds "Compile On-Disk Project" and "Deploy NSF" actions to the context menu, which are shortcuts for the equivalent Maven goals.

NSF Deployment

The NSF deployment service allows for deployment of an NSF to a Domino server without involving the Notes client. Currently, this will only deploy new databases, but the plan is to have this also be able to perform a design replace on an existing database.

XSP Transpiler

The XSP transpile translates XPages and Custom Controls into Java source files in the target/generated-sources/java directory of the project. This is intended for use with non-NSF webapps.

Usage

To use this tooling with an ODP, wrap it in a Maven project with the domino-nsf packaging type. Here is an example pom:

<?xml version="1.0"?>
<project
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
    xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>example-nsf</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>domino-nsf</packaging>

    <pluginRepositories>
        <pluginRepository>
            <id>artifactory.openntf.org</id>
            <name>artifactory.openntf.org</name>
            <url>https://artifactory.openntf.org/openntf</url>
        </pluginRepository>
    </pluginRepositories>
    
    <build>
        <plugins>
            <plugin>
                <groupId>org.openntf.maven</groupId>
                <artifactId>nsfodp-maven-plugin</artifactId>
                <version>3.10.0</version>
                <extensions>true</extensions>
            </plugin>
        </plugins>
    </build>
</project>

Additionally, there are some properties to set in your Maven ~/.m2/settings.xml configuration.

There are two modes of operation: local and remote. In the case of local operations, set the notes-program to the path to a local Notes or Domino installation and notes-platform to the URL of a Domino update site. In practice, I've found that update sites generated from Domino instead of Notes are more reliable.

These are the applicable properties to configure remote or server execution:

<?xml version="1.0"?>
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <profiles>
        <profile>
            <id>nsfodp</id>
            <properties>
                <!-- for local operations, macOS example -->
                <notes-program>/Applications/IBM Notes.app/Contents/MacOS</notes-program>
                <notes-platform>file:///Users/username/path/to/Domino10.0.1</notes-platform>
                <!-- required on Linux -->
                <notes-ini>/var/lib/domino/data/notes.ini</notes-ini>
              
                <!-- for remote operations -->
                <nsfodp.compiler.server>someserver</nsfodp.compiler.server>
                <nsfodp.compiler.serverUrl>https://some.server/</nsfodp.compiler.serverUrl>
                <nsfodp.exporter.server>someserver</nsfodp.exporter.server>
                <nsfodp.exporter.serverUrl>https://some.server/</nsfodp.exporter.serverUrl>
                <!-- Note: deployment operations currently require a server -->
                <nsfodp.deploy.server>someserver</nsfodp.deploy.server>
                <nsfodp.deploy.serverUrl>http://some.server/</nsfodp.deploy.serverUrl>

                <!-- use a remote server even when local properties are set -->
                <nsfodp.requireServerExecution>true</nsfodp.requireServerExecution>
            </properties>
        </profile>
    </profiles>
    <activeProfiles>
        <activeProfile>nsfodp</activeProfile>
    </activeProfiles>
  
    <!-- needed when using remote execution -->
    <servers>
        <server>
            <id>someserver</id>
            <username>builduser</username>
            <password>buildpassword</password>
        </server>
    </servers>
</settings>

ODP Compilation

The ODP compilation process has several properties that can be configured in the plugin's configuration block, with these as the defaults:

<configuration>
	<outputDirectory>${project.build.directory}</outputDirectory>
    <outputFileName>${project.build.finalName}.nsf</outputFileName>
    <odpDirectory>odp</odpDirectory>
    <!-- e.g. ../../releng/some.updatesite.project/target/site -->
    <updateSite></updateSite>
    <compilerLevel>1.8</compilerLevel>
    <!-- Adds the build timestamp to the generated NSF's title -->
    <appendTimestampToTitle>false</appendTimestampToTitle>
    <!-- Creates/updates a $TemplateBuild shared field -->
    <templateName></templateName>
    <!-- Enabled resource aggregation and compressed JS libs in xsp.properties -->
    <setProductionXspOptions>false</setProductionXspOptions>
    <!-- Add jars to the compilation classpath, to mimic jvm/lib/ext deployment -->
    <classpathJars>
        <classpathJar></classpathJar>
    </classpathJars>
</configuration>

ODP Exporter

The ODP exporter is triggered manually, and does not require a Maven project in the current directory (though it will use the settings of an active project if present).

To export an ODP from the command line, execute the mojo directly:

mvn org.openntf.maven:nsfodp-maven-plugin:3.10.0:export-odp -DdatabasePath=names.nsf

This mojo will create or replace the odp directory in the current or project directory with the contents of the specified database. The directory path can be overridden by specifying the nsfodp.exporter.odpDirectory property in the execution.

This process also has several configuration options

<configuration>
    <odpDirectory>odp</odpDirectory>
    <!-- Enable https://openntf.org/main.nsf/project.xsp?r=project/Swiper -->
    <swiperFilter>true</swiperFilter>
    <!-- Export notes in "binary" note format, like the Designer option -->
    <binaryDxl>false</binaryDxl>
    <!-- Export rich text items as Base64 data instead of DXL-ified -->
    <richTextAsItemData>true</richTextAsItemData>
</configuration>

NSF Deployment

To specify a deployment destination and path, expand your project's pom to include configuration information for deployment:

    ...
    <plugin>
        <groupId>org.openntf.maven</groupId>
        <artifactId>nsfodp-maven-plugin</artifactId>
        <version>3.10.0</version>
        <extensions>true</extensions>
        <configuration>
            <!-- This can be on the target Domino server a remote one -->
            <deployDestPath>someserver!!someapp.nsf</deployDestPath>
            <deployReplaceDesign>true</deployReplaceDesign>
        </configuration>
    </plugin>
    ...

By default, compilation binds to the compile phase and deployment binds to the deploy phase, when their parameters are specified.

XSP Transpiler

The XSP transpiler can be invoked with the transpile-xsp goal in an execution block or via the command line:

mvn nsfodp:transpile-xsp

This will search for XPages in odp/XPages or src/main/webapp/WEB-INF/xpages and Custom Controls and definitions in odp/CustomControls or src/main/webapp/WEB-INF/controls. It currently has several restrictions:

  • Extra XPages libraries must still be defined in OSGi plugins and referenced in the updateSites property as in the ODP compiler. Libraries defined in META-INF/services in dependencies are not yet supported
  • Custom controls with property classes from an XPages library additionally require the class's JAR to be a direct dependency on the current project, even if it is included in a referenced update site
    • Currently, transitive dependencies are not supported
  • In-project classes are currently not supported as custom control property classes

Requirements

Maven

The Maven plugin requires Maven 3.0+ and Java 8+.

Eclipse

The Eclipse plugin targets Neon and above, but may work with older releases, as long as they are launched in a Java 8+ runtime.

Domino (For Server Operations)

The Domino plugins require Domino 9.0.1 FP10 or above.

Notes or Domino (For Local Operations)

Local compilation and export require Notes or Domino 9.0.1 FP10 or above on Windows and Linux. On macOS, it requires Notes 10.0.1 through 11.0.1.

Due to changes in Notes V12, local operations do not currently work with that version. Instead, you should use Notes 11.0.1 (which can work as a copy of the .app next to Notes 12) or server-based operations.

Note: if you use local compilation, either your ID file should have no password or you should configure Notes's User Security to allow non-Notes-based programs to execute without prompting for a password.

Compilation on macOS

Due to the way the macOS Notes JVM is set up, the process currently requires that the running user have access to modify the application bundle, which is the default for admin users.

Debugging Local Operations

Local runners launch a separate Java process with an Equinox environment to function. You can pass additional command arguments to this launcher by using the nsfodp.equinoxJvmArgs property, which is a string that is split on whitespace. For example, to enable debug mode during compilation and suspend on launch until a debugger connects:

$ mvn clean install -Dnsfodp.equinoxJvmArgs="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000"

License

This project is licensed under the Apache License 2.0.

org.openntf.nsfodp's People

Contributors

dependabot[bot] avatar jesse-gallagher avatar mpradny avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

org.openntf.nsfodp's Issues

Support non-“raw” DXL ODPs

It's possible that there won't be any work to do if the normal DXL imports work well enough, but it's worth checking for projects that don't have that flag set in Designer.

Investigate auto-adding JRE classpath access rules for projects

When adding jars to the core JRE classpath, Eclipse will by default complain that classes in them are "not API". This can be resolved per-project by editing the access rules to add **, and having the m2e configurator do this automatically would be handy.

Work against a "live" NSF

Rather than just doing an all-or-nothing build, it would be interesting to be able to configure a project to do incremental builds against a live NSF on a server. This could be done by just sending the individual design element over to the server, having it retrieve or build the class path for the NSF, generate the appropriate DXL, and import it.

This wouldn't be able to work as well with OSGi plugin changes, but it may be worth looking in to the OSGi commands on the server to see if there's a way to refresh those too.

This could also be implemented as an alternate mode for compilation instead of the separate "compile" and "deploy" phases, though it'd be primarily useful as something executed as a builder from Eclipse.

ODP creation can double up file resource elements

Since DXL import doesn't de-dupe "plain" file resources, elements like faces-config.xml and xsp.properties already exist in the blank DB and get doubled up on creation.

Ideally, the fix will be to create a truly-blank NSF to start out with. Failing that, file resource import should do a de-dupe check.

Compiled NSFs don't seem to play well with "update existing signatures" option in Administrator

When I replaced design on a DB and did a Sign... in Administrator with the default "update existing signatures only" option checked, it seemed like it skipped over all design elements, and the app didn't launch. Once I unchecked that box and ran it again, the app worked.

I'm not sure if the trouble is that I'm signing design notes too much, signing them not enough, or if server signatures are a bad idea in general.

Support in-NSF-defined controls

Currently, the odpbuilder supports plugin-hosted components and custom controls, but not controls defined via Java+xsp-config locally in the NSF.

NSF projects should list source folders before classpath jars in the Java build path

Properly, the "Order and Export" tab of the "Java Build Path" properties pane in Eclipse lists:

  1. JRE System Library
  2. Plug-on Dependencies
  3. Local source folder(s)
  4. Local jar dependencies

Currently, the embedded jars end up listed first, which causes unexpected behavior when the jars have classes with the same name as those in the source.

Investigate multithreaded import

Depending on how Notes handles writing to the NSF, it may make sense to split standard design imports into threads. Even without splitting the DXL import, generating the byte arrays for design elements can definitely be split apart.

Support XPage/CC-associated .properties files

The org.openntf.xsp.extlibx.bazaar.odpcompiler.odp.XPage class currently only cares about the .xsp, .xsp-properties, and .xsp.metadata files, but it might also need to keep track of the .properties files.

On the other hand, it's possible that the ResourceBundleSource will already take care of this - it'll depend on what the path the translator requests is.

Invalid characters cause the FacesReader to throw up its hands

	at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
	at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
	at javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
	at com.ibm.xsp.util.DomUtil.parse(DomUtil.java:84)
	at com.ibm.xsp.page.parse.FacesReader.<init>(FacesReader.java:74)
	at com.ibm.xsp.extlib.interpreter.DynamicXPageBean.translate(DynamicXPageBean.java:230)
	at org.openntf.xsp.extlibx.bazaar.odpcompiler.ODPCompiler.compileXSP(ODPCompiler.java:702)```

Add "generate missing metadata" mojo

It would be nice to have a way to plop normal resources (Java classes, jars, images, stylesheets, etc.) into the ODP project tree and have a way to generate associated DXL metadata files alongside them without having to load up Designer.

This could be done with a Maven mojo that looks for files without associated .metadata and then spawns one based on their name and position in the project tree. Eclipse could build on this in an incremental builder for newly-created files.

Investigate piggybacking on Eclipse's JSF tooling

Rather than just doing XML autocomplete, it may be possible to present Eclipse with a JSF/JSP tag library. This example covers something similar:

http://help.eclipse.org/kepler/index.jsp?topic=%2Forg.eclipse.jst.jsf.doc.dev%2Fhtml%2Ftutorials%2Fimplementing_a_new_tag_based_el_variable_contributor_for_jsp.html

The org.eclipse.jst.jsf.common.standardMetaDataFiles extension point allows for registering a taglib contributor, and the locator property can specify a class that extends org.eclipse.jst.jsf.common.metadata.internal.StandardMetaDataSourceFileLocator to do so dynamically.

Compilation adds a lot of sign="true" flags on round tripping

This is presumably a side effect of all the "just in case" design-element signing that goes on in compilation. If you replace design of a database with an odpcompiler-compiled version and let Designer sync to the filesystem, a lot of items gain a sign="true" attribute. This isn't really a big issue, but it'd be nice to have proper fidelity on round trips.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.