Hippo CMS Demo project, using Apache Jackrabbit VFSDataStore with SFTP server or WebDAV server as a backend, or using Apache Jackrabbit S3DataStore with Amazon S3 backend, in order to manage binary content (e.g. images, files, etc.).

Jackrabbit VFSDataStore (available since Jackrabbit 2.14.0) is using Commons VFS 2 to get access to various backend storages such as local file system, SFTP, WebDAV, etc. For more details, please read my blog post:

• http://woonsanko.blogspot.com/2016/08/cant-we-store-huge-amount-of-binary.html

Please see highlighted dependencies in cms/pom.xml. Basically you need to include jackrabbit-vfs-ext jar dependency and other VFS2 backend dependencies optionally in your project.

Enable either SFTP server (option 1) or WebDAV server (Option 2) as explained below.

In pom.xml, it is configured to use a SFTP backend by default:

              <repo.config>file://${project.basedir}/conf/repository-vfs2-sftp.xml</repo.config>
              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-webdav.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-db.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-aws-s3.xml</repo.config> -->

NOTE: In addition, VFSFileSystem can also be configured for Repository, Workspace and Versioning even if it's configured only for Versioning in conf/repository-vfs2-sftp.xml at the moment. This is to demonstrate the new feature, JCR-4354: VFS File System. By the example configuration, all the version data will be stored in the SFTP backend.

Open conf/vfs2-datastore-sftp.properties and edit the connection information properly.

If you want to use a WebDAV server as backend, open pom.xml, uncomment the second <repo.config> element and comment out the others:

              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-sftp.xml</repo.config> -->
              <repo.config>file://${project.basedir}/conf/repository-vfs2-webdav.xml</repo.config>
              <!-- <repo.config>file://${project.basedir}/conf/repository-db.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-aws-s3.xml</repo.config> -->

In this demo project, I used Apache Jackrabbit Standalone server as WebDAV backend server.

If you downloaded it somewhere, you can run the following to run it:

    java -jar ./jackrabbit-standalone-2.20.0.jar -p 8888

(-p 8888 sets its port number to 8888; otherwise 8080 by default.)

The above command will start WebDAV server at port 8888 http://localhost:8888/repository/default/.

Open conf/vfs2-datastore-webdav.properties and edit the connection information if necessary.

If you want to use the default Database DataStore, open pom.xml, uncomment the third <repo.config> element and comment out the others:

              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-webdav.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-sftp.xml</repo.config> -->
              <repo.config>file://${project.basedir}/conf/repository-db.xml</repo.config>
              <!-- <repo.config>file://${project.basedir}/conf/repository-aws-s3.xml</repo.config> -->

If you want to use the AWS S3 Database DataStore, open pom.xml, uncomment the third <repo.config> element and comment out the others:

              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-sftp.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-vfs2-webdav.xml</repo.config> -->
              <!-- <repo.config>file://${project.basedir}/conf/repository-db.xml</repo.config> -->
              <repo.config>file://${project.basedir}/conf/repository-aws-s3.xml</repo.config>

Open conf/aws-s3-datastore.properties and edit the connection information if necessary.

This project uses the Maven Cargo plugin to run both CMS (http://localhost:8080/cms/ for "Content Authoring") web application and SITE (http://localhost:8080/site/ for "Content Delivery") web application locally in Tomcat.

From the project root folder, execute:

    mvn clean verify && mvn -P cargo.run -Drepo.path=storage

After your project is set up, access the CMS at http://localhost:8080/cms/ and the site at http://localhost:8080/site/. Logs are located in target/tomcat8x/logs

If you use a WebDAV server as backend, you can take a look at the terminal of the WsgiDAV server. You will see many logs about content uploading. This is because Hippo Repository bootstraps some example binary content to the repository storage during the initialization.

Repository configuration is located at the following:

• VFS2 / SFTP : conf/repository-vfs2-sftp.xml, which sets DataStore using VFSDataStore component.

  More VFS backend specific detail for backend SFTP server is configured in conf/vfs2-datastore-sftp.properties.

  Also, this SFTP configuration demonstrates the separate feature JCR-4354: VFS based FileSystem, by using VFSFileSystem for version storage by default.

  See the New Feature: JCR-4354: VFS File System - for version storage for example section for detail.

• VFS2 / WebDAV : conf/repository-vfs2-webdav.xml, which sets DataStore using VFSDataStore component.

  More VFS backend specific detail for backend WebDAV connection is shown in conf/vfs2-datastore-webdav.properties.

• S3 : conf/repository-aws-s3.xml, which sets DataStore using S3DataStore component.

• Visit http://localhost:8080/site/news.
• Click on a new article link.
• The photo inside the article was retrieved from the backend (SFTP or WebDAV) server through the VFSDataStore component.

• Visit http://localhost:8080/cms/
• Log on by admin/admin.
• Click on the "Content" perspective.
• Select "Image" or "Asset" in the dropdown located near the left top corner.
• Click or create a folder and try to upload image files or asset (e.g, PDF) files.
• Tip: you need to hover your mouse on a tree node to find and click on the action buttons.

• Visit http://localhost:8080/cms/console/
• Log on by admin/admin.
• Find any binary property (under gallery or asset) which contains data bigger than 16KB (by default, a CachingDataStore stores binary only when the data is bigger than 16KB).
• You will be able to find "Content Identity: (Show)" next to the "Upload" button. Click on "(Show)" link to retrieve the content identity.
• You can possibly figure out where the real binary file is stored in the backend (SFTP or WebDAV) by the "Content Identity" value.

• Reference: https://issues.apache.org/jira/browse/JCR-4354.
• Visit http://localhost:8080/cms/
• Log on by admin/admin.
• Open a news document through the Content Perspective / Documents.
• Edit the news document to add some text in the title for example, and save & close it.
• Publish the document and you can see revision history through Document / Revision History menu.
• You may select an old revision, which open a new tab to show the old revision. Under the hood, it reads the versioned content from the backend SFTP server.

The bundlereader submodule provides a tool to investigate bundle files stored in VFS FileSystem.

After building, you can find the executable uber jar file under its build output directory. e.g, bundlereader/target/hippo-davstore-demo-bundlereader-*.jar.

Now suppose you have access a bundle file in your local file system. Then you can run the tool to investigate the content of the file like the following:

# With a file path:
$ java -jar target/hippo-davstore-demo-bundlereader-0.2.0-SNAPSHOT.jar /Users/tester/vfsfs/items/21/2a/4141c7064ac68b2768d6281989c9.n

# Or simply with a UUID after the base path (e.g, /Users/tester/vfsfs/items/):
$ java -jar target/hippo-davstore-demo-bundlereader-0.2.0-SNAPSHOT.jar /Users/tester/vfsfs/items/212a4141-c706-4ac6-8b27-68d6281989c9

Example output:

$ java -jar target/hippo-davstore-demo-bundlereader-0.2.0-SNAPSHOT.jar /Users/tester/vfsfs/items/212a4141-c706-4ac6-8b27-68d6281989c9
ID: 212a4141-c706-4ac6-8b27-68d6281989c9
Parent ID: 9e440e6d-e93a-4d5f-913b-9c3b0867c812
Node type: {http://www.onehippo.org/jcr/hippogallery/nt/2.0}imageset
Mixin types: []
Referenceable? false
Shared set: []
Properties:
  - {http://www.onehippo.org/jcr/hippo/nt/2.0.4}paths (String multiple): [212a4141-c706-4ac6-8b27-68d6281989c9, 9e440e6d-e93a-4d5f-913b-9c3b0867c812, 889e1e98-6bc5-4320-8d0b-31c8fedd9c5c, 6a3a88f5-6e99-4323-ad9a-ca661ca50514, 37aa21b2-d5dd-4ac2-9e1c-9cb0f82dd099, ed307dfc-5774-4ac7-9feb-79b57e058bb2, cafebabe-cafe-babe-cafe-babecafebabe]
  - {http://www.onehippo.org/jcr/hippo/nt/2.0.4}availability (String multiple): [live, preview]
  - {http://www.onehippo.org/jcr/hippogallery/nt/2.0}filename (String): [banner-1.jpg]
Children:
  - {http://www.onehippo.org/jcr/hippogallery/nt/2.0}thumbnail (b962a623-bf1f-4f41-b629-1dccd24e4bbe)
  - {http://www.onehippo.org/jcr/hippogallery/nt/2.0}original (281c722c-53c7-4c67-8292-2c0e65d7dd48)

If you run CMS with VFSFileSystem for both the Workspace and Versioning, but not for Repository, you should be cautious when cleaning up the repository directory (storage/ folder for example as specified by -Drepo.path=storage) because the default FileSystem of Repository is that of LocalFileSystem.

• In that case, if you clean up the existing the repository directory (e.g, storage/), then Jackrabbit could fail to restart due to namespace exceptions. So you must keep and restore the following files at least before restart:

    - repository/namespaces/ns_idx.properties
    - repository/namespaces/ns_reg.properties 
  

• As an example, before cleaning up the the repository directory (storage/ in this example), backup and keep the files (storage/repository/namespaces/*.properties) and clean up the other files and folders.

• Now you can restart with a cleaned up repository directory without any problem:

    mvn clean verify && mvn -P cargo.run -Drepo.path=storage