Code Monkey home page Code Monkey logo

sling-dynamic-include's Introduction

Sling Dynamic Include


Now part of Apache Sling

Sling Dynamic Include (SDI) project has been contributed to the Apache Sling project.

SDI is no longer maintained here - this Github project has been left as an information hub only. All tickets should be raised in the Apache Sling jira.


Purpose

The purpose of the module presented here is to replace dynamic generated components (eg. current time or foreign exchange rates) with server-side include tag (eg. SSI or ESI). Therefore the dispatcher is able to cache the whole page but dynamic components are generated and included with every request. Components to include are chosen in filter configuration using resourceType attribute.

When the filter intercepts request for a component with given resourceType, it'll return a server-side include tag (eg. <!--#include virtual="/path/to/resource" --> for Apache server). However the path is extended by new selector (nocache by default). This is required because filter has to know when to return actual content.

Components don't have to be modified in order to use this module (or even aware of its existence). It's servlet filter, installed as an OSGi bundle and it can be enabled, disabled or reconfigured without touching CQ installation.

Prerequisites

  • CQ / Apache Sling 2
  • Maven 2.x, 3.x

Installation

Add following dependency to your project:

<dependency>
    <groupId>com.cognifide.cq</groupId>
    <artifactId>sling-dynamic-include</artifactId>
    <version>2.0.4</version>
</dependency>

Configuration

Filter is delivered as a standard OSGi bundle. SDI is configured via the configuration factory called SDI Configuration. Following properties are available:

  • Enabled - enable SDI
  • Base path - given SDI configuration will be enabled only for this path
  • Resource types - which components should be replaced with tags
  • Include type - type of include tag (Apache SSI, ESI or Javascript)
  • Add comment - adds debug comment: <!-- SDI include (path: %s, resourceType: %s) --> to every replaced component
  • Filter selector - selector used to get actual content
  • Component TTL - time to live in seconds, set for rendered component (require Dispatcher 4.1.11+)
  • Required header - SDI will be enabled only if the configured header is present in the request. By default it's Server-Agent=Communique-Dispatcher header, added by the AEM dispatcher. You may enter just the header name only or the name and the value split with =.
  • Ignore URL params - SDI normally skips all requests containing any GET parameters. This option allows to set a list of parameters that should be ignored in the test. See the Ignoring URL parameters section in the dispatcher documentation.
  • Include path rewriting -- enable rewriting link (according to sling mappings) that is used for dynamic content including.

Compatibility with components

Filter is incompatible with following types of component:

  • components which handles POST requests or GET parameters,
  • synthetic components which uses suffixes (because suffix is used to pass requestType of the synthetic resource).

If component do not generate HTML but eg. JS or binary data then remember to turn off Comment option in configuration.

Enabling SSI in Apache & dispatcher

In order to enable SSI in Apache with dispatcher first enable Include mod (on Debian: a2enmod include). Then add Includes option to the Options directive in your virtual configuration host. After that find following lines in dispatcher.conf file:

    <IfModule dispatcher_module>
        SetHandler dispatcher-handler
    </IfModule>

and modify it:

    <IfModule dispatcher_module>
        SetHandler dispatcher-handler
    </IfModule>
    SetOutputFilter INCLUDES

After setting output filter open virtualhost configuration and add Includes option to Options directive:

    <Directory />
        Options FollowSymLinks Includes
        AllowOverride None
    </Directory>
    <Directory /var/www/>
        Options Indexes FollowSymLinks MultiViews Includes
        AllowOverride None
        Order allow,deny
        allow from all
    </Directory>

It's also a good idea to disable the caching for .nocache.html files in dispatcher.any config file. Just add:

    /disable-nocache
    {
        /glob "*.nocache.html*"
        /type "deny"
    }

at the end of the /rules section.

Enabling TTL in dispatcher 4.1.11+

In order to enable TTL on Apache with dispatcher just add:

/enableTTL "1"

to your dispatcher configuration.

Enabling ESI in Varnish

Just add following lines at the beginning of the vcl_fetch section in /etc/varnish/default.vcl file:

    if(req.url ~ "\.nocache.html") {
        set beresp.ttl = 0s;
    } else if (req.url ~ "\.html") {
        set beresp.do_esi = true;
    }

It'll enable ESI includes in .html files and disable caching of the .nocache.html files.

JavaScript Include

Dynamic Include Filter can also replace dynamic components with AJAX tags, so they are loaded by the browser. It's called JSI. In the current version jQuery framework is used. More attention is required if included component has some Javascript code. Eg. Geometrixx Carousel component won't work because it's initialization is done in page <head> section while the component itself is still not loaded.

Plain and synthetic resources

There are two cases: the first involves including a component which is available at some URL, eg.

/content/geometrixx/en/jcr:content/carousel.html

In this case, component is replaced with include tag, and nocache selector is added

<!--#include virtual="/content/geometrixx/en/jcr:content/carousel.nocache.html" -->

If the filter gets request with selector it'll pass it (using doChain) further without taking any action.

Plain include

There are also components which are created from so-called synthetic resources. Synthetic resource have some resourceType and path, but they don't have any node is JCR repository. An example is

/content/geometrixx/en/jcr:content/userinfo

component with foundation/components/userinfo resource type. These components return 404 error if you try to make a HTTP request. SDI recognizes these components and forms a different include URL for them in which resource type is added as a suffix, eg.:

/content/geometrixx/en/jcr:content/userinfo.nocache.html/foundation/components/userinfo

If filter got such request, it'll try to emulate <sling:include> JSP tag and includes resource with given type and nocache selector:

/content/geometrixx/en/jcr:content/userinfo.nocache.html

Selector is necessary, because otherwise filter would again replace component with a SSI tag.

External resources

Release notes

2.2.0

#17 Support for time-based (TTL) caching, Dispatcher 4.1.11+ required

Integrated with Sling

Sling Dynamic Include (SDI) project has been contributed to the Apache Sling project as an extension. See the official announcement on the Cognifide website.

This Github project has been left as an information hub.

sling-dynamic-include's People

Contributors

bdelacretaz avatar danchapman-pearson avatar devzbysiu avatar fabio10 avatar mikeyhendy avatar mmajchrzak avatar pacoolsky avatar toniedzwiedz avatar trekawek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sling-dynamic-include's Issues

Configuration not getting picked up correctly

I’ve been working on implementing SDI/SSI with AEM 5.6.1 and found an issue with the current SDI code…

I’ve traced the issue back to the following class:
/src/main/java/com/cognifide/cq/includefilter/Configuration.java

Inside the Configuration method, I believe all the lines for “PropertiesUtil.toString(PROPERTY_XXX,yyy)” should instead be “PropertiesUtil.toString(properties.get(PROPERTY_XXX),yyy)"

When the properties.get() isn’t present, it will just return the property name and not the value of it.

Could someone have a look and confirm this?, as I suspect that it’s probably tripping up other people as well.

Enable TTL support

New version of dispatcher (4.1.11+) supports TTL based cache invalidation. Implement support for TTL, so page fragments (components) can leverage this feature.

Support for more characters in selectors

We SSI-include a component with a selector that sometimes contains special characters.

Example:

Correct URL: /content/site1/en_GB/product/_jcr_content/main/product/par/productinfo.productId=45$190-7547.html

SDI renders:

The special characters in productId get removed which means we can't find the product in the JCR

For us, allowing the following characters would be sufficient, but I'm sure that there are other projects were different special characters would be needed: !$'()+,-<>~_`äöüÄÖÜß

Could you please put the allowed characters into an OSGi-config or remove this restriction altogether?

Secondly, the rendering of the SSI-Include lacks proper escaping. From what I've gathered from http://httpd.apache.org/docs/2.2/howto/ssi.html, at least the characters " and $ have to be escaped by putting a "" before them.

Links not rewritten when including content for synthetic resources

Issue raised through mail support:

When using SDI for synthetic resources, links within the rendered component(s) are not rewritten using the configured sling mappings. They are however rewritten for non-synthetic resources. Can you provide me with an insight on why this might be happening – is this a global issue with SDI? Is there a workaround available that you know off?

So, a concrete example:

An SDI include of an existing resource
/content/mysite/en_US/home/jcr:content/list.nocache.html
=> when this resource is included through SDI the links from the rendered html are rewritten according to the configured sling:mappings – everything is working superb here

/content/mysite/en_US/home/jcr:content/userMenu.nocache.html/mysite/components/nav/userMenu
=> when this resource is included through SDI the links from the rendered html are not rewritten according to the configured sling:mappings – I also do not see this item going through the link checker as I do with the previous url.

Avoid caching of a component based on a loggedIn status?

This does not support a scenario where we can avoid caching of a component whose resourceType is configured in SDI based on a loggin condition? For e.g. if a user is logged in then the component should not cache otherwise for anonymous user, component should cache.

Loop (.nocache.nocache...) when sling:include has no path attribute

When sling:include did not have path attribute ( <sling:include resourceType="..." />) dispatcher was asking for parent /content/application/en/home/page/_jcr_content.nocache.html, and then for /content/application/en/home/page/_jcr_content.nocache.nocache.html and so on (.nocache.nocache.nocache...). Result was: page included about ten times (after tenth .nocache dispatcher gave up in finding component). There could be a mechanism preventing from multiple .nocache with warning/error message notifying about this situation.

SDI issue - Dynamic component also cached !!!

I have integrated SDI in my project. After enabling SDI filter for a component, include tag is being added correctly and the page is also cached. But the issue here is that the component included in SDI is cached as well, Rather, it should fetch during load time.
Could you provide a solution for this issue.

disallowing nocache selector

just wondering: you recommend to disallow the nocache selector in the dispatcher's filter rules.
for me, this disallows the selector for the ssi subrequests as well, effectively making the whole thing disfunctional.
it there a trick that I don't see?

SSI includes are vulnerable to XSS

How to reproduce:

  • Enable SSI Include for the resourceType of a non-synthetic resource
  • Open the page that contains the resource with the following suffix:

--><script>alert('XSS')</script><!--

(example: http://localhost:4503/content/testproject/testsite.html/--><script>alert('XSS')</script><!--)

Result:

the browser executes alert('XSS');

Suggested fix:

in com.cognifide.cq.includefilter.generator.types.SsiGenerator#getInclude, escape "-->" in url before appending it to buf

Default configuration Issue on publish

I have started working on this module recently facing issue on publisher , after installing the bundle.
The component userinfo and carosuel are not displayed on publisher environment.

Dispacther screen shot:
image

Publisher screen shot:
image

Checked the page source for publisher the ssi tags are coming correctly:

<div class="userinfo"><!--#include virtual="/content/geometrixx/en/_jcr_content/userinfo.nocache.html/foundation/components/userinfo" --></div>

carousel:

<div class="parbase carousel list"><!--#include virtual="/content/geometrixx/en/_jcr_content/carousel.nocache.html" --></div>

Dispatcher.log for en page

 [Wed Jan 14 11:58:16 2015] [I] [8528(796)] "GET /content/geometrixx/en.html"  200> + 696ms
[Wed Jan 14 11:58:16 2015] [I] [8528(796)] "GET /content/geometrixx/en/_jcr_content/userinfo.nocache.html/foundation/components/userinfo" 200 1838 19ms
[Wed Jan 14 11:58:16 2015] [I] [8528(796)] "GET /content/geometrixx/en/_jcr_content/carousel.nocache.html" 200 4507 36ms
[Wed Jan 14 11:58:16 2015] [I] [8528(844)] "GET /content/geometrixx/en.topnav.1421209651186.html" 200 + 154ms

Dispatcher is not caching the updated content when using with SDI

Hi,
I am using SDI to cache the experience fragment for header and footer, but I am facing one issue with the content that is,
when I publish the content for the first time it gets cached on the dispatcher but when I updated the same content, it got reflects properly on publisher but on dispatcher it is still showing the old content, that was cached for the first time even after clearing the dispatcher cache.
But when I disable the SDI on publisher it works fine for me.
I ma using TTL for one year and apache ssi as configuration in the config for sdi.
Can anyone provide some insight on it what needs to be done to change this expected behaviour of SDI and dispatcher.
Thanks
Umesh Thakur

IgnoreUrlParams Wildcards in the SlingDynamicInclude Config do not work as expected (AEM 6.4)

Hello

i have the following problem in my AEM instance but i think it might be a general problem.

The issue i am facing is that in my dispatcher configuration there are several parameters defined with a * as wildcard.
This is configured for our tracking engine where users might get a link to a page with one of these parameters.

For example:

https//www.mypage.com/latest-news?testengine-usergroup=10

In this case my dispatcher is configured to ignore the paramers that match the following:

testengine-*

When i configured this in my SlingDynamicInclude config on the server under the ignoreUrlParams it does not see the above parameter from the url as matching for ignoreUrlParams.

include-filter.config.extension=""
include-filter.config.enabled=B"true"
include-filter.config.path="/content"
include-filter.config.required_header="Server-Agent\=Communique-Dispatcher"
include-filter.config.include-type="SSI"
include-filter.config.selector="nocache"
include-filter.config.add_comment=B"true"
include-filter.config.appendSuffix=B"true"
include-filter.config.rewrite=B"false"
include-filter.config.ttl=""
# These are the urlParams from the dispatcher that are ignored by as such and will load from cache (for tracking etc.)
# If you do not ignore them then a page with rendered SDI will be cached if the call contains these parameters
include-filter.config.ignoreUrlParams=[
    "gclid",
    "testengine-*"
]
include-filter.config.resource-types=[
    "mypage/components/slingdynamic/dynamicbanner"
]

Instead the SDI component will completely render and since my dispatcher was configured to ignore this parameter, the response will be cached with an already rendered SDI-Component instead of the SDI-Tag.

I hope this information is enough to understand what my problem is.
If there are some more informations that you need or if something is not clear enough please tell me and i will try to update everything that is needed

SDI encoding ESI

We have to append ESI $(GEO{'country_code'} to end of url

<esi:include src="/master_website/en_US/_jcr_content/hero.nocache.$(GEO{'country_code'}).html"/>

but it is getting encoded like this

hero.nocache.$(GEO%7B'country_code'%7D).html

this is throwing error at the akamai side

How can we avoid this? Since SDI also supports ESI so this should work without issues

Thanks
nitin

When Dispatcher cache is initially created - The includes are not processed

When caching a page that has SSI tags, for some reason the includes are not processed.

I.e. the page is served with
<!--#include virtual="/path/to/content/_jcr_content/root/main/some-component.caching-prevented.html" -->

However, when I then refresh that page, the include is processed and the sub-request ends up in AEM.

Component will be cached if sent request on AEM with /ignoreURLparams

If you will use dispatcher setting /ignoreURLparams and sent request on publish with this settings component will be cached because of SSI logic https://github.com/Cognifide/Sling-Dynamic-Include/blob/master/src/main/java/com/cognifide/cq/includefilter/IncludeTagWritingFilter.java#L86
In SSI thats not possible to exclude all URL params and as result component will be cached even use setting to being excluded from cache by selector.

AEM6.1 with SP1 synthetic include not working

Hi,

I have integrated SDI in my project but finding issues with synthetic include. When the node exist, include is working fine but when the node does not exist, the include does not work based on resource type.

I am getting following error-

unable to include "/content/proj/en/produce/title-of-teaser/placement/level3/_jcr_content/inlineEdit.wkd.nocache.html//apps/proj/components/content/inlineeditingtoolbar" in parsed file C:/Apache22/cache/content/proj/en/produce/title-of-teaser/placement/level3.wkd.html

Tried other way around also-
unable to include "/content/proj/en/produce/title-of-teaser/placement/level3/_jcr_content/inlineEdit.wkd.nocache.html/proj/components/content/inlineeditingtoolbar" in parsed file C:/Apache22/cache/content/proj/en/produce/title-of-teaser/placement/level3.wkd.html

The AEM version is 6.1 with SP1. I have tried changing service ranking for synthetic resource to -2400 (>-2500) and -2600 (<-2500). But no luck.

Can you please suggest what might be wrong.

ignoreUrlParams dispatcher configuration breaks rendering of virtual includes

Dispatcher module supports the definition of ignoreUrlParams, a list of query parameters that should not bypass the caching mechanism with the result being laid into the web servers cache directories. Our main use case for this is to be able to cache readily rendered responses, when certain query parameters exist, i.e for incoming URL's containing tracking links as it happens for Adobe Analytics with all "utm_*" parameters.

The dispatcher module itself then puts the rendered response into the web server cache, as supposed to. Unfortunately, IncludeTagWritingProcessor generally ignores all requests, regardless of which parameters are contained in the query string. Therefore, such requests bypass the SDI feature at all, leading to the situation where SDI-configured components are not rendered with the proper include directive.

The best solution would be to have a configurable String[] on IncludeTagWritingProcessor to be able to define the ignoreUrlParams. These should be checked against in its accepts() method to allow rendering include directives in such scenarios as well, instead of plainly rejecting all URL's containing more than zero parameters:
Enumeration<?> params = request.getParameterNames();
if (params.hasMoreElements()) {
// add check against configured list of allowed parameters (=ignoreUrlParams in dispatcher config)
return false;
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.