mysticfall / pivot4j Goto Github PK

Implement Exp.validate() and Exp.normalize() methods as mentioned in #38.

Need a custom serialization support for StateHolder API to support long term persistence of the model state (i.e. saving as a file for the Pentaho platform).

Improve ChangeSlicer transform to make hierarchical member manipulation easier on slicer axis.

From the JavaDoc API description :

/**

• JPivot by default converts MDX from a logical to a physical representation
• of the metadata after an expansion, from "all children" to a list of
• the specific children. If one wants to save the query, having the query
• represent the current state of the data, the physical state, may mean
• that the saved query will not execute at some future date because children
• and come and go.
  */
  static final String LOGICAL_MDX_PROP = "com.tonbeller.jpivot.mondrian.logical.mdx";

(1) Create a common parent class for axis transformations (i.e. PlaceMembersOnAxes, PlaceHierarchiesOnAxes).

It will help users to implement their own trasnform class which modifies MDX expressions on axes.

(2) Need findVisibleMembers method equivalent for hierarchies.

(3) Support partial modification of hierarchies and members on axis. Currently, to add a hierarchy on an axis for example, one need to manipulate all the existing members on that axis and change the axis expression as a whole.

(4) Add PlaceLevelsOnAxes transform

Invoke to PivotModel.restoreState sometimes fails with the following exception :

java.lang.NullPointerException
at com.eyeq.pivot4j.query.Quax.restoreState(Quax.java:2788)
at com.eyeq.pivot4j.query.QueryAdapter.restoreState(QueryAdapter.java:1016)
at com.eyeq.pivot4j.impl.PivotModelImpl.restoreState(PivotModelImpl.java:812)
at com.eyeq.kona.web.ui.olap.PivotGridHandler.createModel(PivotGridHandler.java:165)
at com.eyeq.kona.web.ui.olap.PivotGridHandler.getDimensions(PivotGridHandler.java:602)
at sun.reflect.GeneratedMethodAccessor380.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at javax.el.BeanELResolver.getValue(BeanELResolver.java:83)
at javax.el.CompositeELResolver.getValue(CompositeELResolver.java:54)
at com.sun.faces.el.FacesCompositeELResolver.getValue(FacesCompositeELResolver.java:72)
at org.apache.el.parser.AstValue.getValue(AstValue.java:123)
at org.apache.el.parser.AstEmpty.getValue(AstEmpty.java:45)
at org.apache.el.parser.AstNot.getValue(AstNot.java:42)
at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:186)
at com.sun.facelets.el.TagValueExpression.getValue(TagValueExpression.java:71)
at javax.faces.component.UIComponentBase.isRendered(UIComponentBase.java:390)
at org.richfaces.component.UITree.processDecodes(UITree.java:472)
at javax.faces.component.UIComponentBase.processDecodes(UIComponentBase.java:1026)
at javax.faces.component.UIComponentBase.processDecodes(UIComponentBase.java:1026)
at javax.faces.component.UIComponentBase.processDecodes(UIComponentBase.java:1026)
at javax.faces.component.UIComponentBase.processDecodes(UIComponentBase.java:1026)
at javax.faces.component.UIForm.processDecodes(UIForm.java:209)
at org.ajax4jsf.component.AjaxViewRoot$1.invokeContextCallback(AjaxViewRoot.java:400)
at org.ajax4jsf.component.AjaxViewRoot.processPhase(AjaxViewRoot.java:240)
at org.ajax4jsf.component.AjaxViewRoot.processDecodes(AjaxViewRoot.java:417)
at com.sun.faces.lifecycle.ApplyRequestValuesPhase.execute(ApplyRequestValuesPhase.java:78)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:100)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:265)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:206)
at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290)
at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388)
at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at fr.xebia.servlet.filter.ExpiresFilter.doFilter(ExpiresFilter.java:1243)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.blogspot.java4it.commons.filters.gzip.GZipFilter.doFilter(GZipFilter.java:114)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.eyeq.kona.web.filter.WelcomePageFilter.doFilter(WelcomePageFilter.java:67)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:378)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:109)
at org.springframework.security.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.ExceptionTranslationFilter.doFilterHttp(ExceptionTranslationFilter.java:101)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.providers.anonymous.AnonymousProcessingFilter.doFilterHttp(AnonymousProcessingFilter.java:105)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.rememberme.RememberMeProcessingFilter.doFilterHttp(RememberMeProcessingFilter.java:116)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.wrapper.SecurityContextHolderAwareRequestFilter.doFilterHttp(SecurityContextHolderAwareRequestFilter.java:91)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.AbstractProcessingFilter.doFilterHttp(AbstractProcessingFilter.java:278)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.preauth.AbstractPreAuthenticatedProcessingFilter.doFilterHttp(AbstractPreAuthenticatedProcessingFilter.java:69)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.ui.logout.LogoutFilter.doFilterHttp(LogoutFilter.java:89)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.context.HttpSessionContextIntegrationFilter.doFilterHttp(HttpSessionContextIntegrationFilter.java:235)
at org.springframework.security.ui.SpringSecurityFilter.doFilter(SpringSecurityFilter.java:53)
at org.springframework.security.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:390)
at org.springframework.security.util.FilterChainProxy.doFilter(FilterChainProxy.java:175)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.eyeq.kona.web.filter.SessionConfigurationFilter.doFilter(SessionConfigurationFilter.java:291)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:722)

Level headers on the row axis are not visible when the showParentMembers property is set to true in TableBuilder class.

Config object passed to the constructor is ignored in PooledOlapDataSource.

We need more flexible way to configure factory classes for various parts of the core API(i.e. Transform, ExpressionEvaluator, SortMode, UICommand, etc).

We can use the service extension SPI(look up via java.util.ServiceLoader), but it need to be able to save and restore its state via #42.

Finish StateHolder.bookmarkState()/restoreState() methods implementation on QueryAdapter and Quax class.

It's useful in certain environment like when one should store and restore PivotModel instance's state between requests.

With the new Aggregator API(#44) combined with the expression language support(#37), we can support client side calculated (or virtual) measure which would provide users much more versatility than the server side alternative.

Generalize drill down & sort related code from the JSF sample project and integrate it into the PivotRenderer API to facilliate implementing common pivot grid UI.

Define an event for query execution. The event object should include the MDX statement and execution time.

Provide convenient methods for ordering members.

For example, when implementing a GUI which lets users to order members in a placed hierarchy on an axis, one needs to find out which member can be moved up or down, and also needs to move the whole descendants along with the member itself.

We need to make this kind of operations easier by adding appropriate helper methods in the PlaceMembersOnAxes transform.

Some transformations assume the axis argument reflects the swapped state of axes while others don't, which lead to inconsistent behavior and confusion.

If user has excluded some members from a parent member, it would be confusing if the aggregated value of measure of its remaining children does not add up to that of parent.

So in this case we need to provide a hint to UI implementation whether a member node contains all of its children or not.

just so you know....

I've added pivot4j to the ci server, hope thats okay:

http://ci.analytical-labs.com/job/pivot4j

Replace static logger initialization codes to prevent possible redeployment issues.

Make BuildContext to provide references to other cells on the same row/column.

For example, when implementing a conditional formatter, it needs to know other cells value on the same row or column to be useful.

Returned level list has incorrect order from PlaceLevelsOnAxes.findVisibleLevels method.

Unable to parse MDX with member names containing ']]' escape character.

For example, if member name is 'AA[BB]', then '[AA[BB]]]' should be recognized as a valid identifier in MDX.

Instead of implementing OlapConnection interface to handle connection pooling, make a proxy at runtime to avoid unnecessary JDBC 4.0/Java 7 dependency.

Incorrect table layout is generated when member levels are not contiguous.

Test MDX :

SELECT {[Measures].[Unit Sales], [Measures].[Store Cost], [Measures].[Store Sales]} ON COLUMNS, CrossJoin({[Promotion Media].[All Media]}, {[Product].[All Products], [Product].[Drink], [Product].[Drink].[Beverages].[Carbonated Beverages], [Product].[Drink].[Beverages].[Drinks], [Product].[Drink].[Beverages].[Hot Beverages], [Product].[Drink].[Beverages].[Pure Juice Beverages], [Product].[Food], [Product].[Non-Consumable]}) ON ROWS FROM [Sales] WHERE [Time].[1997]

Duplicate Order() function is generated when the original MDX already contains one.

JPivot only handles the case when the Order function is added by calling sort() method. While this is not a big problem for JPivot as it lacks saving feature, it's a major issue when saving is involved since everytime the model is loaded with a MDX containing Order() function, it will accumulate whenever the user sort the grid.

Add a generic table model to be used in both UI implementation and drillthrough feature(#3).

Invoking PivotModelImpl.sort() does not have any effect.

Missing member aggregation at the last position of the hierarchy (see column aggregation for Q2 in AggregatorIT test case).

Currently, the table builder API mandate developers to implement specific interfaces for each UI type - table, rows, and cells - which is not suitable for certain type of usage, i.e. building a XML DOM tree.

We need a more flexible API to handle such use cases.

Update pom.xml and site.xml for appropriate release information.

Upgrade to Olap4J 1.1.0

BuildContext doesn't provide position information on cells. It's only available on member nodes.

Typically, pivot grid renders its member cells on the row axis in hierarchical manner (when showParentMembers is set to 'false')using indentation.

Currently, RenderContext interface does not provide reliable information to caculate such indentation except by getMember().getDepth() which is problematic if the top level member represented in the UI is not actually the top level member in its hierarchy.

Though much more elegant way to handle cell formatting is planned to be implemented soon, the traditional 'CELL PROPERTIES' statement also need to supported for compatibility.

Note that, the range of support does not include Mondrian specific 'style' attribute of FORMAT_STRING expression.

Implement drill through feature and an appropriate table model to support it.

With JPivot, one can provide 'level of detail' when bookmarking the model state so it determines whether the state should be vaild upon data modification or not.

Add a generic XML renderer which could be extended and used by markup based renderers and exporters like HtmlRender, FopExporter, and more.

Found a very subtle problem with Mondrian's DelegatingRole, that when it wraps RolapMember instance into RolapHierarchy$LimitedRollupMember, they no longer return true when equals() method is invoked with the other as an argument, which could make various parts of Pivot4J not functioning properly.

So, it would be safe to compare between unique name properties of members instead of the member objects themselves to prevent such a problem.

Add a pooling OlapDataSource implementation, possibly using commons-pool or commons-dbcp.

Migrate more JPivot extensions as transform class, notably PlaceHierarchiesOnAxes, PlaceMembersOnAxes, SwapAxes, ChangeSlicer, and etc.

Those extensions are essential in building real world OLAP frontend applications.

Add member properties support.

Add Libre Office/MS Excel export feature by writing the appropriate PivotRenderer implementation.

Implement visual total feature(based on render strategy API).

Should be able to configure sub/grand total/avg/min/max/etc per each axis and levels.

Requirement

1. Change the existing JPivot based parser definition to Olap4J/Mondrian based one if needed.
2. Remove IDE warnings.
3. Expose parser API for public use.
4. Possibly fix #34 along the way.
5. Provide the basis for implementing #37.

Add PDF export feature by writing the appropriate PivotRenderer implementation using XSL/FO and the Batik library.

Currently, AbstractOLAPDataSource.getConnection() method expects user credentials but as PivotModelImpl class ignores them, they should be provided by connection properties only.

Allow multiple aggregators per position(i.e. - showing both total and average for axis).

Unit test failure since the sorting issue was fixed :

org.junit.ComparisonFailure: MDX has been changed after the state restoration expected:<SELECT BottomCount([{[Measures].[Unit Sales], [Measures].[Store Cost], [Measures].[Store Sales]}], 3, [Measures].[Uni...> but was:<SELECT BottomCount([BottomCount({[Measures].[Unit Sales], [Measures].[Store Cost], [Measures].[Store Sales]}, 3.0, [Measures].[Unit Sales])], 3, [Measures].[Uni...>
at org.junit.Assert.assertEquals(Assert.java:125)
at com.eyeq.pivot4j.impl.PivotModelImplIT.testBookmarkState(PivotModelImplIT.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

Need to support expression language support in MDX query :

Requirements

1. EL implementation should be pluggable.
2. Snippets should be preserved during possible query transformation like drill down, sort, and more.
3. EL is NOT allowed to change the structure of the query.

Provide convinient way to toggle sort modes.

For example, when building a common grid UI which has a sort button on header cells which cycles through the available sort modes,

it's cumbersome to determine what the current status is and what next mode to change without writing lot of if-else statements.

Level members should be added to their respective parent hierarchies when calling PlaceLevelsOnAxes.addLevel() transform.