BlacklightRangeLimit: integer range limiting and profiling for Blacklight applications

The BlacklightRangeLimit plugin provides a ‘facet’ or limit for integer fields, that lets the user enter range limits with a text box or a slider, and also provides area charts giving a sense of the distribution of values (with drill down).

The primary use case of this plugin is for ‘year’ data, but it should work for any integer field. It may not work right for negative numbers, however.

Decimal numbers and Dates are NOT supported; they theoretically could be in the future, although it gets tricky.

A Solr integer field. Depending on your data, it may or may not be advantageous to use a tint (trie with non-zero precision) type field.

If all your integers are the same number of digits, you can use just about any solr type, including string/type, and all will be well. But if your integers vary in digits, strings won’t sort correctly, making your numbers behave oddly in partitions and limits. This is also true if you use a pre-1.4 “integer”/pint/solr.IntField field – these are not “sortable”.

You need to use a “sortable” numeric-type field. In Solr 1.4, the standard “int”/solr.TrieIntField should work fine and is probably prefered. For some distributions of data, it may be more efficient to use “tint” (solr.TrieIntField with non-zero precision).

The pre Solr 1.4 now deprecated sint or slong types should work fine too.

Recent versions of this plugin require Blacklight versions 3.2 or later, which requires Rails 3.1 or later and the Rails asset pipeline.

Add

gem "blacklight_range_limit"

to your Gemfile. Run “bundle install”.

Then run

rails generate blacklight_range_limit

This will install some asset references in your application.js and application.css.

You have at least one solr field you want to display as a range limit, that’s why you’ve installed this plugin. In your CatalogController, the facet configuration should look like:

config.add_facet_field 'pub_date', :label => 'Publication Year', :range => true

You should now get range limit display. More complicated configuration is available if desired, see Range Facet Configuration below.

You can also configure the look and feel of the Flot chart using the jQuery .data() method. On the ‘.facet_limit` container you want to configure, add a Flot options associative array (documented at people.iola.dk/olau/flot/API.txt) as the `plot-config` key. The `plot-config` key to set the `plot-config` key on the appropriate `.facet_limit` container. In order to customize the plot colors, for example, you could use this code:

$('.blacklight-year_i').data('plot-config', { 
  selection: { color: '#C0FF83' }, 
  colors: ['#ffffff'], 
  series: { lines: { fillColor: 'rgba(255,255,255, 0.5)' }}, 
  grid: { color: '#aaaaaa', tickColor: '#aaaaaa', borderWidth: 0 }  
});

You can add this configuration in app/assets/javascript/application.js, or anywhere else loaded before the blacklight range limit javascript.

In order to calculate distribution segment ranges, we need to first know the min and max boundaries. But we don’t really know that until we’ve fetched the result set (we use the Solr Stats component to get min and max with a result set).

So, ordinarily, after we’ve gotten the result set, only then can we calculate the segment ranges, and then we need to do another Solr request to actually fetch the segment range counts.

The plugin uses an AJAX request on the result page to do this. This means that for every application results display that includes any values at all in your range field, your application will get a second AJAX http request, and make a second solr request.

If you’d like to avoid this, you can turn off segment display altogether with the :segment option below; or you can set :assumed_boundaries below to use fixed boundaries for not-yet-limited segments instead of taking boundaries from the result set.

Note that a drill-down will never require the second request, because boundaries on a drill-down are always taken from the specified limits.

Instead of simply passing “true”, you can pass a hash with additional configuration. Here’s an example with all the available keys, you don’t need to use them all, just the ones you want to set to non-default values.

config.add_facet_field 'pub_date', :label => 'Publication Year', :range => {
  :num_segments => 6,
  :assumed_boundaries => [1100, Time.now.year + 2],
  :segments => false    
}

num_segments

Default 10. Approximately how many segments to divide the range into for segment facets, which become segments on the chart. Actual segments are calculated to be ‘nice’ values, so may not exactly match your setting.

assumed_boundaries

Default null. For a result set that has not yet been limited, instead of taking boundaries from results and making a second AJAX request to fetch segments, just assume these given boundaries. If you’d like to avoid this second AJAX Solr call, you can set :assumed_boundaries to a two-element array of integers instead, and the assumed boundaries will always be used. Note this is live ruby code, you can put calculations in there like Time.now.year + 2.

:segments

Default true. If set to false, then distribution segment facets will not be loaded at all.

The selectable histograms/barcharts are done with Javascript, using Flot. Flot requires JQuery, as well as support for the HTML5 canvas element. In IE previous to IE9, canvas element support is added with excanvas.

A ‘require ’blacklight_range_limit’‘ in a Rails asset pipeline manifest file will automatically include all of these things. The blacklight_range_limit adds just this line to your `app/assets/application.js`.

There is a copy of flot vendored in this gem for this purpose. jquery is obtained from the jquery-rails gem, which this gem depends on.

Note this means a copy of jquery, from the jquery-rails gem, will be included in your assets by blacklight_range_limit even if you didn’t include it yourself explicitly in application.js. Flot will also be included.

If you don’t want any of this gem’s JS, you can simply remove the ‘require ’blacklight_range_limit’‘ line from your application.js, and hack something else together yourself.

The excanvas inclusion for IE is handled a bit differently. Coudln’t get conditional inclusion of excanvas in pure JS to work, it does need an actual seperate script line in the HTML document surrounded by IE conditional comments; this gem adds that line using Blacklight’s ‘extra_head_content` feature allowing dependencies to inject content in HTML head; requires your layout to follow BL conventions, or just add excanvas yourself manually. This gem’s attempt to inject the excanvas script line can be turned off in configuration, see Injection below.

The plugin assumes it is in a Blacklight Rails app, and uses Blacklight methods, Rails methods, and standard ruby module includes to inject it’s behaviors into the app.

You can turn off this injection if you like, although it will make the plugin less (or non-) functional unless you manually do similar injection. See lib/blacklight_range_limit.rb#inject! to see exactly what’s going on.

In any initializer, you can set:

BlacklightRangeLimit.omit_inject = true

to turn off all injection. The plugin will be completely non-functional if you do this, of course. But perhaps you could try to re-use some of it’s classes in a non-Blacklight, highly hacked Blacklight, or even non-Rails application this way.

You can also turn off injection of individual components, which could be more useful:

BlacklightRangeLimit.omit_inject = {
  :view_helpers => false,
  :controller_mixin => false,
  :routes => false,
  :excanvas => false
}

:view_helpers

Set to false and the plugin will not insert it’s own rails view helpers into the app. It will raise lots of errors if you do this, you probably don’t want to.

:controller_mixin

The plugin mixes some methods into CatalogController, both over-riding Blacklight methods, and providing a new action of it’s own. Set to false, and the plugin won’t. You’ve basically disabled the plugin if you do this.

:routes

Disable automatic routes loading

:excanvas

Disables injection of a conditionally-commented script tag to load the excanvas library for supporting ‘canvas’ on IE. blacklight_range_limit does this in the controller mixin, using Blacklight’s “extra_head_content” feature to add actual conditional script tag for IE in html <head>.

See Javascript Dependencies above for disabling injection of gem’s js.

Start the Blacklight demo jetty server (on port 8983)

Run the rspec tests by running:

rspec

• StatsComponent replacement. We use StatsComponent to get min/max of result set, as well as missing count. StatsComponent is included on every non-drilldown request, so ranges and slider can be displayed. However, StatsComponent really can slow down the solr response with a large result set. So replace StatsComponent with other strategies. No ideal ones, we can use facet.missing to get missing count instead, but RSolr makes it harder than it should be to grab this info. We can use seperate solr queries to get min/max (sort on our field, asc and desc), but this is more complicated, more solr queries, and possibly requires redesign of AJAXy stuff, so even a lone slider can have min/max.

• tests

• In cases where an AJAX request is needed to fetch more results, don’t trigger the AJAX until the range facet has actually been opened/shown. Currently it’s done on page load.

• If :assumed_boundaries ends up popular, we could provide a method to fetch min and max values from entire corpus on app startup or in a rake task, and automatically use these as :assumed_boundaries.