indicia-team / client_helpers Goto Github PK

When setting up a text input with the Default Control Type set to time_input in the Warehouse survey attribute settings, I see the follow error on the Drupal data entry form

"TypeError: undefined is not an object (evaluating '$.browser.mozilla')"

(Note: this even occurs on other browsers such as Safari, it still says mozilla)

I am wondering if this is Drupal 8 related, seems to be ok in iRecord (which is Drupal 7 at time of writing) where the little arrow icon next to the field are correctly shown.
Going to leave as simple textbox on current project as it isn't worth the budget for me to fix.

If you add a new Indicia page under Drupal 7, then select any prebuilt form and load the settings, the expandable fieldsets containing groups of edit controls won't expand. The HTML output by the call to prebuilt_forms_ajax.php does not contain the additional inside the legend element required for D7.
A brief investigation indicates that the JS which extracts the generator from the meta tags is running inside an iframe (the admin overlay). Therefore it cannot see the generator tag as this only exists on the parent page not the iframe.

The [species map] control available in the dynamic sample occurrence form allows the sample method ID of the subsamples it creates to be specified using a parameter @sampleMethodId=n. It doesn't work without this option even though it should be optional, because the subsamples submissions have sample_method_id=undefined, rather than not specifying the sample_method_id at all.

When using the "Other" option for a termlist based control on a data entry form, it does not currently work if the control type is a drop-down list.
If an "Other" option is setup, it should result in the display of an extra field for the entry of additional information but this fails to work for this control type.

Would be good to have the "Plausible" and all other statuses shown against records in the verifiers' download (and in all other downloads). Currently, plausible records are shown as "Record Submitted", and there is no distinction between Correct and Considered correct, etc.

On a prebuilt form which shows a report there may be a param_ignores option.
This exists to allow parameters of the report to be ignored.
Where parameters are later supplied via an Auto Params Form, the ignored parameters appear as hidden inputs to the form.
If the param_ignores option exists but is blank a spurious hidden input is created.

After some investigation after an issue with NPMS, I discovered some problems with the elegance of the code that shows the Submit/Delete buttons on prebuilt forms

For instance, when I had the dynamic_sample_occurrence running in Wizard Mode it called getSubmitButtons (which is a method that appears in several prebuilt forms).
This always seemed to include a Delete button that appears before the Submit button, I will be committing a fix shortly that swaps the two button around (as this causes the Delete to fire when the Return key is pressed)
It also seems to have no way to remove the Delete button with configuration. Users are always able to delete their data even if we don't want them to (unless we use jQuery to hide).

I switched the screens to tabbed mode (with the "submit button below each tab" option on) and this seemed to call data_entry_helper::wizard_buttons, which doesn't display the delete button in the same situation.

It would be better if the same area of code was called to display the buttons no matter what the screen mode was. Currently very confusing. As I just mentioned, which buttons are displayed also seems to vary with the mode.

According to the documentation at http://indicia-docs.readthedocs.io/en/latest/developing/reporting/report-file-format.html a column element may have a template attribute which provides an HTML template to put the column information in. The img attribute similarly formats image columns as HTML.

It turns out that these are processed by the client website, not by the warehouse report engine, and has only been implemented for report grids. It is also desirable to have this feature for formatting popup information on report maps.

If the Name Filter is set to excludeSynonyms then $options['extraParams']['query'] is set to
{"where":["(preferred='t' or language_iso<>'lat')"]} in species_checklist()
This happens in the lines

      // Apply a filter to extraParams that can be used when loading the initial species list, to get just the correct names.
      if (isset($options['speciesNameFilterMode']) && !empty($options['listId'])) {
        $filterFields = array();
        $filterWheres = array();
        self::parse_species_name_filter_mode($options, $filterFields, $filterWheres);
        if (count($filterWheres))
          $options['extraParams'] += array('query' => json_encode(array('where' => $filterWheres)));
        $options['extraParams'] += $filterFields;
      }

Later, in get_species_checklist_taxa_list(), if there is also a Taxon Filter set, the above is overwritten by the lines

      $qry = array('in'=>array($options['taxonFilterField'], $options['taxonFilter']));
      $options['extraParams']['query']=json_encode($qry);

@johnvanbreda @andrewvanbreda @JimBacon - can anyone help me understand what's going on here?

I'm using the client_helpers lib to invoke a PoMS report on the Warehouse from a module outside iForm. I want to get the current IndiciaUser's records and it all works except that it has become apparent that I am only getting the user's records for the current year. I can't understand where that filter is coming from.

You can get a feel for how I'm calling the report from these PHP extracts:

$opts=array(
  'mode' => 'report',
  'dataSource' => trim($report['value']),
  'readAuth' => $read_auth,
  'sharing' => 'me',
  'userId' => $indiciaUserId,
);
report_helper::request_report($response, $opts, $currentParamValues, false);

The report SQL looks like this:

<query>
  select csn.id as sample,
  cttl.taxon as taxon,
  csn.attrs_json -&gt;&gt; '1050' as flower,
  coalesce((con.attrs_json -&gt;&gt; '666')::int, 0) as pomscount
  from cache_occurrences_functional o
  join cache_occurrences_nonfunctional con on con.id = o.id
  right join cache_samples_nonfunctional csn on o.sample_id = csn.id
  join cache_samples_functional s on s.id = csn.id
  join cache_taxa_taxon_lists cttl on cttl.id = o.taxa_taxon_list_id
  join websites w on w.id=o.website_id
  #agreements_join#
  WHERE #sharing_filter# 
  and o.survey_id in (636, 641, 637)
  and cttl.taxon_group LIKE 'Pollinator%'
  and extract(month from o.date_end) &gt; 3
  and extract(month from o.date_end) &lt; 10
  and array[214127, 1612, 1606, 1605, 1614] &amp;&amp; s.location_ids = true
</query>

There's no filter on year in the SQL, so I'm wondering if it's something to do with the implementation of #agreements_join# and #sharing_filter# but I've looked at the ReportEngine code and I can't see how that would be either.

Does anyone have any insights?

On the "Create an activity" page you can add additional administrators. You have to use "Surname, Forename" to find the person you want. Quite a few surname-forename combinations are non-unique, making it impossible to know if you are assigning the correct user to the admin role. We shouldn't expose email addresses in this drop-down list, but would it be possible to show user names alongside the surname-forename?

If you are invited (or self-invite) to a group that you are already a member of, the code does not check if you are already a member and therefore you can double up your membership. This can result in a group admin seeming to lose admin rights as only one of the memberships will apply.

At least 46 records of the soldierflies group are flagged as "Input still in progress" - does this matter? Most are from source "UKBMS | UKBMS Transects", plus a few from "Norfolk Biodiversity Information Service | CWS digitisation species name check".

On the group report page (prebuilt_forms/group_home.php), the report parameters for any report defined in the page's user interface setting will be updated to reflect the settings for a group.
For example if a group filters to taxon group butterflies, then this setting is automatically added to the extra parameters submitted for every report's query used to build the page.
However, if the page configuration includes its own extra parameters for a report then these parameters will overwrite the group filter so the group filter does not get applied to the report.

For example the following settings define the sort order of a freeform report using the @extraParams option, in this instance the group filter will get ignored:

[report_freeform]
@dataSource=library/taxa/filterable_explore_list
@header=<table><thead><tr><th>Species</th><th>Number of records</th></tr></thead><tbody>
@bands=[{"content":"<tr><th scope='row'>{common}</th><td>{count}</td></tr>"}]
@footer=</tbody></table>
@extraParams={"orderby":"count","sortdir":"desc","limit":10}

If a CSV file has a non-alphanumeric character in one of the titles, then the column's mapping cannot be remembered for future imports.

When we upgraded the NPMS training site to the latest development branch I noticed the common names on our preloaded species grids were not displaying. It was displaying a token {common} instead. This appears to occur on master also
This is actually an npms_sample_occurrence form that extends dynamic_sample_occurrence

I noticed that on a normal dynamic_sample_occurrence that the common name wasn't displaying at all (not even a tag) with exactly the same form settings.

I think the issue started to appear because of the data_entry_helper changes here

4cbb835#diff-776c9b7328c594476e240f1ec163748d

Note that refreshing the screen does not appear to be enough, I seem to need to close and re-open the browser to get the problem to appear/reappear

The behaviour with just an extra species list (not preloaded) appears to be different yet again. When the problem is occurring with the {common} tag on npms_sample_occurrence switching to extra species results in no common name at all

As a last ditch test, I just tried master branch, normal dynamic_sample_occurrence. Both Species and Extra Species list selected, "Include both names in species controls and added rows" select and a species list filter, and the common name still wasn't visible (and yes the species does have a common name)

Hello, I have just installed the Indicia modules for Drupal following this page
https://indicia-docs.readthedocs.io/en/latest/site-building/iform/installation.html#installing-the-indicia-modules-for-drupal

When I try to run these steps:
Now, visit Configuration > Indicia integration > Indicia settings on the admin toolbar and set the following options:

I get the following error message when clicking on 'Settings'

Fatal error: require_once(): Failed opening required 'modules/drupal-8-module-iform-4.0.0/client_helpers/helper_base.php' (include_path='/opt/drupal/vendor/pear/pear_exception:/opt/drupal/vendor/pear/console_getopt:/opt/drupal/vendor/pear/pear-core-minimal/src:/opt/drupal/vendor/pear/archive_tar:.:/usr/local/lib/php') in /opt/drupal/web/modules/drupal-8-module-iform-4.0.0/iform.module on line 1020

I can not see in the code where this should be changed - is this a known issue?

Thanks!

Using the Report Map prebuilt form, when the layer picker option is set to included no map appears.

Selecting the layer picker causes the standard layerSwitcher control to be removed. This can leave a non sequential array of standardControls which, when json-encoded, becomes an object rather than an array. This breaks javascript expecting an array resulting in the map not appearing.

It seems the latest version of google maps has caused a breakage between it and Openlayers where if you use 'zoomToExtent' in the code, then the user cannot zoom in and out properly as the boundary is redrawn for the zoom, but the original boundary is also drawn. This used to be ok.

We have seen this behaviour with NPMS sites when using the "zoom_id" functionality in dynamic location to initially zoom the map to a boundary ghost location given as an id in the URL.

To get round this issue I have had to avoid using zoomToExtent, and centered the map on the boundary using setCenter. Then I used the options supplied to the map control in the form structure to supply the intial_zoom option.
Note that you could center the map on the boundary, and then use zoomTo in code, but this seems to zoom non-centrally to the boundary.
The alternative it to zoom, then set the center, but doing this seems to cause the original issue.

As replacing zoomToExtent with setCenter and then supplying a configuration option is only a workaround (albeit an effective one), I have only committed the fix to the NPMS bitbucket code (commented out old code with a note to explain), not the general GitHub code. Maybe the zoomToExtent will start working again with further updates to openlayers and google maps.

It is possible to set up an IForm instance to use multiple warehouses, each specified at an individual node level. The report picker in the form definition does not work when the node specific warehouse differs from the general IForm warehouse setting.

It should be possible to update an existing sample with information uploaded from a csv file. Because we are performing an update, the sample must already contain values for all the mandatory fields. However the process baulks in two ways if the csv file omits mandatory data.

Firstly, the user interface continues to show required fields and does not enable the upload button after selecting an update method.

Secondly (having hacked the html to enable the upload button) the warehouse rejects the sample upload. In the errors.csv it complains about a missing sample attribute which is required but, in the log there is more:

2018-10-30 17:47:36 +00:00 --- debug: Record did not validate
2018-10-30 17:47:36 +00:00 --- debug: Field date_type: Please supply a date for your observation.
2018-10-30 17:47:36 +00:00 --- debug: Field entered_sref: The spatial reference must be supplied.
2018-10-30 17:47:36 +00:00 --- debug: Field entered_sref_system: The spatial reference system must be supplied.
2018-10-30 17:47:36 +00:00 --- debug: Field geom: The spatial reference must be supplied.
2018-10-30 17:47:36 +00:00 --- debug: No value for Honey time at location in Array

This is not behaviour I would expect.

A post in the NBN forum reports warnings

Strict warning: Declaration of iform_big_sea_survey::getEntity() should be compatible with 
iform_dynamic_sample_occurrence::getEntity(&$args, $auth) in require_once() (line 37 of
iform/client_helpers/prebuilt_forms/big_sea_survey.php).

Strict warning: Declaration of iform_tree_sample_occurrence::getEntity() should be compatible
with iform_dynamic_sample_occurrence::getEntity(&$args, $auth) in 
form_helper::prebuilt_form_picker() (line 68 of iform/client_helpers/form_helper.php).

Looks like this might be a consequence of 6ee3a16

When using New species grid rows use previous row's data option in dynamic_sample_occurrence.php, all occurrence attributes under Columns to include option will be copied into the next row except Determination Date. I tried different notations like DeterminationDate, Determination Date, occAttr1 etc.. Maybe it is because of its date value?

On the standard parameters (report filters) popup, the When panel has 2 date pickers for the start and end dates. These date pickers have no validation applied, so you can enter rubbish and will only be told when the report query returns with an error.
Suggestion is to enable jQuery validation where appropriate on the report filters popups.

If editing an existing record on the sample with occurrences form, the map should always pan and zoom initially to the record's grid square/point/boundary.

Reported by Jürgen Brück

1. leave 'display location from user profile' unticked. Works, when reloading existing data. Map bounds are ok.
2. with ' display location from user profile' ticked. Map Bounds come from boundary from user profile. Never from the record's bounds which is outside the maps's display.
3. with ' display location from user profile' ticked and option remember position ticked. Works, displays existing data centered and zoomed.

If you try and load up Record Details 2 using a training record, it just says that the record cannot be found. I don't see why this type of record should be prevented (although it should say it is training on screen).
It is currently confusing for clients who don't know why it went wrong.

If you have a report grid on dynamic report explorer with a download link such as follows:
[report_grid]
@Datasource=library/occurrences/explore_list
@downloadlink=true

And then you use the column configuration (on the edit tab) to show a column that would otherwise be hidden (because the report has visible='false' on the column), then it will appear on the report grid, but it fails to appear in the download CSV file.

The new Edit Taxon (dynamic taxon) non-warehouse page for websites has an issue with synonyms.
Currently it can be loaded with a synonym, however saving seems to auto-switch this to be the preferred species without informing the user.
This page probably shouldn't allow editing of synonyms apart from in the synonyms box for preferred species.
Perhaps load the preferred species if a synonym load is attempted?

When using the "Other" option for a termlist based control on a data entry form, it does not currently work if the control type is a drop-down list.
If an "Other" option is setup, it should result in the display of an extra field for the entry of additional information but this fails to work for this control type.

A check_or_radio_group can have an 'Other' option which reveals a text box to allow a text attribute to be submitted when none of the given terms are appropriate. It does not work when the 'other' is a location attribute.

If using the standard sample with occurrences form, you can add a control called drawPolygon or drawLine to the map. This allows you to draw a shape to define the location of the record(s) added, as opposed to inputting records against a single grid square or point.

When you reload a record of this nature for editing, the polygon is not correctly drawn on the map so resaving the record results in the polygon being replaced by the grid square/point of the record.

I have recently pull the code from this repository and copy on to my website setup on my machine using iForm module. I access the Indicia page setup on my machine and immediately see this message "Option fieldname not found" and not showing any attributes on the form. I have then tried and repeat same steps on test site which has produced same error message.

Commit 1797e77 and related commits changed the UI and the terminology used when selecting a prebuilt form via the Drupal iform module. This needs to be updated in the documentation.

• Change of some form names (using Customisable rather than Dynamic)
• Description of the new recommended checkbox.

Raising this as a general Indicia note, as I tried to get this to work on the PoMS project, but it is not considered essential for the project so I have no further budget to work on it.

Page
https://ukpoms.org.uk/enter-fit-count

Attribute on page
How much of your 50x50cm patch was occupied by the target flowers?

Sample attribute 1052 in BRC Live Warehouse

Termlist 669

The icons from this termlist don't seem to appear on the website.

On the standard filter panel, the "What" popup allows you to search for the taxa to include by taxon group, species, family and others. When searching for a species or family you are currently shown a drop down to pick from populated with just the taxon name label itself. For example a search for Redshank reveals 3 direct matches but no way of determining which is a plant, moss or bird.

Therefore the proposal is to modify this so you can see the search name, plus either the common name or preferred latin name, plus the taxon group, just like when searching for a taxon on one of the main recording forms.

Requested by FSC.

Notice: Uninitialized string offset: 0 in helper_base::http_post() (line 786 of \iform\client_helpers\helper_base.php) arises when editing an existing record.

helper_base::http_post() is being called with a postarg of sharing=''. This empty value is giving rise to an error when the first character is tested for '@'.

This arises because data_entry_helper::get_population_data() is being called with a sharing value of false from data_entry_helper::load_existing_record_from().

Merge develop into master and make master the active branch on iRecord.

Ability to convert a report page output to PDF format.

On verification 5, redetermining a record shows a popup that includes a search box for choosing the new taxon. On second use, the box is missing so you cannot redetermine.

If a field is locked by a user, and they go in to edit existing data, they the field in the existing data will be overwriten by the locked value.
This is evident for managers, and is further complicated if the field is hidden for certain user types - e.g. if a manager masquerades as someone else to enter their data with fields locked.

See Indicia-Team/warehouse#211.
The client code in get_read_auth should not cache an invalid nonce.

A check_or_radio_group can have an 'Other' option which reveals a text box to allow a text attribute to be submitted when none of the given terms are appropriate. If more than one such pair of controls is added to a page, only the final one will work.

If an app or a website form sends a record with training=true then the record is saved with a training flag, which is correct behaviour, even if the user account on the hosting website (ie irecord) does not have that mode turned on. This allows to turn on the training mode per form/app which is great. But it doesn't work vice versa.

The problem lies when an app or anything else that submits a record through the hosting site without a training flag and that record is flagged as training because of the mode on the hosting site. So now a user using iRecord website with a training mode on and the iRecord App with a training mode off, sends all the records from the app as if they were training ones, which it shouldn't.

I think it would be more correct if the hosting site had a training flag on every form generated (hidden) and not the warehouse going to fetch it after it receives a record.

I think this is the line that does that:

Using Master branches for everything. Using a Drupal-8 site.
Safari 12.0.2
Hierarchical drop-downs do not load their date when loading a data entry page.
Going to identical address in Firefox works perfectly.
Seems to be fine for non-hierarchical.

This has implications as it would mean further saves made by user may result in data loss.

When editing a pre-built form there can be problems with certain html entered in to <textarea> fields because the contents are not html-escaped upon display.

For example, if the form structure field for a recording form contains a </textarea> tag it is interpreted as the end of the area rather than part of the content.

Pull request to follow.

On the prebuilt form, ukbms_sectioned_transects_input_sample, if you enter the details of a new transect and click Next you are shown the species data input grid. There is a Back to Visit Details button. When clicked I'd expect to be able to return and edit the transect sample details I just input. Instead you are shown a fresh transect details entry form for a new sample.

The sample ID is not passed through in the URL causing this problem.

It seems to have disappeared unexpectedly.

I think a bug has been introduced in 47a0598

It has manifested itself during csv upload via http://testwarehouse.indicia.org.uk/index.php/occurrence the symptom being that the chunked upload to the warehouse never runs because the onload javascript output in import_helper::run_upload() is never executed.

Javascript debugging shows this is because there is no document.ready function hence indiciaData.documentReady is never set to done. This would be the case if (!empty($javascript) || !empty($late_javascript) evaluates to false.

The dynamic_sample_occurrence.php prebuilt form includes a get_control_locationautocomplete() which sets a default orderby using the location 'name' field.
This is of dubious value since all autocomplete controls default to sorting by the field they display and it seems unlikely that you would want to sort by any other field.
However, since the feature exists, it should be possible to override the default orderby through options. The use of array_merge_recursive prevents this.
This has been encountered when trying to use the location autocomplete to select locations by centroid_sref rather than by name.

Relates to BiologicalRecordsCentre/iRecord#1047