Code Monkey home page Code Monkey logo

dntp-portal's Introduction

Request portal for Dutch pathology labs

Build Status codecov

This repository hosts the code of a request portal, built for the Dutch National Tissuebank Portal project. The portal is hosted at aanvraag.palga.nl. It allows researchers to submit requests to PALGA, the Dutch pathology database organisation.

Issues

Project members can report issues in JIRA.

Development

Technology

Tool/framework Documentation
IDE Spring Tool Suite
Maven Maven
Web application framework Spring Boot
Activiti business process modelling framework Activiti user guide
Javascript application framework AngularJS
NodeJS package manager npm

Git repository

git clone [email protected]:thehyve/dntp-portal.git
cd dntp-portal
...

Configure PostgreSQL database for development

sudo -u postgres psql
create user thehyve with password 'thehyve';
create database dntp_portal;
grant all privileges on database dntp_portal to thehyve;

Alternatively, edit src/main/resources/application.properties to change the database settings.

Important for performance: setting the indexes appropriately, e.g.:

create index var_procinst_name_index on act_hi_varinst (proc_inst_id_, name_ );
create index var_task_name_index on act_hi_varinst (task_id_, name_ );

Run, test, publish

Make sure you have npm and Maven installed.

To run the application in production mode:

# Start the application in production mode
mvn -Dspring.profiles.active=prod spring-boot:run

There should now be an application running at http://localhost:8092/.

Different profiles are available:

Profile Database Test data
dev H2, in memory Default users and roles
test PostgreSQL Default users and roles
prod PostgreSQL None

To activate these profiles:

# Start the application with test accounts
mvn -Dspring.profiles.active=test spring-boot:run
# Start the application with an in-memory database
mvn -Dspring.profiles.active=dev spring-boot:run

For front-end development, you can start a hot-reloading version of the front-end separately, after starting the application with mvn spring-boot:run:

# Start the front-end application
npm start

This should open the default browser at http://localhost:9000/.

Package

# Create a war package
mvn -Dspring.profiles.active=dev package

There should now be a .war-file in target/dntp-portal-<version>.war.

# Run the packaged application
java -jar target/dntp-portal-<version>.war

Tests

Run all tests

# Run the testNG test suite
mvn -Dspring.profiles.active=dev test

Running front-end unit testing and e2e testing: Get nodejs.

# Start the application (in a separate console)
mvn -Dspring.profiles.active=dev spring-boot:run
# Install dependencies
npm install
# Run unit tests
npm test (currenty failing)
# Prepare webdriver for e2e tests
npx webdriver-manager update
# Run e2e tests
npx protractor
# Run only a selected feature
npx protractor --specs=e2e/scenario_complete_happy_request.feature

To select a particular version of the webdriver (instead of the latest), run:

# For chromium:
npx webdriver-manager update --versions.chrome=$(chromium-browser --version | cut -d ' ' -f 2)
# For Google Chrome:
npx webdriver-manager update --versions.chrome=$(google-chrome --version | cut -d ' ' -f 3)

Publish

The project is configured to publish to the Nexus repository of The Hyve. Credentials are stored in ~/.m2/settings.xml:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                        http://maven.apache.org/xsd/settings-1.0.0.xsd">
    <servers>
        <server>
            <id>nl.thehyve.nexus</id>
            <username>USERNAME</username>
            <password>PASSWORD</password>
        </server>
    </servers>
</settings>

Publish to the repository:

mvn -Dspring.profiles.active=dev deploy

Fetch from repository

mvn dependency:get -Dartifact=nl.thehyve:dntp-portal:<version>:war -DremoteRepositories=https://repo.thehyve.nl/content/repositories/releases/ -Ddestination=dntpportal.jar

Deployment

Instructions on how to set up a production instance of the application.

Dependencies

  • Java JRE 8
  • A PostgreSQL server (version 9.5 or newer) listening on port 5432.
  • An SMTP server, listening on port 25.
    • The email server must be allowed to send emails on behalf of the reply address in the application configuration. (check the DNS records of aanvraag.palga.nl and palga.nl)
  • Package haveged installed.

Setup database

sudo -u postgres psql
create user thehyve with password '<strong random db password>';
create database dntp_portal;
grant all privileges on database dntp_portal to thehyve;

The database schema will be created at application startup.

Configure and run application

  • Create a system user account dntp

  • Create directory /home/dntp with subdirectories (owned by dntp):

    • logs
    • upload
  • Download the war file:

    curl -L -o dntp-portal.war https://repo.thehyve.nl/service/local/repositories/releases/content/nl/thehyve/dntp-portal/1.0.0/dntp-portal-1.0.0.war
  • Copy dntp-portal.war to /home/dntp.

  • Create configuration file /home/dntp/dntp.properties (owned by dntp)

    # Database credentials
    spring.datasource.username=dntp_portal
    spring.datasource.password=<strong random db password>
    
    spring.mail.host = localhost
    spring.mail.port = 25

    See this Guide to Spring Email for more about configuring email for Spring.

  • Create service /etc/systemd/system/dntp.service:

    [Unit]
    Description=DNTP
    After=syslog.target network.target
    
    [Service]
    Type=simple
    ExecStart=/usr/bin/java -jar -server -Xms2g -Xmx2g -XX:MaxPermSize=512m -Djava.awt.headless=true -Dserver.port=8092 -Dspring.profiles.active=prod -Dspring.datasource.url=jdbc:postgresql://localhost/dntp_portal -Djava.security.egd=file:/dev/./urandom -Ddntp.server-name=aanvraag.palga.nl -Ddntp.server-port=443 -Ddntp.reply-address=[email protected] -Ddntp.from-address=[email protected] -Dspring.config.location=/home/dntp/dntp.properties /home/dntp/dntp-portal.war
    WorkingDirectory=/home/dntp
    User=dntp
    UMask=0000
    Restart=always
    StandardOutput=journal
    
    [Install]
    WantedBy=multi-user.target
  • Start the application:

    systemctl start dntp.service
  • Check the application status and logs:

    # Check status
    systemctl status dntp.service
    # Inspect logs
    journalctl -u dntp.service -f
  • Set up a reverse proxy with SSL listening on aanvraag.palga.nl with http://localhost:8092 as target.

  • Test that the application is available at https://aanvraag.palga.nl.

  • Check the SSL configuration using the SSL server test.

Initial user

Create initial lab and Palga user (after the application has started):

  • Create a dummy lab:
    INSERT INTO lab (id, active, name, number, hub_assistance_enabled) VALUES (1, true, 'Dummy', 0, false);
  • Request a user account via the registration page.
  • Find id of the user account:
    SELECT * FROM app_user;
  • Find id of Palga role:
    SELECT id FROM role WHERE name = 'palga';
  • Grant Palga role to user:
    UPDATE app_user_roles SET roles_id = <role_id> WHERE users_id = <user_id>;

Configure indexes

The following indexes are not automatically created, but are important for the performance. They can be created after the application has started.

create index var_procinst_name_index on act_hi_varinst (proc_inst_id_, name_ );
create index var_task_name_index on act_hi_varinst (task_id_, name_ );

Release notes

1.0.0

User specialisms have been changed to be stored and exported in English.

update app_user set specialism = 'Gastroenterology' where specialism = 'Maag-darm-lever-ziekten';
update app_user set specialism = 'Gynaecology' where specialism = 'Gynaecologie';
update app_user set specialism = 'Dermatology' where specialism = 'Dermatologie';
update app_user set specialism = 'Medical Oncology' where specialism = 'Medische Oncologie';
update app_user set specialism = 'Internal Medicine' where specialism = 'Interne geneeskunde';
update app_user set specialism = 'Radiology' where specialism = 'Radiologie';
update app_user set specialism = 'Radiotherapy' where specialism = 'Radiotherapie';
update app_user set specialism = 'Haematology' where specialism = 'Hematologie';
update app_user set specialism = 'Throat-nose-ear' where specialism = 'Keel-neus-oor';
update app_user set specialism = 'Surgery' where specialism = 'Heelkunde';
update app_user set specialism = 'Epidemiology' where specialism = 'Epidemiologie';
update app_user set specialism = 'Primary care' where specialism = 'Eerstelijnsgeneeskunde';
update app_user set specialism = 'Cardiology' where specialism = 'Cardiologie';
update app_user set specialism = 'Pathology' where specialism = 'Pathologie';
update app_user set specialism = 'Lung Disease' where specialism = 'Longziekten';
update app_user set specialism = 'Urology' where specialism = 'Urologie';
update app_user set specialism = 'Neurology' where specialism = 'Neurologie';
update app_user set specialism = 'Endocrinology' where specialism = 'Endocrinologie';

0.0.106

New columns have been added to record the last assigned Palga adviser and the request type, and to distinguish between different sorts of materials requests. To update the database schema, run:

alter table request_properties add column last_assignee character varying(255);
alter table request_properties add column request_type character varying(255);
alter table request_properties add column block_materials_request boolean;
alter table request_properties add column he_slice_materials_request boolean;
alter table request_properties add column others_materials_request character varying(255);

Existing materials requests should be updated to select at least one of the sorts of materials request. The following command selects both blocks and slides for existing materials requests:

-- Update existing materials request to select blocks and slides
update request_properties
    set block_materials_request = true, he_slice_materials_request = true
    where process_instance_id in (
      select proc_inst_id_ from act_ru_variable where name_ = 'is_materials_request' and long_ = 1
    );

0.0.80

alter table lab_request add column return_date timestamp without time zone;
alter table lab_request add column sent_return_email boolean;

alter table request_properties add column biobank_request_number character varying(255);
alter table request_properties add column germline_mutation boolean;

CREATE TABLE request_properties_informed_consent_form_attachments (
    request_properties_id bigint NOT NULL,
    informed_consent_form_attachments_id bigint NOT NULL
);
ALTER TABLE request_properties_informed_consent_form_attachments OWNER TO thehyve;


ALTER TABLE ONLY request_properties_informed_consent_form_attachments
    ADD CONSTRAINT uk_46tqt7cfxfkb9p4coyjq4q6s4 UNIQUE (informed_consent_form_attachments_id);


ALTER TABLE ONLY request_properties_informed_consent_form_attachments
    ADD CONSTRAINT fk_46tqt7cfxfkb9p4coyjq4q6s4 FOREIGN KEY (informed_consent_form_attachments_id) REFERENCES file(id);

ALTER TABLE ONLY request_properties_informed_consent_form_attachments
    ADD CONSTRAINT fk_8i21ti5yf37152ttn4vljo275 FOREIGN KEY (request_properties_id) REFERENCES request_properties(id);

0.0.65

alter table lab_request alter reject_reason type varchar(10000);

0.0.60

alter table excerpt_list add column remark_column int4 not null default -1;

0.0.55

Set date_submitted for existing requests:

alter table request_properties add column date_submitted timestamp;

update request_properties set date_submitted =
(select t.start_time_
	from act_hi_taskinst t
	where request_properties.process_instance_id = t.proc_inst_id_
	and t.task_def_key_ = 'palga_request_review'
	limit 1
)
where date_submitted is null;

0.0.53

Modify these column definitions in an existing database:

# Add columns
alter table lab_request_comments add column comments_order int4 not null default -1;
alter table request_properties_comments add column comments_order int4 not null default -1;
alter table request_properties_approval_comments add column approval_comments_order int4 not null default -1;

# For lab request comments:

# Check if there are existing records that need to be updated
select * from lab_request_comments join comment on lab_request_comments.comments_id = comment.id where comments_order = -1 order by time_created;

# Same for request comments:
select * from request_properties_comments join comment on request_properties_comments.comments_id = comment.id where comments_order = -1 order by time_created;

# Same for request approval comments:
select * from request_properties_approval_comments join comment on request_properties_approval_comments.approval_comments_id = comment.id where approval_comments_order = -1 order by time_created;

To update the tables, an update script is available.

0.0.48

Modify these column definitions in an existing database:

alter table request_properties alter search_criteria type varchar(10000);
alter table request_properties alter laboratory_techniques type varchar(10000);
alter table request_properties alter privacy_committee_rationale type varchar(10000);

0.0.46

Add these new column definitions to an existing database:

alter table excerpt_list add column palga_patient_nr_column int4 not null default -1;
alter table excerpt_list add column palga_excerpt_nr_column int4 not null default -1;
alter table excerpt_list add column palga_excerpt_id_column int4 not null default -1;

Update existing pathology_item records to have the sequence number from the excerpt list:

update pathology_item
set sequence_number = (select e.sequence_number
    from excerpt_entry e
    join excerpt_list l on l.id = e.excerpt_list_id
    join lab_request r on r.process_instance_id = l.process_instance_id
    join pathology_item i on i.pa_number = e.pa_number and i.lab_request_id = r.id
    where i.id = pathology_item.id
    limit 1);

0.0.42

When updating to version 0.0.42, an existing database can be updated with:

alter table lab add hub_assistance_enabled boolean default true;

0.0.5

When updating from 0.0.4 to 0.0.5, an existing database can be updated with:

alter table excerpt_entry add selected boolean;

License

Copyright ยฉ 2016โ€“2021 Stichting PALGA

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.

dntp-portal's People

Contributors

aochagavia avatar berndvdveen avatar dependabot[bot] avatar ewelinagr avatar gijskant avatar metfriet avatar rnugraha avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

slaap123

dntp-portal's Issues

Can't deploy to Tomcat

I'm on Ubuntu 16.04. I can run DNTP just fine with mvn spring-boot:run, but when I deploy it to tomcat 7 I get this exception (and similar more) on startup:

   [ERROR] o.s.b.SpringApplication - Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'embeddedServletContainerCustomizerBeanPostProcessor': BeanPostProcessor before instantiation of bean failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'metaDataSourceAdvisor': Cannot resolve reference to bean 'methodSecurityMetadataSource' while setting constructor argument; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'methodSecurityConfiguration': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire method: public void org.springframework.security.config.annotation.method.configuration.GlobalMethodSecurityConfiguration.setPermissionEvaluator(java.util.List); nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'customPermissionEvaluator': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: business.security.CustomPermissionService business.security.CustomPermissionEvaluator.permissionService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'customPermissionService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private business.services.RequestService business.security.CustomPermissionService.requestService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'requestService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private business.services.LabRequestService business.services.RequestService.labRequestService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'labRequestService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private business.services.RequestFormService business.services.LabRequestService.requestFormService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'requestFormService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: private business.services.RequestPropertiesService business.services.RequestFormService.requestPropertiesService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'requestPropertiesService': Injection of autowired dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Could not autowire field: business.services.FileService business.services.RequestPropertiesService.fileService; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'fileService': Invocation of init method failed; nested exception is java.nio.file.AccessDeniedException: upload
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:478)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
    at org.springframework.context.support.PostProcessorRegistrationDelegate.registerBeanPostProcessors(PostProcessorRegistrationDelegate.java:240)
    at org.springframework.context.support.AbstractApplicationContext.registerBeanPostProcessors(AbstractApplicationContext.java:687)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:523)
    at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:118)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:760)
    at org.springframework.boot.SpringApplication.createAndRefreshContext(SpringApplication.java:360)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:306)
    at org.springframework.boot.context.web.SpringBootServletInitializer.run(SpringBootServletInitializer.java:150)
    at org.springframework.boot.context.web.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:130)
    at org.springframework.boot.context.web.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:85)
    at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:169)
    at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5573)
    at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
    at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
    at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
    at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
    at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:1091)
    at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1980)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

From googling it seems to be a rights issue but which folder is the problem?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.