Allowed for owners and admins.

Bug report
currently license selection in the backend hapens through a hardcoded set of switch statements which resolves the url string.
This url is not unique, and results in errors in the frontend representation as well.

change this to use an enum in the interim. and adapt frontend and backend to use these enums to communicate, instead of the url of the license, which is NOT unique.

See class AbstractTemplateExportScienceEuropeComponent.class
method composeTableDatasetPublication()

Description
We currently do not save/source license information for the reused datasets.
So this information has to be input manually in the export template.

Proposed solution
Expand OpenAIRE API functionality for getting published dtasets by license information.
Then add it to the "reused datasets" table in the export template.

Additional context
At the time of implementing there was an issue getting this information from OpenAIRE API.
OpenAIRE APIs are notoriously tedious to implement/change, as the documentation of the transmitted data is different from the csv file describing it, and both are different from the actual data being received.

After consulting with OpenAIRE directly they suggested the following for finding license data:

the license is available in the metadata you get from the API.
It is at the path //result/instance/license.

Please note that:
- not all datasets have an instance with license;
- some datasets have more than one instance (e.g. manifestations of the same datasets)
- instances of the same dataset may have different licenses. This is more common among results of type "publication" (e.g. a pre-print comes with a license and the published version with a different one) and less common among datasets, but still it can happen.

Consult with Data Steward on how to handle different licenses being returned.

Description
The current instructions are a bit drab.
Improving the presentation of the steps could make it more inviting

Proposed solution
See attached image fr inspiration.

Describe the bug
When exporting a document the datasets listed in the storage section 3a will sometimes have a reversed sequence (i.e. P* IDs go from highest to lowest).

Is this a regression?
No.

Steps to Reproduce
Currently unknown how to reproduce.

Expected behavior
The P* IDs of the datasets should always be in ascending sequence.

Example
"P3 (Marekting materials), P1 (Survey whith Horizon Projects) will be stored in..."

Environment

• Damap version 2.4.0

Changes

• SE and FWF templates have several texts that are either wrong or imprecise, as well as some inconsistently formatted elements.
• add column license to reused dataset table
• export data quality control information from last question step 4 to template placeholder [dataqualitycontrol] (section 2b)
• change order of exported datasets in table 3b to list all P-datasets first, then all reused ones
• set the default accessibility value of produced datasets for "the public" to "no access"
• set the default accessibility value of reused datasets to "read only" for everyone
• fix how [personaldata] text (section 4a) is constructed
• fix how [legalrestriction] text (section 4b) is constructed
• fix how [ethicalissues] text in section 4c is constructed
• fix 3b "other measures" is being ignored
• publication and deleted tables
  • suppress license information for closed datasets in publication table (5a)
  • set default estimated publication date in data publication table (5a, not asked by the tool), set to end of project minus 2 months
  • remove deleted datasets from long term preservation table (5b)
  • set (10 year) default minimum retention period of data (5b)
  • add column ID to overview of deleted datasets (5b)
• Change order of datasets listed in Storage text section to be ascendant instead of descendant
• When composing the storage text the different storages should be more prominently separated (add newline)
• in App
  • Step 4 intro text change
  • landing page text change

Additional notes
These are collected in a document that cannot be shared publicly, but will be resolved in bulk.

Risks? (optional)
Changes to the word template are not tracked by git in detail. instead it will always show the entire document as changed.
Be careful when applying changes when other people are working on the same document.

Describe the bug
When exporting an FWF document and viewing it in Word (Libre office works fine), then the main table will sometimes be wider than the page, and the content clipped.

Is this a regression?
Not sure, FWF export was not as readily available for testing until now.

Steps to Reproduce

1. Create DMP
2. Fill out textfields with the maximum number of characters
3. Export document
4. View in Word

Expected behavior
The template should ideally keep within the limits of its main table and keep its original formatting.

Environment

• Microsoft office Word 365
• Damap version 2.4.0

Additional context
We do suggest that users prefer Word, as that is the format we export to, so it is problematic if that is the application we fail in representing the output correctly.

Add the Horizon Europe DMP template to the list of supported export templates.
Utilize the funder ID to recognize the funder and utilize this template for the export if it corresponds to the science europe program.

Two issues were created to address Christianes feedback mentioned in the comments:

• #141
• #142

When a new project is selected, then the project leader of said project is gathered from the project database and automatically assigned the role/flag of contact. Additionally they should be assigned the "Project Leader" contributor-role.
This helps to differentiate between the two main roles/responsibilities tied to a DMP: that of contact person for the DMP, and that of the Princiupal Investigator of the project. These might be vested onto two different people or be held by the same person. We assume by default the same person, and the user will manually change this if incorrect.

Description
Users must be able to see what kind of personal data Damap has stored about them.

Proposed solution
Add an API, that allows these data to be queried by users (via the frontend), and possibly by other (connected) tools.
Use annotations to mark GDPR related fields, so that it can be easily adapted in the future.

• Include audited data

Description
The default text at the start of the storage section in the export document is currently a hardcoded phrase in the document.

Change it to be a composite phrase after the following pattern:
“For the duration of the project, storage and backup of data will be ensured by [name] (acting as the person responsible for data management and DMP) in cooperation with the system operator. "

where [name] is the contact person in SE template / person responsible for data management and DMP in FWF template (I.2).

Also, allow the phrase to be replaceable, depending on whether internal storage options were selected or not.
This will allow institutions to change this text in order to indicate responsible departments within their organization if such a service was selected.

Proposed solution

1. Remove the existing introductory phrase "For the duration of the project, storage and backup of data will be ensured by the project manager."
2. replace it with a single placeholder [storageintro]
3. compose the new string from resources text and name of the project coordinator.
4. there should be 2 variations of the text in resources depending on whether internal storage options are selected and utilized.

In the frontend the user should be able, like with the search field for their person database, to search for researchers in the ORCID database by using name or ORCID-ID.
this request should be managed by the backend, which will use the institutions ORCID credentials to make the request with ORCID, and then map it to our ContributorDO objects for the frontend to consume.

Changes
Add an admin user in sample-damap-realm-export.json to make testing easier.

Description
Affects science europe table 5a (Data publication and access conditions) and equivalent IV.2 in FWF template.
Insert in the PID column the PID type information ("PID Systems") extracted from r3data.

Proposed solution
Perceive the PID type from r3data.

Alternative solution
If not available in r3data then allow implementing institutions to have a list of repositories that will provide a hardcoded information.
e.g. TUWRD would dictate PID type "DOI"

Additional context
Currently the field is empty, and researchers do not know what input is required.

If this issue requests multiple changes consider using checkboxes or lists.

Update Java to version 17

Changes
Currently, all URLs in the ELicense list follow the pattern of .../licenses/. However, it's been observed that the URL structure is being updated to .../license/, and some URLs have already started redirecting to this new structure.

Additional notes
List any notes you collected that support the original task and include a screenshot if applicable.

Assumptions
List any assumptions that you are aware of but still need to be reviewed.

Acceptance Criteria
Describe how you think a completed task would look like or how it should work.

Risks? (optional)
Potential risks and how to best mitigate them.

When exporting a document, the version table list is not prefilled.
Available DMP versions can be inserted into the table to reduce manual work and possible user errors.

Additionally, the license of the DMP is set automatically. This might not be desired.

The problem

Once a dev wants to pull damap-backend to build a customized project by putting the following in its pom.xml:

<project>
  <repositories>
    <repository>
      <id>github</id>
      <name>GitHub TUWien CSD Apache Maven Packages</name>
      <url>https://maven.pkg.github.com/tuwien-csd/damap-backend</url>
    </repository>
  </repositories>
  ...
  <dependencies>
    <dependency>
      <groupId>at.ac.tuwien</groupId>
      <artifactId>damap-backend</artifactId>
      <version>1.4.0-SNAPSHOT</version>
    </dependency>
   ...
 </dependencies>

this makes Maven fail because of a 403 response from GitHub packages https://maven.pkg.github.com.

is a common issue

which can be seen in the following issues:

• https://github.community/t/how-to-allow-unauthorised-read-access-to-github-packages-maven-repository/115517
• The mother of issues of this kind: https://github.community/t/download-from-github-package-registry-without-authentication/14407/146 leading to the officially accepted workaround (more below).

the officially accepted workaround

In the following issue excerpt, initially created for another bug. I was asking the support engineer about this problem and the following was his reaction:

...

But now I came across to the limitation that only authenticated users can download maven artifacts from GH packages. I know there >> are many discussions around this, but do you think it might be worth to wait for GitHub to allow unauthenticated access? Or is that still >> in the "clouds"?

Unauthenticated access is still very much in the clouds.

If you're looking for a way to allow users to clone and build, you could try this workaround.
https://github.com/jcansdale-test/maven-consume

I'd be interested to hear what solution you end up settling on!

So using https://github.com/jcansdale-test/maven-consume to inject a PAT (Personal Access Token) in each developers settings.xml is the officially accepted workaround.

A link to a very expressive snippet from another user: https://github.community/t/download-from-github-package-registry-without-authentication/14407/148 (just click on the snippets header to unfold it)

but since this is a open source project

this should work as it is intended in Maven, without letting users authenticate to GitHub to pull public packages. And I am pretty sure GitHub wants that too, but it seems to be a matter of time and a matter of interests in solving this.

We should open a separate support ticket for GitHub, just in order to get updates about the current state and work and possible future solutions about this bug, since we would really like to use GitHub Maven package repository for our open source projects, because it would all be under one hood, which makes a few things easier. If there is no chance to get this work as it should in open source environments, we would have to choose another provider for Maven artifacts.

Container packages are well supported on GitHub and are already in use and I think we are happy with it.

Changes
Currently, available licenses are managed on the backend as well as on the frontend. Moving the logic of available licenses to the backend only should reduce duplicate code and allow us to define a single source of truth.

Additionally, license enums can be removed. With the current state, if an institution wants to add licenses, they would have to be added to the frontend code and to the backend code. For simplicity reasons and decoupling reasons, moving licenses from enums to their own database table would allow to simply add/modify database entries without having to touch code.

Additional notes

Assumptions
The license selector in the frontend has to receive all required information. This has to be inspected further.
Incoming requests specifying a license will have to validate the license:

• does the license exist?
• is the license soft-deleted?

Acceptance Criteria
Available licenses are fetched from the backend and displayed on the frontend. The frontend should not have to deal with logic regarding availability of a license but simply receive and display licenses.

Risks? (optional)
Currently specified licenses for datasets will have to be migrated to their own database table and referenced accordingly. In this regard, it also makes sense to think about whether a dataset can be assigned multiple licenses. For example, a dataset only available as zip may contain several files with different licenses applying to different files.

Issues

Feature Requests

• Description
• Proposed solution

Bug Reports

• Is this a regression?
• Description/steps to reproduce

Question or Documentation Bug

• Description

Checks & Notes

If an issue addresses multiple changes create checkbox list.

Pull Requests

• Type (Feature/Bugfix/CI/Refactoring/Config/Documentation/...)
• Description (What does this PR do?)
• Issue reference (GH-reference)
• Breaking changes
• Code review focus

Checks & Notes

• Tests
• Created e2e tests
• Ran e2e tests
• Documentation

Frontend:

• Summary updated
• Version view updated

Backend:

For DB changes note that names should not exceed 30 chars, audit tables have been created/updated and have no FKs on entities.

• Tested on Oracle/PostgreSQL
• Export updated

Changes
Currently the title page will split the tables below onto the next page if the title turns out to be too long.
Change this behavior to always keep the tables on the first page.

Additional notes
If changes to the template itself are necessary (not just the code composing it), then those changes need to be done on any custom templates of implementing institutions as well. This will be up to them, but the changes must be described in detail and the information needs to be propagated.

Assumptions
Assume that any implementing institution might have longer templates, as they might add additional things like a logo.

add field to provide additional information (see word template 6b) → use description field, but if description is too long use additional description below the table

Description
Currently the design of the application does not allow to retire internal storage options from the available list of options.
Simply deleting them will result in a foreign key conflict.
So we must be able to remove them in a way that would allow continued support for old DMPs that have these options selected, while removing it from the list of selectable options available in the frontend.

Proposed solution
Possibly add a column (e.g. "active") to the database, to indicate if a storage option is deprecated.
The frontend needs to hide this option if tagged as inactive.
We still need to communicate this option to the frontend, in order for it to display it if it was selected in a past DMP.
The backend validator needs to alert, in case the user attempts to indicate an inactive storage option in their request.

Alternative solution
The above solution is from the top of my head, please feel free to use different approaches.

Additional context
Internal storage options are added by implementing institutions in their custom implementation. They should have a possibility of changing these options after the fact.

Changes
Currently we have a single file managing all liquibase statements (db/changeLog.yaml).
Instead we want to be able to create new files for each (set of) changeset.
So we should adopt a coordinator/manager file, which includes all newly created files.
The coordinator file will then replace the changeLog.yaml in the configurations.

Additional notes
This will enable implementing institutions to be more flexible with which changesets they want to include.
They can use the default one, or have their own manager which allows them to skip certain changesets.
Also it provides an example of how implementing institutions can add scripts of their own in between the open source changesets.

Assumptions
Ideally a partner will never have to skip any changeset on the official project.
Just idea of keeping the changesets split up into their respective files will be a huge advantage for readability.

As far as adding custom changesets in between:
Liquibase does not care about the sequence of the changesets. It will only check if the hashes of each changeset are correct/unmodified after execution. Therefore addine some in between (e.g. adding data) will not interfere with the project.

Acceptance Criteria
Please do thoroughly check the correct functionality, as this will affect productive systems.
All releases containing changes to how changesets are managed or new changesets should be very explicitly labeled.

In area 4 there is an empty table if no data is reused. This should simply not be present in this case.
There might also be other tables which are empty, if no data is selected.

remove
projects-url: https://next.apps.dev.csd.tuwien.ac.at
persons-url: https://next.apps.dev.csd.tuwien.ac.at

from application.yaml, since they can be hardcoded or otherwise referenced in the custom project.
alternatively, demonstrate how to utilize them properly using the mock services, but do not restrict the customization to these 2 choices.

Changes
Adding a new template for exporting a DMP to a word document currently relies on:

• Adding a new word template and maybe a resource file
• Making sure placeholders are correct
• Implementing logic to fill said document

In this process, several errors might occur. The more manual input is necessary, the more error prone the whole process may become. Additionally, word documents are binary files and changes can not be observed via git. This makes it difficult to track changes in institutional implementations.

In order to simplify this process, the build-in quarkus template engine qute can be used. It allows for template type checking during build time, reducing possible typing errors for placeholders. Utilizing qute would allow to reduce the steps required to introduce a new template, by simply adding a new qute template.

Available templates may be specified via a config variable. When tackling this issue, it also makes sense to move all template logic with regards to availability to the backend, in order to reduce duplication in the frontend. The frontend would then only fetch available templates and display them as options.

In addition, it would also be great to allow for different export options of the document (e.g. pdf, docx, html, markdown). This may be moved to a separate task though.

As an example, user would be shown a modal with available templates (e.g. FWF, Horizon Europe).
After choosing a template, a user will also have the possibility to choose the format of the document (i.e. pdf, docx, html, markdown).
The backend would then render the selected template and convert the document into the specified format.

Additional notes

Assumptions
All current templates can be defined via the qute template engine. qute does not dictate a certain format, so we will have to decide what should be the outcome. HTML would allow for most flexibility with regards to styling (research if inline styling is possible) and is quite common or somehow known. Another option would be to use markdown as a render target, which has easier syntax but does not allow for as much customization. Either way, we should then be able to transform the rendered output into other formats as well (i.e. pdf, docx, markdown, html).

Acceptance Criteria
Implementation of:

• A configuration possibility to easily specify available export formats
• A service taking care of choosing the specified template and render it with the provided DMP
• A converter service receiving the input and converting it to the requested output format
• Current templates converted to qute templates

Risks? (optional)

The DMP version must be entered manually under "Administrative information". An additional description is also added without further comment.

Description
Currently, the dev flag does not do anything - to start the backend without docker, the config files have to be changed manually every time.

Proposed solution
Update dev mode in aplplication.yaml, so that everything is taken care of, when the project ist started via command line with mvn compile quarkus:dev.

Description
In the template export we add the acronym of the license assigned to a specific dataset.
If possible, we make said acronym to also be a hyperlink to the license URL.

Proposed solution
enum ELicense already contains all necessary info.
We need to add the url in the class composing the word document.

Based on Bugreport GH-57, we now know that some fields may be required to be larger than 4000 bytes. As a fix for this bug we will change the limit from 4000 characters to bytes instead, but it still leaves open the possibility, that some input fields might need more characters.
Need to go over all text fields and determine which have to be expanded, create the new columns in the database and migrate the previous data to them.

Describe the bug
At the moment contributors added over ORCID have a list of all their affiliations attached to them. This list contains the affiliation title only and is not sorted.
We should be able to choose the newest/current affiliation only.
ORCID does save dates for the affiliation. The API should be expanded to also perceive this information and use it as criteria for choosing which affiliation to show.

Is this a regression?
No.

Steps to Reproduce

1. Add contributor over ORCID which has several affiliations assigned to them.
2. See error

Expected behavior
Onyl the current/most recent affiliation should show up.

Environment

• Damap version 2.4.0

At Section 3, two paragraphs were added (yellow) which the user did not select and did not write himself.
This text can be added in a specific institutional template but may be undesired in the open source template. Templates should be checked to make sure all such occurrences are found.

For project and person adaption possibilities, the provided resources and services should be streamlined. Generic interfaces should be provided, such that institutions have as much freedom as possible when adapting DAMAP.

Additionally, a pagination mechanism or definition should be implemented

At least read and search interfaces should be provided for resources and services.

Changes
Both export templates insert funder program only in the document.
Science europe template should show funder name as well.
FWF does not need the funder as FWF is the funder.

Assumptions
Assuming that implementing institution might add funder name to the program they will need to rewrite this impleemntation.

Changes
The answer to the question "Please explain why institutional storage will not be used" at the bottom of the storage section (currently step 5) is then inserted into the table listing datasets, for each dataset.
This repetition is not necessary. As such we should move the input below the table and remove it from the table completely.

Additional notes
Possibly it could require an additional default text in the export document as an introduction to that information (only if there is anything to output there. The text can be inquired with the local RDM department.

Changes
After changes to base services we now have an empty page in the frontend project view by default. Add an API for loading a list of "recommended" projects, for this initial view.
The backend will provide this API as a resource. How this API looks like for querying projects in the utilized project database is up to each implementing institution.

Additional notes
each institution should decide for themselves what projects to display here (eg. Person is already involved, newest by date, ...) and can take care of this in the custom project. By default we can just return an empty list.

Acceptance Criteria
The user should have a list of projects when opening the step, in order to facilitate usability. This call will be done once when loading the step. The list can then be replaced by the search results.

connected with frontend issue tuwien-csd/damap-frontend#103

EDITOR:

• Explicit: Must be added byowner, editor or admin.
• Only contributors can be added as editors.
• Cannot delete DMPs.
• Can add editors.
• Must be removed if contributor is deleted. Are kept if contributor is removed.
• Add warning if contributor is removed.

VIEWER:

• Inferred: All contributors are viewers by default.
• Must be contributor.
• Can't see access list.
• TBD: View latest version or form?

Both can:

• view versions
• export DMPs

UI and implementation:

• Separate page with list of contributors, where editor role can be assigned.
• Only people of institution (w/ universityId) can be editors and viewers (add comment to code)
• Use access table
• Consider concurrent edits
• Use hibernate optimistic lock
• Add version property to DOs (DMP etc)
• Make sure error message for failed save is provided (and sufficiently informative to the user)

Currently a dataset must be registered in Openaire to be searchable. It would be very practical if the search could also query other sources and not just use DOIs. Furthermore, it should not be limited to "pure" datasets. Software, code, databases, etc. must also be able to be entered.

Changes
The export templates have a table listing repositories in which datasets are stored.
The PID column is empty/not being filled out. This information should be availyble through the r3data API.
Extract it and add it to the template.

Additional notes
Depending on what is implemented already this task might be easy, by just supplying the information to the template, or more complex, if the PID is not collected already.

Assumptions
The templates themselves should not be changed in order to implement this feature.

Changes
upgrade quarkus to the newest version

Acceptance Criteria
application should work as usual

Describe the bug
When adding a dataset via file upload, the file doesn't show up as dataset source new is not set.

Is this a regression?
Yes.

Steps to Reproduce

1. Go to 'Specify research data > New data > Specify datasets'.
2. Click on 'Upload sample data'.
3. Upload file.
4. Analyzed file doesn't show up in table.

Expected behavior
File should be displayed in table.

Environment

• Damap version 2.3.0