Current Implementation

DEV deployments currently use HTTP.

Suggested Enhancement

Enable HTTPS for dev deployments.

Expected Benefits

Mobile clients can remove "whitelisting" for dev endpoints. Also more S is more better.

Would be cool to have Sonar enabled as well.

Each key bundle that is delivered through the CDN shall represent a 1-hour slice. Therefore, there can be a maximum of 24 of these slices per day, each beginning at the full hour.

Newly submitted diagnosis keys shall be made available through the CDN at every full hour. If there are no new diagnosis keys (db connection error, or there are just none, etc.), a new "empty" file shall still be published.

In addition, each day at 0:00 UTC, a file shall be published that contains all the diagnosis keys that were published through the CDN within the previous 24 hrs.

Current Implementation

The DistributionKeyService I/O (e.g. saveDiagnosisKeys, getDiagnosisKeys, applyRetentionPolicy) is thread blocking.

Suggested Enhancement

Make the DistributionKeyService I/O non-blocking through the use of Futures or other suitable constructs. Adapt SubmissionController and Distribution runners accordingly.

Expected Benefits

Performance improvement / lower resource footprint.

Get Monitoring working on OpenShift

Describe the bug

There are several hard coded credentials for POSTGRESQL_PASSWORD in the docker-compose.yml file.

Expected behaviour

Hard coded credentials shouldn't be part of source code or config files.

Technical details

https://github.com/corona-warn-app/cwa-server/blob/master/docker-compose.yml

Possible Fix

In docker-compose.yml you can specify a file that contains the env variables for the container:

env_file:
- .env

You can set the credentials in the .env file like this:

POSTGRESQL_PASSWORD=xyz

Don't forget to add .env to .gitignore.

The .env file should be placed somewhere local and securely

Additional context

https://cwe.mitre.org/data/definitions/798.html

We need to enforce static code scans to help us find security related issues early.

(on hold due to waiting for Google/Apple spec)

Adjust the current distribution strategy to the following:

Server checks hourly whether new keys need to be published.

• If there are more than N keys available, a new Export should be created.
• Otherwise, no file will be generated
• N must be defined as a parameter (properties file)

The app will use the defined API to check what packages exist for each day. Those packages are not assigned to an hour anymore, but rather just “batchfiles”, e.g. bachfile 1, 2, and 3 for day X. A batch file may or may not be part of a bigger batch.

When an export is generated, it needs to know the exact start timestamp. Since this is now dependent on the creation of the previous export, we need to store this time somewhere. Since we are still using two different time intervals for triggering the export (hourly/daily), we need to store both time indicators.

Mobile needs to be able to fetch additional configuration/settings from the server. The process should be similar to the exposure configuration.

Therefore, add a new app-config.yaml to the codebase. The app-config should contain

risk_classes: 
  - 
    class: LOW
    max: 3
    min: 1
    uri: "https://www..."
  - 
    class: MID
    max: 5
    min: 3
    uri: "https://www..."
  - 
    class: HIGH
    max: 8
    min: 5
    uri: "https://www..."

We need validations & tests (also similar to exposure config). The YAML needs to be parsed to the proto message, so it can used in the distribution service for pushing to CDN.

The S3Publisher is currently not setting Cache Control Headers. We should cache-control: public, maxage=N

N depends on the type of file:

• for hours index files, it should be rather low (5 minutes)
• for days index files, it should be higher (2 hours)
• for all generated aggregate files, it should be 24 hours

E-Tag should already been set, but should also be verified.

Current Implementation

Date, hour, country and version index files are plain, unsigned JSON arrays.

Suggested Enhancement

Wrap index files in SignedPayloads.

Expected Benefits

Attestation that there were no submissions during a time frame etc., basically because of the same reasons for which we sign the diagnosis keys themselves (proof of authenticity and validity, "undeniability").

Any key data that is associated to a point in time older than 14 days shall be removed from the database and will also be removed from the CDN and therefore the index file.

Implement TemporaryExposureKey Persistence Model

Change our current proto spec to reflect the spec mentioned below. Create files based on this spec, instead of the old spec.

Currently, the spec is not available/final.

• Setup central platform for sharing findings/documentation (GitHub, or Jam)

DiagnosisKey shall be extended by a nullable timestamp field that stores the time at which the respective key was first distributed by the distribution service.
This field is required for partitioning the distribution packages.

• kill runner chain execution if single runner fails

• Familiarize with protobuf
• Provide dev server with mock data

These interfaces should later be used for Object Store IO as well.

We need to be able to allow the mobile clients to fetch the parameter file, so they can compute the risk scores accordingly. Therefore, add the file to the repo and the distribution service shall take care of mapping it to the right output file (similar to the aggregate files).

Describe the bug

Need to append /index to all requests to the Object Store. Example: /version/v1/index

Expected behaviour

Appending /index should not be necessary. Example: /version/v1

Possible Fix

Rename files on upload to the object store.

Update README.md to reflect the apha release scope and status ...be creative.

DiagnosisKeyDistributionRunner deletes the files written by ExposureConfigurationDistributionRunner before they are published.

When fake requests are send to the submission service, ensure that the duration of the fake requests take as long as normal requests, so attackers sniffing the traffic are unable to distinguish between fake/normal requests.

Check up on the ObjectStore on the OTC platform & work on connecting it to our service.

• Add object store
• Connect the OS to our service
• Trigger upload of a file and deletion
• Check whether bundle uploads are possible (zip like)

We need to be able to monitor our application. This is about monitoring the logs.

From the looks of it, there is no default support for logging tools like Kibana/Splunk built into OC. However, we need to be able to search the logs in a convenient way.

Find out how we can make tools like Kibana/Splunk work with our current deployment.

• Look into the topic and consolidate findings

After running the testdata profile twice, I noticed that the same file for the same chunk differs on byte level, although the file size is identical. This should not happen, as it will cause problems on CDN.

For the index file, it looks like missing sorting is causing the problem, one example:

[23,18,21,22,19,20]

We need to investigate on what's the preferred way for secret handling for us. OC does provide some features there, but it also has some downsides.

https://www.openshift.com/blog/managing-secrets-openshift-vault-integration

This is a quite interesting read.

This task is about understanding the secret handling in OC and what it would mean to deviate from that (e.g. vault).

Implement unit tests for the diagnosis key submission service according to best practices.

All CWA backend in this repo are currently using HSQLDB regardless of their runtime profile.

• Configure production profile to use Postgres instead of HSQLDB
• Ensure code compatibility
• Re-enable flyway (see application.properties) and create required migration scripts
• Test on OpenShift deployment

Current Implementation

Submission endpoint is /version/{{version}}/diagnosis-keys

Suggested Enhancement

Change submission endpoint to /version/{{version}}/diagnosis-keys/country/{{country}}.

Expected Benefits

At the moment we can not be 100% certain that we will never have to submit diagnosis keys to other countries. However, if we ever get that requirement, and needed to introduce a country differentiating parameter, this would be a breaking change and would force us to bump the overall API version. With the proposed change (and an initial implementation for only /version/v1/diagnosis-keys/country/DE), this issue can already be avoided. Also, the proposed change would align the submission and distribution APIs.

We shall generate an index file that is intended to be accessible through the CDN informs (the frontend app) about which key bundles are available on the CDN at the time of the call.

• basically a list of time stamps (also see Exposure-Notification-App/ena-backend#15 and #5)

• the index file shall be digitally signed

• only key bundles of size > 0 shall be listed in the index file

• index file must be delivered with an ETag header (CDN side)

• specify protocol buffers schema for index file

• implement end point

Validate the submitted temporary exposure keys wrt the latest specification, eg. expected value ranges and things like maximum number of keys per submission. Any violations shall result in an error response, HTTP 400.

The distribution runner shall:

1. Read configuration parameters file (part of this repository).
2. Validate the configuration parameters.
3. Format the configuration parameters according to the current specification.
4. Push the configuration parameters to the CDN.

We need to have a Pipeline in place. @johanneseschrig & @MKusber please work on this and come up with a solution. Ideally, it would include the static code scan tools (fortify&sonar)

Current Implementation

All dates/hours between the first submission and now (last full hour) are included in the index files, even if there were no submissions.

Suggested Enhancement

Only list dates/hours for which there were submissions in the index files. The "empty" date/hour files should, however, still be generated, even if they are not in the index.

Expected Benefits

Mobile clients need to do fewer requests (they can simply skip empty dates/hours).

Before keys are distributed to the object store, run sanity checks on them. In case the checks are failing, the affected key(s) shall not be part of the payload.

Validations should be re-useable based on
#50

• Implement the TanVerifier that calls the verification server.
• Return an appropriate response to the caller in case of an invalid TAN

We would like to store audit events in a separate audit log (similar to what Cloud Platform) provides.

• Adding of DK: When a DK is uploaded. Also involve verification call / result
• Creation of aggregate files (daily/hourly) & push result to CDN

@MKusber

(onhold due to waiting for Google/Apple spec)

When the user sends the diagnosis keys, we need to verify whether the request came from a legitimate source.

• Accept the "platform" attribute in the incoming request. Must either be "ios" or "android". Needed for the device verification payload below:
• Device Verification Payload: This is either Android SafetyNet device attestation or iOS DeviceCheck attestation.

Problem: This highly depends whether data privacy team is OK with this. I will sync with them.

We have prepared an initial proposal for the readme file here: https://github.com/corona-warn-app/cwa-admin/blob/master/drafts/readme-cwa-server.md The goal is that every component uses a similar readme format. This is currently only a rough draft, feel free to comment and give any feedback here in this issue thread. The badges within this readme are not functional at the moment (will only work once the project is public).

Please add a license.
If possible Apache v2.
If you are going to reuse code under MPL license, like DP3T, you can also use MPL, but preference would be Apache.

The exposure configuration parameters are currently not validated after being read from the respective yml file. The validation code in ExposureConfigurationValidator is currently used in tests only.

However, the expected behavior shall be the following:

• The RiskScoreParameters object is constructed based on yml file. (✅)
• ExposureConfigurationValidator performs validation of the parameters.
• In case of a failed validation, do not publish the exposure configuration parameters, but do generate and publish the diagnosis key bundles.
• Clarify whether to keep any previously present (valid) configuration parameters on the object store if a validation error occurs.
• Any validation errors shall be logged appropriately.

Exposure bundles that are published through the CDN shall contain an Ed25519 signature and the respective public key certificate.
(see latest protobuf spec)

Align on the final signing approach (onhold due to waiting for Google/Apple spec)

Randomly sent "fake" diagnosis key submission requests are a privacy preserving measure. They are identified by a non-zero value for the cwa-fake HTTP header.

Currently, this results in the "fake" payload to be discarded and an immediate HTTP 200response.
It shall be evaluated whether a delay should be introduced to ensure a constant processing time, regardless of whether dealing with a "fake" or "real" request.

Applies to all submission and distribution endpoints:
Remove OPTIONS from the allowed response header and reject respective requests with status HTTP 405 Method Not Allowed.

Change the aggregate file creation logic in a way, in which packages are skipped in case the payload (amount of keys) is too small. The threshold should be configurable, and defaulted to 50 for now.