co-epi / coepi-backend-aws Goto Github PK

Mobile app needs to download a list of CENkeys which have reported symptoms. Only if a match is established the app will request specific report for a given CENkey (see https://github.com/Co-Epi/cen-server)

Are there plans to support the above described flow?

All 6 of the unit tests are currently disabled because they are failing.

We should fix them and re-enable them.

Add Terraform definition to use the v4 DynamoDB schema

If the app sends a corrupt or manually edited POST with a non-base64 encoded string, it will crash after the backend returns those values. Rejecting non encoded strings ought to fix this issue and prevent additional potential API problems.

The API supports retrieving reports generated during a fixed time interval. This means that report batches for all intervals that have occurred in the past can be cached in CDN, optimizing latency and cost.
Cloudfront is the CDN product that is available in AWS and can be used for this purpose. Some of the tasks that can be accomplished.

1. API Gateway endpoint to be edge-optimized
2. Enable caching using query string parameters as keys (intervalNumber is passed is as a query string parameter)
3. Automate setting up the Cloudfront infrastructure using Terraform

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/QueryStringParameters.html

Currently if you set up a stack the Terraform state file (which tracks and resolves drift in AWS) is stored on the disk you run Terraform from - the best-practice approach is to store this in an S3 bucket.

Currently, the apps send POST request to the back-end without any authentication, for privacy-preserving reasons. Performing server-side reCaptcha verification would prevent abuse, DOS attacks and fake report generation. We should also consider privacy-pass (https://www.hcaptcha.com/privacy-pass). This requires changes on the client side as well. So recommend doing this in v5

Currently, the v4 API endpoint receives the report data as a POST request, stores the report data buffer as is and serves it in a GET request. It does not perform any report validation and signature verification as per the TCN protocol. There is a Kotlin library that can be used to achieve that. Report validation and signature verification need to be added here -> https://github.com/Co-Epi/coepi-backend-aws/blob/master/src/main/kotlin/org/coepi/api/v4/TCNCloudAPIHandler.kt#L141-L142

This requires knowledge of the TCN protocol:
https://github.com/TCNCoalition/TCN/blob/main/README.md

There is already a Kotlin library that can be re-used:
https://github.com/TCNCoalition/tcn-client-android/tree/develop/tcn-client-android/src/main/java/org/tcncoalition/tcnclient/crypto

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-using-invoke.html

Response payload has to be decoded twice to get to the "plaintext".

Current

 curl -v \
	-X POST \
	-H "Accept-Language: en;q=1.0" \
	-H "Accept-Encoding: br;q=1.0, gzip;q=0.9, deflate;q=0.8" \
	-H "User-Agent: Unknown/Unknown (com.apple.dt.xctest.tool; build:15702; iOS 13.3.0) Alamofire/5.0.5" \
	-d "VGVzdCBwYXlsb2FkIDE1ODczOTg1NzcuNDc3NjA5Mg==" \
	"https://18ye1iivg6.execute-api.us-west-1.amazonaws.com/v4/tcnreport"

When fetching I get something like:
"VkdWemRDQndZWGxzYjJGa0lERTFPRGN6T1RnMU56Y3VORGMzTmpBNU1nPT0="

after decoding twice, the returned string becomes:

Test payload 1587398577.4776092

Expected
Returned payload yields "Test payload ..." after single Base64 decoding operation.

Reference Doc

https://docs.google.com/document/d/1l6ajoKs4En_STtaROlZdHtWI9VZE6c5HLpmBwEqWcXo/edit#

Development Branch

https://github.com/Co-Epi/coepi-backend-aws/tree/unified_backend

API Definition

https://github.com/Co-Epi/coepi-backend-aws/blob/unified_backend/api_definition/coepi_api_0.5.0.yml

Apple/Google Diagnosis Server Reference Implementation:

https://github.com/bhushanRamnani/apple-google-diagnosis-server-aws

Tasks:

1. Implement TCN endpoint with lambda front-end
2. Implement TCN endpoint with ktor front-end
3. Add report and signature validation to TCN endpoint (#9)
4. Implement Apple/Google endpoint with lambda front-end
5. Implement Apple / Google endpoint with ktor front-end

Currently the server endpoints are strongly coupled with Lambda as the compute engine. This prevents the server from being hosted within a Kubernetes cluster or run on EC2. It will be desirable to refactor the front-end handlers so that most of the business logic is abstracted away in cloud-agnostic controllers. Then we can have two shims, one for handling lambda events and another a regular web server framework (ktor or spring). We can then dockerize the server and have an alternate terraform stack deploying it to an EKS cluster.

All the controller business logic is here:
https://github.com/Co-Epi/coepi-backend-aws/blob/master/src/main/kotlin/org/coepi/api/v4/TCNCloudAPIHandler.kt

https://github.com/Co-Epi/coepi-backend-aws/blob/master/src/main/kotlin/org/coepi/api/CoEpiHandler.kt

Merges to master ought to trigger a terraform apply operation and update the "deployed version" automatically - Github actions and S3 state storage should be enough to enable this.

Implement v4 API supporting 0.4 TCN protocol.

Benefits of automated code formatting:

• We'll have a standard style, which helps make the codebase more readable for newbies and oldies alike
• Automation drastically reduces back-and-forth over style; we just go with what the formatter provides
• Reduces merge conflicts from intentional or accidental formatting changes
• Reduces noise in PRs from people fixing formatting errors when they find them (or having to submit separate formatting PRs)

I propose we use ktlint.

• Implements a combination of the official Kotlin and Android style guides.
• Formatters available for major editors, including IntellIJ, Emacs, and Vim
• Minimally configurable, which helps keep us from bikeshedding over small style choices
• A number of options for Gradle integration. (I usually just re-use a simple buildSrc plugin I wrote.)

Just from the description of the API, it looks like backend is intended to support arbitrary time ranges.

I think it might be better to only allow fetching digests of all reports in a fixed time interval (e.g., over 6- or 12-hour chunks of time). There are two main advantages:

1. The backend can randomize the order of reports within a time interval. This means that clients cannot guess that just because two reports are adjacent in the response that they may have come from the same user. In contrast, if the backend allows arbitrary time queries, it provides all clients with a side channel to deanonymize other users.

2. The results of each response are static as soon as they are ready, so they never need to be recomputed. They could just be pushed to a static file service backed by a CDN, rather than requiring any compute.

I think this might already be done but need to verify.