Bulk download attachments on Storage mode about formsg HOT 9 CLOSED

Points we need to communicate to users for this implementation, not final copy

Pre-download

• Downloading x responses will take x minutes or more
• (if threshold exceeded) some message GO BACK vs PROCEED ANYWAY
• (if threshold not exceeded) Downloading attachments for 838 responses
• Downloading may take 30 minutes or more depending on responses selected
• Attachments will be downloaded as one zip file per response. Up to 838 zipped responses will be downloaded into your destination folder.
• Don't use IE11
• Ensure that you have a strong internet connection and sufficient memory on your computer

Eventually we can also detect attachment capacity of form and change the threshold number accordingly, but lets keep it out of 1st iteration as additional code required.

During download

• (If > threshold responses) Download may take x minutes or more
• Do not navigate away from this page
• To stop the download, close your browser

After download

• Download complete
• Attachment folders successfully downloaded: 984
• Attachment folders failed to download: 0
• If failures are >0, refer to .csv file for reference IDs of failed downloads

@liangyuanruo please add on if I missed out anything
@pearlyong @f1zaz

from formsg.

@frankchn @syan-syan @liangyuanruo

Looked at Jotform. They have e2ee forms but not for files. Files are not encrypted.

I'm against bulk downloads for two reasons: a) load on our servers and b) load on agency browsers. This load is non-trivial as we have seen from e2ee text.

How about we:

1. Place https://form.gov.sg/formid/submissionid in downloaded csv
2. On click, open up this response in the UI
   2a. If user is not logged in or authorised to this formid, show a 403/404
   2b. If user is logged in, prompt him for secret key then unlock this submission
   2c. If user has entered secret key before, save it in session storage. Alternatively, save private* key in session storage, which is used to encrypt secret key to temporarily store it on our servers with a short expiry.

Having a link in the Excel is likely the most intuitive way users view files. Google Forms, JotForm, TypeForm all do it that way.

from formsg.

session storage is only persistent within the same browser window. we'll have to use local storage instead, and that will persist the key on the browser for much longer.

if we want to do this, we should build out the caching mechanism within the app first for existing UX first.

from formsg.

@liangyuanruo @leonardloo

What do you think of transferring sharing secret keys by something like BroadcastChannel to request a secret key from another tab if it is already open?

I am envisioning a new tab being opened with a direct link to the response page, and upon load, the new tab will call postMessage({action: requestSecretKey, formId: whatever}) when it displays the prompt for the user to upload/enter the secret key.

Any other tab that already has the secret key loaded can respond by calling postMessage({action: secretKey, formId: whatever, secretKey: secret}). The secret key received from the other tab is treated the same way as if it is entered by the user, and loading proceeds as normal after that.

BroadcastChannel is for scripts from the same origin (unlike window.postMessage) so it should be as secure as secret key handling is now. We don't touch either sessionStorage or localStorage, so there shouldn't be any further security implications.

The only downside I can see is that IE11 doesn't support this API.

from formsg.

Also, if the user opens a bunch of tabs at once, once the user uploads the key to one tab, that tab can postMessage({action: secretKey, formId: whatever, secretKey: secret}) and all the other tabs will receive the same key and proceed with loading too.

from formsg.

to resolve

• web workers for optimisation
• bulk endpoint on server
• error handling

from formsg.

Styling changes

Search icon and field

Position should be on the right, beside the date field. Can see Zeplin for reference.
 For behaviour, can see the Search icon in the Examples page. On click it should display field and focus. There should also be an X to close, which will return it to an icon, and all results will be displayed again.

Dropdown field

Can see Zeplin for reference. Can use the same component as in the Examples page (Agency dropdown sort)

Download attachments modal

Can we use same modal styling as the one that is used while downloading? No border radius, Header styling, centre align all content, and centre in page

Downloading button
Can we change the Downloading button on the responses page to a loading one with a spinner when attachments are downloading?

For copy, i will update this thread again today/tomorrow.

from formsg.

Updates in Figma:
https://www.figma.com/file/ta5tjjBqphML8LPRkmxD53/DesignMaster_v2?node-id=928%3A45

• Copy to reflect above
• Included beta flag which should link to guide.form.gov.sg on what a Beta feature is, and possible consequences (sy: include in later iteration)
• Warning to inform users to download below the threshold allowed to avoid overheating

from formsg.

Updated copy in Figma to reflect the following:

• Warning messages (Do not use Internet Explorer, Your network connectivity is strong, Your device has sufficient disk space for the download)
• Estimated time to download
• Download stopped
• Download complete (with error)

from formsg.