Code Monkey home page Code Monkey logo

wildaid / o-fish-realm Goto Github PK

View Code? Open in Web Editor NEW
23.0 6.0 9.0 52.06 MB

Realm application code and sample data for the Officer's Fishery Information Sharing Hub (O-FISH). The mobile app allows fisheries officers to document and share critical information gathered during a routine vessel inspection. The web app allows agencies to gain insights from the aggregated information.

License: Apache License 2.0

JavaScript 100.00%
realm realm-mobile-database hacktoberfest hacktoberfest2020 mongodb-realm realm-mobile-platform non-profit nonprofit wildaid mongodb

o-fish-realm's Introduction

MongoDB Realm Application for WildAid's O-FISH project

Table of Contents

  1. Overview
  2. O-FISH Installation
  3. Import this repository as standalone code
  4. O-FISH Project Goals
  5. O-FISH Components
  6. O-FISH Infrastructure

Overview

The WildAid Marine Program works to protect vulnerable marine environments.

O-FISH (Officer Fishery Information Sharing Hub) is a multi-platform application that enables officers to browse and record boarding report data from their mobile devices.

The details behind the data architecture, schema, and partitioning strategy are described in Realm Data and Partitioning Strategy Behind the WildAid O-FISH Mobile Apps.



Developers are expected to follow the MongoDB Community Code of Conduct guidelines.

This repo implements the O-FISH Realm serverless backend application and contains sample data. The WildAidSampleBackup directory contains a mongodump of the database, suitable for building and testing. The WildAidDemo directory contains the serverless functions, triggers, values, Realm Sync rules and other code that makes up the backend for the web, iOS and Android applications - together with the connection to the associated MongoDB Atlas database.

See the O-FISH installation guide, which includes instructions for using this repository in the context of building the O-FISH application.

How to import and configure your own MongoDB Realm app

  1. Create an Atlas Cluster.
  2. Configure your Atlas Cluster.
  3. Import sample data (use WildAidMinimalBackup if you don't need thousands of sample documents), and add a Search index to the BoardingReports collection.
  4. Create the Realm App.
  5. Take a note of the Realm App-Id appname-xxxxx
  6. Generate an API key pair for your project and note the public and private IDs. Whitelist your IP address for API access.
  7. Install realm-cli: npm install -g mongodb-realm-cli
  8. Log in to your Atlas/Realm project: realm-cli login --api-key=my-api-key --private-api-key=my-private-api-key
  9. Add your AWS private API key to your app as a Realm Secret: realm-cli secrets add --app-id=appname-xxxxx --name=AWS-secret-key --value=my-aws-secret-api-key - If you are NOT connecting your instance with AWS, you STILL need to run this command but --value can be set to any string.
  10. Import the Realm Code
  11. Create a global administrative user in Realm and add the information to the User collection.
  12. Use the Realm App-Id (appname-xxxxx) in your web, iOS, or Android apps.
  13. Enable Realm Sync through the Realm UI
    Before activating sync, you can add extra security by adding this extra rule to the read and write permissions: { "%%user.custom_data.agency.name": "%%partition" }.
  14. Enable Users/Custom Data (ensure that cluster name = RealmSync, database = wildaid, collection = User and user ID field = realmUserId)
  15. If you're running the web app somewhere than localhost, or you want password reset emails to work on other devices, go to the Autentication section of the UI, and set update Password Reset URL for the username/password service.
  16. Optionally enable additional Triggers through the Realm UI (if you've set up your AWS credentials)

Developer Sandbox Mode

If you want to allow anonymous users to create a new account and agency (only intended for shared development environments/sandboxes), then:

  1. Enable Anonymous Authentication
  2. Set the developerMode Realm value to true so that the regNewAgency function can be called from the web app.
  3. If you want new agencies to be bootstrapped with menu data, reports, and photos then:
    1. Set up an agency (as normal) with the desired menu data, a few sample reports (optional), and a few photos (optional) - note the name of this "donor agency"
    2. Set the donorAgency value to the name of the donor agency – new agencies will now be bootstrapped with the menu data, the photos, and (10 of) the boarding reports from the donor agency.

O-FISH Components

WildAid's O-FISH project has several components:

  1. Android mobile application, used for pre-boarding search for vessel information, and on-board reporting of information
  2. iOS mobile application, used for pre-boarding search for vessel information, and on-board reporting of information
  3. Web application code, used for website navigation and feeding Atlas Search results into MongoDB Charts
  4. Realm serverless platform code (this repo_, used for backend functionality:
    1. Atlas Search queries
    2. Moving photos from the database, where Realm syncs them, to an external bucket
    3. Cryptographically secure verifiable change history
    4. MongoDB Realm Syc configuration - ensuring that the same reports can be accessed from any device This repository contains the Realm serverless platform code in the WildAidDemo directory.
  5. Sample data, for use when building test environments for the project. This repository contains sample data in the WildAidSampleBackup directory.

WildAid O-FISH architecture

Photo lifecycle - a photo is captured in the mobile application and is sync'd to Atlas via Realm. When a record is inserted in the database, a trigger is fired calling a function to save the photo to an S3 bucket, removing the photo from the document in the database, and adding the URL the photo to the document. Realm synchronizes the document to all mobile devices: WildAid O-FISH image management architecture

o-fish-realm's People

Contributors

am-mongodb avatar ayushjainrksh avatar azzoo avatar brittanylau avatar dusch4593 avatar lenmorld avatar mmhuseynov avatar o-fish-wildaid avatar sebawita avatar segh avatar sheeri avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

o-fish-realm's Issues

dump instructions don't result in proper import

Hi

Based on the url https://wildaid.github.io/build/2020/06/09/Data-Import.html. Following these commands:
mongorestore --drop --uri "mongodb+srv://[email protected]/" /Users/sheeri/wildaid_demo/o-fish-realm/WildAidSampleBackup When it is successful, you will see a message like: 6117 document(s) restored successfully. 0 document(s) failed to restore.

(Obviously I'm replacing the user specifics here, for my own setup)

My output for reference is:
2020-06-12T13:54:42.366-0300 preparing collections to restore from 2020-06-12T13:54:42.412-0300 reading metadata for wildaid.BoardingReports from WildAidSampleBackup/wildaid/BoardingReports.metadata.json 2020-06-12T13:54:42.453-0300 reading metadata for wildaid.ChangeHistory from WildAidSampleBackup/wildaid/ChangeHistory.metadata.json 2020-06-12T13:54:42.496-0300 reading metadata for wildaid.Agency from WildAidSampleBackup/wildaid/Agency.metadata.json 2020-06-12T13:54:42.559-0300 restoring wildaid.Agency from WildAidSampleBackup/wildaid/Agency.bson 2020-06-12T13:54:42.611-0300 no indexes to restore 2020-06-12T13:54:42.611-0300 finished restoring wildaid.Agency (3 documents, 0 failures) 2020-06-12T13:54:42.796-0300 finished restoring wildaid.BoardingReports (0 documents, 0 failures) 2020-06-12T13:54:42.796-0300 Failed: wildaid.BoardingReports: error creating collection wildaid.BoardingReports: error running create command: (Location40415) BSON field 'create.recordPreImages' is an unknown field.

It only results in 3 records being added to the Agency collection. The sampledata, also seems to be missing collection files. I only see files for agency, boradingReports and changeHistory. But the images on the aforementioned url seem to suggest other collections. If you could help with this, that would be great 😄

Thanks

Rules - User groups

@Sheeri need to understand the functionality behind user groups. e.g.:

  • Do all members of a user group need to be from the same agency?
  • Can the same group name be reused by different agencies?
  • What are the consequences of 2 users being in the same group?
  • What are the consequences of 2 users not being in the same group?

Move tagged Photos to the associated User document

When the mobile app adds a new headshot for the current user, it adds the photo to the Photo Realm Object (along with the email address of the user).

When that photo hit Atlas, we need a Trigger to copy the image to the User document associated with the email address.

Simplify S3 file names for uploaded photos

The URL for the file in S3 currently contains the agency name - which may include spaces or special characters – resulted in a calculated URL which can't be used. Instead, just use the _id of the Photo document to uniquely identify the file.

Better README

Include and overview of the project: goals, components, infrastructure. And probably a table of contents because it's getting long.....

Could not find realm app

Firstly, please let me know if this is better suited to the Realm Forum at MongoDB.com

Just going through this realm application set up. I'm currently on this page https://wildaid.github.io/build/2020/06/09/Import-Realm-Code.html.

These are the steps and my steps

  1. In the template, you have a field called “Path to Realm code”. Go to that directory - you should be inside a directory called WildAidDemo.
    Correct, I'm in that dir

  2. Edit stitch.json change “name” to your “Realm App Name”
    Changed to my app name of "WildAidApp"

  3. Edit services/RealmSync/config.json services/mongodb-atlas/config.json
    and change the value of “ClusterName” to your “Atlas Cluster Name”
    Changed to "O-Fish" my cluster name

  4. If you are not using AWS, skip this step. If you are, edit the following files and fill in all the relevant details:
    values/awsRegion.json values/destinationEmailAddress.json values/sourceEmailAddress.json
    I'm not using an external AWS service as the previous page asks. My Mongo instance is a free tier on AWS however. Do I still need to complete this piece? I did update the awsRegion to us-east-1 after my ignoring it the first time I tried to import the realm-code

  5. If you are not using AWS, skip this step. If you are, set accessKeyID if using AWS in the file:
    services/AWS/config.json.
    Same here, not using AWS, I did use the secret-key it makes you generate in the previous page, as I didn't exist it to make any difference

Do the import, confirm with ‘y’ when prompted to confirm changes. You will paste your Realm App ID into –app-id:
stitch-cli import --strategy=replace --include-dependencies --app-id=REALM_APP_ID

I adjusted this to realm-cli which I believe is correct and added my REALM_APP_ID too. The same one that was needed from the previous page.

But I get the error could not find realm app

So not exactly sure what I did wrong here?

Again, apologies if this should be in the forum 😄

Chart: Boarding maps to accept filters rather than list of `_id`s

The web app's boarding page currently sends a query with an unlimited list of report IDs to be rendered on the map. That breaks limits on the size of http requests.

Instead, we should have server-side Chart logic that takes filter criteria, which it then uses to find all of the documents that should be rendered on the map.

Once that is implemented, raise a web ticket to use the new Chart.

Schema - partner agencies should be able to access an agency's data

Use the partnerAgencies attribute in an Agency document to indicate which other agencies this agency grants permission to read their boarding reports.

An Agency document should be able to contain an array of sub-documents containing a partner agency name and (optionally) a start and/or end date.

These attributes should be checked by the rule that controls whether a user can read a BoardingReports document.

Enforce ordering of BoardingReports triggers

When creating a Boarding report from the mobile apps (containing Lists such as the crew), this is converted into an INSERT and then a REPLACE into the Atlas collection – causing the trigger to be hit twice.

Need to ensure that the triggers are run sequentially or else the INSERT trigger can fail with a duplicate key (_id) error.

Upload modified Photos to S3

When a Photo document is modified, the newPhoto trigger should replace the existing image in S3 with the new one (if one is there). The URL can stay the same or a new one can be created (which would then be stored in the document).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.