Changes

This list is subject to change based on feedback from Canonn Members

• BT Types
• FG Types
• Merge FM + GY & add types
• Ask users for screenshots for additional types

Additions

• Add Crystalline Shards - (CS)
• Add Gas Vents - (GV)
• Add Cloud - (LG)

Data

These changes are going to seriously break what we have currently and are going to be changing over the next few weeks as we will also have to implement changes on the sheets.

This chart is shown on the website and needs to have the data in a model on Strapi

External Issue: strapi/strapi#1556

Informations

• Node.js version: v10.6.0
• npm version: 6.1.0
• Strapi version: 3.0.0-alpha.12.7.1
• Database: 10.2.16-MariaDB
• Operating system: Linux Mint 19 (Ubuntu 18.04)
• CAPIv2 Version: v2.0.7

What is the current behavior?

Unable to send a POST request on a boolean field using True or False, I have to send 1 or 0

Steps to reproduce the problem

Using Mysql create a model with a boolean field and attempt to send a POST request in Postman using True or False.

Make sure you have debugging turned on in database.json

What is the expected behavior?

you should be able to send True or False in a post request and strapi should handle the conversion in bookshelf/knex

Suggested solutions

It seems this error that was previously fixed has regressed again, related issue and previous PR:

Issue: strapi/strapi#1203
PR: strapi/strapi#1299

@lauriejim

for Athens to exclude automated debug updates

Our goal here is to build postman tests for all models to allow build testing of every endpoint to verify functionality

The flow should go something like this:

1. POST a new item (multiple?)
2. GET the item by ID
3. PUT an update to the entry
4. DELETE an entry
5. GET all entries

Checking should be done to verify the schema as well as response time and response code.

Relational checks should also be included but for now just the core model checking should be the focus.

External Issue: strapi/strapi#1419

First noticed on v2.0.4, however in working on updating to v2.0.5-dev the issue still exists, the issue is due to relations on the model. If the relation is removed, you can delete by ID with no problem.

Tracking issue for updating bodies for EDSM Fields

• edsmBodyID - Integer
• edsmSolidComposition - Json
• edsmSurfacePressure - Float

The POI model is to be used by various Canonn tools and services such as the ED3D Maps to hold important information that may not fit our typical site schemas such as information about our home system, or other systems that may be of interest.

model: pointofinterest

WIP

Create example files based on current GSheets Data.

• BM
• BT
• FG
• FM
• GEN
• GEN - Logs
• GB
• GB - Messages
• GR
• GR - Active Groups
• GR - Active Obelisks
• GS
• GS - Active Groups
• GS - Active Obelisks
• GY
• HD
• LS
• MS
• MS - Flight Ops Route
• NHSS
• OI
• TB
• TS
• TW
• USS Removed v17
• Systems
• Bodies

Add another model that is a sub report of grreport & gsreport for reporting active obelisks and confirming data gathered from an obelisk

• Add grobeliskreport
• Add gsobeliskreport

• Remove Type and just store a string
• Remove Subtype and just store a string
• Update API Docs to reflect changes
• Regenerate seed files/DB (Remove csv seed files for type/subtype)

After thinking about it, there is no reason we should split type and subtype to their own models with relations, we should just store them as a string thus our DB will update as EDSM does and we won't have to check for Anthors alias changes.

to allow us to black list bad clients or updates

Requested layout:

model: patrol

reportType: string
    example: bt
reportID: (Integer?) manyToMany morph?
    example: 125 (could also be linked to the actual report, need to play with morph relationships)
reportVote: integer
    min -10
    max +10
    default 0
    note: calculated based on patrol responses, also mirrored from/to the report
priority: integer
    min 1
    max 5
    default 1
    note: used if we need to set something as high priority
isOrbital: boolean

model: patrolresponse

cmdr: relation - oneWay
    cmdr
patrol: relation - oneToOne
    patrol
dataCorrect: boolean
clientVersion: string

Which Server are you experencing the issue with:

• Live Server - https://api.canonn.tech

• Beta Server - https://api.canonn.tech:2053

• Alpha Server - https://api.canonn.tech:2083

What endpoint(s) are you seeing the error on:

URL: https://api.canonn.tech:2083/admin/plugins/content-manager/reportfss?source=content-manager

Describe the problem in as much detail as possible:

Unable to page through the data.

Steps to reproduce the error:

1. Go to reportfsses
2. Click on the paging controls at the bottom right of the page

Additional context

The problem is the new ? button on the bottom right corner. It is covering the controls so you cant navigate. Opening the DOM editor in firefox I was able to delete the button and with that out of the way I was able to page through the data

NOTE: Upstream Strapi issue: strapi/strapi#1938

Which Server are you experencing the issue with:

• Live Server - https://api.canonn.tech
• Beta Server - https://api.canonn.tech:2053
• Alpha Server - https://api.canonn.tech:2083

What endpoint(s) are you seeing the error on:

URL: All AdminUI URLS

Describe the problem in as much detail as possible:

Steps to reproduce the error:
Install GraphQL Plugin

Expected behavior

GraphQL should not be breaking admin 😉

Screenshots

See Upstream Issue

Additional context

See Upstream Issue

• Remove bodyatmosphere model and update body model field to json
• Remove bodymaterial model and update body model field to json
• Update Documentation to reflect changes

Due to difficulty in matching these models to the body model as a relationship and the fixes done to Strapi to support raw json storage (and how EDSM stores it) we should change the body model to store this as raw json.

Testing will need to be done to ensure this follows the proper format on both the restAPI and GraphQL

To Do List:

• Core update
• Update seed files
• Update tests
• Update Swagger Docs
• Minor README updates
• Testing of new plural routes
• Update public model files for Athena/Athens

Reminder to myself to talk with Sileo about storing his ID on sites.

Example Data:

{
  "Name_Localised": "Amphora Plant",
  "SystemAddress": 1934941425067,
  "Region_Localised": "Norma Expanse",
  "Name": "$Codex_Ent_Vents_Name;",
  "EntryID": 2101400,
  "System": "Prua Phio ZL-T d4-56",
  "SubCategory_Localised": "Organic structures",
  "Category_Localised": "Biological and Geological",
  "Region": "$Codex_RegionName_10;",
  "timestamp": "2019-03-05T02:02:11Z",
  "event": "CodexEntry",
  "Category": "$Codex_Category_Biology;",
  "SubCategory": "$Codex_SubCategory_Organic_Structures;"
}

Get hyperdiction locations and dates from commander logs via EDMC. EDMC will only supply data if the the stats have changed.

Requires new Strapi Report model

Requires aggregation into strapi HyperdictionSite. model

Question: Should we keep the old model with from and to sites and the hyperdiction detector which is a bit unreliable?

Or should we move to the more reliable stats model but risk missing hyperdiction data when hyperdictions happen in quick succession?

Informations

• Node.js version: 10.1.0
• npm version: 6.0.1
• Strapi version: Monorepo 76bb31970f63a86e6976b5406b35d3239d8877b6 (Current Master as of 2018/5/15)
• Database: MariaDB v10.2.14
• Operating system: Ubuntu 16.04 LTS

What is the current behavior?

When using MySQL, and storing a boolean value, the tinyint column type is used thus storing false as 0 and true as 1. However the api response (get, post, and put) use these values. These tinyint values should be used as true and false.

Steps to reproduce the problem

using MySQL, create a boolean field and store some data for testing, Use get, post, and put to try and manipulate the data. Get will return the 0 and 1 values, post/put will not accept true or false and require the use of the numerical values.

What is the expected behavior?

Strapi should translate the boolean tinyint to correctly display and accept false as 0 and true as 1

Suggested solutions

Unsure how bookshelf is handling these values, likely needs a more indepth look and some type of alias to convert back and forth.

External Issue: strapi/strapi#1203

As well as logging Thargoid Bonds we should maintain a count to display in EDMC if requested by the user. The counter should be zeroed if the bounties are cashed.

What needs to be achieved :

• We need to modify the content-managerplugin :
• We need to apply the same filters that the ones mentioned in the documentation
• Harmonize the backend so it can reflects those changes
• Develop the UI of the filters options
• We need to modify the plugin-upload admin since its components are from the strapi-helper-plugin
• Update the plugin-graphql since it uses the content-manager's controllers and services

External Issue: strapi/strapi#1204

• Update Report Model to add Cycles & Defenses
• Add tbcycle model
• Add tbdefense model
• Update tbsite model for oneWay relations to cycle and defense
• Update Seed file and supply initial cycles and defenses
• Update TravisCI Build
• Update Docs for new/modified Fields/Models

Related GDocs: https://docs.google.com/document/d/1rpU2r7HjhShZrLvCwNrzvI6NrxrYK3Ub1KmIWdVtYvU/edit#

Speaking with NetSlayer in relation to the Thargoid Barnacles and the current data I'm making some changes to the API to better reflect (and hopefully ease) the transition from the Meta^2 sheet to the API

We need to create a cron script to either run within strapi or outside as a cronjob.

Our goal is to get the list of systems and bodies and check their updateTime, if it is less than x amount then we query EDSM to pull the data and update it in the database.

There is also the option of doing so before save, but this will require additional testing to see if it works as intended, and how we will handle errors. See the example file here:

I believe we could inject the values before we save, but in this case anytime something is changed, it will ask EDSM for an update. We could use this in conjunction with the cron script

The alert models purpose is to be used as an emergency alert system within Canonn plugins and tools.

model: alert

title: string
body: text
active: boolean
tldr: string
alertLevel: integer
    min: 1
    max: 10

In APIUPDATE model change:

"UpdateTime": {
      ...
},

to

"updateTime": {
     ...
},

(first letter uppercase to first letter lowercase) for consistency with other fields.

On the USS Survey this is done as part of the USS drop manual data entry. In the new plugin this should be a popup interface that comes up after a scan. Something involving clicking on icons to identify the glypgh

We should get the icons from the valkunut page.

It would be handy to detect a screengrab and allow the user to upload an cropped image,

Informations

• Node.js version: v9.11.1
• npm version: 5.6.0
• Strapi version: 3.0.0-alpha.12.1.2
• Database: 10.2.14-MariaDB
• Operating system: Ubuntu 16.04 LTS

What is the current behavior?
When creating a new content type the adminUI still states to create the table manually:

Steps to reproduce the problem
Setup a new strapi instance with mysql, create a new content type

What is the expected behavior?
Message should not show, as it is no longer required.

Suggested solutions
Remove message

External Issue: strapi/strapi#1115

Model:

regionID
name
journalName

EXT Issue: strapi/strapi#1406

Attempting to create a model with the json data type (currently stores it as text in MySQL) and saving data causes strapi/myql to break the json data down and attempt to save it to separate columns.

IE trying to save this: {"a": 5, "b": 10}

Results in this error:

[2018-06-17T21:53:48.801Z] error insert into `testmodels` (`created_at`, `data`, `test`, `updated_at`) values ('2018-06-17 14:53:48.790', {"a":5,"b":10}, 'test2', '2018-06-17 14:53:48.790') - ER_BAD_FIELD_ERROR: Unknown column 'a' in 'field list'

External Issue: strapi/strapi#1387

Currently we are unable to add oneWay relations onto the same model, an example of such a use would be the Thargoid Structures message locations referencing other sites.

The bug was confirmed on an external issue, and I'd image a fix is being worked on.

Request to modify 'apiupdate' model, in line with: canonn-science/CAPIv2-Updater#22

• Remove forced:boolean field (It has been moved to updateLog)
• Add error:boolean field, determining if the update crashed or ended successfuly.

External issue: strapi/strapi#1323

Currently if we try and configure the prefix api.canonn.tech/v2/systems instead of api.canonn.tech/systems this will break the /public folder where we plan to place Static API documentation until Strapi includes automated documentation like swagger.

The issue has been confirmed by the Strapi team.

• Create USS Event Model (Merged with Report Model)
• Create USS Report Model
• Create HD Event Model (Merged with Report Model)
• Create HD Report Model
• Build cron script(s) to migrate data from USS/HD to Site Model
• Test using EDMC plugin

As outlined on Discord:

So for example lets take the USS & HD Sites:

Each have 2 "Report" Models:

USSEvent // USSReport
HDEvent // HDReport

The events are completely automated from journal data, almost Zero user input if any. The reports will be for manual user input and can be tied to the event using a oneToOne relationship. Now talking technical, if we go this route the EDMC plugin will need to "POST" the event first, grab the ID then "POST" the report including the eventID in that request.

Then your script comes into play. This, like the body update script, is going to be fairly complex with a few lookups. I haven't yet even discussed this with @[R&D - Web Dev] AdmlAdama but basically the flow goes something like this:

1. Check if system exists, if not, create it & grab systemID (Our internal, not EDSM)

2. (If applicable) Check if body exists, if not create it and attach it to the system. It already exists grab the bodyID

3. (If applicable) Same as body but for ring (We don't have any use for it right now, could change with the mining crap)

4. Check if location exists, if both system & body do this is almost a given, if not create it using the following:
   systemID
   bodyID
   Site Short code (BT, GR, GS, ect)

5. Check if site exists (Is this an update?) if not then create it and attach it to the location using lookups if needed for type of site (GR Alpha, Beta, Gamma, ect)

6. (Currently only applies to the GR/GS) Lookup/Create sub models such as active obelisks and groups ect.

Informations

• Node.js version: v9.9.0
• npm version: 5.6.0
• Strapi version: 3.0.0-alpha.12.1.2
• Database: MySQL
• Operating system: Windows 10

What is the current behavior?

Deleting a content-type does not drop the database table in MySQL, & when I navigate to roles & permissions, I can still see the deleted content-type and edit its permissions. When i create a new user role however, it only shows the non-deleted content-types. So for some reason, the content-types don't get deleted from the memory of existing user roles

Steps to reproduce the problem

1. create content-type
2. add a string field to it
3. save
4. delete content-type
5. content-type is not in the list of content-types anymore but the database table still exists + the reference to the content-type remains in the roles & permissions folder of a user role, when editing permissions.

What is the expected behavior?

deleting a content-type should remove all references of this content-type ever existing + drop its database automatically. Dropping the database manually still doesn't make the reference to the content-type in roles & permissions disappear. Dropping the whole database and re-creating it does however fix the problem

Suggested solutions

N.A.

External Issue: strapi/strapi#1114

External Issue: strapi/strapi#1657

Did some testing on some VMs it seems the AdminUI isn't sending the jsondata properly and is sending 'undefined'

Testing using postman and the data POST, PUT, DELETE, GET are all fine so it shouldn't affect development of EDMC/Vue/Python script projects. Only dropping this here as a reminder to check for updates from the Strapi team.

Thargoid hyperdictions are different than the typical USS entries in the journal, using the following spreadsheet to build the model.

https://docs.google.com/spreadsheets/d/1grRDdXb6-6W1oxAVzPvvqTQDiVuExqAnvR97Q7cvrMA/edit?ts=5a974486#gid=1746833765

Will likely need to be oneWay relations, will need the following:

• Data (Messages received after correct combo)
• Artifacts & Samples (Items that can be used for combos)
• All Obelisk groups & All Obelisks by site type (Mirror old API for GR)
• Active Obelisk Groups
• Active Obelisks

The survey in the Canonn plugin will log USS drops but will also allow logging of USS from supercruise without drops.

USS Drops can be marked as confirmed however supercruise reports are manual and therefore considered suspect.

We could use USS drop data to prove that the type and threat level are valid combinations and request confirmation if it is in any way suspect.

Counts of USS types/threat are not important.

We will need to update the strapi model with appropriate statuses and we will need a mechanism for validating manual submissions.

Also keeping track of the time spent in supercruise would be useful for the perspective of Hrafnagud which needs to know if a system has been surveyed to count it as complete.

External Issue: strapi/strapi#1392

I have temp committed a fix that enables it: d922f1a

I believe the intended fix will add an additional configuration variable that will allow you to enable it in production use.

This is shown on the website. Its a bit of an odd one as it is live stats we would need to update fairly frequently but it is also time bounded as we are only looking at kills in the current tick Thursday to Thursday for specific systems.

I think we actually need an eagle eye model to make this work

Reminder to add in a Metadata model for use with the following repo: https://github.com/canonn-science/CAPIv2-Updater

Should include the following fields:

• updated: DATETIME
• forced: boolean
• systemsUpdated: json
• bodiesUpdated: json
• updateLog: json

Subject to change based on @coldwhitelight 's input on the reporting structure of the Updater tool.

Tracking Issue for EDSM Python script for pulling body data and caching the data locally as well as scripted for cron updates.

Due to our lack of available javascript devs we should build this in Python for now and move to javascript later.

Breakdown

So most of the other 3rd parties currently store the body name as systemname bodyname however we differ in that we break the two apart. This will be the biggest challenge I believe as you cannot just hit the body table then query edsm. You will need to grab the system ID, grab the systemName, and query EDSM using the systemName. Then you will need to join the systemName with the bodyName and search the response for the data. There should also be an else if clause in that if you do not find the entry, you should then just search the data for the bodyName. I think in most cases those two should pull the correct data and properly handle special case systems like Sol where the body name is a custom named object such as Mars.

Also as a footnote to the above, it is possible that the body data does not exist, in which case we will need to skip that. I can add a special boolean column if needed to help with the cronscript so we can track how many times it has been skipped (may need to add another argument to say don't query if skip > x amount So that the cron script doesn't just keep trying to lookup the same missing body over and over. (see example arguments below)

Required arguments

Similar to the systems script, below are some arguments that should be added to allow for ease of use in specific use cases:

• mode
  • Either development, staging, or production
  • See Systems script to understand, example --development
• config-file-path
  • Used to manually define the location of database.json file for MySQL connection data
• update-missing
  • Used to only grab entries with specific missing data
  • We should only use lookup on specific entries like edsmID not every column
• delay-seconds
  • Used to define a manual delay between batches to help with load on EDSM
• batch-size
  • Used to define the number of bodies in a lookup batch, again to help with load on EDSM
• batch-limit
  • Due to it being possible to having the MySQL server dropping the connection if it is held too long, this is to limit the number of batches, and also works to set a limit on the number of bodies looked up within a given cron timeframe
• fail-skip
  • Should be used to define the number of failed attempts and skip those bodies
  • NOTE: This should be built in, but currently that field does not exist within the bodies model. It should be added if needed.
  • In relation to this if body data isn't found, the script should increase that counter by +1

Breakdown of our columns vs EDSM

(WIP)

We have two of these charts displayed on the lab69 pages. One for each target system. Again this might need some EE Strapi Models

External Issue: strapi/strapi#1435

Issue was created for Jim for tracking. Also making one here as well.

Currently after a NHSS drop the user is presented with an interface that allows the to display the number and composition of thargoids in the instance.

This will require a Strapi Model

It would be useful to be able to configure the interface from the server side so that new Thargoid types can be added quickly

The data can be used for a new map that can show the location of scouts vs interceptors and not just the threat levels

Informations

• Node.js version: v9.11.1
• npm version: 6.0.0
• Strapi version: 3.0.0-alpha.12.1.3
• Database: 10.2.14-MariaDB
• Operating system: Ubuntu 16.04 LTS

What is the current behavior?

When setting a field as unique and using MySQL, strapi still allows for duplicate values to be written to the database. (Note that strapi does not create and apply a UNIQUE key on the specified column in the database)

Steps to reproduce the problem

Using MySQL or MariaDB:
Create a field in the content builder and assign the option unique, then attempt to save a value multiple times. Strapi will write multiple entries without throwing an error.

What is the expected behavior?

Strapi should (for MySQL, unsure about postgres) create and assign a UNIQUE key value to the specified field.

External Issue: strapi/strapi#1189

This is not on the web site so low priority but people have been looking at it on discord