geolytix / xyz Goto Github PK

In order to improve usability, the application offers to remember users so that the next time they visit the application, they do not have to login again. If a user were to select this option on a computer which is accessed by several people, a malicious user would also have access to that account.

Location or cluster of locations on a cluster layer may only be represented by a symbol.

This symbol for location and cluster is currently defined by marker and markerMulti respectively. Each of these are a style object similar to the default / highlight style used for non-cluster layer.

There is a scale property which is used to scale a symbol.

There are markerMin/Max settings which can be defined inside the marker, markerMulti, and theme styles.

The markerMin and markerMax define the size range symbols. A single location will be drawn at the size of markerMin. A cluster will be drawn at a size according to the ratio of locations inside the cluster against the largest cluster in the viewport. The largest cluster in the viewport will be drawn at the markerMax size.

marker definitions are exactly the same as a default style definition for other types of layer. marker should be renamed to default to be in line with other layer style definitions.

markerMin needs to go. markerMin being the size of a single location. scale and size should be added to the default style and any other symbol styles.

Openlayers only understands scale to scale a symbol. Size should be introduced in addition to scale. Size should be defined as pixel. The size should be the average of width and height of a symbol. The size can then be used to scale a symbol.

markerMulti should be renamed to cluster as it only applies to cluster. The scale / size inside the cluster is the maxSize.

This allows for locations to be defined as default, cluster, in a theme, or as highlight.

Object assign applies in this order.

The default style will be applied to single locations.

The cluster style will be assigned to the default style for cluster. As a minimum we only need to define a scale or size in the cluster.

A theme style will either be applied to the default or default + cluster style for graduated themes or if all locations in a categorized theme are of the same type.

The highlight style will be assigned to the previously created style for when a location of cluster is highlighted. As a minimum we only need to set a scale or size in the highlight style.

We replaced the reference for https://api.mapbox.com/mapbox-gl-js/v1.12.0/mapbox-gl.js with a reference to the maplibre mapbox-gl library in the public/js directory.

We also removed the carnery warning for missing style sheets by adding the class to the mapview stylesheet.

.mapboxgl-canary {
  background-color: salmon;
}

Token / Cookie signatures are valid for 8 hours. This could be changed to 30 minutes with each successful call passing through the auth module renewing the token / cookie with a 30 min expiry.

Possible mvt cache query:

WITH 
     n AS (
    /* insert new created tile */
    INSERT INTO postal_district__mvts (z, x, y) VALUES (14, 8185, 5447)
        ON CONFLICT (z, x, y) DO NOTHING RETURNING (z, y, x, mvt) 
   /* returns only when inserted otherwise empty set */
    ),
     c AS (
         /* select existing tile from cache */
         SELECT z, x, y, mvt FROM postal_district__mvts
         WHERE z = 14 AND x = 8185 AND y = 5447
     )
SELECT mvt FROM n FULL JOIN c ON TRUE;

this will run if cache defined. Otherwise mvt created from table.

Vercel allows to define edge cache rules with the cache-control header.

https://vercel.com/docs/serverless-functions/edge-caching

The cache can be set for all routes in the headers definition in the vercel.json.

  "headers": [
    {
      "source": "/(.*)",
      "headers": [
        {
          "key": "Access-Control-Allow-Origin",
          "value": "*"
        },
        {
          "key": "Cache-Control",
          "value": "no-cache"
        }
      ]
   }
]

Or directly on a response header.

res.setHeader('Cache-Control', 's-maxage=1, stale-while-revalidate')

The logic of an edge cache is that the same request should provide the same response. This response must not be processed by the serverless function but can be served from the cloud edge cache, resulting in much improved access speeds.

The stale-while-revalidate directive will serve cached content but also re-evaluate the cache content after sending the cached response. This allows for fast response but makes sure that cache is continuously refreshed.

I understand that the points on the map need to disappear while dragging and zooming for perfomance reasons, but i think it would be good if it was optional, i've commented out the lines of code that do that and it worked perfectly in my scenario.

We want to be able to use dynamic themes based on field statistics of viewport elements.

For vector formats this is relative straightforward.

In #871 we have introduced a field_stats object on the theme.

A plugin can add a custom module which imports the simple-statistics library.

const simple_statistics = import('https://unpkg.com/[email protected]/index.js')

export default (async function () {

    mapp.utils.simple_statistics = await simple_statistics

    mapp.layer.themes.graduated_jenks = function (theme, feature) {

        // Check whether the field_stats field values have changed.
        if (theme.updated) {

            // Generate jenks bins based on the number of cat in the cat_array
            theme.field_stats[theme.field].jenks = mapp.utils.simple_statistics.jenks(theme.field_stats[theme.field].values, theme.cat_arr.length)

            delete theme.updated
        }

        // rest of graduate theme logic but values are checked against the jenks array not the cat_array values.
    }
})()

This can only work for vector layer, not for MVT layer. There are multiple separate issues in regards to the MVT format which must be resolved before dynamic themes are possible with MVT layer.

Setting the environment key APPROVAL_EXPIRY will expire non admin_user accounts X days after they have been approved.

The approved_by field in the ACL will now store the admin user who approves an account with an ISO date-time string of when the account was approved eg. ([email protected]|2020-08-02T14:17:57)

The APPROVAL_EXPIRY value or 0 will be added to that date-time to check whether it exceeds the current data-time upon token generation. If this is the case the account approval will be removed. The user must then re-register. The same account can be re-registered and a user_admin must approve.

Removing the date-time from the approved_by field will prevent the account from expiry.

The value for a location's geometry entry maybe a feature collection returned as geojson from a complex lookup defined as a fieldfx.

It should be possible to assign a theme with the same format as provided in layer styles to the feature collection. The theme should be in the style definition which is already part of the geometry entry definition.

A user can access their account from multiple devices at the same time. While this enhances the user experience, if an attacker were able to gain access to account details they would be able to perform actions undetected as the affected user. Many applications allow this feature, but provide a method for the user to monitor which devices have access to the account.

The map being unresponsive due to a stopevent overlay not allowing interactions to reach the map dom indicates that the Openlayers.css is missing.

I removed the Openlayers.css from the controls.scss. I can't remember the exact reason. I think it is something along the lines of what if I want an Openlayers map but complete custom or no controls with the map.

Make point from navigator location.

The web application is vulnerable to reflected cross-site scripting attacks.

Impact
Cross-site scripting can be exploited to execute HTML or JavaScript code supplied by the attacker.
Examples of attacks that can be conducted include logging of all keystrokes entered by the
compromised user, redirection to attacker-controlled websites and spoofing of login forms to trick
users into divulging their credentials. Cross-site scripting code can be encoded to appear less
suspicious.

Threat Assessment
The malicious code can be entered directly into the URL in multiple locations. Cross-site scripting can
have significant effects against users, as detailed above, and constitutes a high risk to all users of the
application. An attacker would need to trick a user into clicking a link containing the malicious code,
most likely via a phishing attack. Once clicked, the code will execute immediately without further
interaction from the user.

Recommendation
Input validation checks should be made to ensure that only expected data is processed by the
application. All user-supplied data should be encoded before transmission to the client. These checks
should be applied to every field in the application that can possibly accept user input, including those
hidden from an ordinary user.

PostgreSQL does not support parameters for identifiers, eg. database, schema, table, or column names (e.g. in DDL statements). This is a potential security vulnerability. In order to prevent DDL injection in parameters used for identifiers we will only allow a set of whitelisted characters to be substituted in parameters.

Parameters in curly braces prefixed with a percent sign instead of the dollar sign, eg. %{body} are provided as an array with matching parameter placeholder in the query string.

%{body}, ${viewport}, and ${filter} are reserved parameter.

This will only apply to templates without an implicit render method.

The render method will no longer be assigned as default. The current render method will be assigned to view templates in the view module.

The render method for query templates is in this codepen.

55a694c

Really great looking project here, could really make it a lot easier for people to start using mvt tiles with postgis. One question though in:

https://github.com/GEOLYTIX/xyz/blob/master/mod/mvt.js

I see parameters information from the express route being used directly in the sql string literal, and as far as I can see, no attempt at escaping characters in the values, etc. I ended up with the same problem implementing this in my app, I'm curious if there's a good solution for this out there-

At the moment objects require roles for all supported languages in order to be displayed when selected language version is not available.

At the moment empty container returned.

Purple circle was constructed from turf with 4326 centre coordinates and radius of 3km.

When using custom radius plugin where feature is constructed with turf.circle measured radius is correct.
When similar circle is drawn with interaction distance metric tooltip returns significantly distorted value.

Problem possibly lies with the fact that ol length() inside metrics function is calculated from 3857 coordinates.

At the moment the docs are out of sync as I don't believe this is possible, and Zeit are now enforcing new users onto the v2 platform, so the zeroo configuration promise is not fulfilled

Revised Workflow

Revised workflow has created the need to create a separation of the CDN domain used in a workspace, so that we can make use of different AWS CDN domains. That way we can point different instances to different s3 buckets, that are referring to different branches in the same repo.

Example of .env making use of CDN_DOMAIN

 "TITLE": "GEOLYTIX XYZ | Test",
    "DIR": "/test",
    "ALIAS": "geolytix.xyz",
    "PRIVATE": "",
    "SECRET": "",
    "APPROVAL_EXPIRY": "30",
    "CDN_DOMAIN": "blag.cdn.com"
    "TRANSPORT": "",
    "WORKSPACE": "",
    "DBS_MAPP": "",
    "KEY_GOOGLE": "",
    "KEY_HERE": "",
    "KEY_MAPBOX": ""
  }

Example of Workspace using CDN_DOMAIN KEY

{
  "templates": {
    "test_template": {
      "src": "{CDN_DOMAIN}/path/to/s3oject.json"
    }
} 

This can accommodate for the new workflow

Once added to the map the mapbox-gl canvas is 15px thinner than the containing Openlayers map node.

https://gis.stackexchange.com/questions/379976/epsg3857-shift-between-mapbox-gl-and-openlayers

At the moment when map uses zoom control buttons click results in selecting feature underneath.
Click propagation from .ol-control container should be stopped.

I run into problems setting up a numeric filter running on a range from 0 to 0.44.

Chart type: bar, chart type is significant as not all charts have axes (like pie)
Fiddle

var ctx = document.getElementById("myChart");

const aga_plugin = {
afterEvent: function(chartInstance, chartEvent) {

var xAxis = chartInstance.scales['x-axis-0'], // this is default axis, in case of horizontal bar use y-axis
    x = chartEvent.x,
    y = chartEvent.y;
    
    // these conditions will change when other axis used
    // also click here is precise, will be faulty when tick labels are shifted/rotated
    if (chartEvent.type === 'click' &&
    	x <= xAxis.right && x >= xAxis.left &&
      y <= xAxis.bottom && y >= xAxis.top) {
      var index = xAxis.getValueForPixel(x);
      alert(chartInstance.data.labels[index]);
    }
  }
};

Chart.plugins.register(aga_plugin);

var mylineChart = new Chart(ctx, {
    type: 'bar',
    data: {
        labels: ["Time Management", "Career Coach", "Stress & Wellbeing", "Note Taking", "Exam Prep", "Presentations"],
        datasets: [{
            label: 'Module Tracker',
            data: [6, 4, 2, 0, 3, 1],
            backgroundColor: 'rgba(255, 99, 132, 0.2)'
        }]
    },
    options: {
    scales: {
    xAxes: [{
      scaleLabel:{
        display: true,
        labelString: 'Week'
      },
      ticks: {
        maxRotation: 90,
        minRotation: 90
      }
    }]},
    plugins: {
      aga_plugin: {} // plugin kicks in
    }
  }
});

Modules may now be imported dynamically.

This is obviously extremely experimental and wants some testing.

    "n_from_table": {
      "module": "https://geolytix.github.io/public/n_from_table.js"
    }

Setting a module location will import this module as a template.

Template modules must have a render method defined.

module.exports = {
  render: _ => {

    const n = _.n * 2

    return `select * from ${_.table} limit ${n}`
  }
}

The "import" is done by creating a module from the module constructor using the response as source and the url as filename.

      if (entry[1].module) {
        return provider.http(entry[1].module)
        .then(response => {

          const Module = module.constructor;
          const m = new Module();
          m._compile(response, entry[1].module)
          _resolve(m.exports)

        })
      }

While it is possible to define different tables for different zoom levels in the layer tables configuration it is currently impossible to define different geometries for different tables, or indeed the same table with different geometry fields.

It should be possible to define a geometry field with a table in the tables configuration to support multiple geometry fields. This will allow to have simplified geometries for the same location at lower zoom levels.

It would be possible to define the geometry field separated by a pipe or other symbol which cannot be part of a regular table or field name.

We have now added the possibility to override the query timeout of 10 seconds on the node postgres pool used to communicate between the middleware and database cluster.

No timeout is the 4 parameter to be passed with the query statement. This will effectively set the timeout to 0 and then back to it's original value after the query has run.

    env.dbs[dbs] = async (q, arr, no_log, no_timeout) => {

      try {
        no_timeout && await pool.query('SET statement_timeout = 0');
        
        const { rows } = await pool.query(q, arr);

        no_timeout && await pool.query('SET statement_timeout = 10000');

        return rows;

      } catch (err) {
        Object.keys(err).forEach(key => !err[key] && delete err[key]);
        if (!no_log) console.error(err);
        return { err: err };
      }

    };

Logging with Vercel works now rather well with the logflare (beta) integration.

https://vercel.com/integrations/logflare

The integration can be configured in the integrations tab of the vercel dashboard.

With the highlight / select interaction being active, the select event will be fired once click / touch ends after map panning.

In order to improve security the autocomplete should be disabled for the password field on the login form.

The style assigned to the drawing interaction should be applied to the interaction as well as the feature in the drawstart event.

It should also be possible to assign an image in the style param.

It is possible to obtain valid usernames by comparing responses from certain sections of the application. An attacker can use these in combination with a to gain access to accounts. A large number of valid usernames could be discovered via this method, which increases the likelihood of a user's account being compromised.

Using an unlogged table for the cache could provide a non trivial increase in speed.

1. Add custom results limit with default of 10
2. Make source label optional

Sometimes it's useful to hide labels when they no longer contribute to map story.

Finding
The web server does not implement several security headers.

Impact
Omitting these headers leaves users of the application vulnerable to a variety of potential attacks,
including man-in-the-middle attacks, cross-site scripting and phishing attacks that ‘steal’ user clicks or
data entered into malicious forms.

Threat Assessment
The attacks are dependent on certain functionality being made available within applications and
additional prerequisites including suitable network positioning or user interaction. An attacker would
need to entice a victim to click on a phishing link, or visit a malicious website, in order for attacks to
take place. As a result the attacks are highly targeted and unlikely to affect a large number of users,
unless an attacker can find a way of planting a link within the website which is stored and presented to
all users of the application.
The X-Frame-Options dictate where scripts and other resources can be loaded from and can prevent
third-party websites from interacting with the application through attacks such as UI Redress attack
and even cross-site scripting.
In a similar vein, the X-XSS-Protection header prevents malicious cross-site scripting code from
executing and ensures that users are given an additional level of protection against such an attack.
The Expect-CT security header lets sites opt in to reporting certificate transparency requirements, this
is done to prevent the use of mis-issued certificates for that site from going unnoticed.
The X-Content-Type-Options header ensures that browsers do not attempt to display web page code
in an unsafe way. This is known as ‘sniffing’, where the browser will make a guess at how the content
should look based on the code within, rather than rendering it as illustrated by the server.
There is generally no impact on usability to adding these server security headers as they operate
behind the scenes and will not be visible to regular application users. Issues may arise with regard to
Content-Security-Policy and X-Frame-Options if resources are requested from other domains.

Recommendation
The following server security headers should be added to the application.
• X-Frame-Options
• X-XSS-Protection
• Expect-CT
• X-Content-Type-Options

Roles can be set on locales to limit access to that locale.

e.g. Setting the role glx as true in the uk locale will prevent the locale to be returned in the api/workspace/get/locales array.

"uk": {
  "roles": {
    "glx": true
  }
}

api/workspace/get/locale?locale=uk will respond with a 403 if the role glx is not found in the token.

For locales the the role value will be ignored. Filters can only be set on layer, not locales.

Getting any layer from the uk locale will also respond with a 403.

e.g. api/workspace/get/layer?locale=uk&layer=base

Still to implement are role checks on the actual data requests.

I would like to have a JSON type field which will be parsed and stored as a JSON type field in the PostGIS layer. There need to be checks on the input that the field contains valid JSON.

Once this is working the JSON type field should replace the meta type field. Allowing a JSON type field to be used as meta data field or indeed for any other type of configuration. We already have enough controls to hide or style the field in the location view.

Setting the tooltip key of a editable geometries to either area, distance, or length will produce a tooltip popup on the mouse cursor during the construction in the draw interaction.

          "edit": {
            "polygon": {
              "tooltip": "length"
            },
            "freehand": true,
            "circle": {
              "tooltip": "distance"
            },
            "rectangle": {
              "tooltip": "area"
            },
            "line": {
              "tooltip": "length"
            },
            "isoline_mapbox": true,
            "isoline_here": true,
            "delete": true,
            "geometry": true
          },

This metrics are calculated with openlayers sphere methods.

The distance is calculated from the interiorPoint of the geometry to the mapview.position (cursor).

Length is measured along the geometry. For polygons, circles, and rectangles the length is the circumference.

Other API keys for the environment settings are pre_fixed with KEY_; the Cloudinary key isn't.

Cloudname and folder are supplied in this 4 part entry.

Cloudname and folder should possibly be supplied in the workspace or as query parameter not in the environment settings.

Without a fill being specified the symbol in the legend should be a line of the same thickness as the line in the map instead of a square.

Line geometries which are drawn with fills look buggy.

merge query + datasets options

Cross origin resource sharing (also known as CORS) is a way of relaxing the Same-Origin Policy to allow a website loaded from one domain to request data from a website on another domain, via the user's browser. Having a wildcard value (*) allows the application's pages to be read by any other application on any domain. While this does not allow authenticated data to be read cross-domain, it can allow location-based access controls for the website to be bypassed, potentially enabling an attacker to access a website that may otherwise be locked down.

References
https://owasp.org/www-project-web-security-testing-guide/stable/4-Web_Application_Security_Testing/11-Client-side_Testing/07-Testing_Cross_Origin_Resource_Sharing.html
https://enable-cors.org/server.html

In some cases a data source queries may run for a long time. A query to process a complex model may possibly run for hours. The node process can only wait for a certain time as limited by the AWS platform which run the process as a lambda.

Long running queries should be send with a nonblocking=true flag. With this flag, the XYZ query module must not wait for a response from the node-pg module but immediately return to the client.

The status of the long running query must be stored in a table which logs queries and their status.

The Mapp client may then ping this table at an interval to check for a status change.

There is currently no way to select a polygon which is completely covered by another polygon. A check for all features at locations and a multiple features to select dialog (similar to cluster select) needs to be introduced.

also add it to docs.

When we zoom to a location it sometimes feels like it goes too close, especially when a catchment is also displayed. Can we loop through geometries and see what is sensible to display