Code Monkey home page Code Monkey logo

de-australia-map's Introduction

TerriaJS

Build Status Docs

Terria logo

TerriaJS is a library for building rich, web-based geospatial data explorers, used to drive National Map, Digital Earth Australia Map, NSW Spatial Digital Twin and NEII Viewer (and many others). It uses Cesium and WebGL for a full 3D globe in the browser with no plugins. It gracefully falls back to 2D with Leaflet on systems that can't run Cesium. It can handle catalogs of tens of thousands of layers, with dozens of geospatial file and web service types supported. It is almost entirely JavaScript in the browser, meaning it can even be deployed as a static website, making it simple and cheap to host.


We have just released a brand new version of Terria — verson 8!

We've put together a list of things we've removed from version 8 and some steps to help you migrate to the new version in our migration guide

Not ready to move to version 8 yet? You can find terriajs version 7 here: https://github.com/TerriaJS/terriajs/tree/terriajs7


Features

  • Nested catalog of layers which can be independently enabled to create mashups of many layers.
  • Supports GeoJSON, KML, CSV (point and region-mapped), GPX, GeoRSS, CZML and zipped shapefile file types natively.
  • Supports WMS, WFS, WMTS, Esri MapServer, Esri FeatureServer, Bing Maps, Carto Maps, Cesium Ion Imagery, OpenStreetMap-style raster tiles, Mapbox, SDMX, 3D Tiles and GTFS and Sensor Observation Service item types.
  • Supports querying WMS, WFS, Esri MapServer, CSW, CKAN, Socrata, OpenDataSoft and SDMX services for groups of items.
  • 3D globe (Cesium) or 2D mode (Leaflet). 3D objects supported in CZML format.
  • Time dimensions supported for CSV, CZML, WMS. Automatically animate layers, or slide the time control forward and backward.
  • Drag-and-drop files from your desktop to the browser, for instant visualisation (no file upload to server required).
  • All ASGS (Australian Statistical Geographic Standard) region types (LGA, SA2, commonwealth electoral district etc) supported for CSV region mapping, plus several others: Primary Health Networks, Statistical Local Areas, ISO 3 letter country codes, etc.
  • Users can generate a reusable URL link of their current map view, to quickly share mashups of web-hosted data.
  • The interface is internationalised and translations are available for French, Italian and Japanese. Partial translations have also been contributed for other languages (see TerriaJS weblate)

Terria screenshot

Who's using TerriaJS?

Sites developed by Data61

Not Data61

Sites we're aware of that are using TerriaJS. These are not endorsements or testimonials.

Technical

  • NodeJS v16, v18 and v20 are supported
  • Built in TypeScript & ES2020+ JavaScript, compiled with Babel to ES5.
  • Supports modern browsers (recent versions of Microsoft Edge, Mozilla Firefox & Google Chrome).
  • TerriaJS Server component runs in NodeJS and provides proxying for web services that don't support CORS or require authentication. Instead of using TerriaJS-Sever proxy service, an alternative proxying service URL can be specified. See Specify an alternative proxy server URL
  • Dependencies are managed in NPM and assembled using WebPack.

Getting Started

The easiest way to build your own Terria-based map is using the TerriaMap starting point. This gives you the HTML structure, server and build processes you need to get a site up and running immediately.

See Getting Started in the Documentation for all the details.

Components and naming

  • Terria™ is the overall name for the spatial data platform and the team that built TerriaJS.
  • TerriaJS is this TypeScript/JavaScript library consisting of the 2D/3D map, catalog management and many spatial data connectors.
  • Cesium is the 3D WebGL rendering library used by TerriaJS, which provides many low-level functions for loading and displaying imagery and spatial formats such as GeoJSON and KML.
  • TerriaMap is a complete website starting point, using TerriaJS.
  • TerriaJS-Server is a NodeJS-based server that provides proxying and support services for TerriaJS.
  • NationalMap is the flagship Terria deployment, and the origin of the TerriaJS library.

Related components

  • TerriaMapStatic, a pre-built version of TerriaMap, which can be deployed as a static HTML website, such as on Github Pages.

Big Thanks

Hosting and contribution framework for community translations of TerriaJS provided by Weblate under the Libre plan for open source software. See our translation progress for different languages:

Translation status

Join the community

Get in touch!

Publishing TerriaJS

From main using GitHub Actions (preferred method):

  • Checkout and pull main branch of terriajs, run yarn install if necessary.
  • Bump the version number in package.json.
  • Review and edit CHANGES.md.
    • Ensure that entries for new changes are written in the right section by diffing against the last version. Make sure the section name matches the version you set in package.json above.
    • e.g. git diff 8.2.25 HEAD -- CHANGES.md (where 8.2.25 is the previous published version)
    • Or using GitHub web interface https://github.com/TerriaJS/terriajs/compare/8.2.25...main (change the version and master/next to match the previous release and branch)
    • Finalise the heading of the version to be released with version number and date.
    • Add a heading above for upcoming changes to be documented under.
  • Commit and push your changes on a branch and make a PR to main branch.
  • Get someone to review and merge the PR.
  • Wait for slack notification of successful/failed publish.
  • 😄

Publishing to npm without the GitHub Action (not for standard releases):

This method should not be used to publish versions at the tip of main or another branch that is often used for releases. To publish a new version of TerriaJS to npm with this method, you'll need to be listed as a collaborator here. Existing collaborators can add new ones. Then:

  • Checkout and pull the branch of terriajs to be deployed, run yarn install if necessary.
  • Bump the version number in package.json. Follow semver.
  • Appropriately update CHANGES.md, ensuring that changes are listed in their correct sections.
  • Commit and push your changes and make a PR to the branch to be deployed.
  • Get someone to review and merge the PR.
  • Fetch and checkout the merge commit made by merging the PR.
  • rm -rf wwwroot/build
  • Make sure you don't have any changes in your working directory.
  • gulp lint release
  • npm publish --tag your-tag (for a release not at the tip of main a tag must be set, and it must not be latest or next. For releases on old versions you could use e.g. 6-5-x or 6-x-x as a tag, for upcoming features a short name could be used)
  • Ensure that CHANGES.md on main is also updated to list the release and clearly note the nature and reason for release.

The above will publish a new version to npm and also tag that version on GitHub.

de-australia-map's People

Contributors

a-stacey avatar alexgilleran avatar bsimyo avatar chloeleichen avatar danielbaird avatar dependabot[bot] avatar greenkeeper[bot] avatar greenkeeperio-bot avatar hilarycinis avatar jiwalker-usgs avatar keyboardsounds avatar kring avatar layoric avatar meh9 avatar mgcsls avatar mwu2018 avatar na9da avatar nahidakbar avatar peterhassall avatar petertonoli avatar philipgrimmett avatar racingtadpole avatar rdengate avatar reginapramesti avatar rowanwins avatar snayler0 avatar soyarsauce avatar stevage avatar steve9164 avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Forkers

gypsybojangles

de-australia-map's Issues

Update Satellite imagery GSKY blended service (Landsat 5,7,8 and Sentinel 2A, 2B)

Hi All,

The new blended service layer (with true/false styles) has been released. The new layer is called “Multi-sensor (Landsat and Sentinel 2) surface reflectance (Beta)” which can be found at the DEA endpoint (https://gsky.nci.org.au/ows/dea ).

Note the “Multi-sensor (Landsat and Sentinel 2) surface reflectance true Colour (Beta)” and ““Multi-sensor (Landsat and Sentinel 2) surface reflectance false Colour (Beta)” layers are still available.

  • no changes are recommended during the GEO week (4-8 Nov)
  • the above services will live in parallel till the week after

DE Australia - July 2020 release

TerriaJS Onboarding v2

As a non-technical, non-spatial map user,

I want to be on-boarded in my own personally preferred way of processing new information,

so I can get started using a given TerriaMap (DEA Aust/Africa)

Other notes from Vic:

We felt the first step to fixing the on-boarding experience was leaning up the UI so its a little easier to understand, especially for first time users. I tried not to go too far with my designs and so more of a clean up. This is by no mens perfect or the ideal layout but I feel it is an improvement. Please note that when I do the actual on-boarding screens such as modal disclaimer & help, the layout may need to some tweaks.

Design discussion ticket at #39

Gdrive notes: https://docs.google.com/document/d/1cENE8rKUDknTd5EY9NzplAghaV4FetJij-UwIgX2AtU/edit?folder=17hIu2s1LGi2semARtA6h79yBsls3HtxM

Based on designs: https://projects.invisionapp.com/share/VQWC0P9FC3D

On - boarding (4ish) pieces:

(Unrelated to this ticket splitter notes)

  • continue designing split. req for feedback.
  • in the meantime, for delta, leave splitter as it is, re-wire delta to use a hybrid solution of deltav2 and delta v2.5(20200330 Sprint planning vic-demo)

Remaining things from closed v2 tickets:

Add Help in the top right menu

Based on discussions recorded here: https://docs.google.com/document/d/1Y7Z1EwwkzuBrOgrqowxauJ0-p4v70oVGyoDztOSwoUY/edit

  • add help button in the top right hand side menu similar to https://soe.terria.io/

  • fix the issue with users not being able to return to previous step (in case they are at step 2)

Add the following options/guides:

  • Explore map data

  • Interact with the data. To include instructions on general interaction with datasets and more specifically for satellite imagery

  • Zoom in to show satellite, flag to user he needs to wait to load to get imagery

  • Calendar - time

  • Zoom to location and filter by location

  • Splitter

other?

  • Add your own data

  • Change map settings

  • Share/Export/Print

DE Australia - Feb 2021 release

This ticket will incorporate feedback from migration testing post 24/12/2020 and other outstanding issues.

Post Feb 21 release of saas version


  • wrong email in terrain server - #67 this is for v7 - not v8

BoM SOS Stream Guages timeseries often does not work - BoM Tomcat OutOfMemoryError

When trying to use the BoM Stream Guages data, when you click on a station on the map, the timeseries often shows the preview in the feature info popup, but when you click Expand you get a 500 Server Error 95% of the time.

E.g. this URL: https://maps.dea.ga.gov.au/#share=s-qPKEFkC0bO1Ztq99QLAMutiONmW

Clicking on the point (sometimes doesn't work, may have to click again):
Screen Shot 2020-08-26 at 13 36 45

When clicking Expand (usually doesn't work, but sometimes does):
Screen Shot 2020-08-26 at 13 37 09

Looking at what happens on the request, it's actually a Tomcat OutOfMemoryError:

<html><head><title>Apache Tomcat/6.0.24 - Error report</title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>HTTP Status 500 - </h1><HR size="1" noshade="noshade"><p><b>type</b> Exception report</p><p><b>message</b> <u></u></p><p><b>description</b> <u>The server encountered an internal error () that prevented it from fulfilling this request.</u></p><p><b>exception</b> <pre>javax.servlet.ServletException: Servlet execution threw an exception
</pre></p><p><b>root cause</b> <pre>java.lang.OutOfMemoryError: unable to create new native thread
	java.lang.Thread.start0(Native Method)
	java.lang.Thread.start(Thread.java:714)
	sun.rmi.transport.DGCClient$EndpointEntry.&lt;init&gt;(DGCClient.java:266)
	sun.rmi.transport.DGCClient$EndpointEntry.lookup(DGCClient.java:237)
	sun.rmi.transport.DGCClient.registerRefs(DGCClient.java:155)
	sun.rmi.transport.ConnectionInputStream.registerRefs(ConnectionInputStream.java:94)
	sun.rmi.transport.StreamRemoteCall.releaseInputStream(StreamRemoteCall.java:156)
	sun.rmi.transport.StreamRemoteCall.done(StreamRemoteCall.java:312)
	sun.rmi.server.UnicastRef.done(UnicastRef.java:450)
	sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
	java.rmi.Naming.lookup(Naming.java:101)
	de.kisters.kiwis.datasource.wdp.WDPConnector.init(WDPConnector.java:97)
	de.kisters.kiwis.datasource.wdp.WDPConnectionPool.createWDPConnection(WDPConnectionPool.java:159)
	de.kisters.kiwis.datasource.wdp.WDPConnectionPool.getConnection(WDPConnectionPool.java:79)
	de.kisters.kiwis.main.KiWIS.getWDPConnection(KiWIS.java:1705)
	de.kisters.kiwis.services.sos2.wdp.WDPSOS2Operations.doSos2GetObservation(WDPSOS2Operations.java:108)
	de.kisters.kiwis.services.SOAPHandler.handleSOAPRequest(SOAPHandler.java:103)
	de.kisters.kiwis.main.KiWIS.doXMLPost(KiWIS.java:755)
	de.kisters.kiwis.main.KiWIS.doPost(KiWIS.java:673)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
	javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
</pre></p><p><b>note</b> <u>The full stack trace of the root cause is available in the Apache Tomcat/6.0.24 logs.</u></p><HR size="1" noshade="noshade"><h3>Apache Tomcat/6.0.24</h3></body></html>

This is actually a problem with the data service from BoM - we should raise it with them.

DE Australia - Waterbodies - include metadata in chart download

As a user,
When I download a CSV of a chart of Waterbodies data, I want the metadata from the dataset to be included
So that when browsing those files on my computer later, I know which dataset is which and I don't lose the associated metadata.


Follows on from #71

Metadata in a CSV won't currently be included when the data is downloaded from the Chart panel, and GA requests that it should be. Because a chart CSV download could include multiple datasets, the metadata from each dataset should be included along with information on which column (original dataset) it's associated with.

DE Australia - Coastlines - shorter url for demo and broadcasting by DEA team

DEA team would like to start using vanity url shortner for broadcasting work on some specific datasets.
The first example will be the use of the short url for the Coastlines data, to be published and disseminated to Government and other users on 17th of Sep.

There are 2 stages:

  1. 15 Sep 2020 - enable the http://maps.dea.ga.gov.au/**#share=s-DEACoastlines** when DEA team advises the data publication and its url
  2. 28 Sep 2020 - work and release a more elegant shortening of urls which will transform the above url closer to http://maps.dea.ga.gov.au/**DEACoastlines** (TerriaJS/terriajs#4692)

DE Australia on MobX- June 2020 release - End of FY20 delivery

Acceptance criteria for update of DE Australia to MobX TerriaJS:

Development to support the above:

Post 30 June - to release 30 June - 15 July 2020

  • Delta v3 implementation - post user testing undertaken 19-21/05/2020 - TerriaJS/terriajs#4390

  • Delta v3 - has to work with ODC DEA OWS set up as well as with NCI GSKY service

DE Australia - Jun-Sep 2020 migration to MobX

Examples to test:

SaaS
image

v7
image

DE Australia - Waterbodies - dots on chart line

Ability to put dots on the line chart to show where the valid observations are.

This is a symbology change, so quite different to the Water discharge level example.

Instead of a line chart the irrigators would like a line with markers chart. Here is a very basic example using excel. Data is on the left.

Please note that there are fewer points in the DEA waterbodies time series than there are in the imagery stack (surface area isn't calculated for cloud affected / partial coverage of waterbody), so the plot symbology needs to be drawn from the data in the CSV's not from the imagery service stack at that location.

image

Add Disclaimer pop up

Here’s our disclaimer text for the DEA Maps popup window.

The information displayed on the DEA Maps (the “Service”) is for general informational purposes only, and is not intended to provide any commercial, financial, or legal advice.

The Service provides access to a range of data sets and tools, some of which are in development (beta). All data and tools are provided on an “as is” and “with all faults” basis without any warranty whatsoever. Geoscience Australia and Data61 do not warrant that these data sets and tools shall meet any requirements or expectations, or that the will be fit for any intended purposes.

Geoscience Australia and Data61 assumes no responsibility for errors or omissions in the contents of the Service and reserves the right to make additions, deletions, or modification to the contents on the Service at any time without prior notice.

Geoscience Australia and Data61 does not guarantee the accuracy, relevance, timeliness, or completeness of any information or data available through the Service or on linked external websites.

See full terms and conditions [here].

A couple of other bits and pieces:

  • the link in the final sentence of the disclaimer should take users to our existing T&Cs
  • the button to close the modal and progress to the maps portal should say “Acknowledge”
  • the other option should be “Cancel” button with the behaviour being to close the window/tab, i.e. the whole maps window not just the modal, if that option is selected.

DE Australia - share links - comments from DEA team - adding unique IDs

GeoscienceAustralia/dea-config#588

A challenge with using DEA Maps to share links to our data is that small changes to the Terria config can cause Terria share links to break. This occurs when a layer is renamed (e.g. Coastal/NIDEM > Coastal/Intertidal Elevation) or moved to a new directory (e.g. (e.g. Coastal/NIDEM > Intertidal/NIDEM). Given how frequently we change the structure of the DEA Maps catalogue, this means that Terria share links go out of date quickly, which is problematic as it means our documentation or public-facing websites (CMI and DEA website) go out of date too.

A lot of these issues could be resolved if we included a unique Terria id config option in each of the layers in the DEA Maps Terria config. According to the Terria documentation:

id:
An optional unique id for this member, that is stable across renames and moves. Use uniqueId to get the canonical unique id for this CatalogMember, which is present even if there is no id.
https://docs.terria.io/guide/connecting-to-data/catalog-type-details/wms/

This is could be a pretty simple fix for a big reduction in future effort; see the unique IDs used for the DEA Coastlines Terria config below (although see #588 (comment) for a better suggestion for unique IDs):

https://github.com/GeoscienceAustralia/dea-config/blob/master/dev/terria/dea-maps.json#L1004

DE Australia - Oct 19 release

  • workbench user stepping guide - TerriaJS/terriajs#3743
    with this, once the modal is created on front page, the "take a tour" button will have to point to the Help & FAQ pages because it cannot copy the video button from DE Africa (that video will be very specific to Africa map)

  • update related maps - #15

  • add data = explore data: TerriaJS/terriajs#3742 and TerriaJS/terriajs#3707

  • redraft and include dea email address on feedback form #16

  • Leaflet fix - TerriaJS/terriajs#3740

  • update DNS - #18

  • logos - #7

  • about pages - #6

  • data and metadata - #8

  • help menu button to point to help and FAQ - #14

  • social media links in about pages - #19 - only if time permits

Wrong email in terrain server error message

Pretty minor issue, but when the terrain server isn't responding the contact email is [email protected]:

Terrain Server Not Responding
The terrain server is not responding at the moment. You can still use all the features of Digital Earth Australia but there will be no terrain detail in 3D mode. We're sorry for the inconvenience. Please try again later and the terrain server should be responding as expected. If the issue persists, please contact us via email at [email protected].

Delta v2 - designs

Following agreement with DEA and UI developers, Delta v2 designs to be finalised by 20 March if possible.

DE Australia - Mar 21 release

Robbi (DEA team) is preparing the Coastal erosion story for launch by Minister between 12- 16 April (exact date TBC).

DEA team has prepared a change to the Coastlines data needed for the launch.
Here are the changes to config file: https://www.dropbox.com/s/lhlh0xtxkq73wxn/deacoastlines_v8_revised.json?dl=1

The share url also needs to be manually set up as: https://maps.dea.ga.gov.au/#share=s-DEACoastlines

Delta v2 - flat design drafts

Based on discussion with the DEA team, UX will draft flat designs to discuss with DEA, before completion of v2 design.
Meeting booked for 16 March to discuss the workflow approach.
Designs for v2 to be finalised after meeting, until 20 March 2020.

DE Australia - Scope / plan improvement of existing content

Scope/plan improvement of existing content. Final version is HERE

  • Terry the trainer - add more / other workflows
  • Help menu - add more/other videos for workflows: 1) NEII (satellite + water gauges); 2) Delta
  • Help menu - add new/updated welcome video on landing page
  • Catalogue - data group descriptions
  • Catalogue review: 1) organisation; 2) names; 3) metadata
  • About page - more information about DE Australia program; its objectives and scope

DE Australia - related maps button

Related maps button to show only the DE Africa and NationalMap.

Under The NationalMap:

  • The NationalMap is a website for map-based access to spatial data from Australian government agencies. It is an initiative of the Australian Government's Department of the Prime Minister and Cabinet and the software has been developed by Data61 working closely with the Department of the Prime Minister and Cabinet, Geoscience Australia and other government agencies.

  • use this photo:
    image

Under Digital Earth Africa

  • Digital Earth Africa is leveraging international Earth Observation (EO) data and science to produce new information and services that benefit African countries.

  • use this photo:
    image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.