gns-science / nshm-nz-opensha Goto Github PK

As SRM team

we want to evaluate impact of latest upstream changes using test automation

so that we know if our ruptures are affected

Done when:

• metrics pick up the git commit refs for the 4 repos
• test metrics are saved to csv (for gsheets import)
• measures the time-taken per rupture set
• rupture outputs are saved

Background: We kept scratch/kevin/etc for package names but that was a dumb idea - as it confuses this code with those in the opensha-* world.

So, let'sagree some new namespaces and reorg our work to make more sense.

Please Propose/agree new names here, then someone can pick it up.

I just merged in a massive OpenSHA refactor to remove dependence on the FaultSectionPrefData object. That has been replaced with a new FaultSection interface. This will allow you to create your own subduction representation, with whatever rupture surface you require (i.e., it no longer necessitates StirlingGriddedSurface). You can now also mix representations within a single FaultSystemRuptureSet. Once you pull the changes to the OpenSHA repositories, there will surely be some errors that need fixing in your code. Let me know if you have any questions. I can explain more when we talk next week, and will hopefully have some ideas on down-dip-subsection support.

In the XML input data (see example below)

• must the element id / sectionID agree?
• must elementID always start at 0
• must sectionID always start at 0

<?xml version="1.0" encoding="utf-8"?>
 <OpenSHA>
  <FaultModel>
   <i0 aseismicSlipFactor="0.0" aveDip="75.0" aveLongTermSlipRate="0.6" aveLowerDepth="20.0" 
     aveRake="330.0" aveUpperDepth="0.0" connector="false" couplingCoeff="1.0" dipDirection="300.8" parentSectionId="-1"
     sectionId="0" sectionName="Akatarawa" slipRateStDev="0.3">

As SRM Team,

we want to export the rupture subsection data as csv when building ruptures

to make rupture set analysis simple

We want to run Inversions from previously built ruptures sets, rather than building these each and every time an inversion is run.

NB this is probably already in UCERF3 code, so let's use that if we can for the inversion stage.

Done criteria

• decide whether to use Kevins scratch.UCERF3.inversion.CommandLineInversionRunner in opensha-ucerf3 or not
• provide a py4j gateway wrapper for python clients

As SRM team,

we want to apply an NZ model of background (off-fault) seismicity

so this information can be incorporated in rate models.

• where/how is this data applied in ucerf3 (we believe the term "gridded seismicity" is used)
• what data format is required (csep or similar?)

As SRM Team

We want to see the fault names and section ids for each rupture image

So that it's easy to identify these rupture examples

In the example below there'll be a list of 14 fault names + ids

As SRM team

We want an integration test suite to validate that ruptures are built as expected

so that we do not have to execute manual testing repeatedly and bugs are picked up (sooner)

NOTES

• this is a formative test suite so it is expected to be extended beyond this first iteration
• based on XML fixtures created by hand and/or by output from #52

Done When

• three example rupture fixtures are covered (@chrisbc to choose from #52) and for each rupture validate:
  • that the exact rupture is generated (happy case) using given input parameters
  • that the expected count of all ruptures generated is as expected

As SRM team

We want an example of opensha rupture set diagnostics report using NZ CFM data

Se we can understand its usefulness

As SRM team

we want to run many hazard calculations with various parameters

so that we can perform comparative analysis and sensitivity testing.

As SRM Team

we want a plausability filter to constrain outputs to those matching faults involved int the Kaikoura 2016 quake

so that we can see if that rupture is generated using the NSHM rupture builder/config

As the SRM team,

we want to relax (ignore) the upper aspect ratio limit on ruptures when:

• the rupture originates on the upper seismic boundary (row 0), and when
• it exceeds a given depth threshold (count of rows)

so that very large ruptures will be produced once the aspect constraint is relaxed.

Notes:

• we use row 0 to represent the upper fault boundary, although sometimes this is not the minimum absolute depth.
• we use 'count of rows/columns) as a proxy for depth

FUTURE we could use alternatively use the calculated seismic depth, since we have z-cordinates on each subsection tile.

compare :

• single inversion for each
• a combined inversion (but no jumps to/from hikurangi)

considering

• science implications
• computational implications

As SRM Team

We want an automated build pipeline such that commits to master (or otherwise agreed) branch will build the deployables, run the tests, and produce reports on code/test quality

So that we can measure and maintain quality on this codebase.

Assumptions

Done Criteria:

• confirm & agree appropriate build chain

This presents an interesting challenge due to the unique structure of the project

A SRM Team

We want to ensure we can reproduce rupture sets for UCERF3 pre and post recent changes in upstream.

using https://github.com/GNS-Science/opensha-ucerf3/blob/master/src/org/opensha/sha/earthquake/faultSysSolution/ruptures/ClusterRuptureBuilder.java main() method

so that we have a solid baseline for future NZ variations

Notes:

• save a copy of the output rupset zip file for future reference.
• capture basic stats eg number of subsections and number of rupture set generated for future reference
• pls capture git refs for the code used for this testing.

As SRM Team

we want to export the geometries and fields of subsections created in opensha for crustal faults

so that we can visualise these alongside the input geometries and better understand how opensha is treating these.

For example the Wellington, Hutt Valley fault has 5 sections in CFM and these are provided in opensha-xml file. We want to examine the subsections opensha is creating as used in our crustal demo(s).

Done when

• provide simple CLI where....
• user must specify the input file name
• user can optionally specify a list of section IDs to filter by (otherwise, dump everything)
• user can specify other parameters that control the subSection builder (maxSubSectionLength, minSections)
• output data as csv to stdio
• index on parent section ID, subsection ID
• output all the other fields on the subsection objects

so there should be no java source with OS-dependant line-endings in any of these:

• nshm-nz-opensha
• opensha-core
• opensha-commons
• opensha-ucerf3

These are the projects our NSHM builds from.

Bonus points for:

• opensha-dev

For the three opensha upstream repos, please fork into this org (GNS-Science) if they're not forked already.

With scriptCrustalInversionRunner example running NZ CFM crustal data we see rupture permutations grow exponentially as the maxDistance parameter increases above 0.25 (km).

Further investigation is needed to understand what's behind this, and to compare this with the StandaloneRuptureGen approach used before the Ucerf3 refactoring in July. In this case, maxDistance was set to 0.5km and much smaller rupture sets were produced. FYI @mcg-gns

Todo

• refactor (add filter) to focus on 'akatarawa' faults
• scale maxDistance between 0.2 and 0.5 capturing outputs

Originally posted by @chrisbc in #16 (comment)

AS SRM Team

We want to run inversions using deformation (slip rate) from outside the original XML fault sources

So that different slip rate model can be assessed.

• identify file format (and get samples) from SRM team

see https://nshmrevisionproject.slack.com/files/U0135S0NCDD/F018S5P4S1G/geologic_slip_rake_fm_3_2_2013_04_09.csv

For

• https://github.com/GNS-Science/opensha-commons
• https://github.com/GNS-Science/opensha-ucerf3

NB we expect that some slow tests may stop this working, and that we'll have to refactor the test configs somewhat.
Let's keep @kevinmilner posted on this one.

Answer the questions:

• does ./gradlew test run?
• does HikurangiCrustalDemoTwo.java run?

Thoughts: we attempt this non-MPI, with just a single worker node. That at least proves if the fundamental dependencies are met in the environment. I can raise another task for more advanced setups (e.g. for inversions) if this looks promising

we want to modify kevins' example and apply this to NZ data

so that we have run an inversion on Hikurangi

As SRM team

we want to test the alternate ConnectionPointsPermutationStrategy.java rupture builder strategy,

so we can :

• compare the approach with the current UCERF3 approach
• gain experience with this technique
• build understanding of how this might apply to our objective of reducing rupset sizes

Kevin has confirmed that he tests ruptures sets against UCERF3 outputs, and that these should match.

We know we've got issues where old ruptures sets are different than new ones, with approximately equivalent configurations.

• produce a test plan
• execute using smallest possible test data sets
• pin down root causes and fix or agree resolution w team.

Some changes may be required based on Kevins' latest and our updates see GNS-Science/opensha-ucerf3#4

As SRM Team

We want to run an inversion pipeline using just the NS crustal faults , derived from the CFM -> XML pipeline in eq-fault-geom

So that we satisfy the defined TAG milestone

Done When

• produce input XML from CFM using eq-fault-geom tools
• create a job script based on Demo03 code used in previous inversions
• rupture set run and metrics captured
• inversion run and metrics captured

The team want to run the inversion on the beaven cluster,

So that we can test scaling and get some preliminary results on a low-cost MPI cluster

but we're going to need newer software installed there. We need

• java 8 (1.8) JRE on all worker nodes
• latest python3 on all nodes SKIPPED since 3.6 is available already.
• java 8 (1.8) JDK (on master) so we can compile in situ~

NB if anyone else is ever confused by this, java 8 == java 1.8 see https://stackoverflow.com/questions/53642402/why-do-i-see-open-jdk-1-8-instead-of-java-8

NB on beavan the java JDKs have been installed to /opt/sw/java both 8 and 11 are now available.

These figures show the Hikurangi subduction-zone geometry, based on our recent 10*10km tiled dataset.

FIG 1 https://github.com/GNS-Science/nshm-nz-opensha/blob/master/doc/img/Figure_1.png

FIG 2 https://github.com/GNS-Science/nshm-nz-opensha/blob/master/doc/img/Figure_2.png

notable features:

• tiles are arranged in columns and rows of 10km*10km tiles arranged to follow the surface contour
• the surface looks like a sheet
• the first ('top') row 0 is a wavy line.
• the 'down-dip' columns have different row counts - so the bottom edge is a ragged line
• the dip direction near the first row is inverted at one end (interesting but less relevant?)

Story-text

As SRM Team,

we want opensha's rupture sets to support sub-sections surfaces where the row-count can vary,

so that we can support the dataset depicted above

As SRM Team

we want to implement plausability filters as per the UCERF3 model. As of today we have no azimuth filters in our demo code.

So that :

• we can increase the max jump distance to a more reasonable value than 0.25 without "blowing up"
• our baseline is more closely aligned with UCERF3

Done when:

• unit test(s) show that jumps are allowed according to the UCERF3 plausability specs for allowed azimuth change at fault-section jumps TODO: insert link
• we can demonstrate the impact of this filter on our CFM crustal data set

ref: #31 (comment)

We want to make sure that the fault data meets the requirements of the software so that we can avoid runtime errors or silent calculation errors.

• parent fault IDs may not be re-used
• subsection IDs may not be re-used (and they may not use parent IDs)
• dip must be greater than 0 and not greater than 90

As SRM team

We want to extract the fault geometries in XML form for each ruptoon

so that it's simple to create test fixtures for rupture integration tests based on real examples

This exception raised in inversion phase while running new scriptCrustalInversionRunner

The same script has previously run OK with smaller list of fault subsections. It's based on recent Demo03 but using only NZ crustal faults produced with latest NZCFM export.

@kevinmilner I'm cc'ing you in case you know something about this type of error.

Meanwhile I'm trying to get a narrow this down by reducing number of fault sections (down fom 675)

Exception in thread "main" java.lang.IllegalStateException: energy from equality constraints is NaN!
        at com.google.common.base.Preconditions.checkState(Preconditions.java:507)
        at scratch.UCERF3.simulatedAnnealing.SerialSimulatedAnnealing.calculateEnergy(SerialSimulatedAnnealing.java:304)
        at scratch.UCERF3.simulatedAnnealing.SerialSimulatedAnnealing.setup(SerialSimulatedAnnealing.java:147)
        at scratch.UCERF3.simulatedAnnealing.SerialSimulatedAnnealing.<init>(SerialSimulatedAnnealing.java:117)
        at scratch.UCERF3.simulatedAnnealing.ThreadedSimulatedAnnealing.<init>(ThreadedSimulatedAnnealing.java:112)
        at nz.cri.gns.NSHM.opensha.scripts.scriptCrustalInversionRunner.main(scriptCrustalInversionRunner.java:262)
crustal_inversion.pbs.e34867 
PS C:\Users\chrisch\beavan>

Script configuration

#PBS -l nodes=1:ppn=32
#PBS -l walltime=04:30:00

export PATH=/opt/sw/java/java-se-8u41-ri/bin:$PATH
export JAVA_CLASSPATH=~/NSHM/opensha/nshm-nz-opensha/build/libs/nshm-nz-opensha-all.jar
export CLASSNAME=nz.cri.gns.NSHM.opensha.scripts.scriptCrustalInversionRunner
export OPTS="-fsdFile ./data/FaultModels/NZ_CFM_V0.3_crustal_opensha.xml -o ./tmp -maxFaultSections 1000 -maxLength 0.5 -maxDistance 0.1"
export OPTS=${OPTS}" -inversionMins 120 -syncInterval 30 -runInversion true"

cd ~/NSHM/opensha/nshm-nz-opensha
java -Xms4G -Xmx100G -classpath ${JAVA_CLASSPATH} ${CLASSNAME} ${OPTS} > ./tmp/crustal.log

more info

• Rupture outputs and stdout + stderr
  https://drive.google.com/drive/folders/16UKa1E6ewiRpEnwe8zidL55yfy5OP0wn?usp=sharing

• job history
  https://docs.google.com/spreadsheets/d/1pkL_5egm3wZ-7AE12qpb0N6gQKKnvS__5XGxz6fCt0I/edit?usp=sharing

As SRM Team,

we want to restrict our number of single spine crustal ruptures

so that we won't end up with too many total ruptures

Done when

• a pseudo-code version for this strategy is agreed with our SRM science team
• the strategy can still produce the same rupture that the UCERF3 strategy does (i.e. with nil coarsening)
• the rate of coarsening is configurable (the 'epsilon' value)
• we show that the new strategy will produces fewer ruptures than current UCERF3 strategy

See excerpt from this slack thread with Bruce Shaw

The single spine rupture set is going to be similar to the UCERF3 approach, except we are going to be coarser as the events get longer, as we want to restrict our number of single spine ruptures N_1 to be not so large so that when we combine them with compatible second spines into composite N_2 type ruptures we won't end up with too many total ruptures N_1+N_2=N which isn't too big (i.e. we can do the inversion on it). We will want to do some capability studies to see how big N can be (ucerf3 had 250,000, but that was some years ago; for the sake of discussion I'll imagine N=1,000,000 is a doable size, but if we can handle bigger we may want to go bigger). The single spine set is built out similarly to how Kevin built out ucerf3, except as L gets bigger rather than adding 1 more subsection we will add dL=epsilon*L. where epsilon is small. One does this in both directions from a seed subsection (left down the fault for one set, right down the fault for the other. finish sets. move down the fault to the next subsection and repeat). (This describes the upper plate fault process; in NZ the rupture set for the plate interface is currently differently constructed; this is fine, however the set is constructed does not impact the double spine set discussion below).

Workshop, whiteboard, prioritise, storyise

Running StandaloneSubSectRupGenMG raises an exception

using latest CFM faults dataset /data/FaultModels/DEMO2_DIPFIX_crustal_opensha.xml

Exception in thread "main" java.lang.RuntimeException: Problem Mmax: -0.15000000000000002	Ariel East, Subsection 0	BRANCH: LogicTreeBranch[FM3_1, GEOL, Shaw09Mod, DsrTap, CharConst, M5Rate7.9, MMaxOff7.6, NoFix, SpatSeisU3]
	at scratch.UCERF3.analysis.FaultSystemRupSetCalc.getCharSubSeismoOnFaultMFD_forEachSection(FaultSystemRupSetCalc.java:1295)
	at scratch.UCERF3.inversion.InversionTargetMFDs.<init>(InversionTargetMFDs.java:245)
	at scratch.UCERF3.inversion.InversionFaultSystemRupSet.calcRuptureAttributes(InversionFaultSystemRupSet.java:354)
	at scratch.UCERF3.inversion.InversionFaultSystemRupSet.<init>(InversionFaultSystemRupSet.java:158)
	at scratch.kevin.ucerf3.StandaloneSubSectRupGenMG.main(StandaloneSubSectRupGenMG.java:207)

Currently we have examples of ruptures that start out north->south then switch direction to south->north.

TODO attach example.

Done when we can answer these questions:

• are there any constraints on ruptures relating to azimuth in play now?
  Please add links to any relevant code in this issue
• if there are, are they working correctly?

NB After these code questions are answered we can ask SRM team if this is enough, or whether more/better constraints are needed (new feature).

given ruptset and precalcuted cache.

can we package opensha code and run it effectivley on AWS lambda?

As SRM team

we want to automate repetitive testing processes

so that they require less manual work to complete

Done when:

• wrap existing java rupture generator class to support setting, retrieving interesting attributes from test runs (e.g. number of subsections)
• output data is written to unique, temporary location
• provide a py4j so that we can do automation in a python driver script
• provide a simple python example to demonstrate use in test automation (separate project)

• e.g. distinct b values on TVZ vs main NZ

clarify these terms for inversion

• Spatial PDF
• M>5

AS SRM Team

We want the strategy to consider the length of the whole rupture.

So that it produces more effective coarsening effects. NB Highlighted by discussion with Kevin.

Done when:

• reproduce the current incorrect behaviour as a failing test.
• make code modification to pass the test

As SRM team

we want to run many inversions with various parameters

so that we can perform comparative analysis and sensitivity testing.

As SRM Team,

we want to restrict our number of subduction ruptures and constrain the shapes to near-rectangles

so that we won't end up with too many total ruptures

Done when

• the rate of coarsening of growth is configurable
• the rate of coarsening of position is configurable
• only generates ruptures within range of aspect ratio (e.g 2.5 - 3.5)
• allow control of 'incomplete rectangles' caused by ragged edges (e.g. - min-fill-factor)

See also the slack thread with Bruce Shaw

Ref T1 and T2 here https://pubs.usgs.gov/of/2013/1165/pdf/ofr2013-1165_appendixT.pdf

As SRM Team

we want to implement minSectionsInRupture filtering in our demo code.

so that the baseline is more closely aligned with UCERF3, and ruptures sets are smaller

Done when:

• we can demonstrate the impact of this filter on our CFM crustal data set

As NZ Opensha users

We want to be sure we understand how these objects are related

• FaultModel components:
  • FaultSection - the <i{N} XML elements
  • FaultTrace
  • FaultTrace Point (lat lon, in opensha convention)
  • SubSection - from the getSubSectionsList() method
  • StirlingGriddedSurface
• DeformationModel components (ie the container for deformation attributes like avg slip-rate, avg rake etc

So that we can avoid confusion in our opensha travels

Noting that the FaultModel above is the surface-fault model derived from the XML, NOT the pre-subsectioned model derived from our Hikurangi data. @thingbaijam - will we have a deformation model for this too? - We would be having a one (or more?) - it'd be interesting to workout the data format.

Done criteria

• create test data and tests to examine the elements in question
• create some simple visualisations to support the test findings.
• write up a summary of these findings (pref in markdown) for the /doc folder in this repo (based on @thingbaijam doc)

As SRM

we want to calculate hazard

so that we have all 3 high-level features for SRM sensitivity tests

Need for a single site:

• given default GMPE,
• an NSHM Inversion output (either hikurangi or crustal)
• a site (with optional site parameters)

and produce hazard output:

• Spectral Analysis
• PGA curve

Done when

• new package 'hazard'
• add to main script/gateway interface

so that we can compare runtime w tenjin at same settings (ref mod4 config)

• check in the settings for this particular test
• record runtime and/or outputs