ahotovec / redpy Goto Github PK

Repeating Earthquake Detector (Python)

License: GNU General Public License v3.0

Python 100.00%

python repeating-earthquakes volcanology seismology

redpy's Introduction

REDPy has moved!

After over a year of concentrated development work and many hurdles overcome, REDPy has moved to its new home as official U.S. Geological Survey software at https://code.usgs.gov/vsc/REDPy/.

I did a lot of work under the hood to make it more efficient, and I have many plans to add more features in the coming years.

All future development will be hosted in the new repository and this repository will no longer be updated.

New users should download the latest version, and read the updated Wiki for documentation.

A snapshot of the last commit in this repository with an intact README.md file is available here for posterity as "Version 0" of the code. I will no longer be supporting users of Version 0, but I am happy to help you migrate to the new one.

~ Alicia Hotovec-Ellis ([email protected])

redpy's People

Contributors

Stargazers

Watchers

redpy's Issues

error in createPlots

Hi Alicia,

I've created a list of triggered events and once catfill.py has processed them all, I'm running into this error. Do you have any thought of what might go wrong?

Thanks,
Eva

Traceback (most recent call last):
File "catfill.py", line 137, in
redpy.plotting.createPlots(rtable, ftable, ttable, ctable, otable, opt)
File "/data/REDPy/redpy/plotting.py", line 75, in createPlots
plotFamilies(rtable, ftable, ctable, opt) File "/data/REDPy/redpy/plotting.py", line 1094, in plotFamilies
fi, ids, id1, id2, ccc, 'png', 100, cnum, 0, 0, opt)
File "/data/REDPy/redpy/plotting.py", line 1269, in plotSingleFamily
ymin = 0.5*np.min(windowAmp[np.nonzero(windowAmp)]) File "<array_function internals>", line 6, in amin
File "/home/eibl/miniconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2880, in amin
keepdims=keepdims, initial=initial, where=where) File "/home/eibl/miniconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity
Closing remaining open files:redpytable.h5...done

can't open file 'initialize.py'

I installed RedPy via Anaconda (Python2.7) and all dependencies nevertheless, when I run
python initialize.py
I get the following error:

can't open file 'initialize.py': [Errno 2] No such file or directory

Additionally, when I do:
conda env create -f redpy27.yml
I obtain this error:

EnvironmentFileNotFound: '/Users/joaofontiela/redpy27.yml' file not found

My system is MacOS 10.14.6 and anaconda3

Any tips to fix these errors are welcome.

path to local files in subdir

When providing the searchdir path to the top directory of my files, it does not find the files (which are three levels of subfolders down) but just provides a list of folder names in 'flist' that cannot be read in. I had to manually copy and rearrange my files in order to have them just one level below the top directory.

The description just says: 'If using local files, define the path to the top directory where they exist, ending in / or \ as appropriate for your operating system. If there are files in any subdirectories within this directory, they will be found.'

error replotting family figures

when REDPy start creating plots I get get the following error:

...
Length of Orphan table: 212
Number of repeaters: 435
Number of clusters: 22
Creating plots...
Traceback (most recent call last):
  File "catfill.py", line 136, in <module>
    redpy.plotting.createPlots(rtable, ftable, ttable, ctable, otable, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/plotting.py", line 75, in createPlots
    plotFamilies(rtable, ftable, ctable, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/plotting.py", line 1094, in plotFamilies
    fi, ids, id1, id2, ccc, 'png', 100, cnum, 0, 0, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/plotting.py", line 1269, in plotSingleFamily
    ymin = 0.5*np.min(windowAmp[np.nonzero(windowAmp)])
  File "<__array_function__ internals>", line 6, in amin
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2880, in amin
    keepdims=keepdims, initial=initial, where=where)
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity
Closing remaining open files:SaoJorge_2023.h5...done

Then, I force re-plot family figures with python forcePlot.py -f -c settings_saojorge2023.cfgand got another error:

Traceback (most recent call last):
  File "forcePlot.py", line 91, in <module>
    redpy.plotting.plotFamilies(rtable, ftable, ctable, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/plotting.py", line 1094, in plotFamilies
    fi, ids, id1, id2, ccc, 'png', 100, cnum, 0, 0, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/plotting.py", line 1269, in plotSingleFamily
    ymin = 0.5*np.min(windowAmp[np.nonzero(windowAmp)])
  File "<__array_function__ internals>", line 6, in amin
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 2880, in amin
    keepdims=keepdims, initial=initial, where=where)
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation minimum which has no identity

Rendering html files with command python forcePlot.py -l -c settings_saojorge2023.cfg works fine.

It's the first time I'm facing this problem with REDPy. Last days, I had to clone last branch of REDPy due to several errors. Likely the error were caused by the cfg than the code.

Support for Python 3.x?

Hi Alicia,
It seems that the code is not supported in python 3.6. Are you planning to extend the support for Python 3.x?

Thanks!

Add event amplitude to one of the standard output files

In demo'ing some REDpy results to colleagues I mentioned that I thought the events in clusters did not reflect a scale bound process, such as you might see with some earthquakes of volcanic origin. I was asked if I could construct a frequency-magnitude curve to more clearly demonstrate that was a reasonable conclusion. I am wondering if event amplitudes could be added to an appropriate output file, which would make constructing a frequency-magnitude curve for any cluster a trivial matter. Amplitudes are stored and graphed, so I assume outputting wouldn't be too hard. If amplitude was added, would there be an argument for adding other features of cluster events, such as Frequency Index?
Many thanks

populateJunk throughs an error in the 6th time window

The data is read in and the scripts processes the first 5 hours of this day on 28 September without problems. However, the 6th hour is causing the following problem:

Time spent this iteration: 0.1481603701909383 minutes
2015-09-28T06:00:00.000000Z

Traceback (most recent call last):
File "backfill.py", line 123, in
redpy.table.populateJunk(jtable, junk[i], 2, opt) # Both types of junk
File "/data/data/code_python/REDPy/redpy/table.py", line 420, in populateJunk
trigger['waveform'] = trig.data
File "tables/tableextension.pyx", line 1599, in tables.tableextension.Row.setitem
ValueError: could not broadcast input array from shape (258062,) into shape (36872,)
Closing remaining open files:redpytable.h5...done

Does anyone know how what the problem is?

Duplicate server type checking when the type is file

In the following lines:

REDPy/redpy/trigger.py

Lines 43 to 48 in 098495a

    
           if opt.server == 'file': 
        
               # Generate list of files 
        
               if opt.server == 'file': 
        
                   flist = list(itertools.chain.from_iterable(glob.iglob(os.path.join( 
        
                       root,opt.filepattern)) for root, dirs, files in os.walk(opt.searchdir)))

In the line 43, it checks if opt.server equal to file. Then, in the line 46 it also checks the same thing.

Type hinting may also report flist in the line 53 is possibly unbound.

REDPy/redpy/trigger.py

Lines 50 to 55 in 098495a

    
           # Determine which subset of files to load based on start and end times and 
        
           # station name; we'll fully deal with stations below 
        
           flist_sub = [] 
        
           for f in flist: 
        
               # Load header only 
        
               stmp = obspy.read(f, headonly=True)

index out of bounds error with plotting.py

I am running REDpy on a single station using locally stored files. Running Ubuntu 18.04LTS.

[Settings]
title=Savo
filename=Savo.h5
groupName=Savo
groupDesc=Savo
nsta=1
station=SAVO
network=SW
channel=HHZ
location=10
server=file
searchdir=/home/sherburn/Commercial/Savo/SAVOsds/2021/SW/SAVO/HHZ.D/
winlen=512
cmin=0.80
minplot=20
dybin=0.125
nstaC=1
ncor=1
printsta=0

I have a total 10 months of data to process. Have run backfill in blocks of a month or two. Up to 4 months all was fine. Then with the 5th, the clustering was okay, but the plotting of clusters failed with an index out of bounds message. This example is from running forcePlot, but the error message is the same.

python forcePlot.py -c savo.cfg -a -r
Traceback (most recent call last):
  File "forcePlot.py", line 96, in <module>
    redpy.plotting.createPlots(rtable, ftable, ttable, ctable, otable, opt)
  File "/home/sherburn/git/REDPy/redpy/plotting.py", line 64, in createPlots
    plotFamilies(rtable, ftable, ctable, opt)
  File "/home/sherburn/git/REDPy/redpy/plotting.py", line 788, in plotFamilies
    r1 = [np.where(idf==xx)[0][0] for xx in id1[ix]]
  File "/home/sherburn/git/REDPy/redpy/plotting.py", line 788, in <listcomp>
    r1 = [np.where(idf==xx)[0][0] for xx in id1[ix]]
IndexError: index 0 is out of bounds for axis 0 with size 0
Closing remaining open files:Savo.h5...done

The relevant code segment is.

# Plot correlation timeline
            idf = ids[fam]
            ix = np.where(np.in1d(id2,idf))
            C = np.eye(len(idf))
            r1 = [np.where(idf==xx)[0][0] for xx in id1[ix]]
            r2 = [np.where(idf==xx)[0][0] for xx in id2[ix]]
            C[r1,r2] = ccc[ix]
            C[r2,r1] = ccc[ix]
            Cprint = C[np.argmax(np.sum(C,0)),:]

I can't see how the plotting is fine until some point and then fails, after which nothing I do seems to get the cluster families to plot again. Could this be something to do with the large number events detected and that clusters have many elements??

Many thanks.

windowAmp, FI and CCC in catalog.txt

Could you add columns: windowAmp, ccc and FI to the catalog.txt?

This would be very useful for our monitoring tasks in Costa Rica and Colombia.

I tried to do this myself by loading the h5 file, but had difficulties since I don't have a good knowledge on how the h5 tables are structured.

Thank you!

plotJunk.py error with Windows

I was getting an error when trying to use plotJunk.py because filenames can't include ':' on Windows. I was able to get it to run fine by just adding a call to format_fissures.

On lines 1094 and 1095 of plotting.py, I changed it from the original:

        plt.savefig('{0}/junk/{1}-{2}.png'.format(opt.groupName,
            UTCDateTime(r['startTime'])+opt.ptrig,r['isjunk']), dpi=100)

to:

        plt.savefig('{0}/junk/{1}-{2}.png'.format(opt.groupName,
            (UTCDateTime(r['startTime'])+opt.ptrig).format_fissures(),r['isjunk']), dpi=100)

Not sure if this is the "best" fix or not, but it ran without any errors.

P.S. Thanks for all your work on this great tool!

error on delayed STA/LTA

When I select ```delayedstalta´´´ (I also adjusted the remaining triggering settings), I get the following error:

Using config file: settings_saojorge2023.cfg
Opening hdf5 table: SaoJorge_2023.h5
2022-03-26T00:08:01.000000Z
Couldn't find SJ01.EHZ.3k.
Couldn't find SJ02B.EHZ.3k.
Couldn't find SJ08.EHZ.3k.
Couldn't find SJ08B.EHZ.3k.
Couldn't find SJ09.EHZ.3k.
Couldn't find SJ11.EHZ.3k.
Couldn't find SJ12.EHZ.3k.
Couldn't find SJ13.EHZ.3k.
Couldn't find SJ15.EHZ.3k.
Couldn't find SJ16.EHZ.3k.
Couldn't find SJ18.EHZ.3k.
Length of Orphan table: 0
2022-03-26T00:26:50.000000Z
Couldn't find SJ01.EHZ.3k.
Couldn't find SJ02B.EHZ.3k.
Couldn't find SJ08.EHZ.3k.
Couldn't find SJ08B.EHZ.3k.
Couldn't find SJ09.EHZ.3k.
Couldn't find SJ11.EHZ.3k.
Couldn't find SJ12.EHZ.3k.
Couldn't find SJ13.EHZ.3k.
Couldn't find SJ15.EHZ.3k.
Couldn't find SJ16.EHZ.3k.
Couldn't find SJ18.EHZ.3k.
Length of Orphan table: 1
2022-03-26T01:35:39.000000Z
Couldn't find SJ01.EHZ.3k.
Couldn't find SJ02B.EHZ.3k.
Couldn't find SJ08.EHZ.3k.
Couldn't find SJ08B.EHZ.3k.
Couldn't find SJ09.EHZ.3k.
Couldn't find SJ11.EHZ.3k.
Couldn't find SJ12.EHZ.3k.
Couldn't find SJ13.EHZ.3k.
Couldn't find SJ15.EHZ.3k.
Couldn't find SJ16.EHZ.3k.
Couldn't find SJ18.EHZ.3k.
Traceback (most recent call last):
  File "catfill.py", line 74, in <module>
    alltrigs = redpy.trigger.trigger(st, stC, rtable, opt)
  File "/Users/joaofontiela/geophysics/REDPy/redpy/trigger.py", line 207, in trigger
    sta=opt.swin, lta=opt.lwin, details=True)
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/obspy/signal/trigger.py", line 697, in coincidence_trigger
    tmp_triggers = trigger_onset(tr.data, thr_on, thr_off, **kwargs)
  File "/Users/joaofontiela/opt/anaconda3/envs/redpy/lib/python3.7/site-packages/obspy/signal/trigger.py", line 354, in trigger_onset
    ind2_[-1] = True
IndexError: index -1 is out of bounds for axis 0 with size 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "catfill.py", line 77, in <module>
    except (TypeError, obspy.fdsn.header.FDSNException, Exception):
AttributeError: module 'obspy' has no attribute 'fdsn'
Closing remaining open files:SaoJorge_2023.h5...done

This error happened before, in a much more advanced phase of the analysis.
Path to the files are correct on the settings file and I'm using 22 stations.

Version of bokeh raises conflict

When trying to install with the documentation :

conda install -c obspy obspy pytables pandas basemap bokeh=0.9.3 mock nose PIL

It raises a conflict error :

UnsatisfiableError: The following specifications were found to be in conflict:
  - basemap
  - bokeh 0.9.3*
  - navigator-updater
Use "conda info <package>" to see the dependencies for each package.

It works when installing the last version of bokeh (0.12.5) but then, there is an error when trying the Lapine exemple when the script tries to update the plots :

2015-10-28T23:00:00.000000Z
Length of Orphan table: 327
Number of repeaters: 182
Number of clusters: 11
Time spent this iteration: 0.0916738351186 minutes
Caught up to: 2015-10-28T23:58:58.560000Z
Updating plots...
Traceback (most recent call last):
  File "backfill.py", line 169, in <module>
    redpy.plotting.createPlots(rtable, ftable, ttable, ctable, opt)
  File "/home/mhoareau/Logiciels/REDPy/redpy/plotting.py", line 35, in createPlots
    plotTimelines(rtable, ftable, ttable, opt)
  File "/home/mhoareau/Logiciels/REDPy/redpy/plotting.py", line 91, in plotTimelines
    o0.title = 'Repeaters vs. Orphans by {:.1f} Hour Bin'.format(opt.dybin*24)
  File "/home/mhoareau/anaconda2/lib/python2.7/site-packages/bokeh/core/has_props.py", line 269, in __setattr__
    super(HasProps, self).__setattr__(name, value)
  File "/home/mhoareau/anaconda2/lib/python2.7/site-packages/bokeh/core/property/descriptors.py", line 495, in __set__
    self._internal_set(obj, value, setter)
  File "/home/mhoareau/anaconda2/lib/python2.7/site-packages/bokeh/core/property/descriptors.py", line 710, in _internal_set
    value = self.property.prepare_value(obj, self.name, value)
  File "/home/mhoareau/anaconda2/lib/python2.7/site-packages/bokeh/core/property/bases.py", line 265, in prepare_value
    raise e
ValueError: expected an instance of type Title, got Repeaters vs. Orphans by 3.0 Hour Bin of type str
Closing remaining open files:LPtest.h5...done

Any help would be very appreciated :)

can't read files

Hi @ahotovec , I'm trying to run REDPy with local files and output warns that:
2018-02-12T05:00:03.300000Z
No data found for PGRON.PM
No data found for PCALD.PM

Path in settings is pointing to the correct folder. Even when I drag and drop daily waveforms in REDpy root and leave searchdir empty, I get the same warning. Additionally, I'm using three stations but output list two stations. How do I know that the other station is being read?

Could it be related to mseed encoding? Any other cause? I have looked around REDpy to find a clue ... I didn't catch anything.

Any help is welcome.

	if opt.server == 'file':

	# Generate list of files
	if opt.server == 'file':
	flist = list(itertools.chain.from_iterable(glob.iglob(os.path.join(
	root,opt.filepattern)) for root, dirs, files in os.walk(opt.searchdir)))

	# Determine which subset of files to load based on start and end times and
	# station name; we'll fully deal with stations below
	flist_sub = []
	for f in flist:
	# Load header only
	stmp = obspy.read(f, headonly=True)

ahotovec / redpy Goto Github PK

redpy's Introduction

REDPy has moved!

redpy's People

Contributors

Stargazers

Watchers

Forkers

redpy's Issues

Recommend Projects

Recommend Topics

Recommend Org