stanford-futuredata / fast Goto Github PK

First off, thank you for making this code public! I have a question about the goal of this code though.

Is there a reason that this code has been written in such a way that modularizing and using functions as a library was not done? For example, in run_fp.py there is the line

	# Fingerprinting
	chdir('fingerprint')
	for param in fp_params:
		print("Fingerprinting %s" % param)
		process = subprocess.Popen((fpCommand % param),
			stdout=subprocess.PIPE, shell=True)
		output, error = process.communicate()
		print(output.decode('UTF-8'))

there are numerous calls using subprocess throughout this code. If things were written modularly the above code could be replaced by

from fingerprint import gen_fp
...
	# Fingerprinting
	for param in fp_params:
		print("Fingerprinting %s" % param)
		gen_fp.call_mad(param)
		gen_fp.run_fingerprint(param)

where these are two functions within gen_fp.py that can do what the command line call to gen_fp.py does without using the subprocess library to run things from the command line.

Using things like chdir() make this code practically impossible to use outside of the very prescribe directory structure within the FAST directory. Is there any discussion of making things more modular? I can help with this, but I might have some questions as I go and it will be over the winter break in a couple of weeks. I have a student trying to use this FAST code right now, and it is not easy to get our data going.

I install and implement the FAST with python2.7 on Ubuntu 18.04.

Trying to run the following command:
$ python run_simsearch.py -c config.json

Encountering the following error:
Traceback (most recent call last):
File "run_simsearch.py", line 5, in
from fingerprint.util import *
ImportError: No module named util

Any ideas how i can solve this problem?
Thank you.

I am attempting to install FAST on a mac. Using anaconda with Python 3.7, I cannot find sklearn. Instead I find scikit-learn. Does pip recognize sklearn? Should this requirement be updated to scikit-learn?

FYI, the call to imresize() in feature_extractor.py is not working when I am running this on a mac with Python 3.

This function is deprecated, see this page. I found a replacement in skimage.transform.resize.

I am testing the change now. I am running the Hector Mine experiment with the Python 2.8 version and if my fix works I will submit a pull request.

Hi Dev team,

Is this project going to be updated to Python 3 at any point?

Dear FAST developpers:

I am setting up fingerprinting parameters for my channels: fp_lag and spec_lag.

I also know that dt_fp is later required for network detection, and that dt_fp = fp_lag * spec_lag.

But what happens if I set different fp_lag and spec_lag for each station? Would this be a problem for network detection?

I realize that every example provided has the same parameteres in all json files. So I guess this are not station-channel dependent parameters? Am I right?

My general question would be, do fingerprint parameters, like fp_length, need to be equal at all stations for network detection?

Thank you very much

Leo

$../parameters/postprocess/combine_HectorMine_pairs.sh
Sorting ../data/inputs_network/candidate_pairs_HEC_BHN_merged.txt
Sorting ../data/inputs_network/candidate_pairs_HEC_BHZ_merged.txt
('Done sorting ../data/inputs_network/candidate_pairs_HEC_BHN_merged.txt, time taken:', 0.6283180713653564)
('Done sorting ../data/inputs_network/candidate_pairs_HEC_BHZ_merged.txt, time taken:', 0.72080397605896)
Sorting ../data/inputs_network/candidate_pairs_HEC_BHE_merged.txt
('Done sorting ../data/inputs_network/candidate_pairs_HEC_BHE_merged.txt, time taken:', 0.48364806175231934)
Merging files
Merging chunk
Merging chunk
Merging chunk
Traceback (most recent call last):
File "parse_results.py", line 346, in
merge(out_fnames)
File "parse_results.py", line 190, in merge
os.remove(f)
OSError: [Errno 2] No such file or directory: '../data/inputs_network/candidate_pairs_HEC_BHN_merged.txt_sorted_tmp'
Network detection inputs ready at ../data/inputs_network/
I am getting above error please help to solve this ..

dear@kexinrong , when i try to install FAST in python3.10 as the instructions after cloning using "pip install -r requirements.txt" an error is rised, as python3 is not supported for scipy==1.1.0

so,i have create an venv with python2.7, while running "python run_fp.py -c config.json"
i got the error

"Writing global index to data/global_indices/
Traceback (most recent call last):
File "global_index.py", line 36, in
tmp = datetime.datetime.strptime(linecache.getline(ts_fname, 1).strip(), "%y-%m-%dT%H:%M:%S.%f")
File "/usr/lib/python2.7/_strptime.py", line 332, in _strptime
(data_string, format))
ValueError: time data ' ' does not match format '%y-%m-%dT%H:%M:%S.%f' "

please help to rectify it

Dear FAST developers

I already have some earthquake detections using FAST but I want to visualize how their respective fingerprints looks. Is the code (or example) to generate the fingerprints of two similar earthquakes and their respective Jaccard similarity (like the Figure S9 (B) in Yoon et al., 2015) available somewhere ?

Thanks in advance,
Andres

Hi！
I tried to perform a seismic search for a single station, and the file parameters obtained after completing the network were (event_start, event_dt, event_stats[0], event_stats[1], event_stats[2])，I don't understand the meaning of each parameter, and is the search for a single station complete here? Is there a similar function for plotting and final output for multiple station search, I didn't find any guidance for this part.

Thank you for your help
Onebota

It's possible it's just an issue with my machine, but I wasn't able to get the similarity search to compile without some small modifications. I was getting multiple " not specified in enclosing 'parallel'" errors. I fixed this by declaring the variables as shared, although I believe removing the "default(none)" would have accomplished the same thing. It's been a while since I've used OpenMP though so I'm not 100% sure.

Hi,
I'm trying to replicate what you all have done to generate fingerprints but I'm running into an issue. I have generated a .json file for several .sac files containing the data to be used for generating the fingerprints. I edited a global_indices file to contain these fp_input_.json files as well a the config.json file. When running the code the same way you all do in the tutorial, "python run_fp.py -c config.json" I'm given an error saying there is no such directory. The specific error is as follows, "IOError: [Errno 2] No such file or directory: u'../data/waveforms02/Deci5.Pick.20200413000000.01.02.DPZ.sac'"
Do we need to direct it to the waveform file and not the .sac file?

Thanks for the help,

Spencer

Everything run fine until I got to the command "make" after cmake worked well. The error pops out with an error in "start_indice"
It is a problem with the program or my desktop?

Hello, I want to plot the images of the steps of generating fingerprint. like spectogram, spectral image, wavelet transform, top coefficient and the fingerprint itself. Is there any way to do that?

Just set up a conda environment for FAST. Trying to run the following command:

$ python run_fp.py -c config.json

Encountering the following error:

Fingerprinting ../parameters/fingerprint/fp_input_CI_TPC_EHZ.json
Traceback (most recent call last):
File "gen_fp.py", line 5, in
from config import *
ModuleNotFoundError: No module named 'config'

Any ideas what is causing this problem? Probably a very simple solution. Thanks.

i have just pressed enter key. please tell what should be the input file

Why do I get the following error when I run the similarity search for just station CDY? I would be glad if you help

Hi FAST developers,

@sf-wilbur and I have question about the fingerprint json files (e.g. fp_input_CI_CDY_EHZ.json). Is there an automatic routine to set these files up? We could not find one, and it seems quite tedious to set this up for all stations in our network. Can you please give us some guidance on this? Thanks!

Dear all I am getting the following error when I execute the python run_simsearch.py -c config.json command please tell me the solution
Thanking you

$python run_simsearch.py -c config.
Traceback (most recent call last):
File "run_simsearch.py", line 5, in
from fingerprint.util import *
ModuleNotFoundError: No module named 'fingerprint.util'