ai4eps / quakeflow Goto Github PK

Hi everyone,

I have deploy QuakeFlow on my computer with minikube.
So far so good.

However, when I try to connect to QuakeFlow streamlit dashboard,
I get the following error on browser:

$ minikube service streamlit-ui
|-----------|--------------|-------------|---------------------------|
| NAMESPACE |     NAME     | TARGET PORT |            URL            |
|-----------|--------------|-------------|---------------------------|
| default   | streamlit-ui |        8501 | http://192.168.49.2:32348 |
|-----------|--------------|-------------|---------------------------|
🎉  Opening service default/streamlit-ui in default browser...

# in browser
FileNotFoundError: [Errno 2] No such file or directory: 'config_hawaii.pkl'
Traceback:
File "/opt/conda/envs/cs329s/lib/python3.7/site-packages/streamlit/script_runner.py", line 350, in _run_script
    exec(code, module.__dict__)
File "/app/ui_streamlit_iris.py", line 66, in <module>
    with open(CONFIG_PKL, "rb") as fp:

I guess I need config_hawaii.pkl and stations_hawaii.csv.
So are there any suggestions to get these two files (uploaded by maintainer or self-generation)?
Thanks for your help.

I am trying to deploy QuakeFlow with k8s on AWS EKS clusters. Following instructions on page, the second step Prebuilt Kafka, when I run the command

kubectl run --quiet=true -it --rm quakeflow-kafka-client --restart='Never' --image docker.io/bitnami/kafka:2.7.0-debian-10-r68 --restart=Never --command -- bash -c "kafka-topics.sh --create --topic phasenet_picks --bootstrap-server quakeflow-kafka.default.svc.cluster.local:9092 && kafka-topics.sh --create --topic gmma_events --bootstrap-server quakeflow-kafka.default.svc.cluster.local:9092 && kafka-topics.sh --create --topic waveform_raw --bootstrap-server quakeflow-kafka.default.svc.cluster.local:9092"

I always come to the same error below:

WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
[2022-12-06 23:11:44,122] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (quakeflow-kafka.default.svc.cluster.local/10.100.43.152:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2022-12-06 23:11:45,236] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (quakeflow-kafka.default.svc.cluster.local/10.100.43.152:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2022-12-06 23:11:46,356] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (quakeflow-kafka.default.svc.cluster.local/10.100.43.152:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2022-12-06 23:11:47,572] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (quakeflow-kafka.default.svc.cluster.local/10.100.43.152:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2022-12-06 23:11:49,076] WARN [AdminClient clientId=adminclient-1] Connection to node -1 (quakeflow-kafka.default.svc.cluster.local/10.100.43.152:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
.......
[2022-12-06 23:12:43,158] ERROR org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1670368363156, tries=1, nextAllowedTryMs=1670368363257) timed out at 1670368363157 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment. Call: createTopics
(kafka.admin.TopicCommand$)
pod default/quakeflow-kafka-client terminated (Error)

I noticed that we don't have the file kafka-topics.sh. I don't know what to do with this bug. Everything in the instructions runs well except kafka.

And if I further run

kubectl get pods

I see

NAME READY STATUS RESTARTS AGE
quakeflow-kafka-0 0/1 Pending 0 25m
quakeflow-kafka-zookeeper-0 0/1 Pending 0 25m

K8s version:

Client Version: v1.25.3
Kustomize Version: v4.5.7
Server Version: v1.24.7-eks-fb459a0

Hello,

I have been able to get the Demo data and other examples to run just fine but when I look at this particular earthquake I get the following error when running the Gamma section of the notebook. Any idea what my issue is? Thanks for your time.

region: Turkey_99
center: [32, 38.5]
xlim_degree: [24.75, 39.25]
ylim_degree: [35.0, 42.0]
min_longitude: 24.75
max_longitude: 39.25
min_latitude: 35.0
max_latitude: 42.0
degree2km: 111.19492664455873
starttime: 1999-11-12T14:00:00.000
endtime: 1999-11-13T00:00:00.000
networks: ['']
channels: HH,BH*,EH*,HN*
client: IRIS
phasenet: {}
gamma: {}
hypodd: {'MAXEVENT': 10000.0}
use_dbscan: True
use_amplitude: True
method: BGMM
oversample_factor: 4
dims: ['x(km)', 'y(km)', 'z(km)']
vel: {'p': 6.0, 's': 3.468208092485549}
x(km): [-806.16321817 806.16321817]
y(km): [-389.18224326 389.18224326]
z(km): (0, 60)
bfgs_bounds: ((-807.16321817305084, 807.16321817305084), (-390.18224325595554, 390.18224325595554), (0, 61), (None, None))
dbscan_eps: 10
dbscan_min_samples: 3
min_picks_per_eq: 9
min_p_picks_per_eq: 0
min_s_picks_per_eq: 0
max_sigma11: 2.0
max_sigma22: 2.0
max_sigma12: 1.0
Skip 1232 picks: 100%|██████████| 1351/1351 [00:00<00:00, 5262.53it/s]
Traceback (most recent call last):

File ~/qf1022/notebooks/Tur99_Oct.py:1118 in
catalog = gamma(

File ~/qf1022/notebooks/Tur99_Oct.py:1017 in gamma
catalogs[["longitude", "latitude"]] = catalogs.apply(

File ~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py:3643 in setitem
self._setitem_array(key, value)

File ~/anaconda3/lib/python3.8/site-packages/pandas/core/frame.py:3685 in _setitem_array
check_key_length(self.columns, key, value)

File ~/anaconda3/lib/python3.8/site-packages/pandas/core/indexers/utils.py:428 in check_key_length
raise ValueError("Columns must be same length as key")

ValueError: Columns must be same length as key

Region settings:
if region_name == "Turkey_99":
center = (32, 38.5)
horizontal_degree = 14.5
vertical_degree = 7
starttime = obspy.UTCDateTime("1999-11-12T14")
endtime = obspy.UTCDateTime("1999-11-13T00")
client = "IRIS"
network_list = [""]
channel_list = "HH,BH*,EH*,HN*"