Comments (3)
smnorris
commented on May 29, 2024
A more general tool for updating model output based on additions/removals from any of the barrier tables would be valuable too
from bcfishpass.
smnorris
commented on May 29, 2024
crossing_update.sh isn't reliable, it can bail on the stream segmentation. Either remove stream segmentation (re-run the model if this is required), or look into why it is bailing - it should be ok to re-run because there are checks to make sure that redundant breaks are not created.
+ python bcfishpass.py segment-streams bcfishpass.streams bcfishpass.crossings
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/Users/snorris/Projects/repo/bcfishpass/02_model/bcfishpass.py", line 20, in execute_parallel
cur.execute(sql, (wsg,))
psycopg2.errors.NotNullViolation: null value in column "segmented_stream_id" violates not-null constraint
DETAIL: Failing row contains (null, 706840586, null, null, null, null, null, null, null, 1032.1613724488104, 12.117006549682767, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 01020000E0BD0B00000400000052021BC939583B41BAAB5080DC4221413D0CE2..., null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null).
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "bcfishpass.py", line 209, in <module>
cli()
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "bcfishpass.py", line 180, in segment_streams
pool.map(func, groups)
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/local/Cellar/[email protected]/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
psycopg2.errors.NotNullViolation: null value in column "segmented_stream_id" violates not-null constraint
DETAIL: Failing row contains (null, 706840586, null, null, null, null, null, null, null, 1032.1613724488104, 12.117006549682767, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 01020000E0BD0B00000400000052021BC939583B41BAAB5080DC4221413D0CE2..., null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null).
from bcfishpass.
smnorris
commented on May 29, 2024
fixed by using make rather than a shell script. The Makefile is more complex than I'd like but it does detect changes and allows for processing logical groupings of the job rather than reprocessing the entire job.
from bcfishpass.
Related Issues (20)
- check WCRP all species column values HOT 1
- add WCRP tracking tables
- split access model script into two parts
- workflow timeout HOT 2
- All tribs to horsefly upstream of falls at 67,523 are incorrectly processed HOT 4
- do not build individual natural barrier tables
- DRA download error
- consider adding `model_run_date_time` of changes to `bcfishpass.parameters_habitat_thresholds` to `bcfishpass.parameters_habitat_thresholds_log` HOT 3
- add column comments to `bcfishpass.crossings_vw`
- add FTEN road map_label or road_section_id to crossings table/views
- additional WCRP watersheds
- Use observed spawning (not just modelled) when modelling rearing HOT 2
- DRA load failing HOT 3
- model processing workflow should depend on weekly refresh workflow completing successfully
- view refresh failure
- unconnected streams in streams_salmon_vw streams_st_vw HOT 2
- DRA download schedule
- cross db summary comparisons
- model arctic greyling
- include crossings in non-wcrp watersheds in crossings_wcrp_vw? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bcfishpass.