amrex-codes / exaepi Goto Github PK
View Code? Open in Web Editor NEWAn agent-based epidemiological simulation code using AMReX
Home Page: https://exaepi.readthedocs.io/en/latest/
License: Other
An agent-based epidemiological simulation code using AMReX
Home Page: https://exaepi.readthedocs.io/en/latest/
License: Other
We should have a an exaepi_used_inputs
like warpx_used_inputs
that stores the exact parameters used for each run so we'll have it later.
@debog I added OpenMP parallelization to the MFIter loops in AgentContainer.cpp
- could you try to look at the OpenMP scaling again when you get a chance?
We should start to run nightly tests on biollante and garuda for a few inputs files.
HIP tests are currently failing on development branch.
We need to output data on the number of people obeying quarantine so it can be used to calibrate SQIR models. We also should have a few policies that can be toggled between in the inputs.
Currently, the max_grid_size
parameter is exposed and needs to be set to a reasonable value based on the number of cells and the number of MPI ranks. This should be computed automatically, and max_grid_size
should only be used by advanced users.
When the same executable is run multiple times, the answers will be different to a few parts in a thousand - we think the answers should be exactly the same.
I am trying etc/plot.py to extract ExaEpi results. I have update AMReX, ExaEpi, and installed yt from source. But I still get the error:
(pyamrex) [hvandam@lpo-170127 plt00000]$ python ../../../etc/plot.py .
yt : [INFO ] 2023-12-15 13:37:25,009 Parameters: current_time = 0.0
yt : [INFO ] 2023-12-15 13:37:25,009 Parameters: domain_dimensions = [150 149 1]
yt : [INFO ] 2023-12-15 13:37:25,009 Parameters: domain_left_edge = [0. 0. 0.]
yt : [INFO ] 2023-12-15 13:37:25,009 Parameters: domain_right_edge = [1. 1. 1.]
Traceback (most recent call last):
File "/home/hvandam/Documents/AMReX/ExaEpi/examples/test-pbar+1/plt00000/../../../etc/plot.py", line 10, in <module>
ad = ds.all_data()
^^^^^^^^^^^^^
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/data_objects/static_output.py", line 1194, in all_data
self.index
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/data_objects/static_output.py", line 606, in index
self._instantiated_index = self._index_class(
^^^^^^^^^^^^^^^^^^
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/frontends/boxlib/data_structures.py", line 926, in __init__
super().__init__(ds, dataset_type)
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/frontends/boxlib/data_structures.py", line 345, in __init__
GridIndex.__init__(self, ds, dataset_type)
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/geometry/geometry_handler.py", line 42, in __init__
self._setup_data_io()
File "/home/hvandam/pyamrex/lib64/python3.11/site-packages/yt/frontends/boxlib/data_structures.py", line 568, in _setup_data_io
self.io = io_registry[self.dataset_type](self.dataset)
~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
KeyError: 'boxlib_native'
(pyamrex) [hvandam@lpo-170127 plt00000]$
As far as I can see boxlib
is much more common that boxlib_native
but I don't know enough about the details to be able to judge how similar these keys are.
Hey guys,
I recompiled for the new updates and was able to run on a single process all the way through. I then went to use the four gpus and currently getting some issues. Here is some output.
I'm launching the allocation with
salloc --nodes 1 --qos interactive --time 01:00:00 --constraint gpu --ntasks-per-node 4 --gpus-per-task 1 --account=m3623_g
I then run simply with
srun /path/to/agent inputs &> outputs
Sample output:
Initializing AMReX (24.03-35-gb892132f6144)...
MPI initialized with 4 MPI processes
MPI initialized with thread support level 0
Initializing CUDA...
CUDA initialized with 4 devices.
AMReX (24.03-35-gb892132f6144) initialized
Total pop 6783760
Total workers 3342275
Number of communities: 4967
Setting initial case counts in 2630 disease hubs.
Base domain is: ((0,0) (70,69) (0,0))
Max grid size is: 16
Number of boxes is: 25 over 4 ranks.
Infected / FIPS / Total Infected
731 6001 733
630 6013 1365
477 6041 1843
108 6055 1954
344 6075 2298
331 6081 2633
836 6085 3473
338 6095 3814
237 6097 4053
Simulating day 0
Writing plotfile
Simulating day 1
Simulating day 2
Simulating day 3
Simulating day 4
0::Assertion `mmc[3] == counts[4]' failed, file "/global/homes/k/keilbart/git/ExaEpi/src/main.cpp", line 221 !!!
SIGABRT
See Backtrace.0 file for details
MPICH ERROR [Rank 0] [job id 23529530.1] [Tue Mar 26 14:18:22 2024] [nid001565] - Abort(6) (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 6) - process 0
aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 6) - process 0
srun: error: nid001565: task 0: Exited with exit code 255
srun: Terminating StepId=23529530.1
slurmstepd: error: *** STEP 23529530.1 ON nid001565 CANCELLED AT 2024-03-26T21:18:23 ***
srun: error: nid001565: tasks 1,3: Terminated
srun: error: nid001565: task 2: Terminated
srun: Force Terminated StepId=23529530.1
Let me know if I need to provide other information.
Getting these warnings when compiling on Lassen with GCC-12.
/usr/workspace/keilbart/Lassen/ExaEpi/build/_deps/amrex-src/Src/Base/AMReX_Math.H:149:15: note: parameter passing for argument of type 'std::pair<float, float>' when C++17 is enabled changed to match C++14 in GCC 10.1
149 | std::pair<float,float> sincos (float x)
Link to source code documentation in the main readthedocs page.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.