devitocodes / pyrevolve Goto Github PK

View Code? Open in Web Editor NEW

15.0 15.0 6.0 327 KB

Python library to manage checkpointing for adjoints

License: Eclipse Public License 1.0

Python 76.94% C++ 20.86% Cython 2.20%

pyrevolve's People

Contributors

Stargazers

Watchers

Forkers

tjb900 erikajob91 aluamorim ofmla jkwashbourne speglich

pyrevolve's Issues

Add Support to Blocked Timestepping

This issue aims to add Blocked Timestepping to Pyrevolve. Reducing reverse restore timing , storage copy load timing and storage copy save timing

Multistage implementation is incorrect

In initialization of multistage checkpointing

pyrevolve/src/revolve.cpp

Lines 1170 to 1185 in 23dea91

    
           v = get_write_and_read_counts(); 
        
           sort(v.begin(),v.end()); 
        
           mid=v[snaps-sn_ram]; 
        
           // cout << mid << endl; 
        
           for(int i=snaps-1;i>=0;i--) 
        
           { 
        
             if(v[i]>=mid && num<sn_ram) 
        
             { 
        
               where[i]=true; 
        
               num++; 
        
             } 
        
             else 
        
             { 
        
               where[i]=false; 
        
             } 
        
           }

since v is sorted this always sets the last sn_ram elements of where to true, and the other elements to false, irrespective of the result of get_write_and_read_counts.

I think v should be copied before the sort (e.g. into w), and then the conditional replaced with

if(w[i]>=mid && num<sn_ram)

Incorrect initialisation for crevolve offline multistage

The initialisation of offline multistage in pyrevolve/crevolve.pyx appears to have the final two arguments

self.__r = revolve_c.revolve_create_multistage(c_st, c_sr, c_sn)

in the wrong order. I think this should instead be

self.__r = revolve_c.revolve_create_multistage(c_st, c_sn, c_sr)

Bug in revolve implementation

After a lot of testing, I continue to believe that our revolve implementation might not be doing the "right" thing. For 4 timesteps and 4 checkpoints, here is the schedule pyrevolve came up with:

Save
Forward from 0 to 1
Save
Forward from 1 to 2
Save
Forward from 2 to 3
Forward from 3 to 4
Reverse from 4 to 3
Load
Reverse from 3 to 2
Load
Reverse from 2 to 1
Load
Reverse from 1 to 0

Notice that there is no save between F(2, 3) and F(3, 4) and this is, I think, wrong. Even though we had 4 checkpoints, we wasted one by not storing this timestep.
Guillaume's utility, on the other hand, gave the following schedule for the same inputs (4 timesteps, 4 checkpoints):
Sequence: [WM_0, F_0->0, WM_1, F_1->1, WM_2, F_2->2, WM_3, F_3, B_4, RM_3, B_3, DM_3, RM_2, B_2, DM_2, RM_1, B_1, DM_1, RM_0, B_0, DM_0]

Tests needed

No library is complete without a good test.

Pyrevolve fails when using Jupyter Notebook

This Issue is caused by the class output grabber.

When using Jupyter the output of the stdout is not well redirected by him.

pyrevolve installation issue: fatal error: 'iostream' file not found [when creating anaconda environment]

We have a problem with pyrevolve when trying to install devito on a MacBook Pro (os: MacOS Mojave 10.14.3). The error first appears when we create the environment with anaconda. The error message is:

In file included from pyrevolve/crevolve.cpp:628:

  ./pyrevolve/../include/revolve_c.h:5:10: fatal error: 'iostream' file not found

  #include <iostream>

           ^~~~~~~~~~

  1 warning and 1 error generated.

  error: command 'gcc' failed with exit status 1

  

  ----------------------------------------

  Failed building wheel for pyrevolve

Please find attached a file with the complete output of :
conda env create -f environment.yml

We have installed the last version of anaconda3, Xcode, and we took the most recent files of devito from the git repository
devito_pyrevolve_error.pdf

Better driver needed

The current driver function is taken from the example code in the Revolve reference implementation. It does not do anything useful, and simply prints the actions that Revolve is requesting. It would be nice to have a driver that accepts <objects/function pointers/callbacks/something nice and pythonic> to actually do a useful computation, so that the user only needs to provide the worker functions and pyrevolve does all the rest. This is very simple in principle, but not yet sure how to do this nicely so that potential users can embed pyrevolve into their codes with minimum effort. To be discussed.

Harmonise devito's adjoint implementation with standards in AD literature

Devito follows slightly twisted indexing schemes, that are not the same as the standard ones seen in AD literature. We added an extra forward step in pyrevolve to get around that. However, now that we're trying to draw generic conclusions from this implementation, we need to remove this difference to decouple pyrevolve from devito's idiosyncracies. Ideas on how to remove this hard-coded bit?

python setup.py build_ext --inplace

running build_ext
building 'pyrevolve.crevolve' extension
gcc -pthread -B /home/miguel/anaconda3/envs/devito/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./pyrevolve -I. -Ipyrevolve -I/home/miguel/anaconda3/envs/devito/include/python3.7m -c pyrevolve/crevolve.cpp -o build/temp.linux-x86_64-3.7/pyrevolve/crevolve.o
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
error: command 'gcc' failed with exit status 1

memmap might be a faster way to read/write files

https://numpy.org/doc/stable/reference/generated/numpy.memmap.html

`snapshots_disk` argument to `CRevolve` constructor misnamed

This is passed as the third argument to revolve_create_multistage, which sets the number of RAM snapshots.

pyrevolve installation error (g++)

I get this error whenever I try to install devito via pip
I tried different gcc/g++ version, all don't work

Installing collected packages: pyrevolve
  Running setup.py install for pyrevolve ... error
    Complete output from command /bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-woo3yxrq/pyrevolve/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-0yak8udq/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/pyrevolve
    copying pyrevolve/pyrevolve.py -> build/lib.linux-x86_64-3.6/pyrevolve
    copying pyrevolve/_version.py -> build/lib.linux-x86_64-3.6/pyrevolve
    copying pyrevolve/__init__.py -> build/lib.linux-x86_64-3.6/pyrevolve
    UPDATING build/lib.linux-x86_64-3.6/pyrevolve/_version.py
    set build/lib.linux-x86_64-3.6/pyrevolve/_version.py to '1.0.2'
    running build_ext
    building 'pyrevolve.crevolve' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/pyrevolve
    creating build/temp.linux-x86_64-3.6/src
    gcc -pthread -B /compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./pyrevolve -I. -Ipyrevolve -I/include/python3.6m -c pyrevolve/crevolve.cpp -o build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from pyrevolve/../include/revolve_c.h:6:0,
                     from pyrevolve/crevolve.cpp:560:
    pyrevolve/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
    pyrevolve/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
      ~Revolve() { delete f, delete checkpoint; }
                          ^
    gcc -pthread -B compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./pyrevolve -I. -Ipyrevolve -I/include/python3.6m -c src/revolve_c.cpp -o build/temp.linux-x86_64-3.6/src/revolve_c.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from src/../include/revolve_c.h:6:0,
                     from src/revolve_c.cpp:1:
    src/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
    src/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
      ~Revolve() { delete f, delete checkpoint; }
                          ^
    gcc -pthread -B /compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I./pyrevolve -I. -Ipyrevolve -I//include/python3.6m -c src/revolve.cpp -o build/temp.linux-x86_64-3.6/src/revolve.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    In file included from src/revolve.cpp:23:0:
    src/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
    src/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
      ~Revolve() { delete f, delete checkpoint; }
                          ^
    src/revolve.cpp: In member function ‘ACTION::action Revolve::revolve(int*, int*, int*, int, int*)’:
    src/revolve.cpp:1247:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete f;
                ^
    src/revolve.cpp:1254:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete f;
                ^
    src/revolve.cpp: In member function ‘ACTION::action Revolve::revolve()’:
    src/revolve.cpp:1273:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete f;
                ^
    src/revolve.cpp:1280:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete f;
                ^
    src/revolve.cpp: In member function ‘void Revolve::turn(int)’:
    src/revolve.cpp:1304:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete f;
                ^
    src/revolve.cpp:1306:12: warning: deleting object of polymorphic class type ‘Online’ which has non-virtual destructor might cause undefined behavior [-Wdelete-non-virtual-dtor]
         delete g;
                ^
    src/revolve.cpp: In member function ‘int Revolve::get_r(int, int)’:
    src/revolve.cpp:1400:20: warning: unused variable ‘num’ [-Wunused-variable]
       int reps, range, num;
                        ^~~
    src/revolve.cpp: In member function ‘int Revolve::get_r()’:
    src/revolve.cpp:1431:20: warning: unused variable ‘num’ [-Wunused-variable]
       int reps, range, num;
                        ^~~
    g++ -pthread -shared -B compiler_compat -L/ib -Wl,-rpath=/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o build/temp.linux-x86_64-3.6/src/revolve_c.o build/temp.linux-x86_64-3.6/src/revolve.o -o build/lib.linux-x86_64-3.6/pyrevolve/crevolve.cpython-36m-x86_64-linux-gnu.so
    /compiler_compat/ld: build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o: unable to initialize decompress status for section .debug_info
    /compiler_compat/ld: build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o: unable to initialize decompress status for section .debug_info
    /compiler_compat/ld: build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o: unable to initialize decompress status for section .debug_info
    /compiler_compat/ld: build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o: unable to initialize decompress status for section .debug_info
    build/temp.linux-x86_64-3.6/pyrevolve/crevolve.o: file not recognized: file format not recognized
    collect2: error: ld returned 1 exit status
    error: command 'g++' failed with exit status 1
    
    ----------------------------------------
Command "/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-woo3yxrq/pyrevolve/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-0yak8udq/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-woo3yxrq/pyrevolve/

Error about "UnsupportedOperation: fileno"

Hi, I was trying to run Devito on a BP's TTI dataset based on this repo:

https://github.com/ofmla/RTM----segyio-devito-

But after reading necessary velocity parameters, this error came out. Have no clue to solve it, could you help to take a look?

I am running it under Amazon SageMaker, Tensorflow 1.15, Python 3.7

Thanks.

---------------------------------------------------------------------------
UnsupportedOperation                      Traceback (most recent call last)
<ipython-input-45-b0eb6f81bb80> in <module>
----> 1 final_image = forward_modeling_single_shot(records[0], table, par_files) #forward_modeling_multi_shots(par_files, shots)

<ipython-input-44-7f422d0900a0> in forward_modeling_single_shot(record, table, par_files)
    166         wrp = Revolver(cp, wrap_fw, wrap_rev, n_checkpoints, dobs_resam.shape[0]-2)
    167         print('Revolver storage: {}\n'.format(humanbytes(cp.size*n_checkpoints*itemsize)))
--> 168         wrp.apply_forward()
    169         print('{} run finished\n'.format(strng))
    170         summary = wrp.apply_reverse()

/usr/local/lib/python3.7/site-packages/pyrevolve/pyrevolve.py in apply_forward(self)
    100         while(True):
    101             # ask Revolve what to do next.
--> 102             action = self.scheduler.next()
    103             if(action.type == Action.ADVANCE):
    104                 # advance forward computation

/usr/local/lib/python3.7/site-packages/pyrevolve/schedulers.py in next(self)
     43 
     44     def next(self):
---> 45         return Action(self.translations[self.revolve.revolve()], self.capo,
     46                       self.old_capo, self.cp_pointer)
     47 

pyrevolve/crevolve.pyx in pyrevolve.crevolve.CRevolve.revolve()

/usr/local/lib/python3.7/site-packages/pyrevolve/tools.py in __init__(self, stream, threaded)
     16         if self.origstream is None:
     17             self.origstream = sys.stdout
---> 18         self.origstreamfd = self.origstream.fileno()
     19         self.capturedtext = ""
     20         # Create a pipe so the stream can be captured:

UnsupportedOperation: fileno

Compression and Disk Storage can not be used together right now

Maybe Compression should be moved into a mixin for the storage classes so as to work with all kinds of storage. TBD.

pyrevolve depends on a GPL library

Hey,

I think that right now, pyrevolve brings in a GPL dependency in the form of contexttimer (https://pypi.org/project/contexttimer/)

Currently, the contexttimer objects are used, but the results are discarded after computation. The easiest fix would be to just get rid of the dependency and the corresponding code. However, if the corresponding feature is meant to be used in the future, it would be easy enough to reimplement that piece of code in pyrevolve with contextlib and timer directly.

Error with pyrevolve - ImportError: /lib64/libm.so.6: version `GLIBC_2.29' not found

Hi,
I'm getting the following error when I try to import modules from Devito:

Python 3.10.6 (main, Oct 24 2022, 16:07:47) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
>>> 
>>> from examples.seismic import Model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/macelloni/devito/examples/seismic/__init__.py", line 1, in <module>
    from .model import *  # noqa
  File "/home/macelloni/devito/examples/seismic/model.py", line 5, in <module>
    from devito import (Grid, SubDomain, Function, Constant, warning,
  File "/home/macelloni/devito/devito/__init__.py", line 26, in <module>
    from devito.checkpointing import DevitoCheckpoint, CheckpointOperator  # noqa
  File "/home/macelloni/devito/devito/checkpointing/__init__.py", line 1, in <module>
    from .checkpoint import *  # noqa
  File "/home/macelloni/devito/devito/checkpointing/checkpoint.py", line 1, in <module>
    from pyrevolve import Checkpoint, Operator
  File "/home/macelloni/anaconda3/envs/devito/lib/python3.10/site-packages/pyrevolve/__init__.py", line 3, in <module>
    from pyrevolve.pyrevolve import * # noqa
  File "/home/macelloni/anaconda3/envs/devito/lib/python3.10/site-packages/pyrevolve/pyrevolve.py", line 6, in <module>
    from . import crevolve as cr
ImportError: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by /home/macelloni/anaconda3/envs/devito/lib/python3.10/site-packages/pyrevolve/crevolve.cpython-310-x86_64-linux-gnu.so)

I have no problems when I work on Ubuntu 22.04.1 LTS, but I get the above message on CentOS Linux 7 (Core).
From conda list:

devito                    4.7.1+183.gf65800404     dev_0    <develop>
pyrevolve                 2.2                      pypi_0    pypi
python                    3.10.6                   haa1d7c7_1

A colleague of mine is working with the same system and he has never faced this issue using

devito                    4.6+118.gb5d6185         dev_0    <develop>
pyrevolve                 2.2                      pypi_0    pypi
python                    3.9.7                    h12debd9_1

A check of the installed glibc version by using ldd --version command shows ldd (Ubuntu GLIBC 2.35-0ubuntu3.1) 2.35 on Ubuntu and ldd (GNU libc) 2.17 on CentOS.

Is this problem due to the version of python and/or Devito? Does this error means that the older version of Devito (4.6) works with glibc 2.17, but Devito 4.7.1 does not?

Thank you. Best regards,
FM

macos Version 10.11.6

Hi, I cannot make pyrevolve works in macos version 10.11.6 Thanks!

(devito) Elenas-iMac:pyrevolve-1.0.1 elena$ python setup.py install
running install
running bdist_egg
running egg_info
writing pyrevolve.egg-info/PKG-INFO
writing dependency_links to pyrevolve.egg-info/dependency_links.txt
writing top-level names to pyrevolve.egg-info/top_level.txt
reading manifest file 'pyrevolve.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pyrevolve.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.7-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.macosx-10.7-x86_64-3.6
creating build/lib.macosx-10.7-x86_64-3.6/pyrevolve
copying pyrevolve/__init__.py -> build/lib.macosx-10.7-x86_64-3.6/pyrevolve
copying pyrevolve/_version.py -> build/lib.macosx-10.7-x86_64-3.6/pyrevolve
copying pyrevolve/pyrevolve.py -> build/lib.macosx-10.7-x86_64-3.6/pyrevolve
UPDATING build/lib.macosx-10.7-x86_64-3.6/pyrevolve/_version.py
set build/lib.macosx-10.7-x86_64-3.6/pyrevolve/_version.py to '1.0.1'
running build_ext
building 'pyrevolve.crevolve' extension
creating build/temp.macosx-10.7-x86_64-3.6
creating build/temp.macosx-10.7-x86_64-3.6/pyrevolve
creating build/temp.macosx-10.7-x86_64-3.6/src
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I./pyrevolve -I. -Ipyrevolve -I/Users/elena/anaconda2/envs/devito/include/python3.6m -c pyrevolve/crevolve.cpp -o build/temp.macosx-10.7-x86_64-3.6/pyrevolve/crevolve.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from pyrevolve/../include/revolve_c.h:6:0,
                 from pyrevolve/crevolve.cpp:645:
pyrevolve/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
pyrevolve/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
  ~Revolve() { delete f, delete checkpoint; }
                      ^
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I./pyrevolve -I. -Ipyrevolve -I/Users/elena/anaconda2/envs/devito/include/python3.6m -c src/revolve_c.cpp -o build/temp.macosx-10.7-x86_64-3.6/src/revolve_c.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from src/../include/revolve_c.h:6:0,
                 from src/revolve_c.cpp:1:
src/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
src/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
  ~Revolve() { delete f, delete checkpoint; }
                      ^
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:3:11: warning: section "__textcoal_nt" is deprecated
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:3:11: note: change section name to "__text"
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:972:11: warning: section "__const_coal" is deprecated
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:972:11: note: change section name to "__const"
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:979:11: warning: section "__const_coal" is deprecated
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:979:11: note: change section name to "__const"
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:984:11: warning: section "__const_coal" is deprecated
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccA7X8km.s:984:11: note: change section name to "__const"
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I/Users/elena/anaconda2/envs/devito/include -arch x86_64 -I./pyrevolve -I. -Ipyrevolve -I/Users/elena/anaconda2/envs/devito/include/python3.6m -c src/revolve.cpp -o build/temp.macosx-10.7-x86_64-3.6/src/revolve.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
In file included from src/revolve.cpp:23:0:
src/../include/revolve.h: In destructor ‘Revolve::~Revolve()’:
src/../include/revolve.h:441:22: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
  ~Revolve() { delete f, delete checkpoint; }
                      ^
src/revolve.cpp: In member function ‘ACTION::action Revolve::revolve(int*, int*, int*, int, int*)’:
src/revolve.cpp:1247:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete f;
            ^
src/revolve.cpp:1254:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete f;
            ^
src/revolve.cpp: In member function ‘ACTION::action Revolve::revolve()’:
src/revolve.cpp:1273:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete f;
            ^
src/revolve.cpp:1280:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete f;
            ^
src/revolve.cpp: In member function ‘void Revolve::turn(int)’:
src/revolve.cpp:1304:12: warning: deleting object of polymorphic class type ‘Schedule’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete f;
            ^
src/revolve.cpp:1306:12: warning: deleting object of polymorphic class type ‘Online’ which has non-virtual destructor might cause undefined behaviour [-Wdelete-non-virtual-dtor]
     delete g;
            ^
src/revolve.cpp: In member function ‘int Revolve::get_r(int, int)’:
src/revolve.cpp:1400:20: warning: unused variable ‘num’ [-Wunused-variable]
   int reps, range, num;
                    ^
src/revolve.cpp: In member function ‘int Revolve::get_r()’:
src/revolve.cpp:1431:20: warning: unused variable ‘num’ [-Wunused-variable]
   int reps, range, num;
                    ^
src/revolve.cpp: In member function ‘virtual ACTION::action Moin::revolve()’:
src/revolve.cpp:824:20: warning: ‘index’ may be used uninitialized in this function [-Wmaybe-uninitialized]
       check = index;
                    ^
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:3:11: warning: section "__textcoal_nt" is deprecated
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:3:11: note: change section name to "__text"
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:19573:11: warning: section "__textcoal_nt" is deprecated
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:19573:11: note: change section name to "__text"
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:33133:11: warning: section "__textcoal_nt" is deprecated
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:33133:11: note: change section name to "__text"
        .section __TEXT,__textcoal_nt,coalesced,pure_instructions
                 ^      ~~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38355:11: warning: section "__const_coal" is deprecated
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38355:11: note: change section name to "__const"
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38362:11: warning: section "__const_coal" is deprecated
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38362:11: note: change section name to "__const"
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38367:11: warning: section "__const_coal" is deprecated
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38367:11: note: change section name to "__const"
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38375:11: warning: section "__const_coal" is deprecated
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38375:11: note: change section name to "__const"
        .section __TEXT,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38435:11: warning: section "__const_coal" is deprecated
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
/var/folders/wb/bt4160ms6sl4zpcl2dqg406m0000gn/T//ccpBjWbX.s:38435:11: note: change section name to "__const"
        .section __DATA,__const_coal,coalesced
                 ^      ~~~~~~~~~~~~
g++ -bundle -undefined dynamic_lookup -L/Users/elena/anaconda2/envs/devito/lib -arch x86_64 -L/Users/elena/anaconda2/envs/devito/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.7-x86_64-3.6/pyrevolve/crevolve.o build/temp.macosx-10.7-x86_64-3.6/src/revolve_c.o build/temp.macosx-10.7-x86_64-3.6/src/revolve.o -o build/lib.macosx-10.7-x86_64-3.6/pyrevolve/crevolve.cpython-36m-darwin.so
creating build/bdist.macosx-10.7-x86_64
creating build/bdist.macosx-10.7-x86_64/egg
creating build/bdist.macosx-10.7-x86_64/egg/pyrevolve
copying build/lib.macosx-10.7-x86_64-3.6/pyrevolve/__init__.py -> build/bdist.macosx-10.7-x86_64/egg/pyrevolve
copying build/lib.macosx-10.7-x86_64-3.6/pyrevolve/_version.py -> build/bdist.macosx-10.7-x86_64/egg/pyrevolve
copying build/lib.macosx-10.7-x86_64-3.6/pyrevolve/crevolve.cpython-36m-darwin.so -> build/bdist.macosx-10.7-x86_64/egg/pyrevolve
copying build/lib.macosx-10.7-x86_64-3.6/pyrevolve/pyrevolve.py -> build/bdist.macosx-10.7-x86_64/egg/pyrevolve
byte-compiling build/bdist.macosx-10.7-x86_64/egg/pyrevolve/__init__.py to __init__.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/pyrevolve/_version.py to _version.cpython-36.pyc
byte-compiling build/bdist.macosx-10.7-x86_64/egg/pyrevolve/pyrevolve.py to pyrevolve.cpython-36.pyc
creating stub loader for pyrevolve/crevolve.cpython-36m-darwin.so
byte-compiling build/bdist.macosx-10.7-x86_64/egg/pyrevolve/crevolve.py to crevolve.cpython-36.pyc
creating build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying pyrevolve.egg-info/PKG-INFO -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying pyrevolve.egg-info/SOURCES.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying pyrevolve.egg-info/dependency_links.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
copying pyrevolve.egg-info/top_level.txt -> build/bdist.macosx-10.7-x86_64/egg/EGG-INFO
writing build/bdist.macosx-10.7-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
pyrevolve.__pycache__.crevolve.cpython-36: module references __file__
creating dist
creating 'dist/pyrevolve-1.0.1-py3.6-macosx-10.7-x86_64.egg' and adding 'build/bdist.macosx-10.7-x86_64/egg' to it
removing 'build/bdist.macosx-10.7-x86_64/egg' (and everything under it)
Processing pyrevolve-1.0.1-py3.6-macosx-10.7-x86_64.egg
creating /Users/elena/anaconda2/envs/devito/lib/python3.6/site-packages/pyrevolve-1.0.1-py3.6-macosx-10.7-x86_64.egg
Extracting pyrevolve-1.0.1-py3.6-macosx-10.7-x86_64.egg to /Users/elena/anaconda2/envs/devito/lib/python3.6/site-packages
Adding pyrevolve 1.0.1 to easy-install.pth file

Installed /Users/elena/anaconda2/envs/devito/lib/python3.6/site-packages/pyrevolve-1.0.1-py3.6-macosx-10.7-x86_64.egg
Processing dependencies for pyrevolve==1.0.1
Finished processing dependencies for pyrevolve==1.0.1

(devito) Elenas-iMac:pyrevolve-1.0.1 elena$ python -c "import pyrevolve"
Traceback (most recent call last):
  File "/Users/elena/install/pyrevolve-1.0.1/pyrevolve/pyrevolve.py", line 2, in <module>
    import pyrevolve.crevolve as cr
ImportError: dlopen(/Users/elena/install/pyrevolve-1.0.1/pyrevolve/crevolve.cpython-36m-darwin.so, 2): Symbol not found: __ZNKSt5ctypeIcE13_M_widen_initEv
  Referenced from: /Users/elena/install/pyrevolve-1.0.1/pyrevolve/crevolve.cpython-36m-darwin.so
  Expected in: /usr/lib/libstdc++.6.dylib
 in /Users/elena/install/pyrevolve-1.0.1/pyrevolve/crevolve.cpython-36m-darwin.so

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/elena/install/pyrevolve-1.0.1/pyrevolve/__init__.py", line 3, in <module>
    from pyrevolve.pyrevolve import * # noqa
  File "/Users/elena/install/pyrevolve-1.0.1/pyrevolve/pyrevolve.py", line 4, in <module>
    import crevolve as cr
ModuleNotFoundError: No module named 'crevolve'

pyrevolve installation error: Failed building wheel for pyrevolve: conflicting types for ‘_xgetbv’

While creating the conda environment for devito, pip fails with error: conflicting types for ‘_xgetbv’.

Full output at:

https://gist.github.com/dlight/9c4854d9ab49e171f6da39335fd31d48

Undefined symbol in Manjaro installation

We have been having some issues with installing pyrevolve in Manjaro. The pip installation completes as expected, but then the import throws the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/oab18/anaconda3/envs/stride/lib/python3.10/site-packages/pyrevolve/__init__.py", line 3, in <module>
    from pyrevolve.pyrevolve import * # noqa
  File "/home/oab18/anaconda3/envs/stride/lib/python3.10/site-packages/pyrevolve/pyrevolve.py", line 6, in <module>
    from . import crevolve as cr
ImportError: /home/oab18/anaconda3/envs/stride/lib/python3.10/site-packages/pyrevolve/crevolve.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZTVN10__cxxabiv117__class_type_infoE

This was done in a fresh anaconda environment with Python 3.10.5. We are still investigating the issue, but any ideas on what could be causing this?

New release

Would probably need a new release, there is a lot of commits and update since the last one in 2019 including cython fix needed to compile properly with python 3.11

Parameter passing to apply()

The apply_forward() and apply_reverse() methods take no arguments. This should probably be changed, and arguments be passed on to the operator.apply() methods. Before this is implemented, we need to discuss the desired behaviour. Complications:

apply_reverse() calls fwd_operator.apply() and rev_operator.apply() internally, several times. Do we need to distinguish which arguments to apply_reverse() are passed to which operator?
The operator apply routines take t_start and t_end as arguments. These are now set internally by pyrevolve. If a user mis-uses pyrevolve and provides such arguments to apply_reverse() or apply_forward(), we can actively discard the user-provided arguments, throw an error, or blindly forward the user's values and potentially compute wrong results.
A user may try to pass symbols to apply_reverse() that contain data that was modified after running apply_forward(), perhaps in a misguided attempt to compute several adjoint result sets. This would again silently produce wrong results, as the new data provided will be inconsistent with the data inside the checkpoint storage. Do we want to sanity-check things like this, or assume that the user is always right?

The easiest way out (for pyrevolve) is of course to just pass everything without checks, and crash and burn if the user is stupid.

Support data that is not a numpy array

The checkpoint storage in pyrevolve is a big numpy array, and the "pointers" that are given to checkpoint.save(ptr) and checkpoint.load(ptr) are in fact slices of that numpy array. A user who needs to store data that is not numpy will be out of luck. Not sure if this is really a problem in need of a fix, unless someone wants to do mixed-precision computing or checkpoint non-floating-point data. Open for discussion.

	v = get_write_and_read_counts();
	sort(v.begin(),v.end());
	mid=v[snaps-sn_ram];
	// cout << mid << endl;
	for(int i=snaps-1;i>=0;i--)
	{
	if(v[i]>=mid && num<sn_ram)
	{
	where[i]=true;
	num++;
	}
	else
	{
	where[i]=false;
	}
	}