Comments (13)
Can you give a bit more information about your environment (OS, version of python, etc.), and perhaps add stacktrace? I cannot reproduce this in my environment:
$ conda list numpy # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel numpy 1.21.0 pypi_0 pypi numpy-base 1.20.2 py38hfae3a4d_0 $ conda list pvapy # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel pvapy 3.1.0 py38_1 apsu pvapy-boost 1.72.0 py38_1 apsu $ python -c "from pvapy import *; s = NtScalar(PvInt(3)); print(s)" structure int value 3
from pvapy.
Hi.
Well i can get you the basics (see below) but when you talk about backtrace i am lost. The thing is that we have 2 PCs which are in principle (should be identical) installed in the same way. However, they were setup on different dates, and when trying to sort this issue out I by luck noted (when testing reinstalling pvapy) that even if the pvapy version was the same it in turn installed numpy. When I checked the numpy version online it had just been upgraded in the last few weeks, and as it seem one of the PCs had gotten version 1.20.3 and the other had version 1.21.0 probably as they were setup in different days.
When i then installed the older numpy version i did not get the seg fault.
Here is the python test in python CLI i did to check on the different PCs:
$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import pvaccess
pv = pvaccess.PvObject({"hej":pvaccess.INT})
print(pv)
structure
int hej 0
pv = pvaccess.NtScalar(pvaccess.DOUBLE)
Segmentation fault
I am sorry that i cannot provide more. Maybe it is not an issue an dit might just have been a coincidence.
Kind regards
Anders
python3 version 3.7.3
pvapy version 3.1.0
Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 4.19.0-17-amd64
Architecture: x86-64
from pvapy.
I was able to re-produce this on my (MacOS) machine. The stack trace from that was:
0 org.python.python 0x000000010643346b PyDict_GetItemWithError + 11
1 _multiarray_umath.cpython-38-darwin.so 0x0000000107ae6378 PyArray_GetCastingImpl + 40
2 _multiarray_umath.cpython-38-darwin.so 0x0000000107ae6ade PyArray_GetCastSafety + 62
3 _multiarray_umath.cpython-38-darwin.so 0x0000000107ba75c0 PyArray_EquivTypes + 16
4 libboost_numpy38-mt.dylib 0x00000001076df27c boost::python::numpy::(anonymous namespace)::array_scalar_converter<int>::convertible(_object*) + 140
5 libboost_python38-mt.dylib 0x0000000107680215 boost::python::converter::rvalue_from_python_stage1(_object*, boost::python::converter::registration const&) + 69
6 pvaccess.so 0x0000000106b31e15 PyPvDataUtility::updateFieldArrayFromDict(boost::python::dict const&, std::__1::vector<std::__1::shared_ptr<epics::pvData::Field const>, std::__1::allocator<std::__1::shared_ptr<epics::pvData::Field const> > >&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) + 485 (PyPvDataUtility.cpp:1667)
7 pvaccess.so 0x0000000106b31a13 PyPvDataUtility::createStructureFromDict(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 67
8 pvaccess.so 0x0000000106b082a5 PvObject::PvObject(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 101 (PvObject.cpp:54)
9 pvaccess.so 0x0000000106afdfae NtType::NtType(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 14 (NtType.cpp:10)
10 pvaccess.so 0x0000000106af9e5b NtScalar::NtScalar(PvType::ScalarType) + 155 (NtScalar.cpp:33)
11 pvaccess.so 0x0000000106aaf777 boost::python::objects::make_holder<1>::apply<boost::python::objects::value_holder<NtScalar>, boost::mpl::vector1<PvType::ScalarType> >::execute(_object*, PvType::ScalarType) + 71 (make_holder.hpp:95)
12 pvaccess.so 0x0000000106a8be56 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(_object*, PvType::ScalarType), boost::python::default_call_policies, boost::mpl::vector3<void, _object*, PvType::ScalarType> > >::operator()(_object*, _object*) + 102 (py_function.hpp:38)
13 libboost_python38-mt.dylib 0x0000000107686fdb boost::python::objects::function::call(_object*, _object*) const + 651
14 libboost_python38-mt.dylib 0x00000001076894ca boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 26
15 libboost_python38-mt.dylib 0x000000010768f127 boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const + 71
16 pvaccess.so 0x0000000106a59336 boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PvaException, void (*)(PvaException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PvaException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) + 22 (function_template.hpp:137)
17 libboost_python38-mt.dylib 0x000000010768eecd boost::python::handle_exception_impl(boost::function0<void>) + 45
18 libboost_python38-mt.dylib 0x0000000107688fb1 boost::python::objects::function_call(_object*, _object*, _object*) + 81
19 org.python.python 0x0000000106410411 _PyObject_MakeTpCall + 274
20 org.python.python 0x0000000106412e23 method_vectorcall + 415
21 org.python.python 0x000000010641069d PyVectorcall_Call + 108
22 org.python.python 0x0000000106453c95 slot_tp_init + 96
23 org.python.python 0x00000001064508c2 type_call + 172
24 org.python.python 0x0000000106410411 _PyObject_MakeTpCall + 274
25 org.python.python 0x00000001064aff3e call_function + 804
26 org.python.python 0x00000001064ad0b4 _PyEval_EvalFrameDefault + 27252
27 org.python.python 0x00000001064b08a3 _PyEval_EvalCodeWithName + 1958
28 org.python.python 0x00000001064a65b8 PyEval_EvalCode + 42
29 org.python.python 0x00000001064def15 run_eval_code_obj + 102
30 org.python.python 0x00000001064de1a0 run_mod + 82
31 org.python.python 0x00000001064de2cd pyrun_file + 133
32 org.python.python 0x00000001064dca78 PyRun_SimpleFileExFlags + 646
33 org.python.python 0x00000001064f428d Py_RunMain + 1870
34 org.python.python 0x00000001064f460a pymain_main + 360
35 org.python.python 0x00000001064f465f Py_BytesMain + 42
36 libdyld.dylib 0x00007fff6cb84cc9 start + 1
from pvapy.
As Anders downgraded numpy on the (linux) machine where he encountered the crash, I can not re-produce the stack trace on that machine. However, I can tell you that the location of the crash was the same on that machine.
from pvapy.
I talked to Anders (@Anderspp) and was allowed to update numpy on his machine such that I could get the stack trace. The result was as follows (with use of c++filt):
python3(PyDict_GetItemWithError+0x5)[0x5b4be5]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x9dfec)[0x7f99f69fdfec]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x9e988)[0x7f99f69fe988]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x1494e5)[0x7f99f6aa94e5]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_numpy37.so.1.72.0(boost::python::numpy::equivalent(boost::python::numpy::dtype const&, boost::python::numpy::dtype const&)+0x1a)[0x7f99f85fc9ba]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_numpy37.so.1.72.0(+0x4368)[0x7f99f85fd368]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::converter::rvalue_from_python_stage1(_object*, boost::python::converter::registration const&)+0x4e)[0x7f99f881b77e]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::updateFieldArrayFromInt(boost::python::api::object const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::tr1::shared_ptr<epics::pvData::Field const>, std::allocator<std::tr1::shared_ptr<epics::pvData::Field const> > >&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x2b)[0x7f99f9dd57bb]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::updateFieldArrayFromDict(boost::python::dict const&, std::vector<std::tr1::shared_ptr<epics::pvData::Field const>, std::allocator<std::tr1::shared_ptr<epics::pvData::Field const> > >&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x344)[0x7f99f9dd9ee4]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::createStructureFromDict(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x60)[0x7f99f9dda380]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PvObject::PvObject(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x37)[0x7f99f9db79b7]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(NtType::NtType(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x9)[0x7f99f9da6be9]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(NtScalar::NtScalar(PvType::ScalarType)+0x4a)[0x7f99f9da24ca]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::python::objects::make_holder<1>::apply<boost::python::objects::value_holder<NtScalar>, boost::mpl::vector1<PvType::ScalarType> >::execute(_object*, PvType::ScalarType)+0x4c)[0x7f99f9d5dbac]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(_object*, PvType::ScalarType), boost::python::default_call_policies, boost::mpl::vector3<void, _object*, PvType::ScalarType> > >::operator()(_object*, _object*)+0x5a)[0x7f99f9d3c40a]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::objects::function::call(_object*, _object*) const+0xf1)[0x7f99f8823ba1]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(+0x1ef08)[0x7f99f8823f08]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const+0x43)[0x7f99f882e443]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PvaException, void (*)(PvaException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PvaException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)+0x13)[0x7f99f9d108d3]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::handle_exception_impl(boost::function0<void>)+0x2d)[0x7f99f882e21d]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(+0x1d7e3)[0x7f99f88227e3]
python3[0x4d9850]
python3(PyObject_Call+0x56)[0x5dc4f6]
python3[0x5916c8]
python3(_PyObject_FastCallKeywords+0x129)[0x5d9c59]
python3[0x54b3c1]
python3(_PyEval_EvalFrameDefault+0x441d)[0x5524cd]
python3(_PyEval_EvalCodeWithName+0x252)[0x54bcc2]
python3(PyRun_StringFlags+0xaf)[0x5317ef]
python3(PyRun_SimpleStringFlags+0x3d)[0x63177d]
python3[0x654558]
python3(_Py_UnixMain+0x2e)[0x65468e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7f99fa54b09b]
python3(_start+0x2a)[0x5e0e8a]
from pvapy.
Thanks for the additional info. I managed to reproduce the problem with python 3.7 and numpy 1.21.0. It looks like the boost::python::extract object results in segfault when it is constructed with a dictionary-like object if one is using numpy>=1.21. The proper fix will have to be made in boost python, but in the meantime, I think I have a viable workaround on the branch bp-extract-fix (commit 19d9728). I would appreciate if you could verify the fix in your environment.
Here is how I built/tested code from sources, using conda.
- Install conda, downgrade python to 3.7, and install numpy 1.21 via pip (conda would install 1.20.x):
$ export TEST_DIR=/local/PVAPY/TEST $ mkdir -p $TEST_DIR $ cd $TEST_DIR $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh $ sh Miniconda3-latest-Linux-x86_64.sh -b -p $PWD/CONDA $ export PATH=$PWD/CONDA/bin:$PATH $ conda install -y python=3.7 $ pip install numpy==1.21 $ conda list python # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel python 3.7.10 hdb3f193_0 $ conda list numpy # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel numpy 1.21.0 pypi_0 pypi
- Clone pvapy repository, build dependencies locally (epics and boost python):
$ git clone [email protected]:epics-base/pvaPy.git pvapy $ cd pvapy $ mkdir -p $TEST_DIR/opt $ PREFIX=$TEST_DIR/opt make local
- Build pvapy library from sources on the master branch:
$ make configure EPICS_BASE=$TEST_DIR/opt/epics-7.0.5 BOOST_ROOT=$TEST_DIR/opt/boost-python-1.72.0 PYTHON_VERSION=3 $ make
- Verify segfault:
bluegill2> source bin/linux-x86_64/pvapy_setup_full.3.7.sh $ python -c "from pvaccess import *; o = NtScalar(INT); print(o)" Segmentation fault (core dumped) $ python -c "from pvaccess import *; o = NtNdArray(); print(o)" Segmentation fault (core dumped)
- Checkout bp-extract-fix branch, rebuild code:
$ git checkout bp-extract-fix $ make
- Verify that segfault does not occur:
$ python -c "from pvaccess import *; o = NtScalar(INT); print(o)" epics:nt/NTScalar:1.0 int value 0 string descriptor structure alarm int severity 0 int status 0 string message structure timeStamp long secondsPastEpoch 0 int nanoseconds 0 int userTag 0 structure display double limitLow 0 double limitHigh 0 string description string format string units structure control double limitLow 0 double limitHigh 0 double minStep 0 $ python -c "from pvaccess import *; o = NtNdArray(); print(o)" epics:nt/NTNDArray:1.0 union value (none) structure codec string name any parameters (none) long compressedSize 0 long uncompressedSize 0 structure[] dimension int uniqueId 0 structure dataTimeStamp long secondsPastEpoch 0 int nanoseconds 0 int userTag 0 structure[] attribute string descriptor structure alarm int severity 0 int status 0 string message structure timeStamp long secondsPastEpoch 0 int nanoseconds 0 int userTag 0 structure display double limitLow 0 double limitHigh 0 string description string format string units
from pvapy.
try to uninstall the current NumPy and install NumPy 1.20.3
from pvapy.
Right, numpy 1.20.x works, but this issue is about 1.21.x.
from pvapy.
Hi, I have a similar problem. Did you open an issue with numpy or boost-python?
I'm on python 3.7 with boost python 1.69 with boost numpy.
from pvapy.
Hi, sorry, I have not had a chance to do this yet. Do you see the problem with latest pvapy packages? Those should be restricting numpy to 1.20.x until this gets resolved.
from pvapy.
Sorry, I was unclear. I see this in my own work. But I only found this thread here with any information. Seems to me to be a problem in either numpy, or boost python. Latest numpy still fail for me (1.21.2).
from pvapy.
It looks like this problem has been corrected in numpy>1.21.5 See boostorg/python#376. In my environment numpy 1.22 works okay.
from pvapy.
I think this issue can be closed. All recent pip/conda packages avoid numpy 1.21 that was causing the problem.
from pvapy.
Related Issues (20)
- Seg fault with arrays HOT 4
- PvaServer.addRecord not working on Windows 10 HOT 2
- PyPI: pvaPy on Raspberry Pi HOT 11
- Dumping to JSON does not escape strings HOT 4
- Mirror server slow processing large struct arrays
- Error when requesting multiple fields from the same struct HOT 1
- add custom field to NTSCALAR record HOT 1
- Review performance testing HOT 10
- Best method for handling reconnections to large numbers of PVs HOT 7
- Possibility of supporting epicscorelibs for PyPI uploaded wheels HOT 4
- Reading and writing to a single array element of a PVAccess PV HOT 1
- Bug executing`pvget` on unchanged PV in remote server. HOT 15
- Can't install for Python 3.10 on Mac HOT 7
- Unable to install on Macs that use Apple M-series (ARM) chips HOT 5
- pvaPy server listing all fields as 'changed' despite only updating value and timeStamp HOT 4
- Type stubs missing HOT 1
- numpy v2 upgrade causing problems HOT 2
- maximum array size limitation HOT 2
- Processing error: 'Queue' object has no attribute 'size' HOT 2
- EPICS_PVA_NAME_SERVERS: handling reconnection HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pvapy.