Comments (13)
sveseli
commented on August 25, 2024
Can you give a bit more information about your environment (OS, version of python, etc.), and perhaps add stacktrace? I cannot reproduce this in my environment:
$ conda list numpy
# packages in environment at /local/PVAPY/TEST/CONDA:
#
# Name Version Build Channel
numpy 1.21.0 pypi_0 pypi
numpy-base 1.20.2 py38hfae3a4d_0
$ conda list pvapy
# packages in environment at /local/PVAPY/TEST/CONDA:
#
# Name Version Build Channel
pvapy 3.1.0 py38_1 apsu
pvapy-boost 1.72.0 py38_1 apsu
$ python -c "from pvapy import *; s = NtScalar(PvInt(3)); print(s)"
structure
int value 3
from pvapy.
Anderspp
commented on August 25, 2024
Hi.
Well i can get you the basics (see below) but when you talk about backtrace i am lost. The thing is that we have 2 PCs which are in principle (should be identical) installed in the same way. However, they were setup on different dates, and when trying to sort this issue out I by luck noted (when testing reinstalling pvapy) that even if the pvapy version was the same it in turn installed numpy. When I checked the numpy version online it had just been upgraded in the last few weeks, and as it seem one of the PCs had gotten version 1.20.3 and the other had version 1.21.0 probably as they were setup in different days.
When i then installed the older numpy version i did not get the seg fault.
Here is the python test in python CLI i did to check on the different PCs:
$ python3
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
import pvaccess
pv = pvaccess.PvObject({"hej":pvaccess.INT})
print(pv)
structure
int hej 0
pv = pvaccess.NtScalar(pvaccess.DOUBLE)
Segmentation fault
I am sorry that i cannot provide more. Maybe it is not an issue an dit might just have been a coincidence.
Kind regards
Anders
python3 version 3.7.3
pvapy version 3.1.0
Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 4.19.0-17-amd64
Architecture: x86-64
from pvapy.
SkyToGround
commented on August 25, 2024
I was able to re-produce this on my (MacOS) machine. The stack trace from that was:
0 org.python.python 0x000000010643346b PyDict_GetItemWithError + 11
1 _multiarray_umath.cpython-38-darwin.so 0x0000000107ae6378 PyArray_GetCastingImpl + 40
2 _multiarray_umath.cpython-38-darwin.so 0x0000000107ae6ade PyArray_GetCastSafety + 62
3 _multiarray_umath.cpython-38-darwin.so 0x0000000107ba75c0 PyArray_EquivTypes + 16
4 libboost_numpy38-mt.dylib 0x00000001076df27c boost::python::numpy::(anonymous namespace)::array_scalar_converter<int>::convertible(_object*) + 140
5 libboost_python38-mt.dylib 0x0000000107680215 boost::python::converter::rvalue_from_python_stage1(_object*, boost::python::converter::registration const&) + 69
6 pvaccess.so 0x0000000106b31e15 PyPvDataUtility::updateFieldArrayFromDict(boost::python::dict const&, std::__1::vector<std::__1::shared_ptr<epics::pvData::Field const>, std::__1::allocator<std::__1::shared_ptr<epics::pvData::Field const> > >&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&) + 485 (PyPvDataUtility.cpp:1667)
7 pvaccess.so 0x0000000106b31a13 PyPvDataUtility::createStructureFromDict(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 67
8 pvaccess.so 0x0000000106b082a5 PvObject::PvObject(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 101 (PvObject.cpp:54)
9 pvaccess.so 0x0000000106afdfae NtType::NtType(boost::python::dict const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) + 14 (NtType.cpp:10)
10 pvaccess.so 0x0000000106af9e5b NtScalar::NtScalar(PvType::ScalarType) + 155 (NtScalar.cpp:33)
11 pvaccess.so 0x0000000106aaf777 boost::python::objects::make_holder<1>::apply<boost::python::objects::value_holder<NtScalar>, boost::mpl::vector1<PvType::ScalarType> >::execute(_object*, PvType::ScalarType) + 71 (make_holder.hpp:95)
12 pvaccess.so 0x0000000106a8be56 boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(_object*, PvType::ScalarType), boost::python::default_call_policies, boost::mpl::vector3<void, _object*, PvType::ScalarType> > >::operator()(_object*, _object*) + 102 (py_function.hpp:38)
13 libboost_python38-mt.dylib 0x0000000107686fdb boost::python::objects::function::call(_object*, _object*) const + 651
14 libboost_python38-mt.dylib 0x00000001076894ca boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 26
15 libboost_python38-mt.dylib 0x000000010768f127 boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const + 71
16 pvaccess.so 0x0000000106a59336 boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PvaException, void (*)(PvaException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PvaException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&) + 22 (function_template.hpp:137)
17 libboost_python38-mt.dylib 0x000000010768eecd boost::python::handle_exception_impl(boost::function0<void>) + 45
18 libboost_python38-mt.dylib 0x0000000107688fb1 boost::python::objects::function_call(_object*, _object*, _object*) + 81
19 org.python.python 0x0000000106410411 _PyObject_MakeTpCall + 274
20 org.python.python 0x0000000106412e23 method_vectorcall + 415
21 org.python.python 0x000000010641069d PyVectorcall_Call + 108
22 org.python.python 0x0000000106453c95 slot_tp_init + 96
23 org.python.python 0x00000001064508c2 type_call + 172
24 org.python.python 0x0000000106410411 _PyObject_MakeTpCall + 274
25 org.python.python 0x00000001064aff3e call_function + 804
26 org.python.python 0x00000001064ad0b4 _PyEval_EvalFrameDefault + 27252
27 org.python.python 0x00000001064b08a3 _PyEval_EvalCodeWithName + 1958
28 org.python.python 0x00000001064a65b8 PyEval_EvalCode + 42
29 org.python.python 0x00000001064def15 run_eval_code_obj + 102
30 org.python.python 0x00000001064de1a0 run_mod + 82
31 org.python.python 0x00000001064de2cd pyrun_file + 133
32 org.python.python 0x00000001064dca78 PyRun_SimpleFileExFlags + 646
33 org.python.python 0x00000001064f428d Py_RunMain + 1870
34 org.python.python 0x00000001064f460a pymain_main + 360
35 org.python.python 0x00000001064f465f Py_BytesMain + 42
36 libdyld.dylib 0x00007fff6cb84cc9 start + 1
from pvapy.
SkyToGround
commented on August 25, 2024
As Anders downgraded numpy on the (linux) machine where he encountered the crash, I can not re-produce the stack trace on that machine. However, I can tell you that the location of the crash was the same on that machine.
from pvapy.
SkyToGround
commented on August 25, 2024
I talked to Anders (@Anderspp) and was allowed to update numpy on his machine such that I could get the stack trace. The result was as follows (with use of c++filt):
python3(PyDict_GetItemWithError+0x5)[0x5b4be5]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x9dfec)[0x7f99f69fdfec]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x9e988)[0x7f99f69fe988]
/home/mesi/.local/lib/python3.7/site-packages/numpy/core/_multiarray_umath.cpython-37m-x86_64-linux-gnu.so(+0x1494e5)[0x7f99f6aa94e5]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_numpy37.so.1.72.0(boost::python::numpy::equivalent(boost::python::numpy::dtype const&, boost::python::numpy::dtype const&)+0x1a)[0x7f99f85fc9ba]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_numpy37.so.1.72.0(+0x4368)[0x7f99f85fd368]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::converter::rvalue_from_python_stage1(_object*, boost::python::converter::registration const&)+0x4e)[0x7f99f881b77e]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::updateFieldArrayFromInt(boost::python::api::object const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::tr1::shared_ptr<epics::pvData::Field const>, std::allocator<std::tr1::shared_ptr<epics::pvData::Field const> > >&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x2b)[0x7f99f9dd57bb]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::updateFieldArrayFromDict(boost::python::dict const&, std::vector<std::tr1::shared_ptr<epics::pvData::Field const>, std::allocator<std::tr1::shared_ptr<epics::pvData::Field const> > >&, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&)+0x344)[0x7f99f9dd9ee4]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PyPvDataUtility::createStructureFromDict(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x60)[0x7f99f9dda380]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(PvObject::PvObject(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x37)[0x7f99f9db79b7]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(NtType::NtType(boost::python::dict const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x9)[0x7f99f9da6be9]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(NtScalar::NtScalar(PvType::ScalarType)+0x4a)[0x7f99f9da24ca]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::python::objects::make_holder<1>::apply<boost::python::objects::value_holder<NtScalar>, boost::mpl::vector1<PvType::ScalarType> >::execute(_object*, PvType::ScalarType)+0x4c)[0x7f99f9d5dbac]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void (*)(_object*, PvType::ScalarType), boost::python::default_call_policies, boost::mpl::vector3<void, _object*, PvType::ScalarType> > >::operator()(_object*, _object*)+0x5a)[0x7f99f9d3c40a]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::objects::function::call(_object*, _object*) const+0xf1)[0x7f99f8823ba1]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(+0x1ef08)[0x7f99f8823f08]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::detail::exception_handler::operator()(boost::function0<void> const&) const+0x43)[0x7f99f882e443]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/pvaccess.so(boost::detail::function::function_obj_invoker2<boost::_bi::bind_t<bool, boost::python::detail::translate_exception<PvaException, void (*)(PvaException const&)>, boost::_bi::list3<boost::arg<1>, boost::arg<2>, boost::_bi::value<void (*)(PvaException const&)> > >, bool, boost::python::detail::exception_handler const&, boost::function0<void> const&>::invoke(boost::detail::function::function_buffer&, boost::python::detail::exception_handler const&, boost::function0<void> const&)+0x13)[0x7f99f9d108d3]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(boost::python::handle_exception_impl(boost::function0<void>)+0x2d)[0x7f99f882e21d]
/home/mesi/.local/lib/python3.7/site-packages/pvaccess/lib/linux-x86_64/libboost_python37.so.1.72.0(+0x1d7e3)[0x7f99f88227e3]
python3[0x4d9850]
python3(PyObject_Call+0x56)[0x5dc4f6]
python3[0x5916c8]
python3(_PyObject_FastCallKeywords+0x129)[0x5d9c59]
python3[0x54b3c1]
python3(_PyEval_EvalFrameDefault+0x441d)[0x5524cd]
python3(_PyEval_EvalCodeWithName+0x252)[0x54bcc2]
python3(PyRun_StringFlags+0xaf)[0x5317ef]
python3(PyRun_SimpleStringFlags+0x3d)[0x63177d]
python3[0x654558]
python3(_Py_UnixMain+0x2e)[0x65468e]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7f99fa54b09b]
python3(_start+0x2a)[0x5e0e8a]
from pvapy.
sveseli
commented on August 25, 2024
Thanks for the additional info. I managed to reproduce the problem with python 3.7 and numpy 1.21.0. It looks like the boost::python::extract object results in segfault when it is constructed with a dictionary-like object if one is using numpy>=1.21. The proper fix will have to be made in boost python, but in the meantime, I think I have a viable workaround on the branch bp-extract-fix (commit 19d9728). I would appreciate if you could verify the fix in your environment.
Here is how I built/tested code from sources, using conda.
- Install conda, downgrade python to 3.7, and install numpy 1.21 via pip (conda would install 1.20.x):
$ export TEST_DIR=/local/PVAPY/TEST $ mkdir -p $TEST_DIR $ cd $TEST_DIR $ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh $ sh Miniconda3-latest-Linux-x86_64.sh -b -p $PWD/CONDA $ export PATH=$PWD/CONDA/bin:$PATH $ conda install -y python=3.7 $ pip install numpy==1.21 $ conda list python # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel python 3.7.10 hdb3f193_0 $ conda list numpy # packages in environment at /local/PVAPY/TEST/CONDA: # # Name Version Build Channel numpy 1.21.0 pypi_0 pypi
- Clone pvapy repository, build dependencies locally (epics and boost python):
$ git clone [email protected]:epics-base/pvaPy.git pvapy $ cd pvapy $ mkdir -p $TEST_DIR/opt $ PREFIX=$TEST_DIR/opt make local
- Build pvapy library from sources on the master branch:
$ make configure EPICS_BASE=$TEST_DIR/opt/epics-7.0.5 BOOST_ROOT=$TEST_DIR/opt/boost-python-1.72.0 PYTHON_VERSION=3 $ make
- Verify segfault:
bluegill2> source bin/linux-x86_64/pvapy_setup_full.3.7.sh $ python -c "from pvaccess import *; o = NtScalar(INT); print(o)" Segmentation fault (core dumped) $ python -c "from pvaccess import *; o = NtNdArray(); print(o)" Segmentation fault (core dumped)
- Checkout bp-extract-fix branch, rebuild code:
$ git checkout bp-extract-fix $ make
- Verify that segfault does not occur:
$ python -c "from pvaccess import *; o = NtScalar(INT); print(o)"
epics:nt/NTScalar:1.0
int value 0
string descriptor
structure alarm
int severity 0
int status 0
string message
structure timeStamp
long secondsPastEpoch 0
int nanoseconds 0
int userTag 0
structure display
double limitLow 0
double limitHigh 0
string description
string format
string units
structure control
double limitLow 0
double limitHigh 0
double minStep 0
$ python -c "from pvaccess import *; o = NtNdArray(); print(o)"
epics:nt/NTNDArray:1.0
union value
(none)
structure codec
string name
any parameters
(none)
long compressedSize 0
long uncompressedSize 0
structure[] dimension
int uniqueId 0
structure dataTimeStamp
long secondsPastEpoch 0
int nanoseconds 0
int userTag 0
structure[] attribute
string descriptor
structure alarm
int severity 0
int status 0
string message
structure timeStamp
long secondsPastEpoch 0
int nanoseconds 0
int userTag 0
structure display
double limitLow 0
double limitHigh 0
string description
string format
string units
from pvapy.
basilaljamal
commented on August 25, 2024
try to uninstall the current NumPy and install NumPy 1.20.3
from pvapy.
sveseli
commented on August 25, 2024
Right, numpy 1.20.x works, but this issue is about 1.21.x.
from pvapy.
mnordie
commented on August 25, 2024
Hi, I have a similar problem. Did you open an issue with numpy or boost-python?
I'm on python 3.7 with boost python 1.69 with boost numpy.
from pvapy.
sveseli
commented on August 25, 2024
Hi, sorry, I have not had a chance to do this yet. Do you see the problem with latest pvapy packages? Those should be restricting numpy to 1.20.x until this gets resolved.
from pvapy.
mnordie
commented on August 25, 2024
Sorry, I was unclear. I see this in my own work. But I only found this thread here with any information. Seems to me to be a problem in either numpy, or boost python. Latest numpy still fail for me (1.21.2).
from pvapy.
sveseli
commented on August 25, 2024
It looks like this problem has been corrected in numpy>1.21.5 See boostorg/python#376. In my environment numpy 1.22 works okay.
from pvapy.
sveseli
commented on August 25, 2024
I think this issue can be closed. All recent pip/conda packages avoid numpy 1.21 that was causing the problem.
from pvapy.
Related Issues (20)
- Seg fault with arrays HOT 4
- PvaServer.addRecord not working on Windows 10 HOT 2
- PyPI: pvaPy on Raspberry Pi HOT 11
- Dumping to JSON does not escape strings HOT 4
- Mirror server slow processing large struct arrays
- Error when requesting multiple fields from the same struct HOT 1
- add custom field to NTSCALAR record HOT 1
- Review performance testing HOT 10
- Best method for handling reconnections to large numbers of PVs HOT 7
- Possibility of supporting epicscorelibs for PyPI uploaded wheels HOT 4
- Reading and writing to a single array element of a PVAccess PV HOT 1
- Bug executing`pvget` on unchanged PV in remote server. HOT 15
- Can't install for Python 3.10 on Mac HOT 7
- Unable to install on Macs that use Apple M-series (ARM) chips HOT 5
- pvaPy server listing all fields as 'changed' despite only updating value and timeStamp HOT 4
- Type stubs missing HOT 1
- numpy v2 upgrade causing problems HOT 2
- maximum array size limitation HOT 2
- Processing error: 'Queue' object has no attribute 'size' HOT 2
- EPICS_PVA_NAME_SERVERS: handling reconnection HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pvapy.