paulscherrerinstitute / adzmq Goto Github PK

View Code? Open in Web Editor NEW

1.0 4.0 2.0 3.26 MB

ZMQ ADDriver and NDPlugin for areaDetector

Makefile 5.85% Python 2.38% C++ 69.48% C 17.37% Batchfile 4.93%

areadetector zeromq epics

adzmq's Introduction

ADZMQ

This project attempts to integrate ZeroMQ with areaDetector framework, in two folds:

a driver pulls data from a ZeroMQ server and generates NDArray.
a plugin publishes NDArray as ZeroMQ server.

The ZeroMQ message format is detailed here.

Build

If you build under PSI environment,

    cd psiBuild
    make

Otherwise, use the standard EPICS base system. Create a file configure/RELEASE.local with your build environment settings,

SUPPORT=C:/epics/synApps_5_7/support/
EPICS_BASE=C:/epics/base-3.14.12-VC12

# asyn
ASYN=$(SUPPORT)/asyn-4-22
BUSY=$(SUPPORT)/busy-1-6
CALC=$(SUPPORT)/calc-3-2
SSCAN=$(SUPPORT)/sscan-2-9
AUTOSAVE=$(SUPPORT)/autosave-5-1
# AREA_DETECTOR is needed for base and plugins
AREA_DETECTOR=C:/epics/areaDetector
ADSUPPORT=$(AREA_DETECTOR)/ADSupport
ADCORE=$(AREA_DETECTOR)/ADCore

And then run

    make

Test

After building, the examplar IOC zmqIOC is created. Follow the test procedure,

Start the IOC,

cd iocBoot/iocZMQ
../../bin/linux-x86_64/zmqApp st.cmd

Start the ZeroMQ server. It creates an image of 800x600 of int8 type. The update frequency is 1 Hz.
```
cd tests
python zmq_server.py
```

Launch MEDM panel.

medm -x -macro P=13ZMQ1:,R=cam1: ADBase.adl

Start acquisition and observe the image counter increases.

Now enable the plugin to publish NDArray as ZeroMQ server.

caput 13ZMQ1:cam1:ArrayCallbacks 1
caput 13ZMQ1:ZMQ1:EnableCallbacks 1

Launch a ZeroMQ client to receive data.
```
cd tests
python zmq_client.py
```

The data flow is illustrated by the following graph,

adzmq's People

Contributors

Stargazers

Watchers

Forkers

darrent1974 mustafa-zubi

adzmq's Issues

ADZMQ build order on linux-x86_64

When I build synApps areaDetector, I always have do it three time to succeed.
The first time it will fail due to ADZMQ failure (or not yet built), which causes ADAravis driver to fail.
Then I have to build ADZMQ alone, which will succeeded by itself.
Finally build areaDetector again, with ADZMQ already built, then everything will succeeded.

I think I must have the build order incorrectly specified.

Any idea what I need to do to fix this problem?

NDPluginZMQ performance benchmark

Here is an attempt to establish a baseline of NDPluginZMQ (version <= 1.1) performance.

Environment

Hardware: i5-3427U CPU @ 1.80GHz, 8GB RAM
OS: macOS 10.15, clang 12
asyn: 4.39
ADCore: 3.9
ADSimDetector: 2.10

Image Size: 1024 x 1024
Data type: int8
Exposure time: 0.000
AcquirePeriod: 0.000
# Images: 10000
Image mode: Multiple

Results

The baseline is that when 13SIM1:cam1:ArrayCallbacks is "Disable", 13SIM1:cam1:ArrayRate_RBV is ~3400. This is maximum speed of image creation.

PUB/SUB

When no clients are connected, which means ZeroMQ discards all messages, the 13SIM1:cam1:ArrayRate_RBV is ~2000. And there is no dropped arrays 13SIM1:ZMQ1:DroppedArrays_RBV = 0.

13SIM1:ZMQ1:BlockingCallbacks = Yes
13SIM1:cam1:ArrayRate_RBV is ~ 800. All images reached the client.
13SIM1:ZMQ1:BlockingCallbacks = No
When one client is connected, 13SIM1:cam1:ArrayRate_RBV is ~1000. 37% of the images did not reach the client. Note that the additional loss by ZeroMQ (76003 - 63329 = 12674).

13SIM1:cam1:ArrayCounter_RBV  100000
13SIM1:ZMQ1:ArrayCounter_RBV  76003
13SIM1:ZMQ1:DroppedArrays_RBV 23997
Received by client: 63329

PUSH/PULL

When no clients are connected, ZeroMQ blocks the process. All images are dropped.

13SIM1:ZMQ1:BlockingCallbacks = Yes
13SIM1:cam1:ArrayRate_RBV is ~ 800. All images reached the client
13SIM1:ZMQ1:BlockingCallbacks = No
When one client is connected, 13SIM1:cam1:ArrayRate_RBV is ~1000. 30% of the images did not reach the client.

13SIM1:cam1:ArrayCounter_RBV   100000
13SIM1:ZMQ1:ArrayCounter_RBV    70728
13SIM1:ZMQ1:DroppedArrays_RBV   29272
Received by client:             70728

build ADSimDetector and ANZMQ into docker image

Hi , I want to install ADZMQ with ADSimDetector, and I am using linux alpine, epics-base:3.14.12.6 for building areadetector, ADSimDetector and ADZMQ into a docker image.

I followed https://areadetector.github.io/master/install_guide.html to install everything except ADZMQ, I built areadetector and ADSimDetector successfully, but when I added ADZMQ into the dockerfile, I got an error as following when running docker build

#31 584.2 make[6]: *** No rule to make target '../../../lib/linux-x86_64/libADZMQ.a', needed by 'simDetectorApp'. Stop.
#31 584.2 make[5]: *** [/EPICS/base/configure/RULES_ARCHS:61: install.linux-x86_64] Error 2
#31 584.2 make[4]: *** [/EPICS/base/configure/RULES_DIRS:83: src.install] Error 2
#31 584.2 make[3]: *** [/EPICS/base/configure/RULES_DIRS:83: simDetectorApp.install] Error 2
#31 584.2 make[2]: *** [/EPICS/base/configure/RULES_DIRS:83: simDetectorIOC.install] Error 2
#31 584.2 make[1]: *** [/EPICS/base/configure/RULES_DIRS:83: iocs.install] Error 2
#31 584.2 make: *** [/EPICS/base/configure/RULES_DIRS:83: /EPICS/support/areaDetector/ADSimDetector.install] Error 2

my steps for adding ANZMQ is as following:

create RELEASE.local and put into ADZMQ/configure/RELEASE.local

SUPPORT=/EPICS/support
EPICS_BASE=/EPICS/base

# asyn
ASYN=$(SUPPORT)/asyn
BUSY=$(SUPPORT)/busy
CALC=$(SUPPORT)/calc
SSCAN=$(SUPPORT)/sscan
AUTOSAVE=$(SUPPORT)/autosave

# AREA_DETECTOR is needed for base and plugins
AREA_DETECTOR=/EPICS/support/areaDetector
ADBINARIES=$(AREA_DETECTOR)/ADBinaries
ADCORE=$(AREA_DETECTOR)/ADCore

change arch to linux in ADZMQ/iocs/zmqIOC/iocBoot/iocZMQ/Makefile
ARCH = linux-x86_64
add ADZMQ inside areaDetector/Makefile

ifdef ADZMQ
DIRS := $(DIRS) $(ADZMQ)
$(ADZMQ)_DEPEND_DIRS += $(ADCORE)
endif

add ADZMQ in areaDetector/configure/RELEASE_PRODS.local

# ADZMQ  
ADZMQ=$(AREA_DETECTOR)/ADZMQ

add ADZMQ in areaDetector/ADSimDetector/iocs/simDetectorIOC/configure/RELEASE
ADZMQ=$(AREA_DETECTOR)/ADZMQ
add ADZMQ in areaDetector/ADSimDetector/iocs/simDetectorIOC/simDetectorApp/src/Makefile

PROD_LIBS += ADZMQ

ifeq ($(OS_CLASS),WIN32)
LIBZMQ = libzmq
else
LIBZMQ = zmq
endif
PROD_LIBS += $(LIBZMQ)

add ADZMQ in areaDetector/ADSimDetector/iocs/simDetectorIOC/iocBoot/iocSimDetector/st_base.cmd

# add ADZMQ
NDZMQConfigure("NDZMQ1", "tcp://*:1234", 3, 0, "$(PORT)", 0)
dbLoadRecords("$(ADCORE)/ADApp/Db/NDPluginBase.template","P=$(PREFIX),R=ZMQ1:,PORT=NDZMQ1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=$(PORT),NDARRAY_ADDR=0")
 
doAfterIocInit "dbpf 13SIM1:ZMQ1:EnableCallbacks,1"

add ADZMQ in areaDetector/Makefile

ifdef ADZMQ
DIRS := $(DIRS) $(ADZMQ)
$(ADZMQ)_DEPEND_DIRS += $(ADCORE)
endif

am I missing something? Do I need to make ANZMQ before SimDetector? Cause if I don't change the configurations related to simdetector, I can build the image successfully, and the files in /EPICS/support/areaDetector/ADZMQ/lib/linux-x86_64 are libADZMQ.a libADZMQ.so libzmq.a.

does this driver work with EPICS base 7.xx?

Looks like you are using EPICS 3.14.
I got linker error when tried to compile with EPICS 7.0.4:

DPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) protected: virtual enum asynStatus __cdecl NDPluginDriver::setArrayInterrupt(int)" (_imp?setArrayInterrupt@NDPluginDriver@@Meaa?AW4asynStatus@@h@Z)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) protected: virtual enum asynStatus __cdecl NDPluginDriver::connectToArrayPort(void)" (_imp?connectToArrayPort@NDPluginDriver@@Meaa?AW4asynStatus@@xz)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) protected: virtual enum asynStatus __cdecl NDPluginDriver::endProcessCallbacks(class NDArray *,bool,bool)" (_imp?endProcessCallbacks@NDPluginDriver@@Meaa?AW4asynStatus@@PEAVNDArray@@_N1@Z)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) protected: virtual void __cdecl NDPluginDriver::beginProcessCallbacks(class NDArray *)" (_imp?beginProcessCallbacks@NDPluginDriver@@MEAAXPEAVNDArray@@@z)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: virtual enum asynStatus __cdecl NDPluginDriver::start(void)" (_imp?start@NDPluginDriver@@UEAA?AW4asynStatus@@xz)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: virtual void __cdecl NDPluginDriver::run(void)" (_imp?run@NDPluginDriver@@UEAAXXZ)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: virtual void __cdecl NDPluginDriver::driverCallback(struct asynUser *,void *)" (_imp?driverCallback@NDPluginDriver@@UEAAXPEAUasynUser@@Peax@Z)
NDPluginZMQ.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: virtual enum asynStatus __cdecl NDPluginDriver::readInt32Array(struct asynUser *,int *,unsigned __int64,unsigned __int64 *)" (_imp?readInt32Array@NDPluginDriver@@UEAA?AW4asynStatus@@PEAUasynUser@@PEAH_KPEA_K@Z)
Hint on symbols that are defined and could potentially match:

IOC start up with error: Command NDZMQConfigure not found

I compiled ADZMQ into an area detector. When I start the IOC, I get this error:

==============================================
.......
NDZMQConfigure("NDZMQ1", "tcp://*:1234", 3, 0, "SP1", 0)
st.cmd.base line 59: Command NDZMQConfigure not found.
dbLoadRecords("/opt/epics/base-7.0.4/../modules/synApps_6_1_epics7/support/areaDetector-R3-7/ADCore/ADApp/Db/NDPluginBase.template","P=13SP1:,R=ZMQ1:,PORT=NDZMQ1,ADDR=0,TIMEOUT=1,NDARRAY_PORT=SP1,NDARRAY_ADDR=0")
doAfterIocInit ("dbpf 13SP1:ZMQ1:EnableCallbacks,1")
st.cmd.base line 61: Command doAfterIocInit not found.
iocInit()
Starting iocInit
....

==============================================
The driver module I'm using is ADSpinnaker, for a FLIR camera.
thanks for the help.