cms-gem-daq-project / xhal Goto Github PK

Brief summary of issue

Unable to update lmdb presently on CTP7's.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Should not throw an RPC Not Connected error.
Error message thrown should meaningfully reflect the problem.
It should work.

Current Behavior

From gem904daq01:

CTP7 > connect eagle64
eagle64 > update_lmdb /mnt/persistent/gemdaq/xml/gem_amc_top.xml
Caught NotConnectedException: Not connected to rpc service
LMDB address table updated
eagle64 > kw RELEASE
0x6640000c None 	GEM_AMC.GEM_SYSTEM.RELEASE				
0x6640000c r 	GEM_AMC.GEM_SYSTEM.RELEASE.MAJOR			0x00000003
0x6640000c r 	GEM_AMC.GEM_SYSTEM.RELEASE.MINOR			0x00000005
0x6640000c r 	GEM_AMC.GEM_SYSTEM.RELEASE.BUILD			0x00000002
0x66400010 r 	GEM_AMC.GEM_SYSTEM.RELEASE.DATE				0x20180803

Clearly RPC connection is present. Also trying readFW which is defined in the same python file as update_lmdb works so I don't think it's an issue of the rpc conncetion pointer this time?

Same issue also occurs on gem904qc8daq:

eagle60 > update_lmdb /mnt/persistent/gemdaq/xml/gem_amc_top.xml
Caught NotConnectedException: Not connected to rpc service
LMDB address table updated
eagle60 > kw RELEASE
0x6640000c None         GEM_AMC.GEM_SYSTEM.RELEASE
0x6640000c r    GEM_AMC.GEM_SYSTEM.RELEASE.MAJOR                        0x00000003
0x6640000c r    GEM_AMC.GEM_SYSTEM.RELEASE.MINOR                        0x00000005
0x6640000c r    GEM_AMC.GEM_SYSTEM.RELEASE.BUILD                        0x00000002
0x66400010 r    GEM_AMC.GEM_SYSTEM.RELEASE.DATE                         0x20180803

Steps to Reproduce (for bugs)

See above.

Using system installation path of libraries and executables in each case.

Context (for feature requests)

Unable to update lmdb.

Your Environment

• Version used: Honestly not sure how to check this anymore...
• Shell used: zsh

Brief summary of issue

Tried checkout release v3.0.0 from xhal following procedure described at:

https://github.com/cms-gem-daq-project/xhal/wiki/Quick-start-guide

And was unsuccessful.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Looks like the problem is just one of missing libraries that were mistakenly not included in the release.

Expected Behavior

Documentation should work out of the box. Required libraries should be included in release.

Current Behavior

Required libraries are missing in the release.

Steps to Reproduce (for bugs)

From: https://github.com/cms-gem-daq-project/xhal/wiki/Quick-start-guide

1. git checkout tags/v3.0.0
2. python .github/get_binaries -t v3.0.0 -l .github/uploads.cfg This generates an error since get_binaries is a typo.
3. When correcting the typo:

(default)[gemuser@gem904qc8daq xhal]$ python .github/get_binaries.py -t v3.0.0 -l .github/uploads.cfg 
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/ipbus
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxhal.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/librwreg.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libwiscrpcsvc.so
HTTP Error: 404 https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxhal_ctp7.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/libxerces-c.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/liblmdb.so
downloading https://github.com/cms-gem-daq-project/xhal/releases/download/v3.0.0/liblog4cplus.so

The 404 errors are generated because release v3.0.0 does not have those libraries listed:

https://github.com/cms-gem-daq-project/xhal/releases/tag/v3.0.0

Indeed they are missing in the release.

4. When trying to run reg_interface a crash occurs since necessary libraries are not found:

(default)[gemuser@gem904qc8daq reg_interface]$ python reg_interface.py 
Traceback (most recent call last):
  File "reg_interface.py", line 4, in <module>
    from rw_reg import *
  File "/home/gemuser/gemdaq/xhal/python/reg_interface/rw_reg.py", line 17, in <module>
    lib = CDLL(os.getenv("XHAL_ROOT")+"/lib/x86_64/librpcman.so")
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 360, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: libwiscrpcsvc.so: cannot open shared object file: No such file or directory

Possible Solution (for bugs)

Step two above should be: python .github/get_binaries.py -t <tag_name> -l .github/uploads.cfg

Right now I'll just build the required libraries myself but I think this probably defeats the purpose of the release.

Context (for feature requests)

User will complain it doesn't work.

Your Environment

• Version used: https://github.com/cms-gem-daq-project/xhal/releases/tag/v3.0.0
• Shell used: /bin/bash

Is rw_reg.py up-to-date with the latest in GEM_AMC
I know there were some updates when the address tables were adjusted, and you made some in your version.
We should strive to make sure that these files are the same, probably, and only modified in one repository

Brief summary of issue

This repository contains many branches used for past long-lasting developments and quick hot-fixes. I propose to remove all the branches which are no longer useful.

Here is the list of branches I'll delete on Monday except if someone has strong reasons against it. All these branches have been merged in this repository or another GEM repository if the code was moved during refactoring.

• bdorney-patch-1: References a file which no longer exists in this repository. It has been decided not to merge the PR #76
• hotfix/fix-exceptions: Merged in #101
• hotfix/legacy-block-nodes: Merged in #107
• hotfix/simplify-makefiles: Originally in #129, bt then included in #125

Here is the list of branches which I think can safely be deleted, but for which I cannot find the merge commit. Are they still necessary? @mexanick? You created most of them.

• develop-python_package_struct: Cannot find the merge commit. Doesn't seem required anymore to package the Python tools.
• mexanick-hotfix-update_atdb: Not needed anymore. Python imports work correctly in python/reg_interface_gem/core/ri_prompt_extended.py.
• revert-26-feature_rpcOHFunctionality: Not sure why we would want to revert the feature since it works and it is used.
• revert-44-feature_currentPulse: Not sure why we would want to revert the feature since it works and it is used.

Now, about the active branches:

• feature/templated-rpc-methods has become the development branch and contains the only development in progress. It could be merged into develop to simplify the workflow, especially since the two branches already diverged and that the first iteration of the templated code is done.
• legacyPreBoost has technically been renamed release/legacy-1.0 and should be deleted after developer-wide announcement.

Brief summary of issue

Testing compilation of cmsgemos on cc7 with xdaq15 beta release results in a failure because current xhal can't be installed:

Error: Package: xhal-3.2.2-1.0.102.dev.4adab12git.centos7.gcc4_8_5.x86_64 (gemos-extras)
           Requires: liblog4cplus-1.2.so.5()(64bit)

Compiling current xhal fails with errors:

src/common/utils/XHALXMLParser.cpp:10:35: error: no matching function for call to ‘log4cplus::Appender::setLayout(std::auto_ptr<log4cplus::Layout>&)’
   myAppender->setLayout( myLayout );
                                   ^
In file included from /opt/xdaq/include/log4cplus/spi/appenderattachable.h:33,
                 from /opt/xdaq/include/log4cplus/logger.h:36,
                 from /home/sturdy/src/sw/xhal/xhalcore/include/xhal/utils/XHALXMLParser.h:39,
                 from src/common/utils/XHALXMLParser.cpp:1:
/opt/xdaq/include/log4cplus/appender.h:216:22: note: candidate: ‘virtual void log4cplus::Appender::setLayout(std::unique_ptr<log4cplus::Layout>)’
         virtual void setLayout(std::unique_ptr<Layout> layout);
                      ^~~~~~~~~
/opt/xdaq/include/log4cplus/appender.h:216:22: note:   no known conversion for argument 1 from ‘std::auto_ptr<log4cplus::Layout>’ to ‘std::unique_ptr<log4cplus::Layout>’
src/common/utils/XHALXMLParser.cpp: In member function ‘boost::optional<xhal::utils::Node> xhal::utils::XHALXMLParser::getNodeFromAddress(uint32_t)’:
src/common/utils/XHALXMLParser.cpp:359:1: warning: no return statement in function returning non-void [-Wreturn-type]
 }

Types of issue

• Compatibility breaking change

Expected Behavior

Will probably need to set up builds for

• cc7 with default gcc (4.8.5), for "legacy" (14.6) xdaq
• cc7 with devtoolset-8 gcc (8.2.1), for xdaq15
• and when available, the same for cc8 (not sure what the default compiler will be yet)

Current Behavior

Unable to compile xhal for xdaq15 testing, and hence unable to compile cmsgemos

Context (for feature requests)

For compatibility with future development.

Your Environment

cc7 VM with xdaq15 prerelease installed (gemdaq-build-xdaq15)

Brief summary of issue

On the P5 machine I've come across a few problems trying to use the v3 electronics. The release instructions are not successful see updated comments in issue #55.

I have been using the cc7 virtual machine gemvm-daqcc7

After compiling all libraries on gem904qc8daq (os: cc7) and placing them in:

$XHAL_ROOT/lib/arm
$XHAL_ROOT/lib/x86_64

Trying to run reg_interface.py seems to generate the following import errors:

ImportError: No module named lxml.etree
ImportError: No module named cmd

The vm mentioned and the cc7 DAQ machines in 904 seem to use python v2.7.5 while the daq machines in 904 seem to have these packages installed by default?

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

We should have a requirements file to assist with system installation.

Current Behavior

No requirements file is present.

Possible Solution (for bugs)

Add a requirements file.

Context (for feature requests)

Takes awhile to setup the code or a new release.

Your Environment

• Version used: b4980d8
• Shell used: /bin/bash

running outputnode command on a register without a mask crashes the reg_interface (probably just a simple check of "is None" is needed).

can be reproduced e.g. like this:

CTP7 > outputnode GEM_AMC.GEM_SYSTEM.CTRL.LINK_RESET
Name: GEM_AMC.GEM_SYSTEM.CTRL.LINK_RESET
Description: Reset the links to and from OHs and redo VFAT3 sync procedure
Address: 0x00900101
Permission: w
Mask:
Traceback (most recent call last):
  File "/mnt/persistent/texas/apps/reg_interface/reg_interface.py", line 379, in <module>
    prompt.cmdloop('Starting CTP7 Register Command Line Interface.')
  File "/usr/lib/python2.7/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "/mnt/persistent/texas/apps/reg_interface/reg_interface.py", line 17, in do_outputnode
    print node.output()
  File "/mnt/persistent/texas/apps/reg_interface/rw_reg.py", line 49, in output
    print 'Mask:','{0:#010x}'.format(self.mask)
ValueError: Unknown format code 'x' for object of type 'str'

Also please rename the outputnode command to something easier, like help or doc.

@jsturdy , it looks like I don't have the write right for this repo and thus can't merge PR.
Could you please fix this ( I think this happened when I transferred ownership...)?

Using reg_interface.py from the CTP7 when an optical link is not connected results in a Bus Error, which prompts a message memHandle NULL, open it, this second message prints only on the output of the second command.
This message should not be printed to the screen, and if it needs to be logged, should be put into an error/logfile.
Using the reg_interface.py from the host machine this seems to not be the case, but should be verified.

Brief summary of issue

Since the dependencies change I've introduced have been reverted, now when trying to install a following problem appears

...
--> Finished Dependency Resolution
Beginning Kernel Module Plugin
Finished Kernel Module Plugin
Error: Package: xhal-server-4.0.0-1.0.441.dev.94435d1git.centos7.gcc8_2_1.x86_64 (gemos-test)
           Requires: xhal-base
Error: Package: xhal-server-4.0.0-1.0.441.dev.94435d1git.centos7.gcc8_2_1.x86_64 (gemos-test)
           Requires: xerces-c
Error: Package: xhal-client-4.0.0-1.0.441.dev.94435d1git.centos7.gcc8_2_1.x86_64 (gemos-test)
           Requires: xhal-base
Error: Package: xhal-client-4.0.0-1.0.441.dev.94435d1git.centos7.gcc8_2_1.x86_64 (gemos-test)
           Requires: xerces-c
 You could try using --skip-broken to work around the problem

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Should be able to install without any issues

Current Behavior

Apparently xhal-base package is not created at all but listed as a dependency. libxhal-base.so is indeed created, but packages as xhal.
Another (potentially bigger issue) is while xdaq is tied to a particular version of xerces-c and includes it, it is omitted in provides list:

yum provides xerces-c
Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, tsflags, versionlock
Loading mirror speeds from cached hostfile
github_git-lfs                                                                                                                                                                 58/58
github_git-lfs-source                                                                                                                                                          55/55
256 packages excluded due to repository protections
libxerces-c-3_1-3.1.1-2.1.el7.cern.x86_64 : Shared library for Xerces-C++ validating XML parser
Repo        : cern
Matched from:
Provides    : xerces-c = 3.1.1-2.1.el7.cern
libxerces-c-3_1-3.1.4-1.1.el7.cern.x86_64 : Shared library for Xerces-C++ validating XML parser
Repo        : cern
Matched from:
Provides    : xerces-c = 3.1.4-1.1.el7.cern
xerces-c-3.1.1-7.el7_1.i686 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-7.el7_1.x86_64 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-8.el7_2.i686 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-8.el7_2.x86_64 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-9.el7.i686 : Validating XML Parser
Repo        : base
xerces-c-3.1.1-9.el7.i686 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-9.el7.x86_64 : Validating XML Parser
Repo        : base
xerces-c-3.1.1-9.el7.x86_64 : Validating XML Parser
Repo        : updates
xerces-c-3.1.1-10.el7_7.i686 : Validating XML Parser
Repo        : updateserces-c-3.1.1-10.el7_7.x86_64 : Validating XML Parser
Repo        : updates

and as as consequence, we have:

yum info xerces-c
Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, tsflags, versionlock
Loading mirror speeds from cached hostfile
256 packages excluded due to repository protections
Available Packages
Name        : xerces-c
Arch        : i686
Version     : 3.1.1
Release     : 10.el7_7
Size        : 889 k
Repo        : updates/7/x86_64
Summary     : Validating XML Parser
URL         : http://xml.apache.org/xerces-c/
License     : ASL 2.0
Description : Xerces-C is a validating XML parser written in a portable
            : subset of C++. Xerces-C makes it easy to give your application the
            : ability to read and write XML data. A shared library is provided for
            : parsing, generating, manipulating, and validating XML
            : documents. Xerces-C is faithful to the XML 1.0 recommendation and
            : associated standards: XML 1.0 (Third Edition), XML 1.1 (First
            : Edition), DOM Level 1, 2, 3 Core, DOM Level 2.0 Traversal and Range,
            : DOM Level 3.0 Load and Save, SAX 1.0 and SAX 2.0, Namespaces in XML,
            : Namespaces in XML 1.1, XML Schema, XML Inclusions).

Name        : xerces-c
Arch        : x86_64
Version     : 3.1.1
Release     : 10.el7_7
Size        : 879 k
Repo        : updates/7/x86_64
Summary     : Validating XML Parser
URL         : http://xml.apache.org/xerces-c/
License     : ASL 2.0
Description : Xerces-C is a validating XML parser written in a portable
            : subset of C++. Xerces-C makes it easy to give your application the
            : ability to read and write XML data. A shared library is provided for
            : parsing, generating, manipulating, and validating XML
            : documents. Xerces-C is faithful to the XML 1.0 recommendation and
            : associated standards: XML 1.0 (Third Edition), XML 1.1 (First
            : Edition), DOM Level 1, 2, 3 Core, DOM Level 2.0 Traversal and Range,
            : DOM Level 3.0 Load and Save, SAX 1.0 and SAX 2.0, Namespaces in XML,
            : Namespaces in XML 1.1, XML Schema, XML Inclusions).

it is not listed as installed, so in case one has the repos above enabled, another copy of xerces-c will be installed during installation of xhal.
After some investigation found an interesting case of ...hmm... rebranding?

yum provides cmsos-core-xerces
Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, tsflags, versionlock
Loading mirror speeds from cached hostfile
256 packages excluded due to repository protections
cmsos-core-xerces-3.2.0.2-1.r15.centos7.gcc8.x86_64 : C++ is a validating XML parser written in a portable subset of C++.
Repo        : cmsos-core
cmsos-core-xerces-3.2.0.2-2.r15.centos7.gcc8.x86_64 : C++ is a validating XML parser written in a portable subset of C++.
Repo        : cmsos-core
cmsos-core-xerces-3.2.0.2-2.r15.centos7.gcc8.x86_64 : C++ is a validating XML parser written in a portable subset of C++.
Repo        : @cmsos-core

Steps to Reproduce (for bugs)

1. build xhal rpms
2. copy them to the repository
3. run yum install xhal\*

Possible Solution (for bugs)

Brief summary of issue

Need to implement the RPC call wrapper to use the recently added SCA ADC methods of ctp7_modules, in particular for the temperature readings. Target branch: legacyPreBoost

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Read and return the ADC values for various sensors embedded on the GEM front-end electronics

Current Behavior

None

Brief summary of issue

There's a bug when writing the output phase results.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

The output file for subsequent OH's in ohMask should not overwrite the previous one.

Current Behavior

If gbtPhaseScan is provided with a non-None outputFile then saveGBTPhaseScanResults will overwrite the phase scans for all OH's but the last one in ohMask.

Steps to Reproduce (for bugs)

1. Call saveGBTPhaseScanResults with an ohMask with at least 2 unmasked bits

Possible Solution (for bugs)

Either saveGBTPhaseScanResults opens the output file in read/write mode (file is appended) rather than write (in which case the file is overwritten); or one file is made per unmasked OH.

Context (for feature requests)

Trying to track phase scan results but can only see the last one...

Your Environment

• Version used: 3015276
• Shell used:

It would be nice to have a function that offers the possibility to "broadcast" a write to all/some OptoHybrids

Brief summary of issue

memhub uses semaphores to prevent concurrent access to the CTP7 memory. It tries hard to avoid leaving active semaphores behind, but all these efforts are moot if the process gets killed. This is a caveat of semaphores themselves.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

A dying process should release all resources it holds.

Current Behavior

When a process gets killed (SIGKILL) in the middle of a memhub call, it leaves behind an active semaphore. The next process trying to use memhub gets stuck.

Steps to Reproduce (for bugs)

1. Introduce a delay in memhub_read right after the call to memsvc_read
2. Call memsvc_read (eg through an RPC call)
3. While the process is waiting, kill -9 it
4. Call memsvc_read again

Possible Solution

Since this is caused by a caveat in the semaphores API, one has to find another way to synchronize multiple processes. There are basically two ways apart from semaphores:

• Use a pthread mutex in a shared memory region. This would be hard to get right for our use case.
• Create a temporary file (eg /tmp/memhub.lock) and use the advisory locking API. Advisory file locks are released when a process is killed. I have a working prototype.

Context

Want to avoid a CTP7 getting stuck and requiring manual intervention.

Your Environment

• Version used: any
• Shell used: any

Brief summary of issue

After killing the ipbus daemon on the CTP7, the program cannot be restarted immediately but one has to wait some time. During that period the bind sycall fails with the "address in use" error. It can be seen that the netstat tool shows TCP connections in the LAST_ACK state which progressively timeout.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

The ipbus daemon should not keep TCP connections open once the client disconnects.

Current Behavior

While the daemon is running closed TCP connections are blocked in the CLOSE_WAIT state and the ipbus daemon consumes one full CTP7 CPU core after the first TCP connection is closed by the client.

Steps to Reproduce (for bugs)

1. Start the ipbus daemon on the CTP7
2. Connect to and disconnect from the CTP7 IPBus endpoint (using testConnectivity.py for example)
3. Look a the CTP7 CPU usage (e.g. using top)
4. Look a the non-closed TCP connection on the CTP7 (e.g. netstat -an; when writing this issue, 377 such connections are opened on eagle23 used for QC7)

Possible Solution (for bugs)

When the recv call returns 0 the socket must be closed:

Also @mexanick, I was wondering why you implemented this commit? If it was because connections were impossible once the maximum number of client was reached, fixing this issue should have the same effect.

Your Environment

• Version used: d1f8b3d

Brief summary of issue

While working on #126 I noticed that two version of memhub exist. One in xhal/xcompile which does not use the LOGGER and is named libmemhub.so, the other one in the ctp7_modules which uses the LOGGER and is named memhub.so.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Only one version of memhub should exist.

Once and if the ctp7_module logging system is converted to log4cplus (see this issue cms-gem-daq-project/ctp7_modules#139), my proposal would be to use it in memhub and only provide memhub once in xhal.

At the same time memhub could be converted to a complete library which abstracts all calls to libmemsvc and provides a class to manage the memsvc handle lifetime. The API would also be cleaner if everything is managed by memhub instead of open and close function in libmemsvc.

Current Behavior

Two versions of memhub exist.

Context (for feature requests)

Maintainability and good practices.

Brief summary of issue

Trying the sysmon command generates an IndexError error.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

It should work not generate an IndexError.

Current Behavior

Following behavior was reported:

sca.py eagle63 0x2 sysmon
---- Setting JTAG CLK frequency to 10MHz (divider value = 0x1000000) ---- 
Initial value to write: 19, register GEM_AMC.SLOW_CONTROL.SCA.MANUAL_CONTROL.SCA_CMD.SCA_CMD_CHANNEL
Initial value to write: 144, register GEM_AMC.SLOW_CONTROL.SCA.MANUAL_CONTROL.SCA_CMD.SCA_CMD_COMMAND
Initial value to write: 4, register GEM_AMC.SLOW_CONTROL.SCA.MANUAL_CONTROL.SCA_CMD.SCA_CMD_LENGTH
Initial value to write: 16777216, register GEM_AMC.SLOW_CONTROL.SCA.MANUAL_CONTROL.SCA_CMD.SCA_CMD_DATA
Initial value to write: 1, register GEM_AMC.SLOW_CONTROL.SCA.MANUAL_CONTROL.SCA_CMD.SCA_CMD_EXECUTE
Traceback (most recent call last):
  File "/home/gemuser/DAQ/xhal/python/reg_interface/sca.py", line 709, in <module>
    main()
  File "/home/gemuser/DAQ/xhal/python/reg_interface/sca.py", line 128, in main
    coreTemp = ((adc1[oh] >> 6) & 0x3FF) * 503.975 / 1024.0-273.15
IndexError: list index out of range

Steps to Reproduce (for bugs)

1. setup xhal env
2. sca.py eagle63 0x2 sysmon

Context (for feature requests)

Should not generate a crash.

Your Environment

• Version used: 3.2.2
• Shell used: /bin/bash

Brief summary of issue

Attempting to update the LMDB fails.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Should not throw a name error.

Current Behavior

Throws a name error:

CTP7 > connect eagle64
eagle64 > help update_lmdb
Updates LMDB address table at the CTP7. USAGE: update_lmdb <absolute path name to the AMC xml address table at the CTP7>
eagle64 > update_lmdb /mnt/persistent/gemdaq/xml/gem_amc_top.xml
Traceback (most recent call last):
  File "/opt/xhal/bin//gem_reg.py", line 31, in <module>
    prompt.cmdloop_with_keyboard_interrupt()
  File "/usr/lib/python2.7/site-packages/reg_utils/reg_interface/common/ri_prompt.py", line 42, in cmdloop_with_keyboard_interrupt
    self.cmdloop()
  File "/usr/lib64/python2.7/cmd.py", line 142, in cmdloop
    stop = self.onecmd(line)
  File "/usr/lib64/python2.7/cmd.py", line 221, in onecmd
    return func(arg)
  File "/usr/lib/python2.7/site-packages/xhal/reg_interface_gem/core/ri_prompt_extended.py", line 260, in do_update_lmdb
    update_atdb(args)
NameError: global name 'update_atdb' is not defined

Steps to Reproduce (for bugs)

1. export LD_LIBRARY_PATH=/opt/xdaq/lib:$LD_LIBRARY_PATH
2. export LD_LIBRARY_PATH=/opt/wiscrpcsvc/lib:$LD_LIBRARY_PATH
3. export LD_LIBRARY_PATH=/opt/rwreg/lib:$LD_LIBRARY_PATH
4. export LD_LIBRARY_PATH=/opt/xhal/lib:$LD_LIBRARY_PATH
5. export PATH=/opt/xhal/bin/:$PATH
6. export PATH=/opt/reg_utils/bin:$PATH
7. gem_reg.py
8. connect eagle64
9. update_lmdb /mnt/persistent/gemdaq/xml/gem_amc_top.xml

Possible Solution (for bugs)

Perhaps I am missing some setup command for LD_LIBRARY_PATH?

Context (for feature requests)

Unable to update lmdb or debug rpc connection issues.

Your Environment

• Version used: on 904 daq 01 machine

% yum info xhal
Loaded plugins: changelog, fastestmirror, kernel-module, langpacks, protectbase, tsflags, versionlock
Loading mirror speeds from cached hostfile
epel                                                                                                                                                                                                                  12608/12608
791 packages excluded due to repository protections
Installed Packages
Name        : xhal
Arch        : x86_64
Version     : 1.0.0
Release     : centos7
Size        : 1.6 M
Repo        : installed
Summary     : None
URL         : None
License     : __license_
Description : None

• Shell used: /bin/zsh

(Cannot open an issue in the right repository because the functionnality is disabled by default for forks.)

Brief summary of issue

Currently, the only GEM compatible reedmuller-c release in hosted in a personal GitHub repository, jsturdy/reedmuller-c.

A small inconvenience is that issues cannot currently be created in this repository.

Moe importantly, as any other GEM online software related library, it should be owned by the cms-gem-daq-project organization. This is particularly true since that specific library release is mandatory to compile the RPC modules.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

reedmuller-c should be hosted in the cms-gem-daq-project GitHub organization.

Current Behavior

reedmuller-c is hosted in a personal GitHub repository.

Possible Solution (for bugs)

Transfer the ownership of jsturdy/reedmuller-c to the cms-gem-daq-project organization.

Context (for feature requests)

Architecture/infrastructure clean up.

Brief summary of issue

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

The two scans should give the same value of rate vs. scan register (bandwidth considerations not accounted for).

Current Behavior

As shown:

http://cmsonline.cern.ch/cms-elog/1028798

The series and the parallel sbit scans produce different results

Steps to Reproduce (for bugs)

1. Execute: sbitThreshScanParallel.py --shelf=3 -s5 -g0 --vfatmask=0x30 --scanmin=0 --scanmax=256 --stepSize=1
2. Execute: sbitThreshScanSeries.py --shelf=3 -s5 -g0 --vfatmask=0xfffffe --scanmin=0 --scanmax=254 --stepSize=1 --time=1000
3. Analyze the two differtent outputs and compare VFAT0

Possible Solution (for bugs)

The case is the following:

1. There is a bug in the CTP7 FW v3.3.6,
2. There is a bug in the OH FW v3.0.9.A, or
3. There is a bug in sbitRateScanParallelLocal(...) or the corresponding python script sbitThreshScanParallel.py

I will investigate item 3 and update the community.

Context (for feature requests)

Wrong set point

Your Environment

• Version used: 2236dc2
• Shell used: /bin/zsh

Brief summary of issue

Following request from @mexanick.

Based on discussion at camp last week @evka and I discussed how calibration routines could be increased in speed by decreasing the number of RPC calls per routine. I think the simplest implementation would be to provide an interface to genChannelScan.

Types of issue

• Bug report (report an issue with the code)
• (Last) Feature request (request for change which adds functionality)

Expected Behavior

The function would be as genScan but instead take the following arguments:

DLLEXPORT uint32_t genChannelScan(uint32_t nevts, uint32_t ohN, uint32_t dacMin, uint32_t dacMax, uint32_t dacStep, bool useCalPulse, bool currentPulse, uint32_t calScaleFactor, uint32_t mask, char * scanReg, bool useUltra, bool useExtTrig, uint32_t * result)

Here I've just dropped the chan argument.

The format of the input array pointer result would follow the index schema below:

https://github.com/cms-gem-daq-project/ctp7_modules/blob/b358ed18fd77ed0625b5f04082df7d19da2d7ff6/src/calibration_routines.cpp#L1326-L1330

Current Behavior

No interface exists.

Context (for feature requests)

Decrease time for calibration scans by decreasing the number of RPC calls per scan.

Brief summary of issue

Installing the most recent xhal developments with the updated build system (branch feature/templated-rpc-methods) in a local path leads to an unexpected filesystem structure.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

I'm not 100% sure what is the expected behavior, but the local installation seems to be wrong.

Current Behavior

The paths for the cross-compiled libraries in the local installation folder are highly suspicious, i.e.

[lpetre@gem904daq04 local_install]$ tree -L 4
.
├── afs
│   └── cern.ch
│       └── user
│           └── l
├── mnt
│   └── persistent
│       ├── reedmuller
│       │   ├── bin
│       │   ├── etc
│       │   ├── include
│       │   ├── lib
│       │   └── scripts
│       └── xhal
│           ├── bin
│           ├── etc
│           ├── include
│           ├── lib
│           └── scripts
├── opt
│   ├── gem-peta-stage
│   │   └── ctp7
│   │       ├── lib
│   │       └── usr
│   └── xhal
│       ├── bin
│       │   └── reg_interface_gem
│       ├── etc
│       ├── include
│       │   ├── packageinfo.h
│       │   └── xhal
│       ├── lib
│       │   ├── libxhal-base.so -> libxhal-base.so.3.2
│       │   ├── libxhal-base.so.3.2 -> libxhal-base.so.3.2.2
│       │   ├── libxhal-base.so.3.2.2
│       │   ├── libxhal-client.so -> libxhal-client.so.3.2
│       │   ├── libxhal-client.so.3.2 -> libxhal-client.so.3.2.2
│       │   ├── libxhal-client.so.3.2.2
│       │   ├── libxhal-rpcman.so -> libxhal-rpcman.so.3.2
│       │   ├── libxhal-rpcman.so.3.2 -> libxhal-rpcman.so.3.2.2
│       │   ├── libxhal-rpcman.so.3.2.2
│       │   ├── libxhal-server.so -> libxhal-server.so.3.2
│       │   ├── libxhal-server.so.3.2 -> libxhal-server.so.3.2.2
│       │   ├── libxhal-server.so.3.2.2
│       │   ├── xhalpy.so -> xhalpy.so.3.2
│       │   ├── xhalpy.so.3.2 -> xhalpy.so.3.2.2
│       │   └── xhalpy.so.3.2.2
│       └── scripts
└── usr
    ├── lib
    │   ├── debug
    │   │   ├── mnt
    │   │   └── opt
    │   └── python2.7
    │       └── site-packages
    └── src
        └── debug
            ├── reedmuller-..
            └── xhal-3.2.2

42 directories, 16 files

Both the afs and mnt directories contain the arm libraries built for the CTP7. Moreover, mnt contains the header files for development.

I do not expect an afs folder to be created, nor the mnt folder to be outside of the PETA_STAGE directory.

Steps to Reproduce (for bugs)

1. Export the INSTALL_PREFIX environment variable (e.g. export INSTALL_PREFIX=/afs/cern.ch/user/l/lpetre/GEM/develop/local_install)
2. Export the PETA_PATH environment variable (e.g. export PETA_PATH=${INSTALL_PREFIX}/opt/gem-peta-stage)
3. Build the project: scl enable devtoolset-8 -- make
4. Install the project: scl enable devtoolset-8 -- make install
5. Look into the local installation directory.

Context (for feature requests)

Trying to debug the templated RPC code/build/installation.

Your Environment

• Version used: d465fe5

Brief summary of issue

As discussed in cms-gem-daq-project/ctp7_modules#161, there are several missing type (de)serializers.

This is one of the remaining pieces for a released xhal in the land of the templated framework.

Types of issue

• Feature request (request for change which adds functionality)

Expected Behavior

(de)serializers should be provided for all "common" types

Missing are:

• std::vector<T>
• bool
• float
• uintX_t for certain X (8 and 16 at least)
• possible std::unordered_map<K,T>

Brief summary of issue

$XHAL_ROOT/python/reg_interface/ is part of $PATH however reg_interface.py is not executable. Quality of life feature, request it to be executable.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Not needing to go to $XHAL_ROOT/python/reg_interface/ and execute python reg_interface.py and instead just type reg_interface.py from anywhere.

Current Behavior

See above comment.

Context (for feature requests)

Quality of live feature

Your Environment

• Version used: develop
• Shell used: /bin/zsh

Have sca.py take command line argument for which CTP7 it should connect to, right now it looks like it's hardcoded to eagle34.

Brief summary of issue

Sometimes, one types a command and wants to quit before 12 OH links information is printed.
Pressing ctrl+c will quit reg_interface

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Typing ctrl+c or something equivalent will stop printing to the screen the remaining messages (on the host PC this printing is rather slow)

Current Behavior

ctrl+c kills reg_interface

Steps to Reproduce (for bugs)

1. start reg_interface
2. type readKW GEM_AMC.DAQ
3. see the interesting stuff at the top and then realize that you don't care about the link specific stuff
4. press ctrl+c
5. cry

Possible Solution (for bugs)

• Cross session command history could be considered a "workaround"

Brief summary of issue

Counterpart too:

cms-gem-daq-project/ctp7_modules#129

But should target legacyPreBoost branch of this repo (xhal).

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Add following to:

https://github.com/cms-gem-daq-project/xhal/blob/legacyPreBoost/xhalcore/src/common/rpc_manager/calibration_routines.cc

And corresponding header file under xhalcore/include/...

DLLEXPORT uint32_t confCalPulse(uint32_t ohN, uint32_t mask, uint32_t ch, bool toggleOn, bool currentPulse, uint32_t calScaleFactor){
    req = wisc::RPCMsg("calibration_routines.confCalPulse");

    req.set_word("ohN", ohN);
    req.set_word("ch", ch);
    req.set_word("mask", mask);
    req.set_word("toggleOn",toggleOne);
    req.set_word("currentPulse", currentPulse);
    req.set_word("calScaleFactor", calScaleFactor);

    wisc::RPCSvc* rpc_loc = getRPCptr();

    try {
        rsp = rpc_loc->call_method(req);
    }
    STANDARD_CATCH;


    if (rsp.get_key_exists("error")) {
        printf("Caught an error: %s\n", (rsp.get_string("error")).c_str());
        return 1;
    }

    return 0;
} //End confCalPulse()

Current Behavior

No method exists.

Context (for feature requests)

See cms-gem-daq-project/ctp7_modules#129

Brief summary of issue

Unable to issue an SCA reset from the CTP7. Previous functionality allowed this.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Older version of the sca.py tool (e.g. see eagle60) worked from CTP7 without specifying a hostname.

Current Behavior

When attempting to issue an SCA reset from CTP7 without supplying a hostname the help menu is displayed:

eagle61:~$ sca.py 0x3 r                                                                     
Usage: sca.py <card_name> <oh_mask> <instructions>
instructions:
  r:        SCA reset will be done
  h:        FPGA hard reset will be done
  hh:       FPGA hard reset will be asserted and held
  fpga-id:  FPGA ID will be read through JTAG
  sysmon:   Read FPGA sysmon data repeatedly
  program-fpga:   Program OH FPGA with a bitfile or an MCS file. Requires a parameter "bit" or "mcs" and a filename

When providing a hostname a NameError is returned:

eagle61:~$ sca.py eagle61 0x3 r
Open pickled address table if available  /mnt/persistent/gemdaq/xml/gem_amc_top.pickle...
Traceback (most recent call last):
  File "/mnt/persistent/gemdaq/python/reg_interface/sca.py", line 670, in <module>
    main()
  File "/mnt/persistent/gemdaq/python/reg_interface/sca.py", line 71, in main
    rpc_connect(sys.argv[1])
NameError: global name 'rpc_connect' is not defined

Steps to Reproduce (for bugs)

1. Login to eagle61
2. Attempt to issue an SCA reset.

Context (for feature requests)

Cannot program the OH FPGA via JTAG through SCA without issuing an SCA reset. Previously functionality allowed this (unsure which commit because of the over-writing between @evka85 and @mexanick).

Your Environment

• Version used: cac5edf
• Shell used: /bin/sh

Brief summary of issue

To properly handle the new BLASTER RAM, the XML parser (and anything that uses the address table) needs to be able to know the mode and potentially size of any node

Types of issue

• Feature request (request for change which adds functionality)

Expected Behavior

Downstream of the parsing

• if a block transaction is requested on a non-block node, this should result in an error (or at least a warning)
• if a block transaction is requested on a block node, but the requested size is larger than the advertised size, this should result in an error

Current Behavior

No current knowledge about blocks in the parsing/low-level transaction methods.

Brief summary of issue

Release checkout procedure leaves a binary missing.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

All binaries should be present when checking out a release.

Current Behavior

Missing the librpcman.so library.

Steps to Reproduce (for bugs)

1. git clone https://github.com/cms-gem-daq-project/xhal.git
2. cd xhal
3. git checkout tags/3.1.0
4. source setup.sh
5. python .github/get_binaries.py -t 3.1.0 -l .github/uploads.cfg
6. reg_interface.py

Step 6 produces the following error:

$ reg_interface.py 
Traceback (most recent call last):
  File "/home/testbeam/DAQ/xhal/python/reg_interface/reg_interface.py", line 4, in <module>
    from rw_reg import *
  File "/home/testbeam/DAQ/xhal/python/reg_interface/rw_reg.py", line 18, in <module>
    lib = CDLL(os.getenv("XHAL_ROOT")+"/lib/x86_64/librpcman.so")
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 360, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/testbeam/DAQ/xhal/lib/x86_64/librpcman.so: cannot open shared object file: No such file or directory

Indeed the library is not present:

$ ls  /home/testbeam/DAQ/xhal/lib/x86_64/librpcman.so
ls: cannot access /home/testbeam/DAQ/xhal/lib/x86_64/librpcman.so: No such file or directory

Seems it is not present on the release page:

https://github.com/cms-gem-daq-project/xhal/releases/tag/3.1.0

Possible Solution (for bugs)

Add library to the release page or instructions to the Quick Start Guide for how to obtain this library.

Context (for feature requests)

Expect all binaries to be present when checking out a release.

Your Environment

• Version used: cac5edf (HEAD detached at 3.1.0)
• Shell used: /bin/bash

Brief summary of issue

Various Makefile in this project can't handle more than one cpp file.

Steps to reproduce

git checkout d1f8b3d8efa7693418773a923f4c2388129130fb # current develop
make # works
echo "int test() { return 0; }" > xhalcore/common/utils/test.cpp
make # broken

Error message for xhalarm is:

arm-linux-gnueabihf-gcc -fomit-frame-pointer -pipe -fno-common -fno-builtin -Wall -std=c++14 -march=armv7-a -mfpu=neon -mfloat-abi=hard -mthumb-interwork -mtune=cortex-a9 -DEMBED -Dlinux -D__linux__ -Dunix -fPIC --sysroot=/data/bigdisk/sw/peta-stage -I/data/bigdisk/sw/peta-stage/usr/include -I/data/bigdisk/sw/peta-stage/include -std=gnu++14 -I/afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/include -c -o src/linux/arm/utils/XHALXMLParser.o /afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/src/common/utils/test.cpp
arm-linux-gnueabihf-gcc -std=gnu++14 -L/data/bigdisk/sw/peta-stage/lib -L/data/bigdisk/sw/peta-stage/usr/lib -L/data/bigdisk/sw/peta-stage/ncurses -shared  -o /afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalarm/lib/libxhal.so src/linux/arm/utils/test.o src/linux/arm/utils/XHALXMLParser.o -llog4cplus -lxerces-c -lstdc++
src/linux/arm/utils/XHALXMLParser.o: In function `test()':
test.cpp:(.text+0x0): multiple definition of `test()'
src/linux/arm/utils/test.o:test.cpp:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status

If I don't try to compile xhalarm, I get a failure in xhalcore:

mkdir -p src/linux/x86_64/utils
gcc -O0 -g3 -fno-inline -Wall -pthread -fPIC -std=c++11 -m64 -I/usr/include/python2.7 -I/opt/xdaq/include -I/afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/include -c src/common/utils/test.cpp -o src/linux/x86_64/utils/test.o
mkdir -p src/linux/x86_64/utils
gcc -O0 -g3 -fno-inline -Wall -pthread -fPIC -std=c++11 -m64 -I/usr/include/python2.7 -I/opt/xdaq/include -I/afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/include -c src/common/utils/test.cpp -o src/linux/x86_64/utils/XHALXMLParser.o
mkdir -p /afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/lib
gcc -fPIC -std=c++11 -m64 -shared -L/opt/xdaq/lib -L/opt/wiscrpcsvc/lib -o /afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/lib/libxhal.so src/linux/x86_64/utils/test.o src/linux/x86_64/utils/XHALXMLParser.o src/linux/x86_64/XHALDevice.o src/linux/x86_64/XHALInterface.o -llog4cplus -lxerces-c -lwiscrpcsvc -lstdc++
gcc: error: src/linux/x86_64/utils/test.o: No such file or directory
make[1]: *** [/afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore/lib/libxhal.so] Error 1
make[1] : leaving directory "/afs/cern.ch/work/l/lmoureau/GEM/test/xhal/xhalcore"
make: *** [xhalcore] Error 2

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Integrating new source files is seamless.

Current Behavior

Can't understand how to add new source files.

Possible Solution (for bugs)

[troll mode on] Switch over to CMake? [troll mode off]

Context (for feature requests)

Working on build system for #125

Your Environment

• Version used: develop
• Shell used: bash

In setting up scripts capable of being able to set up a CTP7 using a specific release, it came up that the release tagging is not consistent. A list of recent releases is:

• 2.1.0
• v2.0.0
• v1.0.0

The workaround is that the user will have to specify v2.0.0 or 2.1.0, but ideally, the user would specify only x.y.z and the v (if part of the semantic versioning) would be handled by the script

Brief summary of issue

Ran into a problem when launching reg_interface.py on the card. Not sure if this is due to an outdated version or a real bug.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Should be able to launch reg_interface.py no problem.

Current Behavior

eagle60:/mnt/persistent/gemdaq/xml$ reg_interface.py 
Traceback (most recent call last):
  File "/mnt/persistent/gemdaq/python/reg_interface/reg_interface.py", line 15, in <module>
    from ri_prompt_extended import *
  File "/mnt/persistent/gemdaq/python/reg_interface/ri_prompt_extended.py", line 1, in <module>
    import ri_prompt
  File "/mnt/persistent/gemdaq/python/reg_interface/ri_prompt.py", line 2, in <module>
    from rw_reg import *
  File "/mnt/persistent/gemdaq/python/reg_interface/rw_reg.py", line 10, in <module>
    lib = CDLL(os.getenv("GEM_PATH")+"/lib/librwreg.so")
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /mnt/persistent/gemdaq/lib/librwreg.so: file too short

Steps to Reproduce (for bugs)

1. Login eagle60
2. Launch reg_interface.py

Context (for feature requests)

reg_interface.py from the card should work.

Your Environment

• Version used: ??? not sure if we have a way to tell this on the card ???
• Shell used: /bin/sh

Brief summary of issue

When the tools take an argument to connect to some hardware, they should be able to take any hostname that will work, as such the current implementation of the sca.py parsing does not pass this bar, as it assumes an eagleXY connection

Brief summary of issue

While one should not use git add . I think the .gitignore needs to be updated. For example it is not setup to ignore temporary files, e.g.:

Untracked files:
  (use "git add <file>..." to include in what will be committed)

	include/xhal/rpc/optohybrid.h
	src/common/rpc_manager/.vfat3.cc.swp

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Temporary files should not be known to git.

Additionally, I think it would be good to adopt a common .gitignore structure in all our repo's. Looking at:

• cmsgemos .gitignore,
• gem-plotting-tools .gitignore
• vfatqc-python-scripts .gitignore

They seem to have the following structure:

# Ignore everything
*

# Don't ignore git settings
!*.gitignore
!*.gitattributes

# Don't ignore special files
!*.md
!*.rst

# Don't ignore subdirectories
!*/

# Don't ignore special files to this repo that don't fit in the above
<Files/reg expressions>

Current Behavior

Temporary files are known to git.

Brief summary of issue

Right now setup.sh is dependent on the file path, e.g. you need to:

cd $BUILD_HOME/xhal
source setup.sh

Suggest to change L1 too:

#export XHAL_ROOT=<your path>/xhal
if [[ -n "$XHAL_ROOT" ]]; then
    echo XHAL_ROOT $XHAL_ROOT
else
    echo "XHAL_ROOT not set, please set XHAL_ROOT to the directory above the root of your repository"
    echo "(export XHAL_ROOT=<your path>/xhal) and then rerun this script"
    return
fi

This will require the user to setup $XHAL_ROOT themselves but will make the script location independent and conform to the other setup scripts in our project.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Brief summary of issue

The librwreg has to be modified such that it can call the method update_atdb from ctp7 module utils

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

We need to be able to call update_lmdb at the CTP7

Current Behavior

Currently update_lmdb can only be called from CTP7

Context (for feature requests)

We want to update the lmdb address table every time the FW loading scrips are called.

Your Environment

• Version used:
• Shell used:

Brief summary of issue

Unable to program FPGA using JTAG through the SCA. Previous functionality allowed this.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Older version of the sca.py tool allowed programming FPGA without issue from CTP7 (see eagle60).

Current Behavior

When attempting to program the FPGA's using JTAG via SCA from CTP7 a ValueError is returned:

eagle61:~$ sca.py 0x3 program-fpga bit /mnt/persistent/gemdaq/oh_fw/optohybrid_top.bit
Traceback (most recent call last):
  File "/mnt/persistent/gemdaq/python/reg_interface/sca.py", line 670, in <module>
    main()
  File "/mnt/persistent/gemdaq/python/reg_interface/sca.py", line 64, in main
    ohMask = parseInt(sys.argv[2])
  File "/mnt/persistent/gemdaq/python/reg_interface/rw_reg.py", line 357, in parseInt
    return int(string)
ValueError: invalid literal for int() with base 10: 'program-fpga'

Steps to Reproduce (for bugs)

1. Log into eagle61
2. Attempt to program FPGA's using above command

Context (for feature requests)

Cannot program the OH FPGA via JTAG through SCA. Previously functionality allowed this (unsure which commit because of the over-writing between @evka85 and @mexanick).

Your Environment

• Version used: cac5edf
• Shell used: /bin/sh

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

User should be able to call sca reset and related functions from the reg_interface CLI
However reg_interface should also work for other functionality in case sca module is not available

Current Behavior

sca.py is a standalone script

Context (for feature requests)

reg_interface should stay generic as it might be used by projects other then GEM

Your Environment

• Version used:
• Shell used:

Brief summary of issue

Right now the readKW functionality is very useful and allows reading registers matching keyword from all links. Would like to request that this be expanded to allow reading a keyword for just a specific link, e.g. for reading VThreshold1 from all VFATs on link 0 you could execute:

readKW VThreshold1 0

Or more generally:

readKW <keyword> <link #>

And then be given the output. If no link number is specified then it defaults to current behavior where all links are displayed.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

See syntax example above.

Current Behavior

Right now the readKW functionality just queries all links, which may present a large wall of text to the user, and they may only be interested in a specific link.

Context (for feature requests)

Quality of life improvement.

Brief summary of issue

Based on format of other repositories would expect:

cd $BUILD_HOME/xhal
source setup.sh
make

To compile the repository and correctly create the required shared libraries. However when attempting obtain the following error message when calling make:

% make
g++ -O0 -g3 -fno-inline -Wall -fPIC -pthread -m64 -std=gnu++14 -I/opt/xdaq/include -I/opt/rh/devtoolset-6/root/usr/include -I/afs/cern.ch/user/d/dorney/scratch0/CMS_GEM/CMS_GEM_DAQ/xhal/xhal/include -L/opt/xdaq/lib -L/afs/cern.ch/user/d/dorney/scratch0/CMS_GEM/CMS_GEM_DAQ/xhal/xhal/lib/x86_64 -llog4cplus -lxerces-c  -c -o src/common/utils/XHALXMLParser.o src/common/utils/XHALXMLParser.cpp
cc1plus: error: unrecognized command line option "-std=gnu++14"
make: *** [src/common/utils/XHALXMLParser.o] Error 1

Unclear which g++ version should be used. Based on this link it should be either v4.9 or v5.X.

Request that setup.sh correctly configures this by default, and anything else that would required to successfully compile.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Able to compile the repo after calling setup.sh

Current Behavior

Cannot compile the repo.

Context (for feature requests)

Would be helpful when debugging/testing/developing python based scan applications for v3 electronics if we could compile xhal and make tweaks as needed (which would result in new PR's).

Your Environment

• Version used: b7f1812
• Shell used: /bin/zsh

Brief summary of issue

Currently LMDB address table is located under /mnt/persistent/texas
The possible hardcoded hooks should be identified and removed, the address table should go somewhere under $GEM_PATH

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)
• Bad coding style

Context (for feature requests)

Affects:
https://github.com/cms-gem-daq-project/ctp7_modules
https://github.com/cms-gem-daq-project/gemctp7user

Your Environment

• Version used:
• Shell used:

Brief summary of issue

Currently the library is written entirely in C-style with method exported to C-style DLLs via extern C. No classes or namespaces are used per se (however, the language is still C++ and certain parts are c++ code, like wisc::RPCSvc and wisc::RPCMsg). For the ease of use the library should be refactored to provide namespaces and classes. This can be implemented by writing a classes with all the current methods and provide DLLEXPORT wrappers around them for compatibility with python code, like

namespace a {
  class B {
  public:
    cpp_method(){printf("Hello World");}
  };
}
extern "C" c_method(a::B* b)
{
  b->cpp_method();
}

In this case it won't require major refactoring of python code using the xHAL. The other option (requiring more work) is to rewrite everything in regular C++ and use boost_python to provide python wrappers. However this will call upon changes in our python repositories as well.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Your Environment

• Version used:
• Shell used:

Brief summary of issue

I believe the current implementation of RPC methods genScan(...) and thresholdScan(...) will be more difficult to integrate with our existing repositories and use and cause unnecessary file I/O operations.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)
• Discussion

Expected Behavior

The two scan applications genScan(...) and thresholdScan(...) should allow for a pointer of the results array to be supplied as an input argument rather than be a local variable.

This should allow an array in python to be passed with the library is called and then the scan data is stored in the array. Then integration with the current scan tools (and subsequent analysis) would be seamless, e.g.:

if v3El:
    genScan(scanData, ....)
else:
    scanData = getUltraScanResults(ohboard, options.gtx, THRESH_MAX - THRESH_MIN + 1, options.debug)

Then scanData can be parsed accordingly inside the ultra scan script and be neatly stored into the current output TTree.

I think this will offer a more seamless integration with the current vfatqc-python-scripts. As shown in my original PR: https://github.com/cms-gem-daq-project/xhal/pull/13/files. From scanDAC of my original PR:

DLLEXPORT uint32_t scanDAC(uint32_t* result, ssize_t size,
        unsigned int ohN, unsigned int vfatMask, unsigned int chan, std::string scanReg,
        unsigned int nEvts, unsigned int scanMin, unsigned int scanMax, unsigned int stepSize){
    req = wisc::RPCMsg("calibration_routines.genScan");
    req.set_word("ohN",ohN);
    req.set_word("mask",vfatMask);
    req.set_word("ch",chan);
    req.set_word("scanReg",scanReg);
    req.set_word("nevts",nEvts);
    req.set_word("dacMin",scanMin);
    req.set_word("dacMax",scanMax);
    req.set_word("dacStep",stepSize);


    try{
        rsp = rpc.call_method(req);
    }
    STANDARD_CATCH;


    try{
	    if (rsp.get_key_exists("error")) {
            printf("Error: %s",rsp.get_string("error").c_str());
            return 1;
        }
        else {
	        ASSERT(rsp.get_word_array_size("data") == size);
            rsp.get_word_array("data", result);
        }
    }
	STANDARD_CATCH;


    return 0;
} //End scanDAC()

Here the results pointer is an input.

Current Behavior

In PR https://github.com/cms-gem-daq-project/xhal/pull/15/files new scan functions were added:

• genScan(...)
• thresholdScan(...)

However in these functions uint32_t* result is a local variable. I think this will makes integration with vfatqc-python-scripts very challenging. Right now ultra scan results are expecting a return value, take vfatqc-python-scripts/ultraThreshold.py:

            scanData = getUltraScanResults(ohboard, options.gtx, THRESH_MAX - THRESH_MIN + 1, options.debug)
            sys.stdout.flush()
            for i in range(0,24):
            	if (mask >> i) & 0x1: continue
                vfatN[0] = i
                dataNow      = scanData[i]
                trimRange[0] = (0x07 & readVFAT(ohboard,options.gtx, i,"ContReg3"))
                for VC in range(THRESH_MAX-THRESH_MIN+1):
                    vth1[0]  = int((dataNow[VC] & 0xff000000) >> 24)
                    vth[0]   = vth2[0] - vth1[0]
                    Nhits[0] = int(dataNow[VC] & 0xffffff)
                    myT.Fill()
                    pass
                pass
            myT.AutoSave("SaveSelf")
            pass

Here scanData is obtained at each iteration of the loop over all channels.

However, genScan(...) and thresholdScan(...) open an output text file and write the data for the (vfat,ch) pair to it. This implies that to scan over all channels 128 sets of file I/O operations will need to be done and seems a bit excessive. Additionally trying to organize and use this data in the gem-plotting-tools/ is not straight forward.

I think we can strive for a more seamless integration with lower changes.

Context (for feature requests)

The current implementation does not allow for direct access of the scan results inside a scan application and instead causes ~128 sets of file I/O operations.

I'd like to open a discussion on the advantages/disadvantages and the long term strategy of this.

Brief summary of issue

Starting an issue based on discussion in #131 regarding structure of the package.

Types of issue

• Request for comment (discussion to come to a consensus about something)

Current Behavior

Currently, xhal is a package that provides libraries for both PC and AMC.
The libraries have some differences in the symbols, and the arm libraries (and any arm specific headers) need to be present on the PC during development for linking during the cross compilation step.
The current mechanism to do this ensures that the devel RPM package is only built after the arm package is compiled.
It then puts the arm library into the RPMBUILD tree in a xhal/lib/arm folder that can be included in the linker path when compiling for arm on x86_64
Now that there are also headers that have only been placed in the arm package, these would also need some special packaging, possibly a similar copy mechanism into include/arm/xhal

This issue is designed to investigate possible alternate solutions and adopt a uniform policy for any other package that will also provide libraries/headers across architectures (ctp7_modules will be one that only provides the headers)

Brief summary of issue

In order to build e.g., ctp7_modules, the arm libraries from xhal are required to be linked against.
The best way I can think of to accomplish this, would be to add them to the spec template for the xhal-devel package. Then xhal-devel is a build time requirement of ctp7_modules

Types of issue

• Feature request (request for change which adds functionality)

Expected Behavior

Would be best to not have to have 2-3 external packages required as source just to build.

Current Behavior

The build process expects that xhal is checked out in the same working tree as, e.g., ctp7_modules

It would be nice to have a feature to connect multiple CTP7s from within reg_interface.py.
Similar to how AMC13Tool2.exe has the possibility to connect and select different cards from within the CLI

Brief summary of issue

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Current Behavior

After the xhal update on eagle60 I see the following errors when attempting to program the GBT0 link:

eagle60:~/apps/reg_interface$ python gbt.py 0 0 config ~/gbt/GBTX_0_20171120_124635.txt 

See the attached text file for the full list. Example:

Initial value to write: 356, register GEM_AMC.SLOW_CONTROL.IC.ADDRESS
Writing masked reg GEM_AMC.SLOW_CONTROL.IC.ADDRESS failed. Exiting...
wReg output 6

Interestingly the diodes blink on the OHv3a following the usual behavior (e.g. it moves from D6->D13->D6). It is possible to read registers that GBT0 is associated with:

eagle60 > write GEM_AMC.GEM_SYSTEM.CTRL.LINK_RESET 1
Initial value to write: 1, register GEM_AMC.GEM_SYSTEM.CTRL.LINK_RESET
0x00000001(1)   written to GEM_AMC.GEM_SYSTEM.CTRL.LINK_RESET
eagle60 > kw LINK_GOOD 0
0x65800440 r    GEM_AMC.OH_LINKS.OH0.VFAT0.LINK_GOOD                    0x00000001
0x65800448 r    GEM_AMC.OH_LINKS.OH0.VFAT1.LINK_GOOD                    0x00000001
0x65800450 r    GEM_AMC.OH_LINKS.OH0.VFAT2.LINK_GOOD                    0x00000001
0x65800458 r    GEM_AMC.OH_LINKS.OH0.VFAT3.LINK_GOOD                    0x00000001
0x65800460 r    GEM_AMC.OH_LINKS.OH0.VFAT4.LINK_GOOD                    0x00000000
0x65800468 r    GEM_AMC.OH_LINKS.OH0.VFAT5.LINK_GOOD                    0x00000000
0x65800470 r    GEM_AMC.OH_LINKS.OH0.VFAT6.LINK_GOOD                    0x00000001
0x65800478 r    GEM_AMC.OH_LINKS.OH0.VFAT7.LINK_GOOD                    0x00000001
0x65800480 r    GEM_AMC.OH_LINKS.OH0.VFAT8.LINK_GOOD                    0x00000001
0x65800488 r    GEM_AMC.OH_LINKS.OH0.VFAT9.LINK_GOOD                    0x00000001
0x65800490 r    GEM_AMC.OH_LINKS.OH0.VFAT10.LINK_GOOD                   0x00000001
0x65800498 r    GEM_AMC.OH_LINKS.OH0.VFAT11.LINK_GOOD                   0x00000001
0x658004a0 r    GEM_AMC.OH_LINKS.OH0.VFAT12.LINK_GOOD                   0x00000001
0x658004a8 r    GEM_AMC.OH_LINKS.OH0.VFAT13.LINK_GOOD                   0x00000001
0x658004b0 r    GEM_AMC.OH_LINKS.OH0.VFAT14.LINK_GOOD                   0x00000001
0x658004b8 r    GEM_AMC.OH_LINKS.OH0.VFAT15.LINK_GOOD                   0x00000001
0x658004c0 r    GEM_AMC.OH_LINKS.OH0.VFAT16.LINK_GOOD                   0x00000001
0x658004c8 r    GEM_AMC.OH_LINKS.OH0.VFAT17.LINK_GOOD                   0x00000001
0x658004d0 r    GEM_AMC.OH_LINKS.OH0.VFAT18.LINK_GOOD                   0x00000001
0x658004d8 r    GEM_AMC.OH_LINKS.OH0.VFAT19.LINK_GOOD                   0x00000001
0x658004e0 r    GEM_AMC.OH_LINKS.OH0.VFAT20.LINK_GOOD                   0x00000001
0x658004e8 r    GEM_AMC.OH_LINKS.OH0.VFAT21.LINK_GOOD                   0x00000001
0x658004f0 r    GEM_AMC.OH_LINKS.OH0.VFAT22.LINK_GOOD                   0x00000001
0x658004f8 r    GEM_AMC.OH_LINKS.OH0.VFAT23.LINK_GOOD                   0x00000001
eagle60 > kw CFG_RUN 0
0x65400c00 rw   GEM_AMC.OH.OH0.GEB.VFAT0.CFG_RUN                        0x00000000
0x65402c00 rw   GEM_AMC.OH.OH0.GEB.VFAT1.CFG_RUN                        0x00000000
0x65404c00 rw   GEM_AMC.OH.OH0.GEB.VFAT2.CFG_RUN                        0x00000000
0x65406c00 rw   GEM_AMC.OH.OH0.GEB.VFAT3.CFG_RUN                        0x00000000
0x65408c00 rw   GEM_AMC.OH.OH0.GEB.VFAT4.CFG_RUN                        0xdeaddead
0x6540ac00 rw   GEM_AMC.OH.OH0.GEB.VFAT5.CFG_RUN                        0xdeaddead
0x6540cc00 rw   GEM_AMC.OH.OH0.GEB.VFAT6.CFG_RUN                        0x00000000
0x6540ec00 rw   GEM_AMC.OH.OH0.GEB.VFAT7.CFG_RUN                        0x00000000
0x65410c00 rw   GEM_AMC.OH.OH0.GEB.VFAT8.CFG_RUN                        0x00000000
0x65412c00 rw   GEM_AMC.OH.OH0.GEB.VFAT9.CFG_RUN                        0x00000000
0x65414c00 rw   GEM_AMC.OH.OH0.GEB.VFAT10.CFG_RUN                       0x00000000
0x65416c00 rw   GEM_AMC.OH.OH0.GEB.VFAT11.CFG_RUN                       0x00000000
0x65418c00 rw   GEM_AMC.OH.OH0.GEB.VFAT12.CFG_RUN                       0x00000000
0x6541cc00 rw   GEM_AMC.OH.OH0.GEB.VFAT14.CFG_RUN                       0x00000000
0x6541ec00 rw   GEM_AMC.OH.OH0.GEB.VFAT15.CFG_RUN                       0x00000000
0x65420c00 rw   GEM_AMC.OH.OH0.GEB.VFAT16.CFG_RUN                       0x00000000
0x65422c00 rw   GEM_AMC.OH.OH0.GEB.VFAT17.CFG_RUN                       0x00000000
0x65424c00 rw   GEM_AMC.OH.OH0.GEB.VFAT18.CFG_RUN                       0x00000000
0x65426c00 rw   GEM_AMC.OH.OH0.GEB.VFAT19.CFG_RUN                       0x00000000
0x65428c00 rw   GEM_AMC.OH.OH0.GEB.VFAT20.CFG_RUN                       0x00000000
0x6542ac00 rw   GEM_AMC.OH.OH0.GEB.VFAT21.CFG_RUN                       0x00000000
0x6542cc00 rw   GEM_AMC.OH.OH0.GEB.VFAT22.CFG_RUN                       0x00000000
0x6542ec00 rw   GEM_AMC.OH.OH0.GEB.VFAT23.CFG_RUN                       0x00000000
0x6541ac00 rw   GEM_AMC.OH.OH0.GEB.VFAT13.CFG_RUN                       0x00000000

So trying to understand if this is an issue. But this additional text, e.g.

Initial value to write: 356, register GEM_AMC.SLOW_CONTROL.IC.ADDRESS
Writing masked reg GEM_AMC.SLOW_CONTROL.IC.ADDRESS failed. Exiting...

Did not appear before xhal on eagle60 was updated.

Steps to Reproduce (for bugs)

1. Login to eagle60
2. Execute: cd apps/reg_interface
3. Power on v3 electronics
4. Execute: python gbt.py 0 0 config ~/gbt/GBTX_0_20171120_124635.txt

Context (for feature requests)

Your Environment

• Version used: develop
• Shell used: /bin/sh

Brief summary of issue

The CTP7 and OHv3a each have their own xml file which defines the address table. See for example:

https://github.com/evka85/GEM_AMC/releases/tag/v3.2.6
https://github.com/andrewpeck/OptoHybridv3/releases/tag/3.0.7.A

Right now for the XML to be parsed you have to copy/paste the OHv3a xml into the CTP7 xml which is a bit tedious.

Would like that parseXML() is changed to work with two files instead of one.

Types of issue

• Bug report (report an issue with the code)
• Feature request (request for change which adds functionality)

Expected Behavior

Simply making the links (e.g. gem_amc_top.xml and gem_oh_top.xml) for the two xml's in the appropriate location parseXML() should use both of them.

If gem_oh_top.xml is not found it should print a warning but continue parsing the xml to be backwards compatible with older FW versions/v2b electronics.

Current Behavior

You have to copy/paste the OHv3a xml into the CTP7 xml.

Context (for feature requests)

Quality of life improvement.

Brief summary of issue

Here are some starters to get persistent command history in python, possibly able to adapt to a python CLI too