Comments (4)
rosenbaumalex
commented on August 15, 2024
Thanks, we'll take a look.
I'm guessing you addess ip address and solved it for your setup.
Anyway for you so send us a full VMA_TRACELEVEL=4 log file?
This will provide us with most of the information we need to fully reproduce and fix this.
from libvma.
commented on August 15, 2024
cat /tmp/vma_log.txt
Pid: 6210 Tid: 6210 VMA INFO : ---------------------------------------------------------------------------
Pid: 6210 Tid: 6210 VMA INFO : VMA_VERSION: 7.0.9-0 Development Snapshot built on Oct 28 2015 22:33:30
Pid: 6210 Tid: 6210 VMA INFO : Cmd Line: ./a.out
Pid: 6210 Tid: 6210 VMA INFO : Current Time: Mon Nov 2 10:03:56 2015
Pid: 6210 Tid: 6210 VMA INFO : Pid: 6210
Pid: 6210 Tid: 6210 VMA INFO : OFED Version: MLNX_OFED_LINUX-3.0-2.0.1:
Pid: 6210 Tid: 6210 VMA DEBUG : System: 3.10.0-229.7.2.rt56.141.6.el7.centos.x86_64
Pid: 6210 Tid: 6210 VMA INFO : Architecture: x86_64
Pid: 6210 Tid: 6210 VMA INFO : Node: bot5
Pid: 6210 Tid: 6210 VMA INFO : ---------------------------------------------------------------------------
Pid: 6210 Tid: 6210 VMA INFO : Log Level 4 [VMA_TRACELEVEL]
Pid: 6210 Tid: 6210 VMA INFO : Log Details 2 [VMA_LOG_DETAILS]
Pid: 6210 Tid: 6210 VMA DEBUG : Log Colors Enabled [VMA_LOG_COLORS]
Pid: 6210 Tid: 6210 VMA INFO : Log File /tmp/vma_log.txt [VMA_LOG_FILE]
Pid: 6210 Tid: 6210 VMA DEBUG : Stats File [VMA_STATS_FILE]
Pid: 6210 Tid: 6210 VMA DEBUG : Stats shared memory directory /tmp/ [VMA_STATS_SHMEM_DIR]
Pid: 6210 Tid: 6210 VMA DEBUG : Stats FD Num (max) 100 [VMA_STATS_FD_NUM]
Pid: 6210 Tid: 6210 VMA DEBUG : Conf File /etc/libvma.conf [VMA_CONFIG_FILE]
Pid: 6210 Tid: 6210 VMA DEBUG : Application ID VMA_DEFAULT_APPLICATION_ID [VMA_APPLICATION_ID]
Pid: 6210 Tid: 6210 VMA DEBUG : Polling CPU idle usage Disabled [VMA_CPU_USAGE_STATS]
Pid: 6210 Tid: 6210 VMA DEBUG : SigIntr Ctrl-C Handle Disabled [VMA_HANDLE_SIGINTR]
Pid: 6210 Tid: 6210 VMA DEBUG : SegFault Backtrace Disabled [VMA_HANDLE_SIGSEGV]
Pid: 6210 Tid: 6210 VMA DEBUG : Ring allocation logic TX 0 (Ring per interface) [VMA_RING_ALLOCATION_LOGIC_TX]
Pid: 6210 Tid: 6210 VMA DEBUG : Ring allocation logic RX 0 (Ring per interface) [VMA_RING_ALLOCATION_LOGIC_RX]
Pid: 6210 Tid: 6210 VMA DEBUG : Ring migration ratio TX 100 [VMA_RING_MIGRATION_RATIO_TX]
Pid: 6210 Tid: 6210 VMA DEBUG : Ring migration ratio RX 100 [VMA_RING_MIGRATION_RATIO_RX]
Pid: 6210 Tid: 6210 VMA DEBUG : Ring limit per interface 0 (no limit) [VMA_RING_LIMIT_PER_INTERFACE]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP max syn rate 0 (no limit) [VMA_TCP_MAX_SYN_RATE]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx Mem Segs TCP 1000000 [VMA_TX_SEGS_TCP]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx Mem Bufs 200000 [VMA_TX_BUFS]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx QP WRE 3000 [VMA_TX_WRE]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx Max QP INLINE 220 [VMA_TX_MAX_INLINE]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx MC Loopback Enabled [VMA_TX_MC_LOOPBACK]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx non-blocked eagains Disabled [VMA_TX_NONBLOCKED_EAGAINS]
Pid: 6210 Tid: 6210 VMA DEBUG : Tx Prefetch Bytes 256 [VMA_TX_PREFETCH_BYTES]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Mem Bufs 200000 [VMA_RX_BUFS]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx QP WRE 16000 [VMA_RX_WRE]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx QP WRE BATCHING 64 [VMA_RX_WRE_BATCHING]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Byte Min Limit 65536 [VMA_RX_BYTES_MIN]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Poll Loops 100000 [VMA_RX_POLL]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Poll Init Loops 0 [VMA_RX_POLL_INIT]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx UDP Poll OS Ratio 100 [VMA_RX_UDP_POLL_OS_RATIO]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx UDP HW TS Conversion 3 [VMA_RX_UDP_HW_TS_CONVERSION]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Poll Yield Disabled [VMA_RX_POLL_YIELD]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Prefetch Bytes 256 [VMA_RX_PREFETCH_BYTES]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx Prefetch Bytes Before Poll 0 [VMA_RX_PREFETCH_BYTES_BEFORE_POLL]
Pid: 6210 Tid: 6210 VMA DEBUG : Rx CQ Drain Rate Disabled [VMA_RX_CQ_DRAIN_RATE_NSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : GRO max streams 32 [VMA_GRO_STREAMS_MAX]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP 3T rules Disabled [VMA_TCP_3T_RULES]
Pid: 6210 Tid: 6210 VMA DEBUG : ETH MC L2 only rules Disabled [VMA_ETH_MC_L2_ONLY_RULES]
Pid: 6210 Tid: 6210 VMA DEBUG : Select Poll (usec) 100000 [VMA_SELECT_POLL]
Pid: 6210 Tid: 6210 VMA DEBUG : Select Poll OS Force Disabled [VMA_SELECT_POLL_OS_FORCE]
Pid: 6210 Tid: 6210 VMA DEBUG : Select Poll OS Ratio 10 [VMA_SELECT_POLL_OS_RATIO]
Pid: 6210 Tid: 6210 VMA DEBUG : Select Skip OS 4 [VMA_SELECT_SKIP_OS]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Drain Interval (msec) 10 [VMA_PROGRESS_ENGINE_INTERVAL]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Drain WCE (max) 10000 [VMA_PROGRESS_ENGINE_WCE_MAX]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Interrupts Moderation Enabled [VMA_CQ_MODERATION_ENABLE]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Moderation Count 48 [VMA_CQ_MODERATION_COUNT]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Moderation Period (usec) 50 [VMA_CQ_MODERATION_PERIOD_USEC]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ AIM Max Count 560 [VMA_CQ_AIM_MAX_COUNT]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ AIM Max Period (usec) 250 [VMA_CQ_AIM_MAX_PERIOD_USEC]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ AIM Interval (msec) 250 [VMA_CQ_AIM_INTERVAL_MSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ AIM Interrupts Rate (per sec) 5000 [VMA_CQ_AIM_INTERRUPTS_RATE_PER_SEC]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Poll Batch (max) 16 [VMA_CQ_POLL_BATCH_MAX]
Pid: 6210 Tid: 6210 VMA DEBUG : CQ Keeps QP Full Enabled [VMA_CQ_KEEP_QP_FULL]
Pid: 6210 Tid: 6210 VMA DEBUG : QP Compensation Level 256 [VMA_QP_COMPENSATION_LEVEL]
Pid: 6210 Tid: 6210 VMA DEBUG : Offloaded Sockets Enabled [VMA_OFFLOADED_SOCKETS]
Pid: 6210 Tid: 6210 VMA DEBUG : Timer Resolution (msec) 10 [VMA_TIMER_RESOLUTION_MSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP Timer Resolution (msec) 100 [VMA_TCP_TIMER_RESOLUTION_MSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP control thread 0 (Disabled) [VMA_TCP_CTL_THREAD]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP timestamp option 0 [VMA_TCP_TIMESTAMP_OPTION]
Pid: 6210 Tid: 6210 VMA INFO : Exception handling mode 0 (log debug and un-offload) [VMA_EXCEPTION_HANDLING]
Pid: 6210 Tid: 6210 VMA DEBUG : Avoid sys-calls on tcp fd Disabled [VMA_AVOID_SYS_CALLS_ON_TCP_FD]
Pid: 6210 Tid: 6210 VMA DEBUG : Delay after join (msec) 0 [VMA_WAIT_AFTER_JOIN_MSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : Internal Thread Affinity -1 [VMA_INTERNAL_THREAD_AFFINITY]
Pid: 6210 Tid: 6210 VMA DEBUG : Internal Thread Cpuset [VMA_INTERNAL_THREAD_CPUSET]
Pid: 6210 Tid: 6210 VMA DEBUG : Internal Thread Arm CQ Disabled [VMA_INTERNAL_THREAD_ARM_CQ]
Pid: 6210 Tid: 6210 VMA DEBUG : Thread mode Multi spin lock [VMA_THREAD_MODE]
Pid: 6210 Tid: 6210 VMA DEBUG : Buffer batching mode 1 (Batch and reclaim buffers) [VMA_BUFFER_BATCHING_MODE]
Pid: 6210 Tid: 6210 VMA DEBUG : Mem Allocate type 1 (Contig Pages) [VMA_MEM_ALLOC_TYPE]
Pid: 6210 Tid: 6210 VMA DEBUG : Num of UC ARPs 3 [VMA_NEIGH_UC_ARP_QUATA]
Pid: 6210 Tid: 6210 VMA DEBUG : UC ARP delay (msec) 10000 [VMA_NEIGH_UC_ARP_DELAY_MSEC]
Pid: 6210 Tid: 6210 VMA DEBUG : Num of neigh restart retries 1 [VMA_NEIGH_NUM_ERR_RETRIES]
Pid: 6210 Tid: 6210 VMA DEBUG : IPOIB support Enabled [VMA_IPOIB]
Pid: 6210 Tid: 6210 VMA DEBUG : BF (Blue Flame) Enabled [VMA_BF]
Pid: 6210 Tid: 6210 VMA DEBUG : fork() support Enabled [VMA_FORK]
Pid: 6210 Tid: 6210 VMA DEBUG : close on dup2() Enabled [VMA_CLOSE_ON_DUP2]
Pid: 6210 Tid: 6210 VMA DEBUG : MTU 0 (follow actual MTU) [VMA_MTU]
Pid: 6210 Tid: 6210 VMA DEBUG : MSS 0 (follow VMA_MTU) [VMA_MSS]
Pid: 6210 Tid: 6210 VMA DEBUG : TCP CC Algorithm 0 (LWIP) [VMA_TCP_CC_ALGO]
Pid: 6210 Tid: 6210 VMA DEBUG : Suppress IGMP ver. warning Disabled [VMA_SUPPRESS_IGMP_WARNING]
Pid: 6210 Tid: 6210 VMA INFO : ---------------------------------------------------------------------------
Pid: 6210 Tid: 6210 VMA WARNING: *************************************************************
Pid: 6210 Tid: 6210 VMA WARNING: * VMA is currently configured with high log level *
Pid: 6210 Tid: 6210 VMA WARNING: * Application performance will decrease in this log level! *
Pid: 6210 Tid: 6210 VMA WARNING: * This log level is recommended for debugging purposes only *
Pid: 6210 Tid: 6210 VMA WARNING: *************************************************************
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/module/mlx4_core/parameters/log_num_mgm_entry_size, flags=0, mode=0x2) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : sock_redirect_main()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/ipv4/tcp_max_syn_backlog, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/core/somaxconn, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/ipv4/tcp_window_scaling, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/core/rmem_max, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/core/wmem_max, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/ipv4/tcp_timestamps, flags=0, mode=0x18) = 4
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=4)
Pid: 6210 Tid: 6210 VMA DEBUG : ibv_fork_init() succeeded, fork() may be used safely!!
Pid: 6210 Tid: 6210 VMA DEBUG : wakeup_pipe[epfd=0]:48:wakeup_pipe() created wakeup pipe [RD=4, WR=5]
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/tmp//vmastat.6210, flags=0x42, mode=0x1f6) = 7
Pid: 6210 Tid: 6210 VMA DEBUG : vma_shmem_stats_open: file '/tmp//vmastat.6210' fd 7 shared memory at 0x7fe680183000 with 100 max blocks
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a70750' registered PERIODIC timer for 10 msec (user data: 0)
Pid: 6210 Tid: 6210 VMA DEBUG : evh:314:start_thread() VMA Internal thread affinity not set.
Pid: 6210 Tid: 6210 VMA DEBUG : evh:335:start_thread() Started event handler thread
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : evh:251:event_handler_thread() Entering internal thread, id = 140627932108544
Pid: 6210 Tid: 6210 VMA DEBUG : nl_wrapper:122:netlink_wrapper() ---> netlink_route_listener CTOR
Pid: 6210 Tid: 6210 VMA DEBUG : nl_wrapper:126:netlink_wrapper() <--- netlink_route_listener CTOR
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/misc/rdma_cm/abi_version, flags=0, mode=0x1a70b40) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/abi_version, flags=0, mode=0x1a70b40) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/ibdev, flags=0, mode=0x1a78e40) = 9
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=9)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/abi_version, flags=0, mode=0x1a78ef0) = 9
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=9)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/device/vendor, flags=0, mode=0x1a72860) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/device/device, flags=0, mode=0x1a728d0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/device/vendor, flags=0, mode=0x1a72940) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband_verbs/uverbs0/device/device, flags=0, mode=0x1a729b0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband/mlx4_0/node_type, flags=0, mode=0x1a729f0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband/mlx4_0/node_guid, flags=0, mode=0x1a729f0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/dev/infiniband/rdma_cm, flags=0x80002, mode=0x1a78eb0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband/mlx4_0/node_guid, flags=0, mode=0x1a729f0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=8)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/dev/infiniband/uverbs0, flags=0x2, mode=0x1a78eb0) = 8
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_time_converter::get_devices_convertor_status : Checking RX UDP HW time stamp status for all devices [1], ibv_context_list = 0x1a781e0
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_collection[0x1a70a20]:77:map_ib_devices() TS converter status was set to 3
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_collection[0x1a70a20]:79:map_ib_devices() Mapping 1 ibv devices
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a783e8' registered ONE SHOT timer for 1000 msec (user data: 0)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a783e8' registered PERIODIC timer for 10000 msec (user data: 0)
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_handler[0x1a78200]:67:ib_ctx_handler() ibv device 'mlx4_0' [0x1a72a20] has 2 ports. Vendor Part Id: 4099, FW Ver: 2.32.5100, max_qp_wr=16351, hca_core_clock (per sec)=427000000
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_handler[0x1a78200]:136:set_dev_configuration() Setting configuration for the MLX card mlx4_0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : evh:448:priv_prepare_ibverbs_async_event_queue()
Pid: 6210 Tid: 6215 VMA DEBUG : utils:200:set_fd_block_mode() fd[9]: setting to non-blocking mode (0)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/dev/infiniband/rdma_cm, flags=0x80002, mode=0x1a781e0) = 10
Pid: 6210 Tid: 6215 VMA DEBUG : evh:468:priv_prepare_ibverbs_async_event_queue() Emptied 0 Events
Pid: 6210 Tid: 6210 VMA DEBUG : ntm:51:neigh_table_mgr() Creation of neigh_cma_event_channel on fd=10
Pid: 6210 Tid: 6215 VMA DEBUG : evh:488:priv_register_ibverbs_events() 9 added to event_handler_map_t!
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a78528' registered PERIODIC timer for 100000 msec (user data: 0)
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'lo' (addr=1.0.0.0, flags=10049)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('lo') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth0' (addr=2.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('eth0') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth1' (addr=3.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('eth1') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'mlnx0' (addr=4.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('mlnx0') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'mlnx1' (addr=5.0.0.0, flags=1002)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('mlnx1') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'team1' (addr=6.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('team1') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth0.1001' (addr=7.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('eth0.1001') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth0.1002' (addr=8.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('eth0.1002') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'team1.502' (addr=9.0.0.0, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:150:map_net_devices() Blocking offload: Interface ('team1.502') is not of type AF_INET
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'lo' (addr=127.0.0.1, flags=10049)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/dev/infiniband/rdma_cm, flags=0x80002, mode=0x1a77570) = 14
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:190:map_net_devices() Blocking offload: No verbs context in cma_id on interfaces ('lo')
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth1' (addr=172.25.24.102, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:179:map_net_devices() Failed in rdma_bind_addr (src=172.25.24.102) (errno=2 No such file or directory)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth0.1001' (addr=10.20.0.105, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:179:map_net_devices() Failed in rdma_bind_addr (src=10.20.0.105) (errno=2 No such file or directory)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'eth0.1002' (addr=10.30.0.105, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:179:map_net_devices() Failed in rdma_bind_addr (src=10.30.0.105) (errno=2 No such file or directory)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:143:map_net_devices() Checking if can offload on interface 'team1.502' (addr=192.168.31.105, flags=11043)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:639:get_vlan_base_name_from_ifname() found vlan base name 'team1' for interface 'team1.502'
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1/type, flags=0, mode=0x20) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : ndv[]:81:configure()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1.502/mtu, flags=0, mode=0x20) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:639:get_vlan_base_name_from_ifname() found vlan base name 'team1' for interface 'team1.502'
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1.502/address, flags=0, mode=0x100) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:851:get_local_ll_addr() ifname=team1.502 un-aliased-ifname=team1.502 l2_addr_path=/sys/class/net/team1.502/address l2-addr=f4:52:14:22:a2:71
(addr-bytes_len=6)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:864:get_local_ll_addr() found ETH UC addressf4:52:14:22:a2:71 for interface team1.502
Pid: 6210 Tid: 6210 VMA DEBUG : utils:639:get_vlan_base_name_from_ifname() found vlan base name 'team1' for interface 'team1.502'
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1/device/resource, flags=0, mode=0x400) = -1
Pid: 6210 Tid: 6210 VMA ERROR : utils:243:priv_read_file() ERROR while opening file /sys/class/net/team1/device/resource (errno 2 No such file or directory)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/infiniband/mlx4_0/device/resource, flags=0, mode=0x400) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1.502/address, flags=0, mode=0x100) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:851:get_local_ll_addr() ifname=team1.502 un-aliased-ifname=team1.502 l2_addr_path=/sys/class/net/team1.502/address l2-addr=f4:52:14:22:a2:71
(addr-bytes_len=6)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:864:get_local_ll_addr() found ETH UC addressf4:52:14:22:a2:71 for interface team1.502
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/sys/class/net/team1.502/broadcast, flags=0, mode=0x100) = 15
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=15)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:851:get_local_ll_addr() ifname=team1.502 un-aliased-ifname=team1.502 l2_addr_path=/sys/class/net/team1.502/broadcast l2-addr=ff:ff:ff:ff:ff:ff
(addr-bytes_len=6)
Pid: 6210 Tid: 6210 VMA DEBUG : utils:864:get_local_ll_addr() found ETH BR addressff:ff:ff:ff:ff:ff for interface team1.502
Pid: 6210 Tid: 6210 VMA DEBUG : utils:613:get_vlan_id_from_ifname() found vlan id '502' for interface 'team1.502'
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:260:map_net_devices() Offload interface 'team1.502': Mapped to ibv device 'mlx4_0' [0x1a72a20] on port 1
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:90:net_device_table_mgr() registering timer for ring draining with 10 msec intervales
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a78608' registered PERIODIC timer for 10 msec (user data: 0)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:95:net_device_table_mgr() registering timer for cq adaptive moderation with 250 msec intervales
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a78608' registered PERIODIC timer for 250 msec (user data: 1)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : netlink_socket_mgr:104:netlink_socket_mgr()
Pid: 6210 Tid: 6210 VMA DEBUG : netlink_socket_mgr:125:netlink_socket_mgr() Done
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:53:rule_table_mgr()
Pid: 6210 Tid: 6210 VMA DEBUG : rrv[0x7fe680012018]:76:print_val() Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:61:rule_table_mgr() Done
Pid: 6210 Tid: 6210 VMA DEBUG : netlink_socket_mgr:104:netlink_socket_mgr()
Pid: 6210 Tid: 6210 VMA DEBUG : netlink_socket_mgr:125:netlink_socket_mgr() Done
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:51:route_table_mgr()
Pid: 6210 Tid: 6210 VMA ERROR : utils:484:get_ipv4_from_ifname() Failed getting ipv4 from interface 'eth0' (errno=99 Cannot assign requested address)
Pid: 6210 Tid: 6210 VMA WARNING: rtm:143:rt_mgr_update_source_ip() could not figure out source ip for rtv = dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth0 table :main scope 253 type 1 index 2
Pid: 6210 Tid: 6210 VMA ERROR : utils:484:get_ipv4_from_ifname() Failed getting ipv4 from interface 'team1' (errno=99 Cannot assign requested address)
Pid: 6210 Tid: 6210 VMA WARNING: rtm:143:rt_mgr_update_source_ip() could not figure out source ip for rtv = dst: 169.254.0.0 netmask: 255.255.0.0 dev: team1 table :main scope 253 type 1 index 6
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf65160]: dst: 10.20.0.0 netmask: 255.255.255.0 dev: eth0.1001 src: 10.20.0.105 table :main scope 253 type 1 index 7
Pid: 6210 Tid: 6210 VMA ERROR : utils:484:get_ipv4_from_ifname() Failed getting ipv4 from interface 'eth0' (errno=99 Cannot assign requested address)
Pid: 6210 Tid: 6210 VMA WARNING: rtm:206:rt_mgr_update_source_ip() could not figure out source ip for rtv = dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth0 table :main scope 253 type 1 index 2
Pid: 6210 Tid: 6210 VMA ERROR : utils:484:get_ipv4_from_ifname() Failed getting ipv4 from interface 'team1' (errno=99 Cannot assign requested address)
Pid: 6210 Tid: 6210 VMA WARNING: rtm:206:rt_mgr_update_source_ip() could not figure out source ip for rtv = dst: 169.254.0.0 netmask: 255.255.0.0 dev: team1 table :main scope 253 type 1 index 6
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:407:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:144:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:157:update_entry() entry [0x1a78a20]
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:162:update_entry() rule_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:10.20.0.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:185:find_rule_val() found rule val[0x7fe680012018]: Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:148:create_new_entry() new entry 0x1a78a20 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = Destination IP:10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:332:update_entry() entry [0x1a78890]
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:335:update_entry() route_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:10.20.0.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf65160]: dst: 10.20.0.0 netmask: 255.255.255.0 dev: eth0.1001 src: 10.20.0.105 table :main scope 253 type 1 index 7
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:10.20.0.105->eth0.1001]:74:register_to_net_device() register to net device with src_addr 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:339:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:319:get_net_device_val() Can't find net_device for 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:260:register_observer() Failed to allocate new cache_entry_subject with Key = 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:10.20.0.105->eth0.1001]:84:register_to_net_device() route_entry [0x1a78890] tried to register to non-offloaded device ---> registration failed
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:411:create_new_entry() new entry 0x1a78890 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:407:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:144:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:157:update_entry() entry [0x1a79980]
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:162:update_entry() rule_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:10.30.0.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:185:find_rule_val() found rule val[0x7fe680012018]: Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:148:create_new_entry() new entry 0x1a79980 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = Destination IP:10.30.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:332:update_entry() entry [0x1a79820]
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:335:update_entry() route_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:10.30.0.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf652a8]: dst: 10.30.0.0 netmask: 255.255.255.0 dev: eth0.1002 src: 10.30.0.105 table :main scope 253 type 1 index 8
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:10.30.0.105->eth0.1002]:74:register_to_net_device() register to net device with src_addr 10.30.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:339:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:319:get_net_device_val() Can't find net_device for 10.30.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:260:register_observer() Failed to allocate new cache_entry_subject with Key = 10.30.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:10.30.0.105->eth0.1002]:84:register_to_net_device() route_entry [0x1a79820] tried to register to non-offloaded device ---> registration failed
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:411:create_new_entry() new entry 0x1a79820 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:407:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:144:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:157:update_entry() entry [0x1a79f70]
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:162:update_entry() rule_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:172.25.24.102]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:185:find_rule_val() found rule val[0x7fe680012018]: Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:148:create_new_entry() new entry 0x1a79f70 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = Destination IP:172.25.24.102
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:332:update_entry() entry [0x1a79e10]
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:335:update_entry() route_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:172.25.24.102]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf65f78]: dst: 172.25.24.0 netmask: 255.255.255.0 dev: eth1 src: 172.25.24.102 table :main scope 253 type 1 index 3
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:172.25.24.102->eth1]:74:register_to_net_device() register to net device with src_addr 172.25.24.102
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:339:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:319:get_net_device_val() Can't find net_device for 172.25.24.102
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:260:register_observer() Failed to allocate new cache_entry_subject with Key = 172.25.24.102
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:172.25.24.102->eth1]:84:register_to_net_device() route_entry [0x1a79e10] tried to register to non-offloaded device ---> registration failed
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:411:create_new_entry() new entry 0x1a79e10 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:407:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:144:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:157:update_entry() entry [0x1a7a560]
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:162:update_entry() rule_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:0.0.0.0]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:185:find_rule_val() found rule val[0x7fe680012018]: Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:148:create_new_entry() new entry 0x1a7a560 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = Destination IP:0.0.0.0
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:332:update_entry() entry [0x1a7a400]
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:335:update_entry() route_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:0.0.0.0]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf65018]: dst: default gw: 10.20.0.1 dev: eth0.1001 table :main scope 0 type 1 index 7
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:0.0.0.0->eth0.1001]:74:register_to_net_device() register to net device with src_addr 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:339:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:319:get_net_device_val() Can't find net_device for 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:260:register_observer() Failed to allocate new cache_entry_subject with Key = 10.20.0.105
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:0.0.0.0->eth0.1001]:84:register_to_net_device() route_entry [0x1a7a400] tried to register to non-offloaded device ---> registration failed
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:411:create_new_entry() new entry 0x1a7a400 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:407:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:144:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:157:update_entry() entry [0x1a7aba0]
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:162:update_entry() rule_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:192.168.31.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:185:find_rule_val() found rule val[0x7fe680012018]: Priority :32766 lookup table :main
Pid: 6210 Tid: 6210 VMA DEBUG : rrm:148:create_new_entry() new entry 0x1a7aba0 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = Destination IP:192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:332:update_entry() entry [0x1a7aa40]
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:335:update_entry() route_entry is not valid-> update value
Pid: 6210 Tid: 6210 VMA DEBUG : rre[Destination IP:192.168.31.105]:34:get_val()
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:294:find_route_val() found route val[0x7fe67cf660c0]: dst: 192.168.31.0 netmask: 255.255.255.0 dev: team1.502 src: 192.168.31.105 table :main scope 253 type 1 index 9
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:192.168.31.105->team1.502]:74:register_to_net_device() register to net device with src_addr 192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:339:create_new_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:312:get_net_device_val() Found ETH: Net Device: team1.502 for 192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : nde[0x1a7b000]:31:net_device_entry()
Pid: 6210 Tid: 6210 VMA DEBUG : nde[0x1a7b000]:54:net_device_entry() Done
Pid: 6210 Tid: 6210 VMA DEBUG : cache_subject_observer:264:register_observer() Created new cache_entry Key = 192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : rte[Destination IP:192.168.31.105->team1.502]:78:register_to_net_device() route_entry [0x1a7aa40] is registered to an offloaded device
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:411:create_new_entry() new entry 0x1a7aa40 created successfully
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65018]:83:print_val() dst: default gw: 10.20.0.1 dev: eth0.1001 src: 10.20.0.105 table :main scope 0 type 1 index 7
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65160]:83:print_val() dst: 10.20.0.0 netmask: 255.255.255.0 dev: eth0.1001 src: 10.20.0.105 table :main scope 253 type 1 index 7
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf652a8]:83:print_val() dst: 10.30.0.0 netmask: 255.255.255.0 dev: eth0.1002 src: 10.30.0.105 table :main scope 253 type 1 index 8
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf657c8]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth0 table :main scope 253 type 1 index 2
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65910]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth1 src: 172.25.24.102 table :main scope 253 type 1 index 3
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65a58]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: team1 table :main scope 253 type 1 index 6
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65ba0]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth0.1001 src: 10.20.0.105 table :main scope 253 type 1 index 7
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65ce8]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: eth0.1002 src: 10.30.0.105 table :main scope 253 type 1 index 8
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65e30]:83:print_val() dst: 169.254.0.0 netmask: 255.255.0.0 dev: team1.502 src: 192.168.31.105 table :main scope 253 type 1 index 9
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf65f78]:83:print_val() dst: 172.25.24.0 netmask: 255.255.255.0 dev: eth1 src: 172.25.24.102 table :main scope 253 type 1 index 3
Pid: 6210 Tid: 6210 VMA DEBUG : rtv[0x7fe67cf660c0]:83:print_val() dst: 192.168.31.0 netmask: 255.255.255.0 dev: team1.502 src: 192.168.31.105 table :main scope 253 type 1 index 9
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:77:route_table_mgr() Registered to g_p_netlink_handler
Pid: 6210 Tid: 6210 VMA DEBUG : rtm:79:route_table_mgr() Done
Pid: 6210 Tid: 6210 VMA DEBUG : vma_stats_instance_create_bpool_block:539: Added bpool local=0x1a7b460 shm=0x7fe68018337c
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_collection[0x1a70a20]:122:mem_reg_on_all_devices() addr=0x7fe66196b000, length=376000063, pos=0, mr[pos]->lkey=268502784, dev1=0x1a72a20, dev2=0x1a72a20
Pid: 6210 Tid: 6210 VMA DEBUG : bpool[0x1a7b3b0]:96:buffer_pool() Contiguous pages allocation passed successfully
Pid: 6210 Tid: 6210 VMA DEBUG : vma_stats_instance_create_bpool_block:539: Added bpool local=0x1a7b880 shm=0x7fe68018338c
Pid: 6210 Tid: 6210 VMA DEBUG : ib_ctx_collection[0x1a70a20]:122:mem_reg_on_all_devices() addr=0x7fe64b2d6000, length=376000063, pos=0, mr[pos]->lkey=268503040, dev1=0x1a72a20, dev2=0x1a72a20
Pid: 6210 Tid: 6210 VMA DEBUG : bpool[0x1a7b7d0]:96:buffer_pool() Contiguous pages allocation passed successfully
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a7bbf0' registered PERIODIC timer for 100 msec (user data: 0)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : fdc:55:fd_collection() using open files max limit of 4096 file descriptors
Pid: 6210 Tid: 6210 VMA DEBUG : match:223:__vma_print_conf_file() Configuration File:
Pid: 6210 Tid: 6210 VMA DEBUG : match:176:print_instance_id_str() application-id * *:
Pid: 6210 Tid: 6210 VMA DEBUG : match:196:print_instance_conf() tcp_server's rules:
Pid: 6210 Tid: 6210 VMA DEBUG : match:200:print_instance_conf() tcp_clinet's rules:
Pid: 6210 Tid: 6210 VMA DEBUG : match:204:print_instance_conf() udp receiver rules:
Pid: 6210 Tid: 6210 VMA DEBUG : match:208:print_instance_conf() udp sender rules:
Pid: 6210 Tid: 6210 VMA DEBUG : match:212:print_instance_conf() udp connect rules:
Pid: 6210 Tid: 6210 VMA DEBUG : match:215:print_instance_conf()
Pid: 6210 Tid: 6210 VMA DEBUG : lwip:102:vma_lwip()
Pid: 6210 Tid: 6210 VMA DEBUG : utils:460:get_window_scaling_factor() TCP scaling window factor is set to 8
Pid: 6210 Tid: 6210 VMA DEBUG : lwip:123:vma_lwip() LWIP subsystem initialized
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a9deb0' registered PERIODIC timer for 200 msec (user data: 0)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6210 VMA DEBUG : nl_wrapper:161:open_channel() opening netlink channel
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: socket_internal(domain=(16), type=SOCK_RAW(3), protocol=0) = 17
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=1, optname=7)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=1, optname=8)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: bind(fd=17, sa_family=16)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: bind() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: getsockname(fd=17)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: getsockname() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : nl_wrapper:225:open_channel() netlink socket is open
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=270, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=270, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=270, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=270, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=17, level=270, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : evh:179:register_command_event() Register command command_netlink event
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6210 VMA DEBUG : evh:59:register_timer_event() timer handler '0x1a705d8' registered PERIODIC timer for 10000 msec (user data: 0)
Pid: 6210 Tid: 6215 VMA DEBUG : evh:634:priv_register_command_events() Adding new channel (fd 17)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: do_wakeup()
Pid: 6210 Tid: 6215 VMA DEBUG : ENTER: remove_wakeup_fd()
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/proc/sys/net/ipv4/igmp_max_memberships, flags=0, mode=0x20) = 18
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: close(fd=18)
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: socket_internal(domain=AF_INET(2), type=SOCK_STREAM(1), protocol=0) = 18
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: open(file=/dev/infiniband/rdma_cm, flags=0x80002, mode=0x1a705f0) = 19
Pid: 6210 Tid: 6210 VMA DEBUG : fdc:217:addsocket() On-demand creation of cma event channel on fd=19
Pid: 6210 Tid: 6210 VMA DEBUG : fdc:253:addsocket() TCP rules are either not consistent or instructing to use VMA.transport == USE_VMA
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:87:sockinfo_tcp() tcp socket created
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:91:sockinfo_tcp() new pcb 0x1aa2ad8 pcb state 0
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:99:sockinfo_tcp() TCP PCB FLAGS: 0x0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=18, level=6, optname=1)
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:2963:setsockopt() (TCP_NODELAY) nagle: 1
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:3083:setsockopt() going to OS for setsockopt level 6 optname 1
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: setsockopt(fd=18, level=1, optname=2)
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:2981:setsockopt() (SO_REUSEADDR) val: 1
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:3083:setsockopt() going to OS for setsockopt level 1 optname 2
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: setsockopt() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: bind(fd=18, AF_INET, addr=192.168.31.105, port=46357)
Pid: 6210 Tid: 6210 VMA DEBUG : ndtm[0x1a78600]:312:get_net_device_val() Found ETH: Net Device: team1.502 for 192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:1828:bind() socket bound
Pid: 6210 Tid: 6210 VMA DEBUG : EXIT: bind() returned with 0
Pid: 6210 Tid: 6210 VMA DEBUG : ENTER: listen(fd=18, backlog=5)
Pid: 6210 Tid: 6210 VMA DEBUG : match:403:get_family_by_instance_first_matching_rule() MATCHING program name: *, application-id: *
Pid: 6210 Tid: 6210 VMA DEBUG : match:383:get_family_by_first_matching_rule() No matching rule. Using VMA (default)
Pid: 6210 Tid: 6210 VMA DEBUG : match:443:__vma_match_tcp_server() MATCH TCP SERVER (LISTEN): => VMA
Pid: 6210 Tid: 6210 VMA DEBUG : si_tcp[fd=18]:1869:prepareListen() TRANSPORT: VMA, sock state = 0
Pid: 6210 Tid: 6210 VMA DEBUG : si[fd=18]:856:attach_as_uc_receiver() Attaching to specific local if: 192.168.31.105:46357
Pid: 6210 Tid: 6210 VMA DEBUG : match:403:get_family_by_instance_first_matching_rule() MATCHING program name: *, application-id: *
Pid: 6210 Tid: 6210 VMA DEBUG : match:383:get_family_by_first_matching_rule() No matching rule. Using VMA (default)
Pid: 6210 Tid: 6210 VMA DEBUG : match:443:__vma_match_tcp_server() MATCH TCP SERVER (LISTEN): => VMA
Pid: 6210 Tid: 6210 VMA DEBUG : si[fd=18]:339:attach_receiver() Attaching to dst:192.168.31.105:46357, src:0.0.0.0:0, protocol:TCP, local if:192.168.31.105
Pid: 6210 Tid: 6210 VMA DEBUG : ndv[team1.502]:383:reserve_ring() Creating new RING for key 0
Pid: 6210 Tid: 6210 VMA DEBUG : ring_simple[0x1aaa010]:136:create_resources() new ring()
Pid: 6210 Tid: 6210 VMA PANIC : ring_simple[0x1aaa010]:144:create_resources() p_ring_info.p_ib_ctx = NULL. It can be related to wrong bonding configuration
from libvma.
rosenbaumalex
commented on August 15, 2024
We'll make sure to prevent the VMA PANIC on un-supported network configuration
In that case the flows will go to the OS path and will not be offloaded. You can detect such cases with vma_stats utility (https://github.com/Mellanox/libvma/wiki/VMA_Statistics).
Longer path will be to add the team support into VMA core network stack in user space.
from libvma.
rosenbaumalex
commented on August 15, 2024
fixed in 5253c6c
will be part of VMA 8.1.1 tag coming soon
closing issue.
from libvma.
Related Issues (20)
- can't sent/receive between internal threads with libvma HOT 4
- VMA TCP connect() call takes much longer than OS HOT 4
- Libvma : 8.9.4 arp caching HOT 4
- Use MADV_HUGEPAGE as ALLOC_TYPE_HUGEPAGES fallback
- NOT see any rocev2 packet HOT 9
- epoll_ctl EPOLL_CTL_MOD called incorrectly HOT 2
- issue: errno=111 Connection refused
- Rust TcpStream clone cannot send/recv through libvma HOT 5
- Unable to build libvma rpm follow the build instruction wiki HOT 2
- How do I fix the warning "Not enough hugepage resources for VMA memory allocation"？
- Running vma inside docker container HOT 9
- libvma uses huge amount of memory (~4x8G) with max RLIMIT_NOFILE HOT 2
- `libvma` load failed in some situation HOT 1
- `ibv_create_comp_channel` fails with ENOTTY (Inappropriate ioctl for device)
- Trouble Getting Timestamps in vma_recv_callback HOT 3
- failed to build libvma with gcc-12 HOT 5
- How to estimate a number of huge pages required by libvma? HOT 2
- Unable to find the libibverbs-devel header files HOT 2
- VMA support with Connectx-6 VPI HCAs HOT 3
- verify_qp_creation() QP creation failed on ConnectX-6 IPoIB interface HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libvma.