Code Monkey home page Code Monkey logo

ptp-operator's Introduction

PTP Operator

Table of Contents

PTP Operator

Ptp Operator, runs in openshift-ptp namespace, manages cluster wide PTP configuration. It offers PtpOperatorConfig and PtpConfig CRDs and creates linuxptp daemon to apply node specific PTP config.

PtpOperatorConfig

Upon deployment of PTP Operator, it automatically creates a default custom resource of PtpOperatorConfig kind which contains a configurable option daemonNodeSelector, it is used to specify which nodes linuxptp daemon shall be created on. The daemonNodeSelector will be applied to linuxptp daemon DaemonSet nodeSelector field and trigger relaunching of linuxptp daemon. Ptp Operator only recognizes default PtpOperatorConfig, use oc edit PtpOperatorConfig default -n openshift-ptp to update the daemonNodeSelector.

$ oc get ptpoperatorconfigs.ptp.openshift.io default -n openshift-ptp -o yaml

apiVersion: v1
items:
- apiVersion: ptp.openshift.io/v1
  kind: PtpOperatorConfig
  metadata:
    creationTimestamp: "2019-10-28T06:53:09Z"
    generation: 3
    name: default
    namespace: openshift-ptp
    resourceVersion: "2356742"
    selfLink: /apis/ptp.openshift.io/v1/namespaces/openshift-ptp/ptpoperatorconfigs/default
    uid: d7286542-34bd-4c79-8533-d01e2b25953e
  spec:
    daemonNodeSelector: {}
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

Enable PTP events via fast event framework

PTP Operator supports fast event publisher for events such as PTP state change, os clock out of sync, clock class change and port failure. Event publisher is enabled by deploying PTP operator with cloud events framework (based on O-RAN API specifications). The events are published via HTTP or AMQP transport and available for local subscribers.

Enabling fast events

$ oc edit ptpoperatorconfigs.ptp.openshift.io default -n openshift-ptp

apiVersion: v1
items:
- apiVersion: ptp.openshift.io/v1
  kind: PtpOperatorConfig
  metadata:
    creationTimestamp: "2019-10-28T06:53:09Z"
    generation: 4
    name: default
    namespace: openshift-ptp
    resourceVersion: "2364095"
    selfLink: /apis/ptp.openshift.io/v1/namespaces/openshift-ptp/ptpoperatorconfigs/default
    uid: d7286542-34bd-4c79-8533-d01e2b25953e
  spec:
    ptpEventConfig:
      enableEventPublisher: true
      transportHost: "http://ptp-event-publisher-service-NODE_NAME.openshift-ptp.svc.cluster.local:9043"
      storageType: local-sc
    daemonNodeSelector:
      node-role.kubernetes.io/worker: ""
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

PtpConfig

PtpConfig CRD is used to define linuxptp configurations and to which node these linuxptp configurations shall be applied. The Spec of CR has two major sections. The first section profile contains interface, ptp4lOpts, phc2sysOpts and ptp4lConf options, the second recommend defines profile selection logic.

 PTP operator supports T-BC and Ordinary clock which can be configured via ptpConfig

ptpConfig to set up ordinary clock using single interface

NOTE: following ptp4l/phc2sys opts required when events are enabled 
    ptp4lOpts: "-2 -s --summary_interval -4" 
    phc2sysOpts: "-a -r -m -n 24 -N 8 -R 16"
apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: ordinary-clock-ptpconfig
  namespace: openshift-ptp
spec:
  profile:
  - name: "profile1"
    interface: "enp134s0f0"
    ptp4lOpts: "-s -2"
    phc2sysOpts: "-a -r"
  recommend:
  - profile: "profile1"
    priority: 4
    match:
    - nodeLabel: "node-role.kubernetes.io/worker"

ptpConfig to set up boundary clock using multiple interface

NOTE: following ptp4l/phc2sys opts required when events are enabled 
    ptp4lOpts: "-2 --summary_interval -4" 
    phc2sysOpts: "-a -r -m -n 24 -N 8 -R 16"
apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: boundary-clock-ptpconfig
  namespace: openshift-ptp
spec:
  profile:
  - name: "profile1"
    ptp4lOpts: "-s -2"
    phc2sysOpts: "-a -r"
    ptp4lConf: |
      [ens7f0]
      masterOnly 0
      [ens7f1]
      masterOnly 1
      [ens7f2]
      masterOnly 1
  recommend:
  - profile: "profile1"
    priority: 4
    match:
    - nodeLabel: "node-role.kubernetes.io/worker"

ptpConfig to override offset threshold when events are enabled

apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: event-support-ptpconfig
  namespace: openshift-ptp
spec:
  profile:
  - name: "profile1"
    ...
    ...
    ......   
    ptpClockThreshold:
      holdOverTimeout: 24 # in secs
      maxOffsetThreshold: 100 #in nano secs
      minOffsetThreshold: 100 #in nano secs
  recommend:
  - profile: "profile1"
    priority: 4
    match:
    - nodeLabel: "node-role.kubernetes.io/worker"
    

ptpConfig to filter 'master offset' and 'delay filtered' logs

apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: suppress-logs-ptpconfig
  namespace: openshift-ptp
spec:
  profile:
  - name: "profile1"
    ...
    ...
    ......   
    ptpSettings:
      stdoutFilter: "^.*delay   filtered.*$"
      logReduce: "true"
  recommend:
  - profile: "profile1"
    priority: 4
    match:
    - nodeLabel: "node-role.kubernetes.io/worker"
    

ptpConfig to configure as WPC NIC as GM

apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
  name: ptpconfig-gm
  namespace: openshift-ptp
spec:
  profile:
  - name: "profile1"
    ...
    ...
    ......   
    plugins:
      e810:
        enableDefaultConfig: true
    ts2phcOpts: " "
    ts2phcConf: |
      [nmea]
      ts2phc.master 1
      [global]
      use_syslog  0
      verbose 1
      logging_level 7
      ts2phc.pulsewidth 100000000
      #GNSS module s /dev/ttyGNSS* -al use _0
      ts2phc.nmea_serialport  /dev/ttyGNSS_1700_0
      leapfile  /usr/share/zoneinfo/leap-seconds.list
      [ens2f0]
      ts2phc.extts_polarity rising
      ts2phc.extts_correction 0
    synce4lOpts: " "
    synce4lConf: |
     [global]
     logging_level 7
     use_syslog 0
     verbose 1
     message_tag [synce4l]

     [<synce1>]
     dnu_prio 0xFF
     network_option 2
     extended_tlv 1
     recover_time 60
     clock_id 
     module_name ice

     [enp59s0f0np0]
     tx_heartbeat_msec 1000
     rx_heartbeat_msec 500
     allowed_qls 0x4
     allowed_ext_qls 0xFF
     [{SMA1}]
     board_label SMA1
     input_QL 0x1
     input_ext_QL 0x20
  recommend:
  - profile: "profile1"
    priority: 4
    match:
    - nodeLabel: "node-role.kubernetes.io/worker"
    

In above examples, profile1 will be applied by linuxptp-daemon to nodes labeled with node-role.kubernetes.io/worker.

xxx-ptpconfig CR is created with PtpConfig kind. spec.profile defines profile named profile1 which contains interface (enp134s0f0) to run ptp4l process on, ptp4lOpts (-s -2) sysconfig options to run ptp4l process with and phc2sysOpts (-a -r) to run phc2sys process with. spec.recommend defines priority (lower numbers mean higher priority, 0 is the highest priority) and match rules of profile profile1. priority is useful when there are multiple PtpConfig CRs defined, linuxptp daemon applies match rules against node labels and names from high priority to low priority in order. If any of nodeLabel or nodeName on a specific node matches with the node label or name where daemon runs, it applies profile on that node.

ptpConfig to enable High Availability for phc2sys by adding profiles of ptp4l enabled config's under haProfiles

apiVersion: ptp.openshift.io/v1
kind: PtpConfig
metadata:
name: suppress-logs-ptpconfig
namespace: openshift-ptp
spec:
phc2sysOpts: "-a -r"
ptp4lOpts: " "
profile:
- name: "enable-ha"
  ...
  ...
  ......
  ptpSettings:
    stdoutFilter: "^.*delay   filtered.*$"
    logReduce: "true"
    haProfiles: "profile1,profile2"
  recommend:
- profile: "enable-ha"
  priority: 4
  match:
    - nodeLabel: "node-role.kubernetes.io/worker"

Requirements: Two ptp4l configurations must exist with the phc2sysOPts field set to an empty string. The names of these ptp4l configurations will be used and listed under the ptpSettings/haProfiles key in the phc2sys-only enabled ptpConfig.

Quick Start

To install PTP Operator:

$ make deploy

To un-install:

$ make undeploy

ptp-operator's People

Contributors

aneeshkp avatar bnshr avatar dougbtv avatar edcdavid avatar eggfoobar avatar fedepaol avatar greyerof avatar josephdrichard avatar jupierce avatar jzding avatar lack avatar liornoy avatar mrvectorz avatar nishant-parekh avatar openshift-bot avatar openshift-ci-robot avatar openshift-ci[bot] avatar openshift-merge-bot[bot] avatar openshift-merge-robot avatar oribon avatar rpieczon avatar sabinaaledort avatar schseba avatar thiagoalessio avatar thrasher-redhat avatar vitus133 avatar williamcaban avatar ximinhan avatar yselkowitz avatar zshi-redhat avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ptp-operator's Issues

[RFE] Add ability to specify request and limits for ptp daemon

In setups with high density of processes or low latency configuration, the ptp-daemon should be running on QoS guaranteed to avoid the ptp4l from experiencing latency due to be competing with resources.

The asks are:

  • enable the ability to specify resource.request and resource.limits for the ptp-daemon through the CRs
  • enable the ability to annotate the Pod and assign the Pod to runtime classes (eg. like for PAO) such that it can be run with guaranteed resources (e.g. dedicated 1c and memory) and maintain numa alignment with the NIC from which is reading the timing.
  • Alternatively, provide a way to guarantee NUMA alignment between the ptp-daemon and the NICs for timing when running in multi-socket nodes.

test case: "[It] [ptp] PTP e2e tests Running with fifo scheduling Should check whether using fifo scheduling" is skipped in discovery mode although SCHED_FIFO policies were configured

Hi,
I executed the PTP e2e integration tests in discovery mode using ocp version 4.13.7,
and the following test case: "[It] [ptp] PTP e2e tests Running with fifo scheduling Should check whether using fifo scheduling", was skipped although SCHED_FIFO policies were configured.

ptp configs SchedulingPolicies:

[[email protected] ~]$ oc get ptpconfig -n openshift-ptp
NAME            AGE
du-ptp-bc1      2d
du-ptp-slave1   2d
du-ptp-slave2   2d
du-ptp-slave3   2d
 [[email protected] ~]$ oc get ptpconfig -n openshift-ptp du-ptp-bc1 -o yaml | grep ptpSchedulingPolicy
    ptpSchedulingPolicy: SCHED_FIFO
[[email protected] ~]$ oc get ptpconfig -n openshift-ptp du-ptp-slave1 -o yaml | grep ptpSchedulingPolicy
    ptpSchedulingPolicy: SCHED_FIFO
[[email protected] ~]$ oc get ptpconfig -n openshift-ptp du-ptp-slave2 -o yaml | grep ptpSchedulingPolicy
    ptpSchedulingPolicy: SCHED_FIFO
[[email protected] ~]$ oc get ptpconfig -n openshift-ptp du-ptp-slave3 -o yaml | grep ptpSchedulingPolicy
    ptpSchedulingPolicy: SCHED_FIFO

testcase skipped message:

Enter [BeforeEach] [ptp] - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:44 @ 08/01/23 08:39:38.138
 Exit [BeforeEach] [ptp] - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:44 @ 08/01/23 08:39:38.138 (0s)
 Enter [BeforeEach] PTP e2e tests - /home/kni/ptp-operator-testing/ptp-operator/test/pkg/execute/ginkgo.go:9 @ 08/01/23 08:39:38.138
 Exit [BeforeEach] PTP e2e tests - /home/kni/ptp-operator-testing/ptp-operator/test/pkg/execute/ginkgo.go:9 @ 08/01/23 08:39:38.138 (0s)
 Enter [BeforeEach] Running with fifo scheduling - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:576 @ 08/01/23 08:39:38.138
STEP: Refreshing configuration - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:577 @ 08/01/23 08:39:38.138
END STEP: Refreshing configuration - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:577 @ 08/01/23 08:39:38.283 (145ms)
[SKIPPED] No SCHED_FIFO policies configured
In [BeforeEach] at: /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:599 @ 08/01/23 08:39:38.305
 Exit [BeforeEach] Running with fifo scheduling - /home/kni/ptp-operator-testing/ptp-operator/test/conformance/serial/ptp.go:576 @ 08/01/23 08:39:38.305 (167ms)

I would like to know if this is an automation bug, or maybe an error when I configured the environment for the tests.
I did changed the ptpconfigs labels and node label to "ptp/clock-under-test" as it mentioned in: https://github.com/openshift/ptp-operator/tree/master/test#labelling-test-nodes-manually-in-discovery-mode.

Thanks,
Aviv

How does one monitor for PTP events such as CLOCK_REALTIME skew?

I see a reference to an event driven framework, but my google fu is not returning much here. Where can I get more information on how to deal with CLOCK_REALTIME delays. I see mentions of AMQ and cloud-event-proxy, but I just need alerts, from within the cluster.

"make deploy-setup" does not abend when tools are missing

In my example cluster, I ran "make deploy-setup" with go and skopeo missing.

Errors about the missing commands showed on screen but hack/deploy-setup.sh proceeded with a broken installation.

Suggested resolution: deploy-setup.sh should verify all commands it needs are available before proceeding.

conformance test should set phc2sys domain to 24

Currently conformance test set up this ptp config,

apiVersion: v1
items:
- apiVersion: ptp.openshift.io/v1
  kind: PtpConfig
  metadata:
    creationTimestamp: "2022-11-08T15:59:13Z"
    generation: 1
    name: test-slave1
    namespace: openshift-ptp
    resourceVersion: "10761981"
    uid: b4e88bab-dd92-4d3e-ae7e-972a8387930e
  spec:
    profile:
    - interface: ens1f0
      name: test-slave1
      phc2sysOpts: -a -r
      ptp4lConf: |-
        [global]
        tx_timestamp_timeout 50
        ptp_dst_mac 01:1B:19:00:00:00
        p2p_dst_mac 01:80:C2:00:00:0E
        domainNumber 24
        logging_level 7
      ptp4lOpts: -2 -s
      ptpSchedulingPolicy: SCHED_OTHER
      ptpSchedulingPriority: 65
    recommend:
    - match:
      - nodeLabel: ptp/clock-under-test
      priority: 5
      profile: test-slave1
kind: List
metadata:
  resourceVersion: ""

Because `domainNumber 24` is used in ptp4lConf, phc2sysOpts should be updated to `phc2sysOpts: -a -r -n 24`

Implement webhhok validation for scheduler type and priority

This #149 introduces two new fields in the ptpconfig.

The PR was merged to allow vendoring from the linuxptp-daemon project.

Opening this issue to remain the follow-up work needed to complete the feature from the operator point of view

  • adding validating to the fields
  • adding functional tests

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.