Code Monkey home page Code Monkey logo

oaic's People

Contributors

higgigamer19 avatar jgaeddert avatar joshuamoorexyz avatar mrkouchaki avatar nathanstep55 avatar nishaadh avatar pratheeksu avatar romilbalar avatar tangecemsu avatar wchen2654 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

oaic's Issues

Error: could not find a ready tiller pod

Steps to reproduce:
Follow along 'O-RAN Near-Real Time RIC Installation Guide':

command: sudo helm install stable/nfs-server-provisioner --namespace ricinfra --name nfs-release-1

Error: could not find a ready tiller pod

Possible cause of error:
(log after running sudo ./k8s-1node-cloud-init-k_1_16-h_2_17-d_cur.sh)
Not installing Tiller due to 'client-only' flag having been set
++ pwd

  • export HELM_HOME=/root/.helm
  • HELM_HOME=/root/.helm
  • echo HELM_HOME=/root/.helm
  • helm version
    Client: &version.Version{SemVer:"v2.17.0", GitCommit:"a690bad98af45b015bd3da1a41f6218b1a451dbe", GitTreeState:"clean"}
    Error: could not find a ready tiller pod
  • echo 'Waiting for Helm to be ready'
    Waiting for Helm to be ready
  • sleep 15
  • helm version

(8/8 pods were up)

Separate registry for xApps

Current design does not include a separate registry for the xApp images. Since the image is created and is present in the local machine, the tag provided to the xApp image works. But ideally, we need a separate registry for xApps

'Did not find expected key' error

I noticed that on step 4 of the 'Near-Real Time RIC Installation Procedure', I run into the following error:

Error: failed to parse ../RECIPE_EXAMPLE/PLATFORM/example_recipe_oran_e_release_modified_e2.yaml: error converting YAML to JSON: yaml: line 113: did not find expected key

command run: cd RIC-Deployment/bin
sudo ./deploy-ric-platform -f ../RECIPE_EXAMPLE/PLATFORM/example_recipe_oran_e_release_modified_e2.yaml

Is there something missing or is this a non-issue for the overall build?

ricplt-xapp-onboarder fails to start

The service deployment-ricplt-xapp-onboarder fails to start when deploying the ric from the main documentation. There is no error or mention in the output while deploying the ric platform.

  • Occurring on Ubuntu:20.04
  • Same issue occurs on bare metal and kind one node cluster.
  • When undeploying the ric platform it states that the onboarder "does not exist" and continues to undeploy other services.

Error: release r4-e2term failed: Service "service-ricplt-e2term-sctp-alpha" is invalid: spec.ports[0].protocol: Unsupported value: "SCTP": supported values: "TCP", "UDP"

Hi,
when I executed this command to deploy ric, an error occurred. So when I setup my own 5G network, service-ricplt-e2term-sctp-alpha was found.
sudo ./deploy-ric-platform -f ../RECIPE_EXAMPLE/PLATFORM/example_recipe_oran_e_release_modified_e2.yaml
an error occurred:
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "local" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
Saving 1 charts
Downloading ric-common from repo http://127.0.0.1:8879/charts
Deleting outdated charts
Error: release r4-e2term failed: Service "service-ricplt-e2term-sctp-alpha" is invalid: spec.ports[0].protocol: Unsupported value: "SCTP": supported values: "TCP", "UDP"
when I execuyed this command : kubectl get svc -n ricplt --field-selector metadata.name=service-ricplt-e2term-sctp-alpha
No resources found in ricplt namespace.

kubectl get svc -n ricplt
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
aux-entry ClusterIP 10.111.240.79 80/TCP,443/TCP 47m
r4-infrastructure-kong-proxy NodePort 10.109.94.119 32080:32080/TCP,32443:32443/TCP 47m
r4-infrastructure-prometheus-alertmanager ClusterIP 10.96.113.213 80/TCP 47m
r4-infrastructure-prometheus-server ClusterIP 10.102.8.5 80/TCP 47m
ricplt-influxdb ClusterIP 10.97.139.248 8086/TCP,8088/TCP 18m
service-ricplt-a1mediator-http ClusterIP 10.107.241.10 10000/TCP 28m
service-ricplt-a1mediator-rmr ClusterIP 10.103.24.82 4561/TCP,4562/TCP 28m
service-ricplt-alarmmanager-http ClusterIP 10.102.208.6 8080/TCP 20m
service-ricplt-alarmmanager-rmr ClusterIP 10.102.188.206 4560/TCP,4561/TCP 20m
service-ricplt-appmgr-http ClusterIP 10.107.175.50 8080/TCP 39m
service-ricplt-appmgr-rmr ClusterIP 10.108.153.62 4561/TCP,4560/TCP 39m
service-ricplt-dbaas-tcp ClusterIP None 6379/TCP 45m
service-ricplt-e2mgr-http ClusterIP 10.106.190.81 3800/TCP 34m
service-ricplt-e2mgr-rmr ClusterIP 10.98.227.250 4561/TCP,3801/TCP 34m
service-ricplt-e2term-prometheus-alpha ClusterIP 10.107.94.0 8088/TCP 31m
service-ricplt-e2term-rmr-alpha ClusterIP 10.109.190.178 4561/TCP,38000/TCP 31m
service-ricplt-jaegeradapter-agent ClusterIP 10.109.193.226 5775/UDP,6831/UDP,6832/UDP 17m
service-ricplt-jaegeradapter-collector ClusterIP 10.98.168.34 14267/TCP,14268/TCP,9411/TCP 17m
service-ricplt-jaegeradapter-query ClusterIP 10.106.84.73 16686/TCP 17m
service-ricplt-o1mediator-http ClusterIP 10.96.142.139 9001/TCP,8080/TCP,3000/TCP 23m
service-ricplt-o1mediator-tcp-netconf NodePort 10.98.37.37 830:30830/TCP 23m
service-ricplt-rtmgr-http ClusterIP 10.108.156.252 3800/TCP 37m
service-ricplt-rtmgr-rmr ClusterIP 10.105.255.237 4561/TCP,4560/TCP 37m
service-ricplt-submgr-http ClusterIP None 3800/TCP 26m
service-ricplt-submgr-rmr ClusterIP None 4560/TCP,4561/TCP 26m
service-ricplt-vespamgr-http ClusterIP 10.99.186.75 8080/TCP,9095/TCP 25m
service-ricplt-xapp-onboarder-http ClusterIP 10.104.85.80 8888/TCP,8080/TCP 42m

server localhost:8080 was refused during step 3

I followed it step by step but I stuck by step 3 .
When I Run the script "k8s-1node-cloud-init-k ... .sh," I received the below message:
The connection to the server localhost:8080 was refused - did you specify the right host or port?

Flannel will not deploy in O-RAN near-RT RIC installation

Issue: When running the k8s-1node-cloud-init-k_1_16-h_2_17-d_cur.sh script as part of the installation for the near-RT RIC, when applying the kube-flannel.yml located here, Flannel never deploys and CoreDNS will remain pending.

Reason: A commit last month in the flannel-io/flannel repository removes the PodSecurityPolicy in kube-flannel.yml and moves it to kube-flannel-psp.yml

Solution: In the k8s_vm_install.sh script used in RIC-Deployment, replace the kube-flannel.yml link with one from before the problematic commit: https://raw.githubusercontent.com/flannel-io/flannel/9de10c12c8266b0cfe09bc0d5c969ae28832239f/Documentation/kube-flannel.yml
An alternative (and more futureproof solution) is to apply both kube-flannel.yml and kube-flannel-psp.yml from the repository. I have not tested this, so I cannot confirm if it works or not.

NextRAN xapp

  • Integrate NextRan xapp with 5G NSA
  • Modify the mac layer scheduler

Kong ingress-controller causes CrashLoopBackOff

When the near-RT RIC is deployed, the Kong ingress-controller never becomes ready and ultimately causes a CrashLoopBackOff.

One of the PCs failed on:

  • Ubuntu 18.04 host after reinstallation of near-RT RIC
  • Ubuntu 20.04 VirtualBox VM after reinstallation of near-RT RIC (twice)
  • Ubuntu 22.04 VirtualBox VM on new install of near-RT RIC

Another laptop failed on:

  • Ubuntu 20.04 host on new install of near-RT RIC using Kind w/ Docker

Not all of my installations have this issue. Here is a list of successful configurations on other PCs:

  • Ubuntu 20.04 Virt Manager VM on new install of near-RT RIC
  • Ubuntu 20.04 host on new install of near-RT RIC
  • Windows 10 host on new install of near-RT RIC using Kind w/ Docker

One Kong log (18.04 host) shows the following error (repeated for every ricplt service):

-------------------------------------------------------------------------------
Kong Ingress controller
  Release:    0.7.0
  Build:      e4605db
  Repository: [email protected]:kong/kubernetes-ingress-controller.git
  Go:         go1.13.1
-------------------------------------------------------------------------------

I0623 18:03:57.598271       1 main.go:407] Creating API client for https://10.96.0.1:443
I0623 18:03:57.604753       1 main.go:451] Running in Kubernetes Cluster version v1.16 (v1.16.0) - git (clean) commit 2bd9643cee5b3b3a5ecbd3af49d09018f0773c77 - platform linux/amd64
I0623 18:04:12.741756       1 main.go:187] kong version: 1.4.3
I0623 18:04:12.741772       1 main.go:196] Kong datastore: off
I0623 18:04:12.867656       1 controller.go:224] starting Ingress controller
I0623 18:04:12.881103       1 status.go:201] new leader elected: r4-infrastructure-kong-646b68bd88-pltr2
E0623 18:04:42.869142       1 controller.go:119] unexpected failure updating Kong configuration: 
posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout
W0623 18:04:42.869187       1 queue.go:112] requeuing , err posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout
E0623 18:05:12.869925       1 controller.go:119] unexpected failure updating Kong configuration: 
posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout
W0623 18:05:12.869946       1 queue.go:112] requeuing ricplt/ingress-ricplt-a1mediator, err posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout
I0623 18:05:12.893015       1 status.go:342] updating Ingress ricplt/ingress-ricplt-xapp-onboarder-chartmuseum status to [{10.193.124.136 }]
I0623 18:05:12.893063       1 status.go:342] updating Ingress ricplt/ingress-ricplt-e2mgr status to [{10.193.124.136 }]
I0623 18:05:12.893099       1 status.go:342] updating Ingress ricplt/ingress-ricplt-a1mediator status to [{10.193.124.136 }]
I0623 18:05:12.894140       1 status.go:342] updating Ingress ricplt/ingress-ricplt-appmgr status to [{10.193.124.136 }]
I0623 18:05:12.894642       1 status.go:342] updating Ingress ricplt/ingress-ricplt-xapp-onboarder-server status to [{10.193.124.136 }]
E0623 18:05:42.870662       1 controller.go:119] unexpected failure updating Kong configuration: 
posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout

Ultimately, it ends up in a loop trying to update every ricplt service:

W0623 18:10:29.541816       1 queue.go:112] requeuing ricplt/service-ricplt-appmgr-http, err posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout
E0623 18:10:59.542263       1 controller.go:119] unexpected failure updating Kong configuration: 
posting new config to /config: making HTTP reqeust: Post https://localhost:8444/config?check_hash=1: dial tcp 129.174.106.106:8444: i/o timeout

Another installation (20.04 host using Kind) simply shows the following Kong log:

-------------------------------------------------------------------------------
Kong Ingress controller
  Release:    0.7.0
  Build:      e4605db
  Repository: [email protected]:kong/kubernetes-ingress-controller.git
  Go:         go1.13.1
-------------------------------------------------------------------------------

I0810 22:34:46.447659       1 main.go:407] Creating API client for https://10.96.0.1:443
I0810 22:34:46.456529       1 main.go:451] Running in Kubernetes Cluster version v1.16 (v1.16.15) - git (clean) commit 2adc8d7091e89b6e3ca8d048140618ec89b39369 - platform linux/amd64

It crashes because Kubernetes' liveness probe fails:

Events:
  Type     Reason     Age                  From     Message
  ----     ------     ----                 ----     -------
  Warning  BackOff    38h (x472 over 40h)  kubelet  Back-off restarting failed container
  Warning  Unhealthy  55s (x113 over 40h)  kubelet  Liveness probe failed: Get http://10.244.0.32:10254/healthz: dial tcp 10.244.0.32:10254: connect: connection refused

The majority of my near-RT RIC installations have faced this problem, mostly crashing without meaningful logs like in the latter installation shown above. Reinstallation (./undeploy-ric-platform, running k8s script, then ./deploy-ric-platform) does not help. The former issue in the logs may or may not be related to the latter issue. It is worth noting that the Kong pod has two containers: ingress-controller and proxy, and proxy seems to work fine.

5G Network script has invalid lines

oaic/setup5GNetwork.sh

Lines 17 to 19 in e2659a2

sudo srsenb --enb.n_prb=50 --enb.name=enb1 --enb.enb_id=0x19B \
--rf.device_name=zmq --rf.device_args="fail_on_disconnect=true,tx_port0=tcp://*:2000,rx_port0=tcp://localhost:2001,tx_port1=tcp://*:2100,rx_port1=tcp://localhost:2101,id=enb$" \
--ric.agent.remote_ipv4_addr=${E2TERM_IP} --log.all_level=warn --ric.agent.log_level=debug --log.filename=enbLog.txt --ric.agent.local_ipv4_addr=${E2NODE_IP} --ric.agent.loc$

It looks as though the ends of these lines were clipped when copied from a terminal window. Compare to the commands in the documentation

Nexran unable to deploy on e-release of near real time ric using dms_cli tool

I'm trying to run nexran xapp on the e-release of the near real time ric using the dms_cli xapp_onboarder tool. Here's the problem that I'm facing:

When running this command dms_cli install --xapp_chart_name=nexran --version=0.1.0 --namespace=ricxapp, it states an error showing how there is a missing chart name, contradicting the fact that there is already a chart name stated in the --xapp_chart_name variable.

How to reproduce:

  1. Obtain the xapp-embedded-schema.json from the nexran repo on github # I put this file under the nexran directory which is located under the oaic directory

  2. Create a local helm repository with a port other than 8080 on host
    docker run --rm -u 0 -it -d -p 8090:8080 -e DEBUG=1 -e STORAGE=local -e STORAGE_LOCAL_ROOTDIR=/charts -v $(pwd)/charts:/charts chartmuseum/chartmuseum:latest

  3. Set an environment variable for Chart repo
    export CHART_REPO_URL=http://0.0.0.0:8090

  4. Install the dms_cli tool # Website that I used: https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-ric-dep/en/latest/installation-guides.html

sudo -i # I had to go to super user to download dms_cli due to a permission error later on
git clone "https://gerrit.o-ran-sc.org/r/ric-plt/appmgr" # I cloned this under the oaic repository
cd appmgr/xapp_orchestrater/dev/xapp_onboarder
sudo apt install python3-pip # Python3 is required for this
pip3 install ./ # Installs xapp_onboarder

  1. Set permissions to non-root users for certain files
    sudo chmod 755 /usr/local/bin/dms_cli
    sudo chmod -R 755 /usr/local/lib/python3.6
    sudo chmod -R 755 /usr/local/lib/python3.6

  2. Onboard the xApp
    dms_cli onboard ~/home/<user_name>/oaic/nexran/nextran-config-file.json ~/home/<user_name>/oaic/nexran/xapp-embedded-schema.json

  3. Deploying the xApp The Error
    dms_cli install --xapp_chart_name=nexran --version=0.1.0 --namespace=ricxapp

coredns pod not in ready state

Hello,
When trying to bring up the kubernetes pods during RIC Kubernetes Cluster Installation, I am facing an issue where pods related to coreDNS are running but not in the ready state. After checking the logs related to CoreDNS, I can see that there is a taint which is not tolerated which is stopping the coreDNS from moving to ready state.
I have tried to debug this but unable to resolve the issue successfully. Could you please check the issue and let me know how to make sure that the CoreDNS pods are up and ready.
Attached the relevant log files.

aerpawops@aerpaw17:~$ sudo kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-5644d7b6d9-twr2q 0/1 Running 1 7m52s 10.244.0.7 aerpaw17
kube-system coredns-5644d7b6d9-wrlvh 0/1 Running 1 7m52s 10.244.0.6 aerpaw17
kube-system etcd-aerpaw17 1/1 Running 1 7m9s 10.153.13.85 aerpaw17
kube-system kube-apiserver-aerpaw17 1/1 Running 1 7m1s 10.153.13.85 aerpaw17
kube-system kube-controller-manager-aerpaw17 1/1 Running 1 6m45s 10.153.13.85 aerpaw17
kube-system kube-flannel-ds-wdrcx 1/1 Running 1 7m52s 10.153.13.85 aerpaw17
kube-system kube-proxy-9jqm9 1/1 Running 1 7m52s 10.153.13.85 aerpaw17
kube-system kube-scheduler-aerpaw17 1/1 Running 1 7m8s 10.153.13.85 aerpaw17
kube-system tiller-deploy-7d7bc87bb-wfsdn 1/1 Running 1 6m39s 10.244.0.5 aerpaw17

aerpawops@aerpaw17:~$ sudo kubectl describe pod/coredns-5644d7b6d9-twr2q -n kube-system

Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True

Events:
Type Reason Age From Message


Warning FailedScheduling default-scheduler 0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
Warning FailedScheduling default-scheduler 0/1 nodes are available: 1 node(s) had taints that the pod didn't tolerate.
Normal Scheduled default-scheduler Successfully assigned kube-system/coredns-5644d7b6d9-twr2q to aerpaw17
Normal Pulled 10m kubelet, aerpaw17 Container image "k8s.gcr.io/coredns:1.6.2" already present on machine
Normal Created 10m kubelet, aerpaw17 Created container coredns
Normal Started 10m kubelet, aerpaw17 Started container coredns
Warning Unhealthy 9m6s (x6 over 9m56s) kubelet, aerpaw17 Readiness probe failed: HTTP probe failed with statuscode: 503
Warning FailedCreatePodSandBox 7m31s kubelet, aerpaw17 Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "85032397a0f2c6b6c86aa9510cdf7ae27e90c415d9908932b7489306dd7576a6" network for pod "coredns-5644d7b6d9-twr2q": networkPlugin cni failed to set up pod "coredns-5644d7b6d9-twr2q_kube-system" network: open /run/flannel/subnet.env: no such file or directory, failed to clean up sandbox container "85032397a0f2c6b6c86aa9510cdf7ae27e90c415d9908932b7489306dd7576a6" network for pod "coredns-5644d7b6d9-twr2q": networkPlugin cni failed to teardown pod "coredns-5644d7b6d9-twr2q_kube-system" network: fork/exec /opt/cni/bin/flannel: text file busy]
Warning FailedCreatePodSandBox 7m30s kubelet, aerpaw17 Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "0961e5369fdb041d6ccc84c5c70de16bbce37143798f275716c4fe11c457f850" network for pod "coredns-5644d7b6d9-twr2q": networkPlugin cni failed to set up pod "coredns-5644d7b6d9-twr2q_kube-system" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 7m29s kubelet, aerpaw17 Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "5032a83ded37f0dfbfa4e93222c6f62796f5af885ef2f722fb1d1c22b25db79f" network for pod "coredns-5644d7b6d9-twr2q": networkPlugin cni failed to set up pod "coredns-5644d7b6d9-twr2q_kube-system" network: open /run/flannel/subnet.env: no such file or directory
Warning FailedCreatePodSandBox 7m28s kubelet, aerpaw17 Failed create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "00106dc0a7a73b0698838534baf2d5f9d44549f73eb47cbb1d4e14d8e70d11be" network for pod "coredns-5644d7b6d9-twr2q": networkPlugin cni failed to set up pod "coredns-5644d7b6d9-twr2q_kube-system" network: open /run/flannel/subnet.env: no such file or directory
Normal SandboxChanged 7m27s (x5 over 7m32s) kubelet, aerpaw17 Pod sandbox changed, it will be killed and re-created.
Normal Pulled 7m27s kubelet, aerpaw17 Container image "k8s.gcr.io/coredns:1.6.2" already present on machine
Normal Created 7m27s kubelet, aerpaw17 Created container coredns
Normal Started 7m27s kubelet, aerpaw17 Started container coredns
Warning Unhealthy 2m20s (x31 over 7m20s) kubelet, aerpaw17 Readiness probe failed: HTTP probe failed with statuscode: 503

srsenb "unrecognized option '--gw.netns=ue1'" in ZMQ mode

Tagging @pratheeksu for support. Running without this option seems to work.

sudo srsenb --enb.n_prb=50 --enb.name=enb1 --enb.enb_id=0x19B --gw.netns=ue1 \
--rf.device_name=zmq --rf.device_args="fail_on_disconnect=true,tx_port0=tcp://*:2000,rx_port0=tcp://localhost:2001,tx_port1=tcp://*:2100,rx_port1=tcp://localhost:2101,id=enb,base_srate=23.04e6" \
--ric.agent.remote_ipv4_addr=${E2TERM_IP} --log.all_level=warn --ric.agent.log_level=debug --log.filename=stdout --ric.agent.local_ipv4_addr=${E2NODE_IP} --ric.agent.local_port=${E2NODE_PORT}

Ubuntu 20.04

When running ./gen-cloud-init.sh command ubuntu 20.04 is not allowed in generated shell script " ./k8s-1node-cloud-init-k_1_16-h_2_17-d_cur.sh " file. I added below lines manually and it worked for me.

elif [[ ${UBUNTU_RELEASE} == 20. ]]; then
echo "Installing on Ubuntu $UBUNTU_RELEASE (Bionic Beaver)"
if [ ! -z "${DOCKERV}" ]; then
DOCKERVERSION="${DOCKERV}-0ubuntu1~20.04.4"
fi
*

change-1
IMG-4339 (1)

Fatal Errorr

fatal: No url found for submodule path 'asn1c' in .gitmodules

RIC EC2 Termination docker registry / base image is unavailable (timeout)

Hey, thanks for your work.

Problem

I am trying to setup the testbed as described in the docs and it went really well until I had to set the RIC-EC2-TERMINATION feature.

More specifically this is the image in question: nexus3.o-ran-sc.org:10002/o-ran-sc/bldr-ubuntu20-c-go:1.0.0.
Trying to pull it manually yield the following:

Step 1/70 : FROM nexus3.o-ran-sc.org:10002/o-ran-sc/bldr-ubuntu20-c-go:1.0.0 as ubuntu
Get "https://nexus3.o-ran-sc.org:10002/v2/": context deadline exceeded (Client.Timeout exceeded whileawaiting headers)

Possible Reason

The reason is that the base image in the dockerfile found in oaic/ric-plt-e2/RIC-E2-TERMINATION is either inaccessible, too slow or a pre-setup is necessary, not really sure.

Reproduce

The step that is failing can be found in this section of the docs:
https://openaicellular.github.io/oaic/oran_installation.html#creating-docker-image

System Info

Not that is matters but here are the host details:

OS: Ubuntu 20.04 focal
Kernel: x86_64 Linux 5.4.0-126-generic
CPU: Intel Xeon Gold 6242R @ 8x 3.093GHz

Issue with multiUE simulation

Hello,

When I am trying with a multi-UE setup with the commands as listed, I am getting the following error. What should be the port values to be entered for Port0 and Port1. Could you please provide the modified command for setting up srsue

root@shreyas-VirtualBox:~# sudo srsue /home/shreyas/.config/srsran/ue.conf --rf.device_name=zmq --rf.device_args="fail_on_disconnect=true,tx_port=tcp://*:2010,rx_port=tcp://localhost:2008,id=ue1,base_srate=23.04e6" --gw.netns=ue1 --usim.algo=xor --usim.imsi=001010123456789
Reading configuration file /home/shreyas/.config/srsran/ue.conf...

Built in RelWithDebInfo mode using commit eee2bbf on branch HEAD.

Opening 2 channels in RF device=zmq with args=fail_on_disconnect=true,tx_port=tcp://:2010,rx_port=tcp://localhost:2008,id=ue1,base_srate=23.04e6
Available RF device list: UHD zmq
CHx base_srate=23.04e6
CHx id=ue1
Current sample rate is 1.92 MHz with a base rate of 23.04 MHz (x12 decimation)
CH0 rx_port=tcp://localhost:2008
CH0 tx_port=tcp://
:2010
CH0 fail_on_disconnect=true
[zmq] ue1 Tx port not specified. Disabling transmitter.
[zmq] ue1 Rx port not specified. Disabling receiver.
[zmq] Error: Neither Tx port nor Rx port specified.
Error initializing radio.

kpimon metrics not OK

metrics are Not OK, I receive "-1" for available PRBs, QCI=0
We investigated issue from KPIMON & srsRAN side, itseems both are working well but there should be an issue, maybe srsRAN are not sending metrics to E2 interface coreectly,

Convoluted Setup - Further deployment automation

Hey all, thanks a lot for putting effort into this testbed setup, I would like to to start a discussion regarding developer experience.

What's wrong?

Lot's of manual processes can be automated using proper scripts or proper server config and provision systems (Ansible / Terraform). I had colleagues struggling with setting up the testbed.
Despite using cloud methods, "Cloud Native" mindset is not present and the setup is much messier than is should be imho.

Solution

  • Make the scripts "sane" - embedding k8s resources into bash scripts as strings seem sloppy
  • Perhaps bundling the RIC into a properly made helm chart
  • Setup of xApp with an external Ngnix server only to server config is not a great solution in my book
  • Using ansible / puppet / Terraform to have a fully automated setup

What do I propose?

  • I can submit a PR that lay out what I have in mind.
    My current setup is much friendlier and less prone to errors.

srsepc says that : T3413 expired -- Could not page the ue.

Dears,
Recently I received an error message during the Setup of my own 5G Network. I'm not sure about the cause and side effects of the issue.

`EPC terminal:
ubnt@ubnt-VirtualBox:$ sudo -i
[sudo] password for ubnt:
root@ubnt-VirtualBox:
# sudo srsepc

Built in RelWithDebInfo mode using commit eee2bbf on branch HEAD.

--- Software Radio Systems EPC ---

Couldn't open , trying /root/.config/srsran/epc.conf
Reading configuration file /root/.config/srsran/epc.conf...
Couldn't open user_db.csv, trying /root/.config/srsran/user_db.csv
HSS Initialized.
MME S11 Initialized
MME GTP-C Initialized
MME Initialized. MCC: 0xf001, MNC: 0xff01
SPGW GTP-U Initialized.
SPGW S11 Initialized.
SP-GW Initialized.
Received S1 Setup Request.
S1 Setup Request - eNB Name: enb1, eNB id: 0x19b
S1 Setup Request - MCC:001, MNC:01
S1 Setup Request - TAC 7, B-PLMN 0xf110
S1 Setup Request - Paging DRX v128
Sending S1 Setup Response
Initial UE message: LIBLTE_MME_MSG_TYPE_ATTACH_REQUEST
Received Initial UE message -- Attach Request
Attach request -- IMSI: 001010123456789
Attach request -- eNB-UE S1AP Id: 1
Attach request -- Attach type: 1
Attach Request -- UE Network Capabilities EEA: 11110000
Attach Request -- UE Network Capabilities EIA: 01110000
Attach Request -- MS Network Capabilities Present: false
PDN Connectivity Request -- EPS Bearer Identity requested: 0
PDN Connectivity Request -- Procedure Transaction Id: 1
PDN Connectivity Request -- ESM Information Transfer requested: false
Downlink NAS: Sending Authentication Request
UL NAS: Received Authentication Response
Authentication Response -- IMSI 001010123456789
UE Authentication Accepted.
Generating KeNB with UL NAS COUNT: 0
Downlink NAS: Sending NAS Security Mode Command.
UL NAS: Received Security Mode Complete
Security Mode Command Complete -- IMSI: 001010123456789
Getting subscription information -- QCI 7
Sending Create Session Request.
Creating Session Response -- IMSI: 1010123456789
Creating Session Response -- MME control TEID: 1
Received GTP-C PDU. Message type: GTPC_MSG_TYPE_CREATE_SESSION_REQUEST
SPGW: Allocated Ctrl TEID 1
SPGW: Allocated User TEID 1
SPGW: Allocate UE IP 172.16.0.2
Received Create Session Response
Create Session Response -- SPGW control TEID 1
Create Session Response -- SPGW S1-U Address: 127.0.1.100
SPGW Allocated IP 172.16.0.2 to IMSI 001010123456789
Adding attach accept to Initial Context Setup Request
Sent Initial Context Setup Request. E-RAB id 5
Received Initial Context Setup Response
E-RAB Context Setup. E-RAB id 5
E-RAB Context -- eNB TEID 0x1; eNB GTP-U Address 127.0.1.1
UL NAS: Received Attach Complete
Unpacked Attached Complete Message. IMSI 1010123456789
Unpacked Activate Default EPS Bearer message. EPS Bearer id 5
Received GTP-C PDU. Message type: GTPC_MSG_TYPE_MODIFY_BEARER_REQUEST
Sending EMM Information
Received UE Context Release Request. MME-UE S1AP Id 1
There are active E-RABs, send release access bearers request
Received GTP-C PDU. Message type: GTPC_MSG_TYPE_RELEASE_ACCESS_BEARERS_REQUEST
Received UE Context Release Complete. MME-UE S1AP Id 1
UE Context Release Completed.
Initial UE message: NAS Message Type Unknown
Received Initial UE message -- Service Request
Service request -- S-TMSI 0xef7d585c
Service request -- eNB UE S1AP Id 2
Service Request -- Short MAC valid
Service Request -- User is ECM DISCONNECTED
UE previously assigned IP: 172.16.0.2
Generating KeNB with UL NAS COUNT: 2
UE Ctr TEID 0
Sent Initial Context Setup Request. E-RAB id 5
Found UE for Downlink Notification
MME Ctr TEID 0x1, IMSI: 001010123456789
Received Initial Context Setup Response
E-RAB Context Setup. E-RAB id 5
E-RAB Context -- eNB TEID 0x2; eNB GTP-U Address 127.0.1.1
Initial Context Setup Response triggered from Service Request.
Sending Modify Bearer Request.
Received GTP-C PDU. Message type: GTPC_MSG_TYPE_MODIFY_BEARER_REQUEST
Modify Bearer Request received after Downling Data Notification was sent
T3413 expired -- Could not page the ue.
Received GTP-C PDU. Message type: GTPC_MSG_TYPE_DOWNLINK_DATA_NOTIFICATION_FAILURE_INDICATION
`

nexRAN xapp can't send control request to enb

Steps to reproduce the problem:

  1. Get 5G tesbed ready following OAIC documentation and have epc, enB and ue running and nexRAN xapp deployed.
  2. Do POST request to add eNB so that eventually, you will be able to bind "fast" and "slow" slices to the eNB and bind UE to fast or slow slice.

Expected output with nexran logs VS real output:
On nexran logs I should have got message "sent control request" after initialing subscription request.
Actual output: No such message

On Base Station side,
I should have got,
Actual missing . Instead, I get first, RicSubscription request, message for "Received RICsubscription Request", and just metrices from UE in loop.

Here are nexran logs and base station logs along with actual expected outcome:

Actual expected outcome:
Nexranlog
Screenshot 2022-10-28 155809

Base Station Side
Screenshot 2022-10-28 155935

Nexran logs:
nexran-logs.txt

Screenshot: See after requestor ID, "sent control request message is missing" See above logs for full log.
Screenshot 2022-10-28 160418

Base station logs
bs_log_nexran.txt

Note that I am able to create slice, ues, add enbs. But since, enb and xapp's connection has some problem. Resources are not being allocated as required. So, even if I allocate UE to slow slice there is not change in data transfer rate between UE and network.

Arrange documentation

Installation Procedures Section:

  • Xapp Deployment Section:
    +Create new page "Xapp Development"
  • include getting started developing xapps
  • include helloworld xapp code and deployment
  • xapp frame
  • Software Installation and deployment
  • Create new page "Xapp deployment"
    +new page with explicit instructions for kpimon
    -include xapp deployment - general guidelines

remove "Create an xApp Image Registry๏ƒ

First we need to create a registry to host our xApp images. Here we create a local docker registry called xApp-registry.local:5008. 5008 indicates the port number on which the registry is listening"

remove "DMS-CLI Deployment"

  • make new page

waiting for 5/8 pods running in namespace

In step 3, after running sudo ./k8s-1node-cloud-init-k_1_16-h_2_17-d_cur.sh , script struggled in the loop:

  • echo '> waiting for 5/8 pods running in namespace [kube-system] with keyword [Running]'

waiting for 5/8 pods running in namespace [kube-system] with keyword [Running]

  • '[' 5 -lt 8 ']'
  • sleep 5

Symbolic links to config files - Web server

The xApp config files are copied to the directory accessible by the web server. This means that every time we change the config file we will need to replace it in the web server accessible directory. This is fine for users planning to test out our platforms but during development of new xApps this can prove to be a tedious task.

Instead we can use symbolic links to config files, so that whenever a change is made we do not need to copy paste the updated file. We can explore other more efficient methods as well.

Thanks to Joao Santos for suggesting this.

Recipe Example file not found

The file name is not OK: ../RECIPE_EXAMPLE/PLATFORM/example_recipe_oran_e_release_modified_e2.yaml
I used : example_recipe_oran_e_release.yaml

Problem with step: "Setup your own 5G Network"

command:
sudo srsepc

Error encountered:
Couldn't open , trying /home/srsran2/.config/srsran/epc.conf
Reading configuration file /home/srsran2/.config/srsran/epc.conf...
Couldn't open user_db.csv, trying /home/srsran2/.config/srsran/user_db.csv
HSS Initialized.
MME S11 Initialized
MME GTP-C Initialized
MME Initialized. MCC: 0xf001, MNC: 0xff01
SPGW GTP-U Initialized.
SPGW S11 Initialized.
SP-GW Initialized.

Steps to reproduce:
Finish steps 1 & 2 successfully (O-RAN and srsRAN setup respectively)
run command: sudo srsepc

Deploying OAIC using Docker and kind/minikube

Current steps:

  • Install Docker and kind to the host computer
    • Docker can be installed using Docker Desktop or Docker Engine, and kind can be installed using Homebrew or as a Golang package
  • Download the Dockerfile and config.yaml from the oaicinstaller repository (currently located here)
  • Create a kind cluster with kind create cluster --image=kindest/node:v1.16.15 --config=config.yaml
  • Build the Dockerfile with docker build -t oaic . which will configure kind through kubectl, install Helm, deploy the near-RT RIC and the srsRAN software

Current issues:

  • TLS certificate doesn't include the address host.docker.internal, which prevents Helm from being configured inside a Docker container (a new TLS certificate needs to be issued)
    • Configuring the Kubernetes cluster could be done outside of Docker, but requires extra steps
  • It may not be a good idea to run all of the scripts inside of a Dockerfile like it is currently as the end user will have to rebuild the Dockerfile instead of being able to pull the Docker image directly from Docker Hub, etc.
    • The best we could currently do is allow the end user to use Docker Compose to pull multiple OAIC images from Docker Hub at once
    • We should look to see if kind supports running commands directly, either through its config.yaml, a Dockerfile, or some other solution.

KPIMON Xapp

  • Integration of kpimon with 5G NSA
  • Collect metrics like prb usage, 5 qi, number of bytes

OAIC Installation Error

While trying to configure OAIC, I am stuck while trying to run the script k8s-1node-cloud-init-k_1_16-h_2_17-d_cur.sh in the directory oaic/RIC-Deployment/tools/k8s/bin/. I have added the script and the stack trace file below.

stacktrace.txt

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.