pccproject / pcc-uspace Goto Github PK

The latency values here are doubles. Treating as long variables will make them zeros and lead all RTT samples to be zeros. Thus, this will cause a wrong observation to be fed into the RL model.

Hi Author,

Thanks for your great job.
I had a problem. When I am training the PCC-RL online. I choose history length as 4:

cd src
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:pwd/core/ ./app/pccclient send 127.0.0.1 9000 --pcc-rate-control=python -pyhelper=shim -pypath=/path/to/pcc-rl/src/udt-plugins/training/ --history-len=4 --pcc-utility-calc=linear

However, the trained model is still using len = 10. It seems that the argument (len=4) does not used in the training. Can I have your help?

Thanks and regards,
Tianbo

Hi,when I build project on my mac,some errors occurred.I don't know how to solve it.

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/functional:688:21: error: invalid operands to binary expression ('const timespec' and 'const timespec')
{return __x == __y;}
~~~ ^ ~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/unordered_map:516:17: note: in instantiation of member function 'std::__1::equal_to::operator()' requested here
{return static_cast<const _Pred&>(*this)(__x.__get_value().first, __y);}
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/__hash_table:2472:24: note: in instantiation of member function 'std::__1::__unordered_map_equal<timespec, std::__1::__hash_value_type<timespec, int>, std::__1::equal_to, true>::operator()' requested here
&& key_eq()(__nd->__upcast()->_value, __k))
^

Hi,

I'm trying to run Vivace between two machines (running Ubuntu 16.04) and the throughputs are around 7Mbps. If the flow runs for a long time, it achieves throughputs of over 200Mbps, but that's still very low - in comparison, iperf gets more than 700Mbps. But even when it reaches higher throughputs, it will still revert to 10-20 Mbps. Do you have any idea why this behavior occurs?

Update: The problem seems to occur only if the RTT between the two machines is very low (~0.2ms), otherwise, everything looks fine (e.g., with RTT ~190ms).

i follow Readme and test PCC

$ ./app/pccserver recv 1234
$ ./app/pccclient send 127.0.0.1 1234 Vivace

but pccclient soonly run error and stop. the message is shown below

Reset to Vivace Sender
Using Utility Function: Vivace
connect
finished connect
SendRate(Mb/s)  RTT(ms) CTotal  Loss    RecvACK RecvNAK
1       401.236 0.204015        37021   109
pccclient: pcc_vivace_sender.cpp:306: QuicBandwidth PccVivaceSender::ComputeRateChange(const std::vector<UtilityInfo>&): Assertion `!delta_sending_rate.IsZero()' failed.
Aborted (core dumped)

how can i slove this bug? Thanks if you would like to give some advice.

Hi, is there any way to change parameters such as bandwidth or loss rate ？I just could not find the interface of setting bandwidth of link like appointting port and IP address . Should I set it by tools about udt？

here are three questions:
1、what are the class packetTracker the udt Library?
2、have you modified some code of udt source code such as /src/core.cpp?
3、if i want write a new cc，do i have to modify the code just like pcc implement？

thanks！

Is the Vivace code in the master branch, or should I work from the NSDI-2018 branch to review Vivace utility implementation?

When I run the command `sunifdef -r -UQUIC_PORT -UQUIC_PORT_LOCAL ./pcc/\*" in directory "src", I get the following error:

sunifdef: abend 0x08280: No such file or directory as "./pcc/*"

I have no idea why I get this error...

sunidef version 3.1.3 for Unix
OS: CentOS 7.5.1804 with kernel version 4.16.13

I tried to run the implementaton of PCC in the QUIC framework in ns3 and have some questions:

1. In ComputeMonitorDuration fun, is the unit inconsistency? (the unit of the second variable in max fun is second).
2. Is the CalculateUtility2 fun the lagency-based utility function introduced in the NSDI'18 paper? and is the CalculateUtility fun the loss-based utility function? If so, why are they different from the function in the paper?
3. Is the pacing section very important for PCC？I found that the calculation of utility use the actual send rate as the throughput, so if the actual send rate is far more than the pacing rate(target send rate), what effect does it have on the result? In the QUIC framework, if there is no byte in flight, it also can send datas, so the actual send rate is higher than than the target one.

Hi author,

Thanks very much for your great work!

-I'm following the online training instruction in Deep_Learning_Readme.md, but when I finished the third step(start the UDT side), I couldn’t see any update at the PCC-RL training side.
-I'm using Ubuntu 18.04, opened two terminals inside Visual Studio for PCC-Uspace and one terminal inside the PyCharm for the PCC-RL.
-I didn’t change anything of the code, though I'm not sure if I should change something here: "pwd/core/ ./app/pccclient send 127.0.0.1 9000 --pcc-rate-control=python -pyhelper=shim -path=/path/to/pcc-rl/src/udt-plugins/training/ --history-len=10 --pcc-utility-calc=linear".

May I ask for your opinion that if I did anything wrong to run the online training?
Sincerely looking forward to your reply!

Hello,
I run the client and the server as instructed, but it seems like the connection finished before the first rate request even takes place. I get the following output:

connect
NO RESPONSE!!
finished connect
Rate (Mbps) RTT (ms) Sent Lost
Getting rate
TypeError: must be real number, not NoneType
ERROR: Output from python get_rate() is not a float

( I added the NO RESPONSE!! and Getting rate prints).

Do you know why this might occur?

Thank you!

Hi,
I couldn't find in the readme or in the code, where I can change the bandwidth of the link, or the random loss. I would appreciate it if anyone directs me to the right place. Thanks!

Hi,
Thank you for your dedicated work of PCC-Uspace.
When I followed the instruction in Deep_Learning_Readme.md, I found that values of both Reward and Ewma Reward were so high as the snapshot below:
Reward: 1360096.79, Ewma Reward: 21968834.42
Reward: 1013840.44, Ewma Reward: 21759284.48
Reward: 425067.66, Ewma Reward: 21545942.31
Reward: 327400.01, Ewma Reward: 21333756.89
Reward: 154455.32, Ewma Reward: 21121963.88
Reward: 115554.43, Ewma Reward: 20911899.78
Reward: 140730.04, Ewma Reward: 20704188.08
Reward: 112697.73, Ewma Reward: 20498273.18
Reward: 107894.34, Ewma Reward: 20294369.39
...
Worsestill, values of loss_vf_loss were also unexpected, one of which reached "4512207000000.0 ".
Did you ever stumbled across this problem and could you please tell me the possible reason behind this phenomenon? Thanks!

I've started reviewing the PCC code and need some clarification on how PCC uses or sets the congestion window (cwnd).

I've setup my local repository for UDT running the command sunifdef -r -UQUIC_PORT -UQUIC_PORT_LOCAL ./pcc/\* . But looking at the code I do not see anywhere that PCC actually sets or changes the congestion window. I can see PCC calculates the rate and applies the utility function etc, but nowhere in the code or in the paper can I see how PCC manages the congestion window.

If I look at the QUIC code there does seem to be a PccSender::GetCongestionWindow() function, that appears to set the congestion window to BDP.

Can anyone explain to me how PCC should be managing the congestion window? Or point me to a paper that gives more information on how it should work?

is there any new url for vivace_proof.pdf downloading? thanks!

Hello folks,

Good day. This is Tergel Munkhbat.

We are testing PCC Vivace over 100G network, and the throughput cannot exceed 150Mbps. We suppose that the issue comes from the CPU (a core) because it (a core) reaches 100% when we start testing.

Is there anyone whose throughput exceeds more than 150 Mbps or 200 Mbps? Is there any other reason that could cause this issue? (There is no limitation or ACL on the intermediate devices, and on the sender and receiver side we configured 2G buffer size)

Sincerely,
Tergel

hi, when I run the program following the readme.md . I saw the clinet's loss and RecAck with the increase in time, and they get really big. So can you explain this phenomenon for me? I would appreciate if you could offer me any help.

I want to compare the performance of Allegro and Vivace.
So how can I choose to use Allegro or Vivace in this project?

Does this project only implement Vivace ? Then should I find Allegro in "https://github.com/modong/pcc"?

Dear Authors,

We found the PCC codes under this repo can run on QUIC framework. We tried to deploy it on top of QUIC. We use the latest QUIC version (https://github.com/google/proto-quic) and we put the corresponding PCC codes to the net/quic/core/congestion_control folder. We also modified net/quic/platform/impl/quic_pcc_sender_impl.h under QUIC to make it really enable PccSender. Then we met many compiling problems and solved them. However, PCC cannot modify the sending rate correctly.
We guess it may be because of the QUIC version we used and PCC implementation cannot correctly be ported to it. Could you tell us which QUIC version you use on your testbed implementation? Thanks a lot.

Best,
Libin Liu

Env

• PCC-Uspace: branch deep-learning, commit b4eaa60
• PCC-RL: branch master, commit e292c31
• Mininet virtual machine Ubuntu 14.04 LTS - 64 bit
• Miniconda 64-bit with conda version 4.7.11
• sudo apt install python3.5-dev

I create a new conda env pcc, and install all modules required.

pip list prints that:

Package              Version  
-------------------- ---------
absl-py              0.7.1    
astor                0.8.0    
certifi              2019.6.16
cloudpickle          1.2.1    
cycler               0.10.0   
future               0.17.1   
gast                 0.2.2    
google-pasta         0.1.7    
grpcio               1.16.1   
gym                  0.14.0   
h5py                 2.9.0    
joblib               0.13.2   
Keras-Applications   1.0.8    
Keras-Preprocessing  1.1.0    
kiwisolver           1.1.0    
Markdown             3.1.1    
matplotlib           3.1.1    
mkl-fft              1.0.14   
mkl-random           1.0.2    
mkl-service          2.0.2    
mpi4py               3.0.2    
numpy                1.16.4   
opencv-python        4.1.0.25 
pandas               0.25.1   
pip                  19.2.2   
protobuf             3.8.0    
pyglet               1.3.2    
pyparsing            2.4.2    
python-dateutil      2.8.0    
pytz                 2019.2   
scipy                1.3.1    
setuptools           41.0.1   
six                  1.12.0   
stable-baselines     2.7.0    
tensorboard          1.14.0   
tensorflow           1.14.0   
tensorflow-estimator 1.14.0   
termcolor            1.1.0    
Werkzeug             0.15.5   
wheel                0.33.4   
wrapt                1.11.2

Problem

In terminal A(Server)

conda activate pcc
pcc_uspace_path='/home/mininet/src/PCC-Uspace'
server_port='9000'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"${pcc_uspace_path}/src/core/"
"${pcc_uspace_path}/src/app/pccserver" recv ${server_port}

Starting sending rate = 2.24e+06
server is ready at port: 9000

In terminal B(Client)

conda activate pcc
pcc_uspace_path='/home/mininet/src/PCC-Uspace'
pcc_rl_path='/home/mininet/src/PCC-RL'
model_path='/home/mininet/src/pcc-model/20190725/'
# ls "${model_path}" outputs that:
# saved_model.pbtxt  variables
server_ip='127.0.0.1'
server_port='9000'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:"${pcc_uspace_path}/src/core/"
"${pcc_uspace_path}/src/app/pccclient" send ${server_ip} ${server_port} --pcc-rate-control=python -pyhelper=loaded_client -pypath="${pcc_rl_path}/src/udt-plugins/testing/" --history-len=10 --pcc-utility-calc=linear --model-path="${model_path}"

It prints:

Starting sending rate = 2.24e+06
ERROR: Could not load python module: loaded_client
Traceback (most recent call last):
  File "/home/mininet/src/PCC-RL/src/udt-plugins/testing/loaded_client.py", line 26, in <module>
    from common import sender_obs
  File "/home/mininet/src/PCC-RL/src/common/sender_obs.py", line 15, in <module>
    import numpy as np
ImportError: No module named 'numpy'

What I have tried

It seems that CPython can not find the correct Python interpreter.

In file PCC-Uspace/src/pcc/rate_control/pcc_python_rc.cpp line 8-12, I find:

void PccPythonRateController::InitializePython() {
    Py_Initialize();
    PyRun_SimpleString("import sys");

    std::stringstream set_argv_ss;

If I modify this file to

    Py_Initialize();
    PyRun_SimpleString("import sys");
    PyRun_SimpleString("print(sys.path)");  // +

In terminal B, I get

['/usr/lib/python35.zip', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-x86_64-linux-gnu', '/usr/lib/python3.5/lib-dynload', '/usr/local/lib/python3.5/dist-packages', '/usr/lib/python3/dist-packages']

If I modify this file to

    Py_SetPythonHome(L"/home/mininet/miniconda3/envs/pcc/");  // +
    Py_Initialize();
    PyRun_SimpleString("import sys");

or

    Py_SetProgramName(L"/home/mininet/miniconda3/envs/pcc/bin/python");  // +
    Py_Initialize();
    PyRun_SimpleString("import sys");

B always prints:

Starting sending rate = 2.24e+06
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'

Current thread 0x00007fbc435c7780 (most recent call first):
Aborted