Code Monkey home page Code Monkey logo

vnf_placement_optimization_rl's Introduction

Virtual Network Function placement optimization with Deep Reinforcement Learning

Code of the paper: Virtual Network Function placement optimization with Deep Reinforcement Learning.

In this project, an attentional sequence-to-sequence model is used to predict real-time solutions on a highly constrained environment. For that purpose, additional reward signals are provided to estimate the parameters of the agent. The reward signal indicating the energy consumption of the infrastructure is complemented with additional feedback signals, indicating the degree of constraint dissatisfaction. These constraints are incorporated into the cost function using the Lagrange relaxation technique.

Paper

Pending to be published...

Requirements

  • Python 3.6
  • Tensorflow 1.8.0
  • Minizinc 2.1.1 (optional -> --enable_performance)
    pip install -r requirements.txt

Usage

Learn your own model from scratch (model is saved in the default location save/model):

    python main.py --learn_mode=True --save_model=True

Continue learning a previously saved model:

    python main.py --learn_mode=True --save_model=True --load_model=True

Test pretained model performance:

    python main.py --learn_mode=False --save_model=False --load_model=True (--enable_performance=True)

Debug

To visualize training variables on Tensorboard:

    tensorboard --logdir=summary/repo

To activate Tensorflow debugger in Tensorboard, uncomment TensorBoard Debug Wrapper code. Execute Tensorboard after running the model.

    tensorboard --logdir=summary/repo --debugger_port 6064

Results

The models used in the paper are stored in save/model/, to test them agains Gecode solver run the script:

    script_test

The output of script_test are the test.csv files, which contain the placement results from both, the solver and the neural network. These output files are also available in save/. To visualize the comparison between Gecode and the model model run:

    python graphicate_test.py -f save/model_test.csv

To learn the models used in the paper from scratch run the script:

    script_learning

The learning process is monitored in a learning_history.csv file, available in save/model/. To visualize the learning process run:

    python graphicate_learning.py -f save/model/learning_history.csv

Miscelanea

Test Minizinc instalation solving the model placement.mzn fed with a demo input data placement.dzn:

    minizinc placement.mzn placement.dzn -a

Author

Ruben Solozabal, PhD student at the University of the Basque Country (UPV/EHU) - Bilbao

Date: June 2019

Contact me: [email protected]

vnf_placement_optimization_rl's People

Contributors

rubensolozabal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

vnf_placement_optimization_rl's Issues

get problem when runnning "solver.py"

Hello, author! I install the "pymzn==0.17.0" as the requirement in the project. But when I run the "solver.py", it said that "No such file or directory: 'mzn2fzn': 'mzn2fzn' "

some problem about running main.py

Your code is very helpful to me !
When I run the main.py, I met an error line 351 , it display as follow:
Total_parameters: 22015
penalty_m = np.vstack(penalty_m)
File "E:\Program Files (x86)\anaconda3\envs\gym\lib\site-packages\numpy\core\shape_base.py", line 234, in vstack
return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: need at least one array to concatenate.
I guess at the first epoch, the variable 'penalty_m' is an empty list,it cannot be used 'np.vstack',can you tell me how to solve the problem? Thank you

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.