Tempesta FW configuration framework about tempesta HOT 5 CLOSED

I'm suggesting the following:

1. Use ioctl and copy binary structures to the kernel.
Rationale: de-serialization of data in the user-space.
We need it because:
Although configfs allows to work with unordered sets, everything beyond that is quite tricky.
We will have to de-serialize non-primitive data types (like lists) by hands.
In user-space we may use a ready-to-use C library that will de-serialize configuration files for us.
Also, even though we have files in /proc/sys now, we anyway perform the configuration using a file like tempesta.sysctl.conf, so we end with a configuration file in both cases.

2. Split configuration into different C sturctures and thus provide granularity.
For instance, we may do a separate C structure for each of: backend list, listen lsit, cache, filter, scheduler.
Rationale: modularity and granularity.

• Modularity means each "module" will access only his own configuration (not a kernel module but a logical one).
• Granularity means we can update only part of the configuration, and in addition we may subscribe to such events,
  say invoke a callback when a list of back-end servers is changed.
  So we end up with an array of structures instead of just one global structure.
  In user space it will be represented as a separate config file or a separate section for each such structure.

An open question is: how to store the config structure declaration together with a kernel module?
For instance, we may want to put scheduler configuration to a separate structure, but the scheduler is a
separate kernel module. How to decouple the ioctl interface and the structure defined in the kernel module?

3. Allocate these structures in local memory, put them to per-CPU variable and protect with RCU instead of a lock.
Rationale: performance.
Drawback: slow write, only asyncrhonous write from a softirq context.

from tempesta.

The configuration system must satisfy following requirements:

1. to be expressed in plain configuraion file which is common for any system administrator (including standard comments, includes and other common things);
2. Currently Tempesta starts as soon as the module is loaded, so we have to support defaiult configuration for everything, start the server with the defauilts and then load actual configuration. That's bad;
3. We need to be able to specify many new relatively complex entities in configuration file (e.g. many backed server with different settings for each of them);
4. To be able efficiently manage very large lists (e.g. blocked IP addresses).

Due to 3 sysfs is not a good choice.

2 means that we need to move starting logic from the module initialization and introduce /proc/tempesta/state interface (not sysfs) which starts on "start" written to the file or stops on "stop" the server correspondingly.

4 is not doable by any of standard configuration opprotunities, so userpace tool to adjust and query Tempesta DB must be introduced. So it should be able at least add, del and show all current data for particular table/collection.

Surely it's possible to parse a configuration file and create/adjust files in configfs for each item of the file. But it has no sense to have the file parser and in-kernel configuration routines at the same time. So configfs is also not so promising thing.

User-space parser which loads the configuration to the system as a binary C-structure via ioctl seems akward:

• open file and parse it is very straightforward classic and non performance critical issue. Why to move it to user space? Moreover, the parser is generally plain C and can be (and must be) tested using user space unit test;
• the configuration data structures are assumed to be complex, so it's not an easy task to pass them to the kernel. Basically, you need to invent some data representation which can place in continous memory region or to write one more parser for the binary data to handle pointers;
• there is no need to introduce one more executable just to parse and load configuration - we should keep the architecture as simple as possible.

Thus the module should parse the config file in kernel by writting "1" to appropriate /proc file (like /proc/tempesta/config_reload). Path to config file must be passed as an argument to tempesta_fw.ko and handled by tempesta.sh.

If the configuration becomes complex, then we can introduce subdirectries for reloading of particular configuration file pieces. E.g. /proc/tempesta/config/reload_full, /proc/tempesta/config/reload_backends, /proc/tempesta/config/reload_foo etc.

from tempesta.

If I understood you right, you suggest to read a plain-text configuration file via VFS and parse it inside the kernel, right?

I'm seeing the following problems:

• We can't use already existing libraries as we could do that in user-space. We can pull some code from them, but usually they use a lot of user-space stuff, so we have to almost write everything from scratch. It may cost a decent amount of time to write a parser for complex configuration entities.
• It may be quite painful to pull it to user-space and cover by unit tests if we'll use kernel-specific data structures such as linked lists, red-black and radix trees, and hash-tables. Their source code is tightly coupled with the rest of the kernel, so we have to mock a lot of stuff. So again, it may cost us some additional time.

What do you think?

from tempesta.

Also, if we are about to write our own parser for configuration files, then we need to agree on the format of configuration files to parse.

I suggest "SDL", the Simple Declarative Language: http://www.ikayzo.org/display/SDL/Language+Guide

It was originally implemented with Ruby in mind, but we may adopt it with some restrictions.

For me, the advantages are:

• support for collections: trees, lists and dictionaries;
• support for "classes" of objects (called "tags" there) - useful because we can map them to C structures;
• it is already documented, we don't need to invent our own syntax

Alexander, please review.

from tempesta.

Configuration file parser is a quick task for first year student, there is no real need for large libraries. Also it just reads text data and fill in some C strucure. Usually there is no problem to move a parser to user space (we did this for HTTP parsers in DPI). Very straightforward.

The main point about synthax is administration simplicity since we don't provide Web UI for now. So we should use the same synthax as Nginx, so users are comfortable with it. SDL seems very similar to it.

from tempesta.