This is a pure-Python implementation of Apache-like configuration file parser into Python built-in types. Similar file format is utilized by Perl's Config::General module.
With apacheconfig you can build a tree of Python objects from Apache configuration file.
For example, the following Apache configuration:
<cops>
name stein
age 25
<colors>
color \#000000
</colors>
</cops>Would be transformed by apacheconfig into:
{
'cops': {
'name': 'stein',
'age': '25',
'colors': {
'color': '#000000'
}
}By running the following code:
from apacheconfig import *
with make_loader() as loader:
config = loader.load('httpd.conf')
print(config)You can also dump the config dict back into Apache configuration file form
by calling dump or dumps method:
from apacheconfig import *
with make_loader() as loader:
config = loader.load('httpd.conf')
# ...potentially modify the `config` dict anyhow...
print(loader.dumps(config))Please, note that dumped config file might differ in its representation from the original
because dict form does not contain all pieces of the original file (like comments),
some items may get modified on load (like variable expansion) and included files
rendered into the single dict.
Parser behavior can be modified by passing it one or more options. The options are passed as a dictionary:
from apacheconfig import *
options = {
'lowercasenames': True
}
with make_loader(**options) as loader:
config = loader.load('httpd.conf')
print(config)The options are largely patterned after Perl's Config::General module, documentation for the options is also borrowed there.
The following options are currently supported:
If the value is False, then multiple identical options are disallowed. The default is True in which
case values of the identical options are collected into a list.
You may force a single config line to get parsed into a list by turning on the option forcearray and by surrounding the value of the config entry by [].
Example:
hostlist = [foo.bar]Will result in a single value array entry if the forcearray option is turned on.
If set to True, then all options found in the config will be converted to lowercase. This allows you to
provide case-in-sensitive configs. The values of the options will not lowercased.
If set to False, then each value found in the config will not be right-stripped. This allows you to gather the options as long as their trailing whitespaces.
If set to True, the parser will consider "include ..." as valid include statement (just like the well known
Apache include statement).
It also supports apache's "IncludeOptional" statement with the same behavior, that is, if the include file doesn't exist no error will be thrown.
If set to True, you will be able to include a sub-configfile multiple times. With the default, False, duplicate
includes are silently ignored and only the first include will succeed.
Reincluding a configfile can be useful if it contains data that you want to be present in multiple places in the data tree.
Note, however, that there is currently no check for include recursion.
If set to True, included files with a relative path (i.e. "cfg/blah.conf") will be opened from within the location
of the configfile instead from within the location of the script ($0). This works only if the configfile has a
absolute pathname (i.e. "/etc/main.conf").
If the configpath option has been set and if the file to be included could not be found in the location relative to the current config file, the module will search within configpath for the file.
If set to True, you may specify include a directory, in which case all files inside the directory will be loaded
in ASCII order. Directory includes will not recurse into subdirectories. This is comparable to including a directory
in Apache-style config files.
If set to True, you may specify a glob pattern for an include to include all matching files
(e.g. <<include conf.d/*.conf>>).
An include option will not cause a parser error if the glob didn't return anything.
You can use this variable to specify a search path for relative config files which have to be included. The apacheconfig tool will search within this path for the file if it cannot find the file at the location relative to the current config file.
To provide multiple search paths you can specify an array reference for the path. For example:
options = {
'configpath': ['dira', 'dirb']
}If set to True, then duplicate blocks, that means blocks and named blocks, will be merged into a single one
The default behavior is to create an array if some junk in a config appears more than once.
If set to True, then duplicate options will be merged. That means, if the same option occurs more than once, the
last one will be used in the resulting config dictionary.
If set to True, then options in your config file, whose values are set to true or false values, will be
normalised to 1 or 0 respectively.
The following values will be considered as true:
- yes
- on
- 1
- true
The following values will be considered as false:
- no
- off
- 0
- false
This effect is case-insensitive, i.e. both Yes or No will result in 1.
This option enables tag splitting by the first whitespace and turning the trailing
piece into a nested block. True by default.
This option takes one required parameter, which must be a dictionary defining variables for which you want to preset values. Each variable you have defined in this dictionary and which occurs in your config file, will cause this variable being set to the preset values to which the value in the config file refers to.
Multiple flags can be used, separated by the pipe character |.
For example, this option:
options = {
'flagbits': {
'mode': {
'CLEAR': '1',
'STRONG': '1',
'UNSECURE': '32bit'
}
}
}In this example we are defining a variable named mode which may contain one or more of CLEAR, STRONG and UNSECURE as value.
The appropriate config entry may look like this:
mode = CLEAR | UNSECURE
The parser will create a dictionary which will be the value of the key mode. This dictionary will contain all flags which you have pre-defined, but only those which were set in the config will contain the pre-defined value, the other ones will be undefined.
The resulting config structure would look like this after parsing:
config = {
'mode': {
'CLEAR': '1',
'UNSECURE': '32bit',
'STRONG': None
}
}This method allows the user to set multiple pre-defined values for one option.
Please beware, that all occurrences of those variables will be handled this way, there is no way to distinguish between variables in different scopes. That means, if mode would also occur inside a named block, it would also parsed this way.
Values which are not defined in the dictionary supplied to the parameter flagbits and used in the corresponding variable in the config will be ignored.
The value of this option should be a dictionary holding default options-values. This causes the module to populate the resulting config dictionary with the given values, which allows you to set default values for particular config options directly.
Note that you probably want to use this with mergeduplicateoptions, otherwise a default value already in the configuration file will produce an array of two values.
If set to True, variable interpolation will be done on your config input.
Variables can be defined everywhere in the config and can be used afterwards as the value of an option. Variables cannot be used as keys or as part of keys.
If you define a variable inside a block or a named block then it is only visible within this block or within blocks which are defined inside this block.
For example:
# sample config which uses variables
basedir = /opt/ora
user = t_space
sys = unix
<table intern>
instance = INTERN
owner = $user # "t_space"
logdir = $basedir/log # "/opt/ora/log"
sys = macos
<procs>
misc1 = ${sys}_${instance} # macos_INTERN
misc2 = $user # "t_space"
</procs>
</table>This will result in the following structure:
{
'basedir': '/opt/ora',
'user': 't_space'
'sys': 'unix',
'table': {
'intern': {
'sys': 'macos',
'logdir': '/opt/ora/log',
'instance': 'INTERN',
'owner': 't_space',
'procs': {
'misc1': 'macos_INTERN',
'misc2': 't_space'
}
}
}
}As you can see, the variable sys has been defined twice. Inside the block a variable ${sys} has been used, which then were interpolated into the value of sys defined inside the
block, not the sys variable one level above. If sys were not defined inside theVariables inside double quotes will be interpolated, but variables inside single quotes will not interpolated unless allowsinglequoteinterpolation option is set.
In addition you can surround variable names with curly braces to avoid misinterpretation by the parser.
If set to True, environment variables can be referenced in configs, their values will be substituted in place of
their reference in the value.
This option enables interpolatevars.
By default variables inside single quotes will not be interpolated. If you turn on this option, they will be interpolated as well.
By default this is set to True, which causes parser to fail if an undefined variable with interpolatevars turned
on occurs in a config. Set to False to avoid such error messages.
The parser will process C-style comments as well as hash-style comments. By default C-style comments are processed,
you can disable that by setting ccomments option to False.
If you want to preserve escaping special characters ($"#) in the configuration data, turn this parameter on. It is
set to False by default.
When this option is enabled, the parser would collect insignificant whitespaces
into AST. This information could then be used for code generation. The default
is False.
Escaped newlines at the end of hash comments add the following line to the comment. Apache's native config parser processes hash comments this way. Set to False by default.
When this option is enabled, the parser does not recognize empty element tags
such as <block name/>. The former, for instance, would be re-interpreted
as opening tag with tag name block, and argument name/.
The default is False.
You can alter the behavior of the parser by supplying callables which will be invoked on certain hooks during config file processing and parsing.
The general approach works like this:
def pre_open_hook(file, base):
print('trying to open %s... ' % file)
if 'blah' in file:
print('ignored')
return False, file, base
else:
print('allowed')
return True, file, base
options = {
'plug': {
'pre_open': pre_open_hook
}
}Output:
trying to open cfg ... allowed
trying to open x/*.conf ... allowed
trying to open x/1.conf ... allowed
trying to open x/2.conf ... allowed
trying to open x/blah.conf ... ignoredAs you can see, we wrote a little function which takes a filename and a base directory as parameters. We tell the parser via the plug option to call this sub every time before it attempts to open a file.
General processing continues as usual if the first value of the returned array is True. The second value of that
tuple depends on the kind of hook being called.
The following hooks are available so far:
Takes two parameters: filename and basedirectory.
Has to return a tuple consisting of 3 values:
TrueorFalse(continue processing or not)- filename
- base directory
The returned basedirectory and filename will be used for opening the file.
Takes two parameters: the source of the contents read from and a string containing the raw contents. This hook gets the unaltered, original contents as it's read from a file (or some other source).
Has to return an array of 3 values:
TrueorFalse(continue processing or not)- the source of the contents or
Noneifloads()is invoked rather thanload() - a string that replaces the read contents
You can use this hook to apply your own normalizations or whatever.
The library comes with a simple command-line tool apacheconfigtool which can convert Apache-style
config files into JSON. The tool is also useful for playing with config file formats and parser
options.
$ apacheconfigtool --help
usage: apacheconfigtool [-h] [-v] [--json-input] [--allowmultioptions]
[--forcearray] [--lowercasenames] [--nostripvalues]
[--useapacheinclude] [--includeagain]
[--includerelative] [--includedirectories]
[--includeglob] [--mergeduplicateblocks]
[--mergeduplicateoptions] [--namedblocks] [--autotrue]
[--interpolatevars] [--interpolateenv]
[--allowsinglequoteinterpolation] [--strictvars]
[--noescape] [--ccomments]
[--configpath CONFIGPATH] [--flagbits <JSON>]
[--defaultconfig <JSON>]
file [file ...]
Dump Apache config files into JSON
positional arguments:
file Path to the configuration file to dump
optional arguments:
-h, --help show this help message and exit
-v, --version show program's version number and exit
--json-input Expect JSON file(s) on input, produce Apache
configuration
parsing options:
--allowmultioptions Collect multiple identical options into a list
--forcearray Force a single config line to get parsed into a list
by turning on this option and by surrounding the value
of the config entry by []
--lowercasenames All options found in the config will be converted to
lowercase
--nostripvalues All values found in the config will not be right-
stripped
--useapacheinclude Consider "include ..." as valid include statement
--includeagain Allow including sub-configfiles multiple times
--includerelative Open included config files from within the location of
the configfile instead from within the location of the
script
--includedirectories Include statement may point to a directory, in which
case all files inside the directory will be loaded in
ASCII order
--includeglob Include statement may point to a glob pattern, in
which case all files matching the pattern will be
loaded in ASCII order
--mergeduplicateblocks
Duplicate blocks (blocks and named blocks), will be
merged into a single one
--mergeduplicateoptions
If the same option occurs more than once, the last one
will be used in the resulting config dictionary
--namedblocks Do not split tags by the first whitespace and turn the
trailing part into a nested block
--autotrue Turn various forms of binary values in config into "1"
and "0"
--interpolatevars Enable variable interpolation
--interpolateenv Enable process environment variable interpolation
--allowsinglequoteinterpolation
Perform variable interpolation even when being in
single quotes
--strictvars Do not fail on an undefined variable when performing
interpolation
--noescape Preserve special escape characters left outs in the
configuration values
--ccomments Do not parse C-style comments
--configpath CONFIGPATH
Search path for the configuration files being
included. Can repeat.
--flagbits <JSON> Named bits for an option in form of a JSON object of
the following structure {"OPTION": {"NAME": "VALUE"}}
--defaultconfig <JSON>
Default values for parsed configuration in form of a
JSON object
$ apacheconfigtool --includedirectories include-directory-test.conf
{
"final_include": "true",
"seen_first_config": "true",
"seen_second_config": "true",
"inner": {
"final_include": "true",
"seen_third_config": "true"
},
"seen_third_config": "true"
}The apacheconfig package is distributed under terms and conditions of 2-clause BSD license. Source code is freely available as a GitHub repo.
You could pip install apacheconfig or download it from PyPI.
If something does not work as expected, open an issue at GitHub.
Copyright (c) 2018-2020, Ilya Etingof. All rights reserved.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent