Code Monkey home page Code Monkey logo

apteryx's Introduction

Centralized configuration database.

Stores data in a tree like structure with nodes referenced by "paths" that have a file system-like format. i.e. /root/node1/node2/node3 = value

API

  • SET - set the value for the specified path
  • VALIDATE - accept / deny sets that match the specified path
  • WATCH - watch for changes in the specified path
  • GET - get the value stored at the specified path
  • REFRESH - refresh database just in time when someone does a get for the specified path
  • PROVIDE - provide the value stored at the specified path
  • SEARCH - look for sub-paths that match the requested root path
  • INDEX - provide search results for the specified root path
  • PRUNE - from a requested root path, set values for all sub-paths to NULL
  • PROXY - proxy gets and sets to the requested path via the specified URL

Paths

Apteryx paths are similar to unix paths.

  • Use forward-slash / as a separator
  • Start with a separator
  • Spaces are prohibited
  • Double separator is prohibited (i.e. "/test//example" is invalid)
  • Some functions take a path and a key, this is treated as if they were joined with a separator, i.e. func(path, key, ...) called with ("/test/example", "name",...) would access "/test/example/name"
  • Avoid collisions by selecting a starting path that is unique and not shorthand, i.e. "/av" is not acceptable, but "/antivirus" is, preferably the name of the library also matches the path used.
  • Full paths include the Apteryx instance url e.g.
UNIX       "unix:///<unix-path>[:<apteryx-path>]"    e.g. unix:///tmp/apteryx:/system/hostname
TCP(IPv4)  "tcp://<IPv4>:<port>[:<apteryx-path>]"    e.g. tcp://192.168.1.2:9999:/system/hostname
TCP(IPv6)  "tcp:[<IPv6>]:<port>[:<apteryx-path>]"    e.g. tcp://[fc00::1]:9999:/system/hostname

Validating

Care must be taken when registering validation functions with apteryx_validate. Calls made to apteryx_set will block until the apteryx_validate callback is processed - this introduces a possible loop that can only be broken with a timeout. In order to avoid this, a process should avoid setting a value that it validates itself, and particularly avoid doing this from a watch callback.

Simple Example

#define _GNU_SOURCE
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <apteryx.h>

#define SYSTEM_PATH "/system"
#define SYSTEM_HOSTNAME SYSTEM_PATH "/hostname"
#define SYSTEM_TIMEZONE SYSTEM_PATH "/timezone"
#define SYSTEM_TIME SYSTEM_PATH "/time"

bool watch_timezone (const char *path, const char *value)
{
	char *cmd = NULL;

	/* When the timezone is unset, use UTC */
	if (value == NULL)
	{
		value = "UTC";
	}

	/* Create symlink from /etc/localtime to /usr/share/zoneinfo/(value) */
	asprintf (&cmd, "ln -sf /usr/share/zoneinfo/%s /etc/localtime", value);
	system (cmd);
	free (cmd);
	return true;
}

char *provide_time (const char *path)
{
	char *ret = NULL;
	time_t rawtime;

	time ( &rawtime );
	ret = strdup (ctime (&rawtime));
	char *nl = strchr (ret, '\n');
	nl[0] = '\0';
	return ret;
}

int main (int argc, char *argv[])
{
	char *value;

	apteryx_init (false);
	apteryx_watch (SYSTEM_TIMEZONE, watch_timezone);
	apteryx_provide (SYSTEM_TIME, provide_time);
	apteryx_set (SYSTEM_HOSTNAME, "host1");
	value = apteryx_get (SYSTEM_TIME);
	printf ("%s\n", value);
	free ((void*)value);
	while (1)
	{
		sleep (100);
	}
}
sudo apt-get install libglib2.0-dev libcunit1-dev liblua5.2-dev
gcc -o clockd clockd.c -I. -L. -lapteryx -std=c99 `pkg-config --cflags glib-2.0` `pkg-config --libs glib-2.0`
LD_LIBRARY_PATH=. ./clockd &
apteryx -g /clock/time
apteryx -s /clock/timezone NZ # This might require you to run clockd as root
apteryx -g /clock/time

Unit tests

# Run unit tests on the release binary
make unit [TESTS="filter to match one or more tests"]
# Run the unit tests on the gcov and address sanitiser compiled binary
make test [TESTS="filter to match one or more tests"]
# GCOV output
google-chrome  .test/gcov/index.html

Client

Usage: apteryx [-h] [-s|-g|-f|-q|-t|-r|-w|-p|-x|-l|-m|-c|-u<filter>] [<path>] [<value>]
  -h   show this help
  -d   debug
  -s   set <path> to <value>
  -g   get <path>
  -f   find <path>
  -q   query <path>?<query>
  -t   traverse database from <path>
  -r   prune <path>
  -w   watch changes to the path <path>
  -p   provide <value> for <path>
  -x   proxy <path> via url <value>
  -l   last change <path>
  -m   display memory usage for <path>
  -c   display counters and statistics
  -u   run unit tests (optionally match only tests with <filter>)

  Internal settings
    /apteryx/debug
    /apteryx/sockets
    /apteryx/watchers
    /apteryx/providers
    /apteryx/validators
    /apteryx/proxies
    /apteryx/counters

Examples:

./apteryxd -b -p apteryx.pid
LD_LIBRARY_PATH=. ./apteryx -s /interfaces/eth0/description "our lan"
LD_LIBRARY_PATH=. ./apteryx -s /interfaces/eth0/state "up"
LD_LIBRARY_PATH=. ./apteryx -g /interfaces/eth0/description
/interfaces/eth0/description/ = our lan
LD_LIBRARY_PATH=. ./apteryx -t /interfaces/eth0/
/interfaces/eth0/description                                    our lan
/interfaces/eth0/state                                          up
./apteryxd -b -p apteryx2.pid -l tcp://127.0.0.1:9999
LD_LIBRARY_PATH=. ./apteryx -s tcp://127.0.0.1:9999:/test/dog cat
LD_LIBRARY_PATH=. ./apteryx -g /remote/node/test/dog
LD_LIBRARY_PATH=. ./apteryx -x /remote/node/* tcp://127.0.0.1:9999
LD_LIBRARY_PATH=. ./apteryx -g /remote/node/test/dog

apteryx's People

Contributors

anthonylineham avatar ashmillar avatar bernieh-atlnz avatar blairsteven avatar carlgsmith avatar coledishington avatar cpackham-atlnz avatar davidt-thomson avatar farseer285 avatar felix-jia avatar gcampbell512 avatar gregatl avatar lokeshdh avatar luukp avatar macka601 avatar psiyengar avatar samuelvarley avatar sparlane avatar thomasjwinter avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

apteryx's Issues

Add tree API support for int and string

/**

  • Add a leaf to the tree under parent with name key, converting value into a string as the
  • value.
  • This function places an allocated pointer into the tree which must be freed later and
  • will not add a leaf if the asprintf call fails.
    *
  • @param tree_parent The GNode to add the leaf under.
  • @param key The name for the leaf.
  • @param value An integer value for the leaf.
    */

void
apteryx_leaf_int (GNode *tree_parent, const char *key, int value)
{
char *value_str = NULL;
if (asprintf (&value_str, "%d", value) > 0)
{
APTERYX_LEAF (tree_parent, key, value_str);
}
}

/**

  • Add a string leaf to the tree under parent with name key.
  • This function will strdup the value if it is not NULL so the pointer must be freed
  • after the tree is no longer used.
    *
  • @param tree_parent The GNode to add the leaf under.
  • @param key The name for the leaf.
  • @param value A string value for the leaf.
    */

void
apteryx_leaf_string (GNode *tree_parent, const char *key, const char *value)
{
APTERYX_LEAF (tree_parent, key, value ? strdup (value) : NULL);
}

/**

  • GNode traversal function used to free the data stored in the tree.
  • Used to free allocated values in apteryx set functions.
  • @param tree_node Node to free data of.
  • @param unused
  • @return FALSE as this never wants to stop traversal.
    */

gboolean
apteryx_tree_free_values (GNode *tree_node, gpointer unused)
{
free (tree_node->data);
tree_node->data = NULL;
return FALSE;
}

Also, a way to automatically do this:

/* Set the populate tree in Apteryx and free any memory allocated */
if (g_node_n_children (entry_tree) != 0)
{
apteryx_set_tree (entry_tree);
g_node_traverse (entry_tree, G_PRE_ORDER, G_TRAVERSE_LEAVES, -1, free_values, NULL);
}

g_node_destroy (entry_tree);

Build errors with GCC 11

mips64,apteryx: Done (20)
mips64,apteryx: Running Commands
mips64,apteryx: Compiling apteryx.c
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: apteryx.c: In function 'apteryx_prune':
mips64,apteryx: internal.h:64:9: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    64 |         syslog (LOG_ERR, fmt, ## args); \
mips64,apteryx:       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:450:9: note: in expansion of macro 'ERROR'
mips64,apteryx:   450 |         ERROR ("PRUNE: invalid path (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:450:38: note: format string is defined here
mips64,apteryx:   450 |         ERROR ("PRUNE: invalid path (%s)!\n", path);
mips64,apteryx:       |                                      ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: internal.h:69:13: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    69 |             fprintf (stderr, fmt, ## args); \
mips64,apteryx:       |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:450:9: note: in expansion of macro 'ERROR'
mips64,apteryx:   450 |         ERROR ("PRUNE: invalid path (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:450:38: note: format string is defined here
mips64,apteryx:   450 |         ERROR ("PRUNE: invalid path (%s)!\n", path);
mips64,apteryx:       |                                      ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: apteryx.c: In function 'apteryx_path_node':
mips64,apteryx: internal.h:64:9: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    64 |         syslog (LOG_ERR, fmt, ## args); \
mips64,apteryx:       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:980:9: note: in expansion of macro 'ERROR'
mips64,apteryx:   980 |         ERROR ("PATH_NODE: invalid path (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:980:42: note: format string is defined here
mips64,apteryx:   980 |         ERROR ("PATH_NODE: invalid path (%s)!\n", path);
mips64,apteryx:       |                                          ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: internal.h:69:13: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    69 |             fprintf (stderr, fmt, ## args); \
mips64,apteryx:       |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:980:9: note: in expansion of macro 'ERROR'
mips64,apteryx:   980 |         ERROR ("PATH_NODE: invalid path (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:980:42: note: format string is defined here
mips64,apteryx:   980 |         ERROR ("PATH_NODE: invalid path (%s)!\n", path);
mips64,apteryx:       |                                          ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: apteryx.c: In function 'apteryx_search':
mips64,apteryx: internal.h:64:9: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    64 |         syslog (LOG_ERR, fmt, ## args); \
mips64,apteryx:       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:1341:9: note: in expansion of macro 'ERROR'
mips64,apteryx:  1341 |         ERROR ("SEARCH: invalid root (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:1341:39: note: format string is defined here
mips64,apteryx:  1341 |         ERROR ("SEARCH: invalid root (%s)!\n", path);
mips64,apteryx:       |                                       ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: internal.h:69:13: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    69 |             fprintf (stderr, fmt, ## args); \
mips64,apteryx:       |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:1341:9: note: in expansion of macro 'ERROR'
mips64,apteryx:  1341 |         ERROR ("SEARCH: invalid root (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:1341:39: note: format string is defined here
mips64,apteryx:  1341 |         ERROR ("SEARCH: invalid root (%s)!\n", path);
mips64,apteryx:       |                                       ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: apteryx.c: In function 'apteryx_find':
mips64,apteryx: internal.h:64:9: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    64 |         syslog (LOG_ERR, fmt, ## args); \
mips64,apteryx:       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:1451:9: note: in expansion of macro 'ERROR'
mips64,apteryx:  1451 |         ERROR ("FIND: invalid root (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:1451:37: note: format string is defined here
mips64,apteryx:  1451 |         ERROR ("FIND: invalid root (%s)!\n", path);
mips64,apteryx:       |                                     ^~
mips64,apteryx: In file included from apteryx.c:33:
mips64,apteryx: internal.h:69:13: error: '%s' directive argument is null [-Werror=format-overflow=]
mips64,apteryx:    69 |             fprintf (stderr, fmt, ## args); \
mips64,apteryx:       |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mips64,apteryx: apteryx.c:1451:9: note: in expansion of macro 'ERROR'
mips64,apteryx:  1451 |         ERROR ("FIND: invalid root (%s)!\n", path);
mips64,apteryx:       |         ^~~~~
mips64,apteryx: apteryx.c:1451:37: note: format string is defined here
mips64,apteryx:  1451 |         ERROR ("FIND: invalid root (%s)!\n", path);
mips64,apteryx:       |                                     ^~
mips64,apteryx: cc1: all warnings being treated as errors
mips64,apteryx: make: *** [Makefile:55: apteryx.o] Error 1
mips64,apteryx: Error Running make (path = output/mips64/apteryx/work/apteryx-4.47, return code = 512)
mips64,apteryx: Path: output/mips64/apteryx/work/apteryx-4.47

Improve watch support

  • verify that watches are efficient at high numbers, or make them so
  • don't re-send a watch when the value didn't change

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.