Code Monkey home page Code Monkey logo

zlog's Introduction

zlog

zlog is a reliable, high-performance, thread safe, flexible, clear-model, pure C logging library.

Actually, in the C world there was NO good logging library for applications like logback in java or log4cxx in c++. Using printf can work, but can not be redirected or reformatted easily. syslog is slow and is designed for system use. So I wrote zlog. It is faster, safer and more powerful than log4c. So it can be widely used.

1. Install

Downloads: https://github.com/HardySimpson/zlog/releases

tar -zxvf zlog-latest-stable.tar.gz
cd zlog-latest-stable/
make 
sudo make install

Or

make PREFIX=/usr/local/
sudo make PREFIX=/usr/local/ install

PREFIX indicates the installation destination for zlog. After installation, refresh your dynamic linker to make sure your program can find zlog library.

$ sudo vi /etc/ld.so.conf
/usr/local/lib
$ sudo ldconfig

Before running a real program, make sure libzlog.so is in the directory where the system's dynamic lib loader can find it. The command metioned above are for linux. Other systems will need a similar set of actions.

2. Configuration file

There are 3 important concepts in zlog: categories, formats and rules.

Categories specify different kinds of log entries. In the zlog source code, category is a zlog_cateogory_t * variable. In your program, different categories for the log entries will distinguish them from each other.

Formats describe log patterns, such as: with or without time stamp, source file, source line.

Rules consist of category, level, output file (or other channel) and format. In brief, if the category string in a rule in the configuration file equals the name of a category variable in the source, then they match. Still there is complex match range of category. Rule decouples variable conditions. For example, log4j must specify a level for each logger(or inherit from father logger). That's not convenient when each grade of logger has its own level for output(child logger output at the level of debug, when father logger output at the level of error)

Now create a configuration file. The function zlog_init takes the files path as its only argument.

$ cat /etc/zlog.conf

[formats]
simple = "%m%n"
[rules]
my_cat.DEBUG    >stdout; simple

In the configuration file log messages in the category my_cat and a level of DEBUG or higher are output to standard output, with the format of simple %m - usermessage %n - newline. If you want to direct out to a file and limit the files maximum size, use this configuration

my_cat.DEBUG            "/var/log/aa.log", 1M; simple

3. Using zlog API in C source file

3.1 Sample code

$ vi test_hello.c

#include <stdio.h> 

#include "zlog.h"

int main(int argc, char** argv)
{
    int rc;
    zlog_category_t *c;

    rc = zlog_init("/etc/zlog.conf");
    if (rc) {
    printf("init failed\n");
    return -1;
    }

    c = zlog_get_category("my_cat");
    if (!c) {
    printf("get cat fail\n");
    zlog_fini();
    return -2;
    }

    zlog_info(c, "hello, zlog");

    zlog_fini();

    return 0;
}

3.2 Compile, and run it

$ cc -c -o test_hello.o test_hello.c -I/usr/local/include
$ cc -o test_hello test_hello.o -L/usr/local/lib -lzlog -lpthread
$ ./test_hello
hello, zlog

4. Advanced Usage

  • syslog model, better than log4j model
  • log format customization
  • multiple output destinations including static file path, dynamic file path, stdout, stderr, syslog, user-defined output
  • runtime manually or automatically refresh configure(safely)
  • high-performance, 250'000 logs/second on my laptop, about 1000 times faster than syslog(3) with rsyslogd
  • user-defined log level
  • thread-safe and process-safe log file rotation
  • microsecond accuracy
  • dzlog, a default category log API for easy use
  • MDC, a log4j style key-value map
  • self debuggable, can output zlog's self debug&error log at runtime
  • No external dependencies, just based on a POSIX system and a C99 compliant vsnprintf.

5. Reference

zlog's People

Contributors

almichala avatar bjoern-r avatar bmanojlovic avatar darrenfang avatar deemar avatar deemic avatar efrgmbh avatar gringolito avatar guillaumefanene avatar hardysimpson avatar hy0kl avatar jkelley avatar junbo-zheng avatar kraj avatar lorewalkerzhou avatar mofaph avatar moiri avatar overdrivegain avatar pcorbes avatar phoppermann avatar polnico avatar ruipacheco avatar timgates42 avatar timwitvliet avatar tuduongquyet avatar xiangfeich avatar yegorich avatar yonghuhaige avatar zaxbbun avatar zhentao-huang avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zlog's Issues

test_hashtable: 段错误

在提交 5d6b304 中,编译源码,然后运行 test_hashtable 这个测试用例出现段错误。

$ uname -srv
Linux 3.2.0-45-generic #70-Ubuntu SMP Wed May 29 20:12:06 UTC 2013

$ lsb_release --all
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.2 LTS
Release: 12.04
Codename: precise

Ubuntu 12.04 Server Error

虽然可以搞,但还是报个 issuse 吧

make

cd src && make all
make[1]: Entering directory /tmp/zlog-master/src' cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb buf.c cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb category.c cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb category_table.c cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb conf.c conf.c: In function ‘zlog_conf_build_with_file’: conf.c:246:2: error: implicit declaration of function ‘lstat’ [-Werror=implicit-function-declaration] conf.c:251:2: error: implicit declaration of function ‘localtime_r’ [-Werror=implicit-function-declaration] cc1: all warnings being treated as errors make[1]: *** [conf.o] Error 1 make[1]: Leaving directory/tmp/zlog-master/src'
make: *** [all] Error 2

zlog_fini multi thread dead lock

HI,
Thank your library first.It's very good log library.
I am use for my work.
I use it in a multi thread program.
I found a question that when in function will not exit.

In my main thread ,when i got a signal such as SEGEV then call clean up function. In clean function it will cal zlog_fini(void).
but some time it will hung there and not exit,then the program will not exit.
I use debug message found that 👍 in zlog_fini(void)
rc = pthread_rwlock_wrlock(&zlog_env_lock);
is waiting and not return.because of another thread got the lock somewhere.
if when cleanup i call zlog_fini(void) in that thread it will success and every thing is ok.

So i guess here is a multi thread bug in multi thread call zlog_fini(void).

special thanks for your the good library.

Thanks!

Jeremy Li

VZLOG_ segfaults

hi,
when I try to use VZLOG_ macros i get a segfault:

==2485== Invalid read of size 1
==2485== at 0x40240E8: strlen (mc_replace_strmem.c:282)
==2485== by 0x4089107: vfprintf (in /lib/i686/cmov/libc-2.7.so)
==2485== by 0x40AD323: vsnprintf (in /lib/i686/cmov/libc-2.7.so)
==2485== by 0x403C71E: zlog_buf_vprintf (buf.c:234)
==2485== by 0x403690A: zlog_spec_write_usrmsg (spec.c:253)
==2485== by 0x403641F: zlog_spec_gen_msg_direct (spec.c:351)
==2485== by 0x4037B3A: zlog_format_gen_msg (format.c:165)
==2485== by 0x403971A: zlog_rule_output_stdout (rule.c:339)
==2485== by 0x4038177: zlog_rule_output (rule.c:754)
==2485== by 0x403E59E: zlog_category_output (category.c:240)
==2485== by 0x404169E: vzlog (zlog.c:646)
==2485== by 0x804879D: main (in /usr/src/zlog-1.1.2/tmp/test_hello)

the code is:

include <stdio.h>

include "zlog.h"

include "malloc.h"

int main(int argc, char** argv)
{
int rc;
zlog_category_t *c;
char *message;

message=malloc(200);
strcpy(message,"test");

rc = zlog_init("/tmp/zlog.conf");
if (rc) {
    printf("init failed\n");
    return -1;
}

c = zlog_get_category("rv_server");
if (!c) {
    printf("get cat fail\n");
    zlog_fini();
    return -2;
}
ZLOG_INFO(c, "hello, zlog");
ZLOG_DEBUG(c, "debug message");
VZLOG_INFO(c, "vzlog %s",message);

zlog_fini();

return 0;

}

Am I doing something wrong here ?

有没有支持其它平台的考虑(如macos)?又可否支持交叉编译呢?

我在mac下编译正常,安装给出:

cd src && make install
mkdir -p /usr/local/include /usr/local/lib
cp -a zlog.h /usr/local/include
cp -a libzlog.dylib /usr/local/lib/libzlog...dylib
cd /usr/local/lib && ln -sf libzlog...dylib libzlog..dylib
cd /usr/local/lib && ln -sf libzlog..dylib libzlog.dylib
cp -a libzlog.a /usr/local/lib

加多个.dylib的这种后缀是设计如此?

zlog一直不释放内存

在服务器上用zlog,程序要到退出的时候才调用zlog_fini(),所以,程序不退出时,内存一直不会释放,服务器占的内存越来越大,这个问题怎么解决呢?按理说,这个需求应该做服务器的都有,是不是我哪里用错了呢?
后来发现zlog_init原来可以调用多次,我到一定的时候zlog_fini,释放内存,然后再zlog_reload,就解决了内存一直不释放的问题,但是这样又会出现另外的问题:当zlog_fini、 zlog_init 、 zlog_reload和dzlog的函数在不同的线程的情况下,在打日志的时候会崩溃,原因是zlog_default_category为空。即在zlog_fini、 zlog_init之间的那段时间打日志会崩溃,改怎么解决呢?

make doc fails due to xelatex error "kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'"

invoking make doc after installation of lyx and hevea and ./configure && make fails with

$ make doc
Entering auto-apt mode: make doc
Exit the command to leave auto-apt mode.
cd doc && make
make[1]: Entering directory '/mnt/DATA/sources/zlog/doc'
lyx -f -e pdf2 UsersGuide-EN.lyx
lyx -f -e pdf4 UsersGuide-CN.lyx
lyx -f -e pdflatex UsersGuide-EN.lyx
lyx -f -e pdflatex UsersGuide-CN.lyx

Running: xelatex  "UsersGuide-CN.tex" > /dev/null
hevea book.hva -s UsersGuide-EN.tex
hevea book.hva -s UsersGuide-CN.tex

Running: pdflatex  "UsersGuide-EN.tex" > /dev/null
This is XeTeX, Version 3.14159265-2.6-0.99991 (TeX Live 2014/Debian) (preloaded format=xelatex)
 restricted \write18 enabled.
This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./UsersGuide-EN.tex
LaTeX2e <2014/05/01>
Babel <3.9k> and hyphenation patterns for 79 languages loaded.

entering extended mode
(./UsersGuide-CN.tex
LaTeX2e <2014/05/01>
Babel <3.9k> and hyphenation patterns for 79 languages loaded.


Running: pdflatex  "UsersGuide-EN.tex" > /dev/null
This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./UsersGuide-EN.tex
LaTeX2e <2014/05/01>
Babel <3.9k> and hyphenation patterns for 79 languages loaded.


Running: pdflatex  "UsersGuide-EN.tex" > /dev/null
This is pdfTeX, Version 3.14159265-2.6-1.40.15 (TeX Live 2014/Debian) (preloaded format=pdflatex)
 restricted \write18 enabled.
entering extended mode
(./UsersGuide-EN.tex
LaTeX2e <2014/05/01>
Babel <3.9k> and hyphenation patterns for 79 languages loaded.

hevea book.hva -s UsersGuide-EN.tex
hevea book.hva -s UsersGuide-CN.tex
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
kpathsea:make_tex: Invalid fontname `SimSun:', contains ':'
support/Systemcall.cpp (273): Systemcall: 'xelatex  "UsersGuide-CN.tex"' finished with exit code 1
makefile:17: recipe for target 'UsersGuide-CN.pdf' failed
make[1]: *** [UsersGuide-CN.pdf] Error 1
make[1]: Leaving directory '/mnt/DATA/sources/zlog/doc'
makefile:15: recipe for target 'doc' failed
make: *** [doc] Error 2

with ca6162b on Ubuntu 14.10 amd64.

zlog_reload can cause a crash by freeing memory used by zlog_spec_write_time.

The last_time_format stored per thread in a_thread->event->last_time_fmt points to memory allocated in a zlog_spec_t. When zlog_reload deletes and recreates the the spec array it frees this memory but the thread storage still holds a pointer to it. The quick fix is to disable caching the last time format in spec.c:

static int zlog_spec_write_time(zlog_spec_t * a_spec, zlog_thread_t * a_thread, zlog_buf_t * a_buf)
{
/* do fetch time every event once */
zlog_spec_fetch_time;

/* strftime %d() is slow too, do it when 
 * time_fmt changed(event go through another spec) */
if (a_thread->event->last_time_fmt != a_spec->time_fmt) {
            /* The last_time_fmt memory can be free'd when zlog_reload deletes the formats */
            /* disable this for now.                                                       */
    //a_thread->event->last_time_fmt = a_spec->time_fmt;    <----- Comment out this line

test_syslog veiled out

test_syslog failed with this message
"init failed"

  • 1a160f7
  • latest-stable on master
  • System info
    $ uname -a
    Linux 3.2.0-33-generic #52-Ubuntu SMP Thu Oct 18 16:29:15 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
    $ gcc --version
    gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
  • "zlog-chk-conf" output
    $ zlog-chk-conf test_syslog.conf
    02-04 20:12:30 ERROR (31592:conf.c:508) not in any section
    02-04 20:12:30 ERROR (31592:conf.c:324) parse configure file[test_syslog.conf]line_no[1] fail
    02-04 20:12:30 ERROR (31592:conf.c:325) line[@ignore_error_format_rule false]
    02-04 20:12:30 ERROR (31592:conf.c:170) zlog_conf_build_with_file fail
    02-04 20:12:30 ERROR (31592:zlog.c:83) zlog_conf_new[test_syslog.conf] fail
    02-04 20:12:30 ERROR (31592:zlog.c:142) zlog_init_inner[test_syslog.conf] fail
    02-04 20:12:30 ERROR (31592:zlog.c:156) ------zlog_init fail end-----

how are zlog_category_t objects to be managed?

Hello,

It would be helpful to have some better real-world examples of how to manage the zlog_category_t objects. In the documentation and test examples, a local variable is set by zlog_get_category(). In a real-world program the zlog logging macros are called in many different source files.

There are two options;

(a) have global extern variables (which is troubling)

(b) call zlog_get_category() in every function which logs (which is slow)

It's not clear from the documentation and the tests which method is to be used, or if there is some other way that has not occurred to me. Searching for projects that use zlog, I see cases of both (a) and (b) in use in the wild.

Method (b) is shown in the documentation. Looking at the zlog_get_category() code, I see that hits a read-write lock (!?), so performance would suffer if zlog_get_category() was called in every function that needed to log.

Also, the zlog_get_category() returns a pointer to some created object, but it's not clear if that needs to be cleaned up some how. The fact that it returns a pointer implies there should be some cleanup method for the object returned by zlog_get_category().

Of course, I would prefer not to have global extern variables, and it seems zlog should be managing the categories, not my application. It seems to me the application should be able to specify a category by an application defined integral constant when logging. zlog already obtains a read-lock when logging, and can map the constant to the object it needs at that time. An indexed lookup would be very fast.

Thanks,
Mike

Latest Zlog won't compile with gcc 4.7.2

I downloaded the latest zlog tonight (2012-Nov-15) and tried compiling it with gcc 4.7.2 and it went like shown below:

todd@todd-Desk ~/Downloads/zlog-master $ make
cd src && make all
make[1]: Entering directory /home/todd/Downloads/zlog-master/src' cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb buf.c In file included from /usr/include/string.h:642:0, from zc_xplatform.h:41, from zc_defs.h:26, from buf.c:26: In function ‘memset’, inlined from ‘zlog_buf_adjust_append’ at buf.c:615:11: /usr/include/x86_64-linux-gnu/bits/string3.h:82:30: error: call to ‘__warn_memset_zero_len’ declared with attribute warning: memset used with constant zero length parameter; this could be due to transposed parameters [-Werror] cc1: all warnings being treated as errors make[1]: *** [buf.o] Error 1 make[1]: Leaving directory/home/todd/Downloads/zlog-master/src'
make: *** [all] Error 2

The same exact code compiles fine on another computer with gcc 4.6.3

P.S. I've been using zlog for a few months and love how it is lightweight with a simple API and flexible rules, formats, etc.

Remove auto reload feature in zlog 2.0 ?

Auto reload is a good feature.
It make people spend less time on when zlog reloads its config file.

In zlog 1.2, auto reload is controlled by logging count. Each time when zlog_xxx() is called, this count add one, util reach the reload point.

In zlog 2.0, I considered make time as the reload flag. But the problem is, the time is fetched by log action which calls gettimeofday(). If there is no rules or formats need to output time, gettimeofday() will not be called and there will be no time. Then I can't know whether the time reach the reload point...

So the trade off of performance and feature comes again?

Should I give up the auto reload feature, to make everything clear?
Or should I call gettimeofday() in every zlog_xxx() function? which will make log without time slow but keep the auto reload feature.

Thanks for any ideas, or any other advice or opinion for zlog 2.0.

zlog的性能瓶颈问题 --- the speed's bottleneck of zlog problem, any help?

zlog的性能一直在提高
the speed of zlog get up and up through every version:

./test/speed.record.txt
-------------------------------------------------
v1.0.6
$ time ./test_press_zlog 1 10 100000
real    0m1.798s
user    0m3.140s
sys     0m0.170s

$ time ./test_press_zlog2 1 10 100000
real    0m0.740s
user    0m1.390s
sys     0m0.020s
-------------------------------------------------
v1.0.5
$ time ./test_press_zlog 1 10 100000
real    0m2.776s
user    0m4.300s
sys     0m0.800s

$ time ./test_press_zlog2 1 10 100000
real    0m1.822s
user    0m2.610s
sys     0m0.560s
-------------------------------------------------
v1.0.3
$ time ./test_press_zlog 1 10 100000
real    0m6.418s
user    0m6.240s

$ time ./test_press_zlog2 1 10 100000
real    0m5.359s
user    0m4.410s
sys     0m5.750s
-------------------------------------------------

不过最大的瓶颈,在linux操作系统上,还是strftime,见图
But The biggest bottleneck on my Linux 2.6.32-41 is strftime, see pic

Linux 2.6.32-41 benchmark

在aix就没有这个问题
On Aix 5.3, there is no such problem

AIX 5.3 benchmark

不知大家有没有什么好的办法来解决这个问题?
Any one knows is there a way to solve this problem?

split log by size, add time tag to archive file SIGSEVF if no sequence number

I've been trying to rotate and timestamp log files, and unfortunately both zlog and zlog-chk-conf SIGSEV when reading the following line:

*.ERROR "aa.log", 10MB ~ "aa-%d(%Y%m%d-%H%M%S).log"

*.ERROR "aa.log", 10MB ~ logs/aa-%d(%Y%m%d-%H%M%S).#2s.log

The commented out line is fine. Adding the "#2s" is the key, it appears that your code does something horrid if there's no sequence number in the filename.

Even if the sequence number is required I think a SIGSEVF is a pretty brutal way to indicate an error :)

zlog features

  1. Is it possible to completely disable logging in zlog at compile/run time leading to zero performance overhead? If yes, how?
  2. Does zlog supports crash handling(signal handling)

File rotation is not working

Hi!

I'v the following configuration file but rotation does not work as it expected (nothing has done):

[global]
strict init = true
reload conf period = 0
rotate lock file = /tmp/zlog.myapp.lock
buffer min = 1KB
buffer max = 64KB
file perms = 640
fsync period = 64KB

[formats]
default = "%d.%ms %v [%F:%L] %m%n"

[rules]
*.debug "/mnt/data/myapp.log",100KB; default

ls -l /mnt/data

drwxr-xr-x 1 root root 2048 Jan 1 1970 app
drwx------ 1 root root 2048 Jan 1 2007 lost+found
-rw-r----- 1 root root 361335 Mar 16 06:29 myapp.log

Unicode support?

This is not an issue, its more of a quick question. I looked at user guide and also all issues' logs, I did not find any mention for unicode support. Is logging message in Unicode supported(non-ASCII)?

Any help on this is much appreciated!

zlog_init() fails with configuration file that does not have write permission

Hi,

I noticed that zlog_init(), when given a configuration file that does not have write permission, fails. zlog-chk-conf also fails on such a configuration file with the following output

11-28 21:53:34 ERROR (16208:rotater.c:108) open file[test_hello.conf] fail, errno[13]
11-28 21:53:34 ERROR (16208:conf.c:391) zlog_rotater_new fail
11-28 21:53:34 ERROR (16208:conf.c:314) parse configure file[test_hello.conf]line_no[4] fail
11-28 21:53:34 ERROR (16208:conf.c:315) line[[rules]]
11-28 21:53:34 ERROR (16208:conf.c:160) zlog_conf_build_with_file fail
11-28 21:53:34 ERROR (16208:zlog.c:93) zlog_conf_new[test_hello.conf] fail
11-28 21:53:34 ERROR (16208:zlog.c:135) zlog_init_inner[test_hello.conf] fail
11-28 21:53:34 ERROR (16208:zlog.c:150) ------zlog_init fail end------

When write permission is granted to the configuration file, everything works without changing the code. In case you're curious why I'd need this: during make distcheck, the source dir is made read-only.

Is this by design? Thank you for looking at my post!

Best regards, Edward

程序析构时出现段错的问题

我将程序中的zlog更换为其他日志系统时,没问题,或是程序中不使用log时也没问题,使用zlog的时候每次主程序退出时将段错。我将zlog单独写个简单程序也没问题,我的链接库有pthread gtest zlog rt pcap,执行代码如下:
29 zlog_category_t *my_log;
30
31 int main(int argc, char *argv[])
32 {
33 if (zlog_init("analysistcp_log.conf"))
34 {
35 if (zlog_init(NULL))
36 return -1;
37 }
38
39 if ((my_log = zlog_get_category("analysistcp_main")) == NULL)
40 {
41 zlog_fini();
42 zlog_init(NULL);
43 my_log = zlog_get_category("analysistcp_main");
44 }
45
46 if (argc != 2)
47 {
48 zlog_error(my_log, "Usage: %s \n", argv[0]);
49 zlog_fini();
50 return 0;
51 }

详细的栈信息如下:
Program terminated with signal 11, Segmentation fault.
#0 __libc_free (mem=0x20) at malloc.c:3709

3709 if (chunk_is_mmapped(p)) /* release mmapped memory. */
(gdb) bt full
#0 __libc_free (mem=0x20) at malloc.c:3709

    ar_ptr = <value optimized out>
    p = <value optimized out>
    hook = 0

#1 0x00007f12fb181d0e in zc_hashtable_del (a_table=0x129d3c0) at zc_hashtable.c:80

    i = <value optimized out>
    p = <value optimized out>
    q = <value optimized out>

#2 0x00007f12fb17c1c6 in zlog_mdc_del (a_mdc=0x12c1000) at mdc.c:36

No locals.
#3 0x00007f12fb1812fb in zlog_thread_del (a_thread=0x129d470) at thread.c:45

No locals.
#4 0x00000036584369f1 in __run_exit_handlers (status=0) at exit.c:78

    atfct = <value optimized out>
    onfct = <value optimized out>
    cxafct = <value optimized out>

#5 exit (status=0) at exit.c:100

No locals.
#6 0x000000365841ee64 in __libc_start_main (main=0x41e8c0 <main(int, char**)>, argc=1, ubp_av=0x7fff4b937d38, init=,

fini=<value optimized out>, rtld_fini=<value optimized out>, stack_end=0x7fff4b937d28) at libc-start.c:258
    result = <value optimized out>
    unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 401847908166741622, 4219392, 140734461345072, 0, 0, -401610168829006218, 
            432088316090986102}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x451e80, 0x7fff4b937d38}, data = {prev = 0x0, 
          cleanup = 0x0, canceltype = 4529792}}}
    not_first_call = <value optimized out>

#7 0x0000000000406229 in _start ()

No symbol table info available.

在linux下静态库的问题

cc -o zlog-chk-conf -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb -fPIC -lpthread zlog-chk-conf.o libzlog.a 时出错,
出现类似于zlog.c:52: undefined reference to `pthread_key_delete' 之类的错误信息, 用-pthread可以解决

但如果写一个最简单的程序链接静态库,用valgrind跑一下会产生几千个内存错误:

但链接静态库却没有这个问题,请释疑
==15205== Use of uninitialised value of size 8
==15205== at 0x464701: __strftime_internal (in ...)
==15205== by 0x463D3B: __strftime_internal (in ..)
==15205== by 0x465695: strftime_l (in ..)
==15205== by 0x421F21: zlog_spec_write_time_D (spec.c:128)
==15205== by 0x41D862: zlog_format_gen_msg (format.c:164)
==15205== by 0x41FEDF: zlog_rule_output_stderr (rule.c:421)
==15205== by 0x41B7FD: zlog_category_output (category.c:240)
==15205== by 0x41A84F: zlog (zlog.c:904)
==15205== by 0x402938: main (main.c:15)
==15205==
==15205== Use of uninitialised value of size 8
==15205== at 0x4646DD: __strftime_internal (in ..)
==15205== by 0x463D3B: __strftime_internal (in ..)
==15205== by 0x465695: strftime_l (in ..)
==15205== by 0x421F21: zlog_spec_write_time_D (spec.c:128)
==15205== by 0x41D862: zlog_format_gen_msg (format.c:164)
==15205== by 0x41FEDF: zlog_rule_output_stderr (rule.c:421)
==15205== by 0x41B7FD: zlog_category_output (category.c:240)
==15205== by 0x41A84F: zlog (zlog.c:904)
==15205== by 0x402938: main (main.c:15)
==15205==
==15205== Use of uninitialised value of size 8
==15205== at 0x425046: zlog_buf_append (buf.c:532)
==15205== by 0x41D862: zlog_format_gen_msg (format.c:164)

makefile issue and memory leak fix

Hi, I tried the latest stable zlog library (v1.2.3), it is nice and easy to configure. But during the trial, I have encounter two issues and try to fix them.

Makefile issue:
I am not familiar with makefile, but if I do not modify makefile, I can't make successfully. The error occurs when linking libzlog.a with zlog-chk-conf.o to become executable file. My environment is Ubuntu 12.04 Desktop and default compiler use "cc". The root cause of link error is because "-lpthread" can't go before "zlog-chk-conf.o libzlog.a". Here is my modification.

a. Separate $(REAL_LDFLAGS) to two variables

  • Before: REAL_LDFLAGS=$(LDFLAGS) -lpthread
  • After: REAL_LDFLAGS=$(LDFLAGS)
    LIBS=-lpthread

b. Change the script of build zlog-chk-conf

  • Before: $(CC) -o $@ $(REAL_CFLAGS) $(REAL_LDFLAGS) zlog-chk-conf.o $(STLIBNAME)
  • After: $(CC) -o $@ $(REAL_CFLAGS) $(REAL_LDFLAGS) zlog-chk-conf.o $(STLIBNAME) $(LIBS)

Memory leak issue:
I use valgrind to do memcheck and found there are memory leaks, here is part of memcheck result,

==3803== 140,160 (72 direct, 140,088 indirect) bytes in 3 blocks are definitely lost in loss record 3 of 3
==3803== at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==3803== by 0x50471BF: zc_arraylist_new (zc_arraylist.c:31)
==3803== by 0x5043FF4: zlog_rule_parse_path.constprop.7 (rule.c:498)
==3803== by 0x5044B81: zlog_rule_new (rule.c:743)
==3803== by 0x503F32F: zlog_conf_build_with_file (conf.c:470)
==3803== by 0x50400A7: zlog_conf_new (conf.c:169)
==3803== by 0x50483AD: zlog_init_inner (zlog.c:70)
==3803== by 0x50485FF: zlog_init (zlog.c:135)
==3803== by 0x4028D3: dispatcher_new (main.c:123)
==3803== by 0x405435: main (main.c:753)

After checking source code, there is an error when freeing memory at line 909, zlog_rule_del(), src/rule.c. It looks like a copy-n-paste error. You check archive_specs points to allocated memory or not but free dynamice_specs. Here is the modification.
Before:
if (a_rule->archive_specs) {
zc_arraylist_del(a_rule->dynamic_specs);
a_rule->dynamic_specs = NULL;
}
After:
if (a_rule->archive_specs) {
zc_arraylist_del(a_rule->archive_specs);
a_rule->archive_specs = NULL;
}
After that, there is no memory leak by valgrind's memcheck.

No config file

Is there a way to load the config settings as a static definitions inside the program without depending of the actual config file?

typo error in DZLOG_FATAL define (zlog.h)

in file: libzlog/zlog.h, 2 DZLOG_FATAL definations've been mis-typed as "ddzlog...", which should be "dzlog.." instead.

define DZLOG_FATAL(format, args...) \

ddzlog(__FILE__, sizeof(__FILE__)-1, __func__, sizeof(__func__)-1, __LINE__, \
ZLOG_LEVEL_FATAL, format, ##args)

PS This libzlog works smoothly for me, thanks for your excellent code.

I am now working on zlog 2.0, which provides....

  1. zlog_category_t is no longer cross-thread. I found a cross thread category is inefficient. So, the multi-thread code will looks like.

    // in thread a
    static zlog_category_t *a_cat;
    
    {
    a_cat = zlog_get_category("my_cat");
    zlog_info(a_cat, "blablaa...");
    }
    
    // in thread b
    static zlog_category_t *b_cat;
    
    {
    b_cat = zlog_get_category("my_cat");
    zlog_info(b_cat, "blablaa...");
    }
    

each thread should call zlog_get_category to get a category. This may need a litter more code, but will make log faster.

2.Buffer & permission abstract. I add a section in zlog.conf. The new configure file will looks like this:

[global]
  strict init = true
  rotate lock file = self
  auto reload = 3600 # reload configure file once every 3600 second # file mtime

[deepness]
# sign = permisson, buffer size, flush size, flush count, fsync count
  + = 600, 10KB, 8KB, 20, 1000
  - = 600, 10KB, 1KB, 1, 20
 ++= 777, 10KB, 8KB, 100, 0

[levels]
  TRACE = 10
  CRIT = 130, LOG_CRIT

[formats]
  * = "%d %V [%p:%F:%L] %m%n"
  simple = "%m%n"
  normal = "%d(%F %T.%l) %m%n"

[rules]
  *.*                             +"%12.2E(HOME)/log/%c.log"; simple
  my_cat.!ERROR     -"aa.log"
  my_.info                     >stderr;
  my_dog.=DEBUG     >syslog, LOG_LOCAL0; simple

As you see, I name the new section deepness, which is a combination of buffer control and file permission. I am still doubt about the name. If you have a good name, please tell me. The +, -, ++ signs in rules tell the buffer policy of file output.

This is my design. I need some cool advice.

zlog2.0 开发中,求建议,求现实使用反馈。

zlog2.0的目标是:
1.节省内存
2.节省CPU
3.增强性能

这3者是可能实现的吗?我想是的,软件开发,永远都会又更加优雅合适的解决方法,随着时间不断涌现出来。

当然,为了达到这3个目标,zlog的API语义也要有一定的变化

  1. zlog_category_t ,分类,不再是跨线程可用的,多线程下的代码会是这样
    // in thread a
    static zlog_category_t *a_cat;
    {
    a_cat = zlog_get_category("my_cat");
    zlog_info(a_cat, "blablaa...");
    }

    // in thread b
    static zlog_category_t *b_cat;

    {
    b_cat = zlog_get_category("my_cat");
    zlog_info(b_cat, "blablaa...");
    }

2.增加了一个“深度”的配置文件项,更透明的由用户来控制缓存、速度和安全性的平衡。配置文件就像这样:

[global]
  strict init = true
  rotate lock file = self
  auto reload = 3600 # reload configure file once every 3600 second # file mtime

[deepness]
# sign = permisson, buffer size, flush size, flush count, fsync count
  + = 600, 10KB, 8KB, 20, 1000
  - = 600, 10KB, 1KB, 1, 20
 ++= 777, 10KB, 8KB, 100, 0

[levels]
  TRACE = 10
  CRIT = 130, LOG_CRIT

[formats]
  * = "%d %V [%p:%F:%L] %m%n"
  simple = "%m%n"
  normal = "%d(%F %T.%l) %m%n"

[rules]
  *.*                             +"%12.2E(HOME)/log/%c.log"; simple
  my_cat.!ERROR     -"aa.log"
  my_.info                     >stderr;
  my_dog.=DEBUG     >syslog, LOG_LOCAL0; simple

[deepness]决定了zlog给每个文件开多大的用户态缓存、缓存到多少write出去、多久fsync一次,然后以加减号的形式写在规则前面。

  1. 考虑自动重载配置文件这个特性该怎么设计,原先在zlog1.2中开一个计数器,然后每次写日志+1。现在在zlog 2.0中,自动重载打算由时间来触发。问题是,每次写日志的时候,只有在rule或者format里面碰到需要输出时间的时候,才会调用gettimeofday()。如果没有任何一个rule或者format需要时间,那zlog就不会知道现在是什么时候,也没办法去做自动重载。

所以,还是性能和功能的平衡点

我是否应该抛弃自动重载配置文件这个特性呢?
或者在每次写日志函数被调用时,都去跑一次gettimeofday(),无论后面是否需要。然后用这个时间来做自动重载呢?

希望大家能给我一些反馈,任何想法,或者其他希望在zlog 2.0中实现的特性或改变都行。

helper function interpret message as format zlog_info, zlog_debug...

Simple code (message) from bellow will raise segfault.
It looks like it tries to interpret it instead of writing as is.

#include <stdio.h>
#include "zlog.h"

int main(int argc, char** argv)
{
        int rc;
        zlog_category_t *zc;

        rc = zlog_init("test_hello.conf");
        if (rc) {
                printf("init failed\n");
                return -1;
        }

        zc = zlog_get_category("message_type");
        if (!zc) {
                printf("get cat fail\n");
                zlog_fini();
                return -2;
        }

        //zlog_info(zc, "% n   %  n");
        zlog_info(zc,"%n %s");
        zlog_fini();
        return 0;
}

make -jN and makefile dependency

With the latest build, make -j8 results in the following error on Linux:

cc -o zlog-chk-conf -O2 -fPIC  -Wall -Werror -Wstrict-prototypes -g -ggdb zlog-chk-conf.o -L. -lzlog  -lpthread
./libzlog.so: file not recognized: File truncated

I updated src/makefile as follows, and it now works:

zlog-chk-conf: zlog-chk-conf.o $(STLIBNAME) $(DYLIBNAME)

Stop logging with rotating date parameter while corssing two days

I’ve tested zlog-1.2.9 on linux platform, it’s a great logging library, but I found it stops logging with following configuration:

I think the key is the rotating with date parameter %d(%Y%m%d)

[rules]

rtu_agent.* "/var/log/rtu_agent.log", 1kb * 10 ~ "/var/log/rtu_agent-%d(%Y%m%d).#s.log"; normal

Testing steps:

  1. Set the system time to 23:59:30 for testing file rotating while crossing two days

  2. Create a test program in C with an infinite loop to write the log file without any delay(sleep()):

    while(1)
    
    {
    
            zlog_debug(log, "test debug1");
    
            zlog_debug(log, "test debug2 super long message test for rtu_agent for debug purpose");    
    
            zlog_debug(log, "test debug3:%d-%s-%f",1234,"string",13.555);                 
    
    }
    
  3. run the test program and zlog will stop after a while.

Wrong log files names for 1.2.5

Here is my log.conf:

[global]
strict init = true
reload conf period = 0
buffer min = 1024
buffer max = 2MB
rotate lock file = /tmp/zlog.lock
default format = "%D %-5V [%p:%f:%L] %m%n"

[formats]
simple = "%m%n"
normal = "%D %m%n"

[rules]
*.INFO >stdout ; normal
worker.DEBUG "worker-%H-%d(%F).log", 10MB * 10 ~ "worker-%H-%d(%F).#r.log" ;
client.DEBUG "client-%H-%d(%F).log", 10MB * 10 ~ "client-%H-%d(%F).#r.log" ;
dispatcher.DEBUG "dispatcher-%H-%d(%F).log", 10MB * 10 ~ "dispatcher-%H-%d(%F).#r.log" ;

For 1.2.3, the output log files are

dispatcher-pj-VirtualBox-2012-11-17.log
dispatcher-pj-VirtualBox-2012-11-17.0.log
dispatcher-pj-VirtualBox-2012-11-17.1.log
dispatcher-pj-VirtualBox-2012-11-17.2.log

But for 1.2.5, the output becomes

dispatcher-pj-VirtualBox-2012-11-17.log
dispatcher-pj-VirtualBox-.0.log
dispatcher-pj-VirtualBox-.1.log
dispatcher-pj-VirtualBox-.2.log

support windows

First of all, thank you so much for this excellent library. Decent c logging libraries are quite rare. Have you thought about doing a windows port? I would be happy to help out.

Thanks,
Aaron

zlog_get_category, zlog_init

Hi,

I believe it would good to define as follows (use const char) at least in these two functions...
zlog_category_t *zlog_get_category(const char *cname)
zlog_init(const char *confpath)

do you agree ?

Support for zlog config initialization from string instead of config file

Currently zlog supports (as far as I know) initialize from config file or using default settings hard-coded in the source code if no config file path supplied . Is there any chance that we can initialize zlog using config strings? for example:

zlog_init("[global]\nstrict init = true\n[rules]\nlib.* >stdout");

As C has no java-like jar package system, this way would be easier for a shared library developer to help library users to config its logging option (eg: from environment variable)

Comparison of unsigned expression < 0 is always false

When building the latest stable release (1.2.6), this error occurs. The issue is because the size_real which is of type size_t, "according to the 1999 ISO C standard (C99), size_t is an unsigned integer type of at least 16 bit (see sections 7.17 and 7.18.3)." Therefore this comparison will always be false, triggering this error.

I fixed the error by going to the buf.c file and replacing "->size_real < 0" with "->size_real == 0", which gets rid of the build error, however I am not a 100% sure if that's what the author really intended. The error message that's supposed to be triggered is:

zc_error("pre-use of zlog_buf_resize fail, so can't convert");

Which I do not understand, being new to ZLog.

Here's the full error:

abbadon:zlog-1.2.6 csueiras$ make
cd src && make all
cc -std=c99 -pedantic -c -O2 -fPIC -Wall -Werror -Wstrict-prototypes -g -ggdb buf.c
buf.c:227:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
buf.c:289:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
buf.c:355:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
buf.c:447:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
buf.c:527:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
buf.c:575:23: error: comparison of unsigned expression < 0 is always false [-Werror,-Wtautological-compare]
if (a_buf->size_real < 0) {
~~~~~~~~~~~~~~~~ ^ ~
6 errors generated.
make[1]: *** [buf.o] Error 1
make: *** [all] Error 2

categories matched but not written because of level are not caught by "uncaught" rules

I have a lot of no-category logging (via dzlog_init)and I'd like to use categories to move some entries into their own log file. Right now I've had to add scheduled tasks to a server program and I don't want those entries in the main log file.

What I want is the default category/uncategorised stuff in one file, the rest in a second file, and all errors duplicated into a third.

So I've done this:

# all levels of categorised stuff go into own log file and all categories start with c_
c_.*        "%E(HOME)/code/listener_schedule.log",10M*2 
# errors also go into a special file:
*.ERROR     "%E(HOME)/code/listener-errors.log"
# everything else goes into the default file
!.*     "%E(HOME)/code/listener.log",10M*10

zlog-chk-conf says that's fine and it runs with log output, but I don't get anything in the last file - the ! category that's supposed to collect everything not matched.

If I comment out the *.ERROR file I get the output I want (but not the error file). It looks as though the * matches all categories, so the ! doesn't see anything. Levels apply only after that.

Is there a way to get what I want without having to prefix my categories (so I'd have "default" and "S_ThisCat", "S_ThatCat" and so on, then rules to put only the default stuff into one file, the S_* stuff in another? Or to have multiple matches in the category part of the rule (which would make parsing harder so I'm not sure it's worth while)?

Ideally I'd like to be able to reset the "category matched" flag but at the moment it seems as though rule matching is not ordered except in the implicit "look for unmatched categories last".

Otherwise, is it possible to have another symbol, like ~ to mean "don't consider this as a match for the unmatched rules flag"? Then I could use:

# errors also go into a special file:
~*.ERROR        "%E(HOME)/code/listener-errors.log"

To get that file without that counting as matching everything.

thanks.

'short' is not callable'

when trying to print log using zlog, i got the error message: "lua entry thread aborted: runtime error: /home/bilin/rtb/src/lua/ shared/log.lua:34: 'short' is not callable", can someone explain what caused this? And maybe a solution plz.
Thank you.
and the question never happens, but happen the messege " [alert] 50158#0: worker process 50161 exited on signal 11 (core dumped)"

auto validate param list when use gcc

27,32d26
< #if defined GNUC
< #define ZLOG_CHECK_PRINTF(FORMAT, ARGS)
< attribute ((format (printf, FORMAT, ARGS)))
< #else
< #define ZLOG_CHECK_PRINTF(FORMAT, ARGS)
< #endif
57c51

< const char *format, ...) ZLOG_CHECK_PRINTF(8, 9);

const char *format, ...);
75c69

< const char *format, ...) ZLOG_CHECK_PRINTF(7, 8);

const char *format, ...);

Latest stable release doesn't compile

GCC throws an error because nscan isn't being used after being initialized in rotater.c, line 172.
I'm assuming this is because of your recent fix regarding nread at the same location.

phtread_rwlock_rdlock undefined

编译环境是ubuntu 12.04,gcc 4.6.3。
zlog版本使用的是2月21日的latest-stable。

现象:
链接libzlog.so报错,pthread的相关函数没有定义。
检查ldd libzlog.so,发现zlog没有链接pthread。
解决方法:
重新编译zlog,并加入-lpthread后,问题消失。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.