odygrd / quill Goto Github PK

View Code? Open in Web Editor NEW

1.1K 26.0 129.0 4.77 MB

Asynchronous Low Latency C++ Logging Library

License: MIT License

CMake 2.33% C++ 97.40% Python 0.19% C 0.01% Starlark 0.04% Meson 0.04%

cpp low-latency logging-library logging cpp14 asynchronous async logger cpp17 log-library

quill's People

Contributors

Stargazers

Watchers

Forkers

graydon sdmg15 yaoxinliu kensou24 islc 12at7 5l1v3r1 awen162 vermosen haoxj jyjatbupt linecode jh0x cppguru russellsch liblebs buzzia sandyharvie gitplcc zhangzq liuyf5231 fnrizzi lioneldaniel externalrepositories therustmonk bigcookieliu generalzzd jeremyong ashgen bhardwajs dpocock anluoma mayursankpal druidfund museghost muse117 sammyjames yesudeep tillalex ac-ctrl lengcenyang dawnpatlod lucpelletier dkuo123 cateatfish108 alphanso chenhongquan taiheioki yanrbts whitelilis gerhobbelt tearnote yongliz opizil tonyischen shiyi23 baptistapedro mayhemheroes ajunlonglive jf-0 emattiza fengjixuchui helloworld100times gg-big-org freeacetone bygo7 guitarhua vochlea e-kwsm ma-9999 gmh5225 przybysh alishewn andrewman2 mister-good-deal sunxm123 jjqcat overfittingstudyroom bhatt-r sarvex crazystone1314 marcellomello2 adfernandes usefulcat stephanie-eng imxiaobo tetsuh walkthetalk picknikrobotics ros2middleware shabbirhasan1 zhuoqiang jacekpoz abelianwang krogk heal-bot itsbasi magiruuvelvet guanglei tinyik

quill's Issues

[feature request] bind log level to handlers

Hi, I use a single log with two handlers, one is file handle, the other is stdout.

I need to log all to file (set_log_level=TraceL3) but only log error (and critical) on stdout(set_log_level=Error).

However, currently, log levels are binded to log instance but not handlers. It will be great if log levels are binded to handlers (just like formatter, and also in Python).

got an assert error `BaseEvent has a std::chrono timestamp, but the backend thread is using rdtsc timestamp`

Please help. Error message:

quill/detail/BackendWorker.h:396: std::chrono::nanoseconds quill::detail::BackendWorker::_get_real_timestamp(const quill::detail::BaseEvent*) const: Assertion `base_event->using_rdtsc() && "BaseEvent has a std::chrono timestamp, but the backend thread is using rdtsc timestamp"' failed.

calculation should ignored when should_log() = false

e.g.

std::string very_cost_calc() {
    // ...
    return "costly result";
}

LOG_TRACE("{}", very_cost_calc());

When we set log level in info, then the very_cost_calc() shouldn't be called.

The problem is check_format(FMT_STRING(fmt), ##__VA_ARGS__); in QUILL_LOGGER_CALL. We need to remove or fix the implement.

Document gotcha, unsafe classes [Documentation]

Logging Mutable classes with std::shared_ptr even on a single thread or classes that contain a raw pointer or reference that is mutated can be problematic since the backend worker thread might log the updated value.

Those classes need to be converted to a string first before being logged

A similar case to this : https://github.com/odygrd/quill/wiki/5.-FAQ

The data can be mutated after the log call and before the formatting.
The data can be mutated (surprise) during formatting.

Recent change breaks logging in nested generic lambdas

Here's a fairly minimal repo (both lambdas must be generic). On gcc 8.3.0, the following compiles on v1.2.3 but breaks on master (bisect points to c3cd06c as the breaking commit)

#include <quill/Quill.h>

void f(quill::Logger* log) {
    [log](auto){
        [log](auto){
            LOG_INFO(log, "Hi");
        }(1);
    }(0);
}

This is because (for whatever reason), __FUNCTION__ isn't a constant expression in that context on gcc 8.3 (I think this is gcc bug 66639):

void f() {
    [](auto){
        [](auto){
            static constexpr char const* function_name = __FUNCTION__;
            static constexpr int value = function_name[0]; // error on 8.3, ok on 9.2
            (void)value;
        }(1);
    }(0);
}

Add more information when an exception is thrown in the backend logger thread

User's can not catch the exceptions that are thrown in the backend worker thread, and there is no information printed out.

Portion of pattern after %(message) is ignored if it contains no format specifiers

[feature request] Add option for replacing '::' with '.' in %(function_name) formatter

When I have a function several namespaces deep, the :: symbol becomes hard to read when outputted line after line, several times in each line. It would be nice to be able to specify an option to replace :: with a ..

Don't automatically use verbose makefiles

Currently, that flag is set unconditionally if quill detects that it's the primary project. Can it... not?

If I want verbose makefiles, that's easy for me to specify.

Compilation for __ARM_ARCH < 6

Currently quill does not compile for __ARM_ARCH < 6 because x86 intrinsics are used in rdtsc.h. Using chrono clock does not help. Perhaps defaulting to chrono clock in this case is a solution.

Custom handlers

Is there support for custom handlers? I am not sure if I just missed them in the documentation / examples. If not, are you planning to support custom handlers?

How to add a file handler for default logger ?

e.g:

auto logger = quill::get_logger();
logger->add_handler(file_handler);

Thx.

Compile on arm/arm64

Attempt to compile and test the logging library on arm/arm64.
At the moment it won't compile.

Backend worker thread, read all messages from the queue as a batch, to keep the queue as empty as possible

At the moment the backend worker thread processes the log record messages from each caller via a queue, but it only reads one message each time.

In situations where there is a burst of messages to the queue, the queue can get full quickly and cause a re-allocation in the hot path.

The backend logging thread should read everything from the queue first as a batch, then store it locally and then process the messages one by one from the local copy instead.
This will keep the queue shared with the hot patch as empty as possible eliminating the possibility of the queue being full.

wstring_to_utf8 is not a member of quill::detail

On Mingw-w64, using GCC 10.1.0, the following error comes up.

In file included from _deps/quill-src/quill/include/quill/handlers/Handler.h:8,
                 from _deps/quill-src/quill/include/quill/handlers/StreamHandler.h:11,
                 from _deps/quill-src/quill/src/handlers/StreamHandler.cpp:1,
                 from _deps/quill-build/quill/CMakeFiles/quill.dir/Unity/unity_2_cxx.cxx:3:
_deps/quill-src/quill/include/quill/PatternFormatter.h: In member function 'std::enable_if_t<quill::detail::any_is_same<std::__cxx11::basic_string<wchar_t>, void, Args ...>::value, void> quill::PatternFormatter::format(std::chrono::nanoseconds, const char*, const char*, const quill::detail::LogRecordMetadata&, const Args& ...) const':
_deps/quill-src/quill/include/quill/PatternFormatter.h:442:11: error: 'wstring_to_utf8' is not a member of 'quill::detail'
  442 |   detail::wstring_to_utf8(_w_memory_buffer, _formatted_log_record);
      |           ^~~~~~~~~~~~~~~

I'm using commit a5ad7b0 at the moment.

Header-only?

It would be an awesome feature!

Quill 1.2.2 includes intrinsic headers in its own namespace

Recent changes mean that Rtdsc.h now does something like this:

namespace quill::detail {
#include <x86intrin.h>
}

Which can break name lookup in all sorts of weird contexts. Can you do the includes outside of the namespace?

Shadowed field in TransitLogRecord constructor

As of Quill v1.3.1, Clang 10.0.0 generates warnings for two shadowed fields in the TransitLogRecord constructor.
I've pasted the output below.

In file included from _deps/quill-src/quill/include/quill/Quill.h:10:
_deps/quill-src/quill/include/quill/detail/BackendWorker.h:145:37: error: constructor parameter 'thread_context' shadows the field 'thread_context' of 'TransitLogRecord' [-Werror,-Wshadow-field-in-constructor]
    TransitLogRecord(ThreadContext* thread_context, std::unique_ptr<RecordBase> base_record)
                                    ^
_deps/quill-src/quill/include/quill/detail/BackendWorker.h:155:20: note: previous declaration is here
    ThreadContext* thread_context;
                   ^
_deps/quill-src/quill/include/quill/detail/BackendWorker.h:145:81: error: constructor parameter 'base_record' shadows the field 'base_record' of 'TransitLogRecord' [-Werror,-Wshadow-field-in-constructor]
    TransitLogRecord(ThreadContext* thread_context, std::unique_ptr<RecordBase> base_record)
                                                                                ^
_deps/quill-src/quill/include/quill/detail/BackendWorker.h:156:33: note: previous declaration is here
    std::unique_ptr<RecordBase> base_record;

timestamp point

Hi,

At what point does Quill calculate the timestamp? is it at the time the caller sends to the log thread? or when the worker thread eventually gets it from the work queue?

[feature request] user defined type support member function `std::string to_string() const`

Currently, quill only support operator<<() for user defined type. But << is rarely used (and is not compatiable with fmt and quill grammar). Could you support std::string to_string() const?

Here is an example:

struct User {
    std::string name;
    std::string password;

    std::string to_string() const {
        return fmt::format("{{user:{}, password:{}}}", user, password);
   }
};

int main() {
    // ....
    User user{"foo", "xxx"};
    LOG_INFO("{}", user);
}

example_custom_formatter.cpp causes assertion failure

example_custom_formatter.cpp fails with the following message:

$ ./example_custom_formatter
libc++abi.dylib: terminating with uncaught exception of type quill::QuillError: set_default_logger_handler needs to be called before quill::start()
zsh: abort      ./example_custom_formatter

Support non-guaranteed logging (drop messages)

Add an option to drop messages instead. In low-latency settings, allocating is not always an option.
In this case use a bounded fixed size queue instead of an unbounded one.
Notify the user how many messages were dropped.

"emit" function names conflict with Qt macro

There are many compilation errors when used with Qt.
I guess emit function names conflict with the Qt macro emit.

[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(59): error C2143: syntax error: missing ')' before 'const'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(59): error C2143: syntax error: missing ';' before 'const'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(59): error C2838: 'memory_buffer': illegal qualified name in member declaration
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(59): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(60): error C3646: 'log_record_timestamp': unknown override specifier
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(60): error C2059: syntax error: ')'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/Handler.h(60): error C2838: 'nanoseconds': illegal qualified name in member declaration
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C2143: syntax error: missing ')' before 'const'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C2143: syntax error: missing ';' before 'const'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C2838: 'memory_buffer': illegal qualified name in member declaration
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C3646: 'log_record_timestamp': unknown override specifier
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C2059: syntax error: ')'
[build] D:\tests\cmaketest\quill\quill\include\quill/handlers/StreamHandler.h(32): error C2838: 'nanoseconds': illegal qualified name in member declaration

Give an option to disable exceptions

Single logger with multiple handlers

Is it possible to have a single logger with multiple handlers? For example, I would like to have a logger with two handlers: one to stdout (with colors), and the other to a file (without colors).

Can you provide some instructions to guide how to use make to compile quill？

My system does not use cmake, but make

Logging third party types that are not trivial and require QUILL_COPY_LOGGABLE

I have a 3rd party type—a custom string wrapper—and it has a method GetText(), which returns a const char *. But since I can't modify the class definition of this type I can't add the QUILL_COPY_LOGGABLE macro.

Is there any way I can create a custom formatter function for this type?

Missing Documentation for Function Arguments

When compiling with Clang's "-Wdocumentation" flag, there is a slew of warnings output for Doxygen documentation for function arguments which are missing documentation strings. I would like to have these warnings treated as errors when compiling my application, but the included headers cause my build to fail. I can ignore the warning for now, but would it be possible to either remove the undocumented function parameters or document them to fix this warning?

Support safe and unsafe logging modes as a workaround to formatting classes with mutable objects gotchas

Currently Quill copies all user defined types and then formats them on a backend thread. This leads problems when user defined types include a mutable reference or pointer to another object.

I am thinking to have two modes available. It should be able to switch between them with a preprocessor definition.

Safe Logging

If this mode is used without providing a tag to all the safe objects it will degrade performance.
However, if used correctly it can achieve the same performance as unsafe logging in a safer way.

This mode will copy all safe to copy types that accepted by printf to the spsc queue and format them later on the backend logger thread.
For any user defined types check if copy_loggable_t tag is defined and copy the object without formatting, otherwise if the tag is not defined fallback to calling operator<< on the caller thread.
For all related stl objects and containers check the underlying type against 1) and 2) and decide whether to copy or format the container on the caller thread.

example :

class foo
{
    public:

      // Enables copying the object instead of formatting it in the caller thread. 
      // This object is safe to copy. But we are in SAFE_LOGGING and it would 
      // be formatted on the caller thread without this tag.

      using copy_loggable_t = std::true_type; 

      foo(int m) : mo(m) {};
      friend ostream& operator<<(ostream& os, const foo& f)
      {
          os << f.mo;
          return os;
      }

    int mo;
};

class foo
{
    public:

      // With SAFE_LOGGING this class will always be formatted by default 
      // on the caller thread and a string will be copied to the queue

      foo(int m) : mo(m) {};
      friend ostream& operator<<(ostream& os, const foo& f)
      {
          os << f.mo;
          return os;
      }

    int mo;
};

Unsafe Logging (Current Mode)

This mode will always attempt to copy all objects and format them in a backend logger thread.
The user has to be extra careful about objects that contain a mutable reference or a pointer as those members can change at any time :

after LOG_() was called but before formatting happened
during formatting

Pattern Format Segfault

Using the set_pattern function handler appears to cause a runtime crash.
A stack trace of the segmentation fault is below.
This occurs on GCC 9.1.1 on CentOS 7.
I'm using the recent commit eb56fe8, but it occurs on v1.2.0 as well.
The issue does not occur using Clang 9.0.1 on macOS 10.15 Catalina.
My program crashes on launch.
Perhaps it has to do with the QUILL_STRING macro?

#0  0x0000000000498114 in __gnu_cxx::__exchange_and_add (__mem=0x200543c, __val=-1) at /opt/rh/devtoolset-9/root/usr/include/c++/9/ext/atomicity.h:49
#1  0x00000000004981ab in __gnu_cxx::__exchange_and_add_dispatch (__mem=0x200543c, __val=-1) at /opt/rh/devtoolset-9/root/usr/include/c++/9/ext/atomicity.h:82
#2  0x00000000004a5340 in std::string::_Rep::_M_dispose (this=0x200542c, __a=...) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:3311
#3  0x000000000049e39a in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::~basic_string (this=0x2122458, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/basic_string.h:3706
#4  0x000000000049bd6e in quill::detail::TimestampFormatter::~TimestampFormatter (this=0x2122418, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/detail/TimestampFormatter.h:29
#5  0x00000000004a8a34 in quill::PatternFormatter::~PatternFormatter (this=0x21221f0, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/PatternFormatter.h:188
#6  0x00000000004a8a86 in std::default_delete<quill::PatternFormatter>::operator() (this=0x2121608, __ptr=0x21221f0) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/unique_ptr.h:81
#7  0x00000000004a166e in std::unique_ptr<quill::PatternFormatter, std::default_delete<quill::PatternFormatter> >::~unique_ptr (this=0x2121608 = {...}, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/unique_ptr.h:289
#8  0x00000000004f84aa in quill::Handler::~Handler (this=0x2121600, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/handlers/Handler.h:29
#9  0x00000000004fd416 in quill::StreamHandler::~StreamHandler (this=0x2121600, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/handlers/StreamHandler.h:26
#10 0x00000000004fd432 in quill::StreamHandler::~StreamHandler (this=0x2121600, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/handlers/StreamHandler.h:26
#11 0x000000000050553a in std::default_delete<quill::StreamHandler>::operator() (this=0x2121670, __ptr=0x2121600) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/unique_ptr.h:81
#12 0x000000000050345c in std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> >::~unique_ptr (this=0x2121670 = {...}, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/unique_ptr.h:289
#13 0x000000000050b1dc in std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >::~pair (this=0x2121668, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/stl_pair.h:208
#14 0x000000000050b208 in __gnu_cxx::new_allocator<std::__detail::_Hash_node<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, true> >::destroy<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > > > (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56>, __p=0x2121668) at /opt/rh/devtoolset-9/root/usr/include/c++/9/ext/new_allocator.h:153
#15 0x0000000000509f6f in std::allocator_traits<std::allocator<std::__detail::_Hash_node<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, true> > >::destroy<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > > > (__a=..., __p=0x2121668) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/alloc_traits.h:497
#16 0x00000000005085f9 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, true> > >::_M_deallocate_node (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56>, __n=0x2121660) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/hashtable_policy.h:2102
#17 0x0000000000507c24 in std::__detail::_Hashtable_alloc<std::allocator<std::__detail::_Hash_node<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, true> > >::_M_deallocate_nodes (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56>, __n=0x0) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/hashtable_policy.h:2124
#18 0x0000000000505e06 in std::_Hashtable<std::string, std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, std::allocator<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > > >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::clear (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/hashtable.h:2028
#19 0x0000000000503d98 in std::_Hashtable<std::string, std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > >, std::allocator<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > > >, std::__detail::_Select1st, std::equal_to<std::string>, std::hash<std::string>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::~_Hashtable (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56>, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/hashtable.h:1352
#20 0x0000000000502324 in std::unordered_map<std::string, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> >, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::unique_ptr<quill::StreamHandler, std::default_delete<quill::StreamHandler> > > > >::~unordered_map (this=0x5794b8 <quill::detail::LogManagerSingleton::instance()::instance+56> = {...}, __in_chrg=<optimized out>) at /opt/rh/devtoolset-9/root/usr/include/c++/9/bits/unordered_map.h:102
#21 0x0000000000502344 in quill::detail::HandlerCollection::~HandlerCollection (this=0x5794a0 <quill::detail::LogManagerSingleton::instance()::instance+32>, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/detail/HandlerCollection.h:24
#22 0x00000000005023b6 in quill::detail::LogManager::~LogManager (this=0x579480 <quill::detail::LogManagerSingleton::instance()::instance>, __in_chrg=<optimized out>) at _deps/quill-src/quill/include/quill/detail/LogManager.h:27
#23 0x00000000004ff350 in quill::detail::LogManagerSingleton::~LogManagerSingleton (this=0x579480 <quill::detail::LogManagerSingleton::instance()::instance>, __in_chrg=<optimized out>) at _deps/quill-src/quill/src/detail/LogManagerSingleton.cpp:9
#24 0x00007faa19fd2ce9 in __run_exit_handlers () from /lib64/libc.so.6
#25 0x00007faa19fd2d37 in exit () from /lib64/libc.so.6
#26 0x00007faa19fbb55c in __libc_start_main () from /lib64/libc.so.6
#27 0x00000000004761c9 in _start ()

Please support local timezone

Thank you for your great work!
I love this library.

One thing to prevent using quill is supporting local time zone.
Please support local timezone.
Currently quill::PatternFormatter::_timezone_type is private and hard coded.

Add Process ID to PatternFormatter

Would it be possible to include the process ID in the log messages? Something like %(pid) for example.

Are there plans to implement crash safe feature like g3log?

Awesome lib!
Are there plans to implement crash safe feature like g3log?
This is the only reason why I choose g3log other than other tools.

Conflicting declaration of strlen with Mingw-w64

There is a conflicting declaration of the function strlen with C-Linkage when compiling with Mingw-w64 on Windows 10.
This occurs in version v1.2.3 and on the most recent commit a4ae296.
The compiler error is as follows.

C:/msys64/mingw64/x86_64-w64-mingw32/include/intrin.h:269:5: error: conflicting declaration of 'size_t quill::detail::strlen(const char*)' with 'C' linkage
  269 |     __MACHINE(size_t __cdecl strlen(const char *))
      |     ^~~~~~~~~
compilation terminated due to -Wfatal-errors.

The conflicting function is defined in Utilities.h.

Is it possible to create multiple stdout loggers with different formatters?

As the question says, I want to create multiple quill::Loggers, that all log to stdout, but have different formatters. Basically such that:

LOG_INFO(log_a, "hello"); // [A] hello
LOG_INFO(log_b, "hello"); // [B] hello

But all to stdout, not to different files.

The documentation seems to suggest there is a singleton stdout handler with a single formatter. Is it possible to do what I'm suggesting or do I have to manually write the [A] and [B] for each log line?

quill flush policy: call flush() every X seconds

I am very new to quill. I am not sure if this is an issue for quill.

If an application prints just 1 log message ("program starts"), will quill use '\n' or std::endl here?
Will the log be buffered for a long time if the application doesn't log any more messages?
Sometimes it is not obvious to see whether a program doesn't hit that line of code or the log message is buffered, which causes some debugging anxiety.

Probably it will be better if quill backend thread can have an option to flush every X seconds, this can minimize lost messages during an unexpected crash. quill backend will do some unnecessary work here, but the main app thread can avoid paying the cost of calling quill::flush(). So main app thread can be fast, without worrying too much about logging messages being buffered.

Add fmt library version number in changelog

Our application also uses fmt library, which is a great library. Quill has its copy of fmt in include/bundled/ directory. Right now our fmt library(6.1.2) seems to co-exist well with bundled/fmt. But in the future, quill can upgrade to a different fmt version. Users would like to know what version of fmt is supported/tested with a new quill release. It will be convenient to just read changelog.md for this information.

Support {fmt} v7.1.1

Recently, {fmt} version 7.1.0 was released.

When using quill with vcpkg after microsoft/vcpkg#14217 and microsoft/vcpkg#14239, I am now experiencing the following issues when building:

quill.lib(FileUtilities.cpp.obj) : error LNK2019: unresolved external symbol "struct fmt::v7::detail::dragonbox::decimal_fp<float> __cdecl fmt::v7::detail::dragonbox::to_decimal<float>(float)" (??$to_decimal@M@dragonbox@detail@v7@fmt@@YA?AU?$decimal_fp@M@0123@M@Z) referenced in function "class fmt::v7::detail::buffer_appender<wchar_t> __cdecl fmt::v7::detail::write<wchar_t,class fmt::v7::detail::buffer_appender<wchar_t>,float,0>(class fmt::v7::detail::buffer_appender<wchar_t>,float)" (??$write@_WV?$buffer_appender@_W@detail@v7@fmt@@M$0A@@detail@v7@fmt@@YA?AV?$buffer_appender@_W@012@V3012@M@Z)
quill.lib(FileUtilities.cpp.obj) : error LNK2019: unresolved external symbol "struct fmt::v7::detail::dragonbox::decimal_fp<double> __cdecl fmt::v7::detail::dragonbox::to_decimal<double>(double)" (??$to_decimal@N@dragonbox@detail@v7@fmt@@YA?AU?$decimal_fp@N@0123@N@Z) referenced in function "class fmt::v7::detail::buffer_appender<wchar_t> __cdecl fmt::v7::detail::write<wchar_t,class fmt::v7::detail::buffer_appender<wchar_t>,double,0>(class fmt::v7::detail::buffer_appender<wchar_t>,double)" (??$write@_WV?$buffer_appender@_W@detail@v7@fmt@@N$0A@@detail@v7@fmt@@YA?AV?$buffer_appender@_W@012@V3012@N@Z)

...which I believe to be related to the switch from Grisu3 to Dragonbox for the default floating-point formatting, as quoted from the release page:

Switched from Grisu3 to Dragonbox for the default floating-point formatting which gives the shortest decimal representation with round-trip guarantee and correct rounding (#1882, #1887, #1894).

I've opened an issue on vcpkg (microsoft/vcpkg#14331) thinking the issue was vcpkg, but now I realize that a new version of {fmt} was released and I think that is the problem.

Compile Time Checking of Log Messages

The {fmt} library provides the FMT_STRING macro to validate formatting constructs at compile-time. Is it possible to check the log message format at compile-time in Quill?

A huge time gap between the first log and the second log

Thanks for your great work! I can compile quill using make now. I write a wrapper for using quill in my project. It works well.
But now I notice some performance issues, for example:

13:29:44.906777105 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906781955 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906781971 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906781987 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906781996 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782006 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782015 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782025 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782034 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782044 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782054 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782063 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782072 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782082 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782090 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782098 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782106 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782115 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782125 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782132 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782141 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782148 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782157 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782165 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782173 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782183 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782190 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782198 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782205 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782214 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782222 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782231 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782238 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782249 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782256 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782266 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782274 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782282 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782290 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782300 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .
13:29:44.906782308 [27908] index_base.cpp:99            LOG_INFO      root         - ok  this is index base .

Why is there a huge time gap between the first log and the second log? Hope your help, thx.

Please add prefix QUILL_ to log macros like LOG_INFO

This is causing macro conflicts for me, as 1 of the libraries I use has the same log macro. It will be better to always have a prefix for macros like LOG_INFO, which is too easy to have a naming conflict without prefix.

Thanks a lot.

Support {fmt} Version 7

Quill needs updated to support {fmt} 7.

When using a an external {fmt} of version 7 or higher, compilation fails.

_deps/quill-src/quill/include/quill/detail/LogMacros.h:37:8: fatal error: no member named 'internal' in namespace 'fmt'
  fmt::internal::check_format_string<std::remove_reference_t<Args>...>(format_str);
  ~~~~~^

The internal namespace was renamed to detail in {fmt} version 7.
Updating the namespace used in LogMacros.h fixes the issue, as shown below.

/**
 * Check in compile time the correctness of a format string
 */
template <typename S, typename... Args, typename Char = fmt::char_t<S>>
constexpr void check_format(const S& format_str, Args&&...)
{
  fmt::detail::check_format_string<std::remove_reference_t<Args>...>(format_str);
}

std::bad_alloc exception on Mingw-w64 - Invalid timestamp formatting due to strftime failure on MinGw

My application crashes when compiled with Mingw-w64 (GCC 9.3) on Windows 10 when run using the recent commit 6248396.
An unexpected std::bad_alloc exception is thrown.
I am able to re-create the issue by running the example_custom_formatter example target.
I tried to run the tests on Mingw-w64 but there were compilation issues with the use of std::ifstream and std::wstring in the quill::testing::file_contents function.

Formatting timestamp fractional seconds in PatternFormatter

Currently, formatting the time always appends the fractional part of the seconds to the string formatted with strftime. I want to include the fractional seconds immediately after the seconds, but also include time zone information at the end of the date-time.

The following example should illustrate this.
I have the date-time format string "%FT%T%z%Z".
This produces 2020-04-15T11:31:08-0500CDT.543.
I would like to be able to format the date-time to look like this: 2020-04-15T11:31:08.543-0500CDT.

Support external fmt

At the moment Quill is using a bundled version of lib fmt.

If a project is already using it's own fmt version, both fmt libraries have to be of the same version. Otherwise it won't work

There can be a CMake option FMT_EXTERNAL that when specified Quill will try to find the user's already installed fmt library and use that instead of the bundled one

Add CMAKE_INSTALL_PREFIX in README for installation directory?

This is to save some users(who didn't know cmake well enough) some google time for this CMake feature. My first google solution is 'make DESTDIR=/home/john install', then /usr/local is still created under /home/john. Not a big deal, I manually copied them from /home/john/usr/local.

My current google solution is 'cmake -DCMAKE_INSTALL_PREFIX=/home/john/quill-1.0/'. I didn't try the command as my installation was done. It will be nice to add this to README if it works.

QUILL_USE_BOUNDED_QUEUE cause assertion error : "Object should always be cache aligned"

Thank you for fast development. I love using this library.
When I try to use QUILL_USE_BOUNDED_QUEUE in In quill 1.3.1,
I met the assertion error message:

.../include/quill/detail/BoundedSPSCQueue.h:362: quill::detail::BoundedSPSCQueue<TBaseObject, Capacity>::Handle quill::detail::BoundedSPSCQueue<TBaseObject, Capacity>::try_pop() [with TBaseObject = quill::detail::RecordBase; long unsigned int Capacity = 262144]: Assertion `(reinterpret_cast<uintptr_t>(buffer_pos) % CACHELINE_SIZE == 0) && "Object should always be cache aligned"' failed.

Please tell me workaround.

my environment : Fedora 32, gcc 10.1, glibc 2.31
quill version: 1.3.1

Cannot compile example_trivial.cpp in Linux

I try to compile example_trivial.cpp in Linux using clang++-9.0.1 and get below compilation errors. Do you have any suggestion?

Thanks,
Hung

[ 50%] Building CXX object CMakeFiles/test_quill.dir/test_quill.cpp.o                                                                                                                                                                 
In file included from /home/hungptit/working/cppidioms/src/test_quill.cpp:1:                                                                                                                                                          
In file included from /home/hungptit/working/cppidioms/src/../3p/include/quill/Quill.h:8:                                                                                                                                             
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:39:62: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    QUILL_LOGGER_CALL(logger, quill::LogLevel::TraceL3, fmt, ##__VA_ARGS__)                                                                                                                                                           
                                                             ^                                                                                                                                                                        
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:33:54: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    logger->log<log_statement_level>(&log_line_info, ##__VA_ARGS__)...                                                                                                                                                                
                                                     ^                                                                                                                                                                                
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:46:62: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    QUILL_LOGGER_CALL(logger, quill::LogLevel::TraceL2, fmt, ##__VA_ARGS__)                                                                                                                                                           
                                                             ^                                                                                                                                                                        
/home/hungptit/working/cppidioms/src/../3p/include/quill/detail/LogMacros.h:33:54: error: token pasting of ',' and                                                                                                                    
      __VA_ARGS__ is a GNU extension [-Werror,-Wgnu-zero-variadic-macro-arguments]                                                                                                                                                    
    logger->log<log_statement_level>(&log_line_info, ##__VA_ARGS__)...

And this is the compilation command

{                                                                                                                                                                                                                                     
  "directory": "/home/hungptit/working/cppidioms/src",                                                                                                                                                                                
  "command": "clang++   -I/home/hungptit/working/cppidioms/src/../3p/include -I/home/hungptit/working/cppidioms/src/../src    -O3  -march=native  -std=c++17  -Wall  -pedantic  -pedantic-errors  -fstrict-aliasing  -Wstrict-aliasing  -DFMT_HEADER_ONLY  -DCEREAL_RAPIDJSON_HAS_CXX11_RVALUE_REFS  -flto  -static -O3 -DNDEBUG   -o CMakeFiles/test_quill.dir/test_quill.cpp.o -c /home/hungptit/working/cppidioms/src/test_quill.cpp",                                  
  "file": "/home/hungptit/working/cppidioms/src/test_quill.cpp"                                                                                                                                                                       
},

to support compiling dynamic library

Hi Quill team,

We are currently using your quill logger for our C++ projects which works brilliantly. Thanks.

Now we would like to utilize quill logger for JAVA applications as well by loading it as a dynamic library. In this way, Quill need to dynamically depend on fmt lib as well instead of being bundled together, as we are also using fmt-cpp's dynamic library.

Is that possible you can help to add the support of generating dynamic library as well, together with existing support of static library for quill?

Thanks in advance,
Xiong

Run benchmark failed

BENCHMARK_quill_hot_path_rdtsc_clock

terminate called after throwing an instance of 'quill::QuillError'
what(): failed to call set_cpu_affinity, with error message "Invalid argument", errno "22"
Aborted (core dumped)