how do i Use loguru in multi modules? about loguru HOT 8 CLOSED

So I think you can use the snippet from the Logging Cookbook: Sending and receiving logging events across a network.

That would look like this:

# server.py
import time
import socketserver
import threading
import pickle
import struct
from loguru import logger

class LogRecordStreamHandler(socketserver.StreamRequestHandler):

    def handle(self):
        while True:
            chunk = self.connection.recv(4)
            if len(chunk) < 4:
                break
            slen = struct.unpack('>L', chunk)[0]
            chunk = self.connection.recv(slen)
            while len(chunk) < slen:
                chunk = chunk + self.connection.recv(slen - len(chunk))
            record = pickle.loads(chunk)
            logger.opt(raw=True).log(record['levelno'], record['msg'] + '\n')

address = ('localhost', 9999)
server = socketserver.TCPServer(address, LogRecordStreamHandler)

thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()

while 1:
    time.sleep(1)
    logger.info("Message from server")

# client.py
import time
from loguru import logger
from logging.handlers import SocketHandler

socket_handler = SocketHandler('localhost', 9999)

logger.remove()
logger.add(socket_handler)

while 1:
    time.sleep(1)
    logger.info("Message from client")

I tested it and it seems to work, but this is not as elegant as I would like. Ideally, you would just log the unpickled record, so that formatting depends on your server (and not client) handlers. Keep in mind that pickle is unsafe, so you maybe better should serialize the logs using JSON.

from loguru.

Oh, yes you're correct, thanks for pointing this out! The above snippet was a "translation" of the existing logging cookbook, but it needs to be adapted to accept multiple clients at the same time, indeed.

from loguru.

Hi @popeaaa.

You can log to the same file from different modules (i.e. Python files composing your application), just call .add() only once, the imported logger is the same object so share the same handlers.

Do you mean you have two applications run independently and executed by two different processes?

Well, if you .add("file.log") from the two apps, it risks to produce corrupted logs obviously. So you need some kind of communication for synchronization between the two processes. Maybe using the built-in SocketHandler with one server (writing logs to file) and one client (sending logs to server)?

from loguru.

Hi @popeaaa.

You can log to the same file from different modules (i.e. Python files composing your application), just call .add() only once, the imported logger is the same object so share the same handlers.

Do you mean you have two applications run independently and executed by two different processes?

Well, if you .add("file.log") from the two apps, it risks to produce corrupted logs obviously. So you need some kind of communication for synchronization between the two processes. Maybe using the built-in SocketHandler with one server (writing logs to file) and one client (sending logs to server)?

thank you ,it is 2 processes

from loguru.

So I think you can use the snippet from the Logging Cookbook: Sending and receiving logging events across a network.

That would look like this:

# server.py
import time
import socketserver
import threading
import pickle
import struct
from loguru import logger

class LogRecordStreamHandler(socketserver.StreamRequestHandler):

    def handle(self):
        while True:
            chunk = self.connection.recv(4)
            if len(chunk) < 4:
                break
            slen = struct.unpack('>L', chunk)[0]
            chunk = self.connection.recv(slen)
            while len(chunk) < slen:
                chunk = chunk + self.connection.recv(slen - len(chunk))
            record = pickle.loads(chunk)
            logger.opt(raw=True).log(record['levelno'], record['msg'] + '\n')

address = ('localhost', 9999)
server = socketserver.TCPServer(address, LogRecordStreamHandler)

thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()

while 1:
    time.sleep(1)
    logger.info("Message from server")

# client.py
import time
from loguru import logger
from logging.handlers import SocketHandler

socket_handler = SocketHandler('localhost', 9999)

logger.remove()
logger.add(socket_handler)

while 1:
    time.sleep(1)
    logger.info("Message from client")

I tested it and it seems to work, but this is not as elegant as I would like. Ideally, you would just log the unpickled record, so that formatting depends on your server (and not client) handlers. Keep in mind that pickle is unsafe, so you maybe better should serialize the logs using JSON.

thanks a lot! happy Chinese new year!

from loguru.

Im not sure the above example will work from multiple clients. I think you would have to remove after each message sent.

from loguru.

@Mango-kid What do you mean? What should be removed?

from loguru.

If you want to have multiple log clients connected you will have to

logger.remove()
logger.debug("some log message")
logger.add(socket_handler)

for every logging event. I dont think the server in the above cookbook code will handle more that one connection because each logger leaves the connection open.

from loguru.