pickle doesn't work on mac m1 about sqs-workers HOT 3 CLOSED

Thank you for your response. But I already figure out the problem and even forget how =)) Sory did not close this issue earlier.
Have a nice day

from sqs-workers.

Hi there 🙂 I happened to have encountered the same problem as originally mentioned (TypeError: cannot pickle 'module' object)

I was wondering if you had any ways of helping to figure this out, whether it's an issue with my environment or implementation?

So far I've tried the minimal code mentioned in #16 (comment) and it still produces the problem. It's possible to avoid this with multiprocessing.set_start_method("fork") however that seems to present other issues affecting stability (at least for me, on this mac)

Calling something like this from the python repl (based on what the process_queues implementation does) will also produce the exception, I'm just at the limits of where to look further into detail as a relative newbie to python.

>>> import pickle
>>> from queue_handlers import sqs
>>> fn = sqs.queue("email").process_queue
>>> fn
<bound method GenericQueue.process_queue of RawQueue(name='email', backoff_policy=<sqs_workers.backoff_policies.ExponentialBackoff object at 0x10d630710>, batching_policy=<sqs_workers.batching.NoBatching object at 0x10d5e6ad0>, processor=<function send_email at 0x10dcb91c0>)>
>>> pickle.dumps(fn)
...the computer screams into the void ...
TypeError: cannot pickle 'module' object

This is with sqs-workers = 0.5.11 on python 3.11

If you have any tips for investigating problems like this, I'd appreciate any pointers or direction!
Like the original reporter, I too hope you have (or are already having) a nice day 😄

from sqs-workers.

Hey @1k2s3v, sorry for the delay.
Which version of sqs-workers are you using?

Could you please try to split your code in 2 files:

# queue_handlers.py
from sqs_workers import SQSEnv, create_standard_queue

sqs = SQSEnv()

create_standard_queue(sqs, "emails")
queue = sqs.queue("emails")

@queue.processor("send_email")
def send_email(to, subject, body):
    print('asd')

queue.add_job("send_email", to="[email protected]", subject="Hello world", body="hello world")

# processor.py
from queue_handlers import sqs

def run():
    sqs.process_queues()

if __name__ == "__main__":
    run()

and run python processor.py?

from sqs-workers.