Comment by rtpg

Comment by rtpg 10 months ago

2 replies

You really should dive more into the `multiprocess` support option and highlight how this gets around issues with the GIL. This feels like a major value add, and "does this help with CPU-bound work" being "yes" is a big deal!

I don't really need pipelining that much, but pipelining along with a certain level of durability and easy multiprocessing support? Now we're talking

t43562 10 months ago

...although python 3.13 can be built without the GIL and it really does make threading useful. I did some comparisons with and without.

I suppose one excellent thing about this would be if you could just change 1 parameter and switch from multiprocessing to threaded.

  • rtpg 10 months ago

    I think you could build off of threading. I do think here it's good to acknowledge that Python async is fundamentally single-threaded (or rather, "single thread per event loop"), so if you do go a multi-thread version you might have to do some bookkeeping to make it all work well.

    I'm not sure how well async Python libs are tested against working in a world with multiple event loops, but I bet there are a _lot_ of latent bugs in that space.