Comment by rtpg

Comment by rtpg 3 days ago

2 replies

You really should dive more into the `multiprocess` support option and highlight how this gets around issues with the GIL. This feels like a major value add, and "does this help with CPU-bound work" being "yes" is a big deal!

I don't really need pipelining that much, but pipelining along with a certain level of durability and easy multiprocessing support? Now we're talking

t43562 3 days ago

...although python 3.13 can be built without the GIL and it really does make threading useful. I did some comparisons with and without.

I suppose one excellent thing about this would be if you could just change 1 parameter and switch from multiprocessing to threaded.

  • rtpg 3 days ago

    I think you could build off of threading. I do think here it's good to acknowledge that Python async is fundamentally single-threaded (or rather, "single thread per event loop"), so if you do go a multi-thread version you might have to do some bookkeeping to make it all work well.

    I'm not sure how well async Python libs are tested against working in a world with multiple event loops, but I bet there are a _lot_ of latent bugs in that space.