Comment by reactordev
Comment by reactordev 2 days ago
This is not analogous. It’s just someone beating the Erlang drum. You can’t PyTorch in Erlang.
Comment by reactordev 2 days ago
This is not analogous. It’s just someone beating the Erlang drum. You can’t PyTorch in Erlang.
There are similar libraies in Elixir. Is the ecosystem for ML as developed as for python? Nope, but not every ML project needs the most obscure libraries etc.
(For the record I don't really see Erlang clusters as a replacement for k8s)
You aren’t going to teach a research scientist Erlang in spite of Python and R. You aren’t going to win that fight, ever.
Oh totally, that is not anyone's goal I think. The goal is more about enabling teams already running Elixir to also do their ML stuff in Elixir or teams that needs to do something that would benefit from the BEAM actor model but also need to do some AI stuff to do it all in Elixir. Lots of neural networks created in Python or R can then run in Elixir.
Kubernetes is way heavier than Erlang’s lightweight processes, so for millions of tasks at scale, a middle-ground solution could blend Erlang’s concurrency efficiency with k8s’ orchestration power, dodging containers’ overhead while keeping flexibility for diverse workloads. That's if you don't actually need the strict isolation of pods/containers and you're just trying to run something at massive scale. I don't get why so many people want to run everything as heavy container processes or pods vs coming up with a better solution. The point is we don't have to fit every problem into the shoe called kubernetes if it doesn't seem to fit, and we should look at other ways to spin up millions of processes