Comment by cyberax

Comment by cyberax 3 days ago

I tried to use K8s several times, and I just can't make it work. It's fine as a deployment platform, but I just can't justify its complexity for local development.

We're using Docker/Podman with docker-compose for local development, and I can spin up our entire stack in seconds locally. I can attach a debugger to any component, or pull it out of the Docker and just run it inside my IDE. I even have an optional local Uptrace installation for OTEL observability testing.

My problem is that our deployment infrastructure is different. So I need to maintain two sets of descriptions of our services. I'd love a solution that would unify them, but so far nothing...

stackskipton 3 days ago

I wouldn't use K8s for local development unless you have some system where there is a dev cluster and you can route traffic for particular pod to your local workstation.

Docker Compose for local development is fine. If your K8s setup is crazy complex that you need to test it locally, please stop.

Reply View 0 replies

fragmede 3 days ago

Tilt? Skaffold? configuration isn't free, but a debugger on a local k8 cluster that's at least somewhat representative of prod is pretty handy once you do.

Reply View 1 reply

cyberax 3 days ago

I tried Tilt, but it's still too complicated. For example, we have a computer vision model that is a simple Python service. When developing on macOS, it's not possible to use GPUs inside containers, so we need to run it locally on the host.
It's trivial with my current setup, but not really possible with Tilt.

Reply View | 0 replies

ghthor 3 days ago

I know this pain, though we’re running nomad not k8s as our cluster control plane. But local devs are stuck with docker-compose, so 2 different configurations for running locally versus in the production environment.

Reply View 0 replies