Comment by nucleardog

I have tools I've developed that do some/most of this, but they're internal/proprietary so I can't share them directly. What I _can_ share is how it works. Maybe somebody with more time/energy/will to live than me can take a crack at the problem.

Every developer runs Rancher Desktop as a local k8s cluster.

There's a controller + nginx container in the cluster.

For any appropriately annotated ingress the controller's mutatingwebhook patches the ingress to be backed by its own proxy service. It then reaches in and reconfigures the cluster's CoreDNS to resolve the domain to its proxy service as well.

Then as pods are started/stopped, it tracks whether the service your ingress was supposed to point at has any running pods behind it. If it does, it configures nginx to forward the request to the local service. If it doesn't, it configures nginx to proxy it to the upstream URL.

That all comes together in three main ways:

1. We start chromium with, among other things, a flag to set host rules that rewrite all connections to 127.0.0.1. So going to `oursite.com` loads our site through your cluster. API requests the page makes to `service.oursite.com` get routed through the local cluster.

2. Any requests your containers make to other services can request `oursite.com` and because of the CoreDNS stuff they'll hit the local proxy and get routed appropriately.

3. ... And for anything else we just have a real `localdev.cloud` domain with a wildcard subdomain that resolves to 127.0.0.1 and include that host on all the ingresses as well. So Postman can hit a service at `service.localdev.cloud`.

This puts us in a good place to "easily swap". There's a pile of bash scripts calling themselves a justfile that manages most of this. Run `system start` to bring up the controller and proxy as well as deploy all of the ingresses and services (so it knows what it's rewriting/proxying). Then you just do `project whatever up` and `project whatever down` to create/destroy the deployment and other resources. Mounting the project code into the container is `project whatever mount`--this is a separate step so in a situation where, e.g., a FE guy wants to test a specific BE build he can just throw the container tag in a .env and start it up and keep working on what he was working on. (And the QA can just start up any build of anything without any extra fuss.)

As for SSL, we're mostly solving for "on the same machine". The controller generates a root certificate on first start (so every developer has their own and it couldn't be used to intercept anyone else's traffic), then uses that to issue + sign certificates for everything else. You could add that to any other devices if you wanted. What we do is just slip chromium an extra flag to tell it to treat our certificates as valid.

So I can run a just command to open up chromium with the novel-worth of extra flags, go to `https://oursite.com`, and everything works. If there's a specific BE service I need to poke at, I just `project my-service up` and go back to chromium and keep doing stuff. If I wanted to make some FE changes, `project my-fe up && project my-fe mount` and start editing code.

There's a lot more to all of this but this comment's already way too long (but feel free to ask and I can talk your ear off). End of the day, though, we went from it taking like 1-2 days to get people to a point where they could start editing code to (and I tested this when a computer died one day) 45 minutes--most of which was waiting for rancher to download and then starting it and waiting for it to download more stuff. Went from it taking a day to get some rarely-touched legacy service limping along enough that you could debug something to it being consistently a single command and like 30 seconds. Went from spending a bunch of time reconfiguring all the services for each combination you might try and run to... just not anymore. Bugs/issues/misalignments getting caught much earlier because turns out making it easy to actually run the software together means people will do it more.

I have most of what you're asking for and you're definitely on the right track--it's a way nicer way to live.