Comment by bb88 5 days ago
After spending a lot of my time on Django, it's fine for simple to moderately complex things. The ORM mostly good. DRF is fine for APIs. And the admin is super nice as well.
But once something gets significantly complex, the ORM starts to fall down, and DRF becomes more of a hindrance.
But if you're just doing simple CRUD apps, Django is perfectly serviceable.
Significantly complex means when ORM starts to become bigger and bigger and you need multiple threads and more complex processes that run in workers. When you start to run into scaling problems, your solution is within that framework and that becomes a limiting factor from my experience.
Then as a programmer, you have to find workarounds in Django instead of workarounds with programming.
PS: Dealing with a lot of scaling issues right now with a Django app.
You can absolutely scale Django.
The framework itself is not the limiting factor. The main constraint of performance usually comes from Python itself (really slow). And possibly I/O.
There are well established ways to work around that. In practice, lots of heavy lifting happens in the DB, can you can offload workloads to separate processes as well (whether those are Python, Go, Rust, Java etc).
You need to identify the hotspots, and blindly trusting a framework to "do the job for you" (or for that matter, trusting an LLM to write the code for you without understanding the underlying queries) is not a good idea.
I'm not saying you are doing that, but how often do you use the query planner? Whenever I've heard someone saying Django can't scale, it's not Django's fault.
> When you start to run into scaling problems, your solution is within that framework and that becomes a limiting factor from my experience.
Using Django doesn't mean that everything needs to run inside of it. I am working on an API that needs async perf, and I run separate FastAPI containers will still using Django to maintain the data model + migrations.
Occasionally I will drop down to raw SQL, or materialized views (if you are not using them with Django, you are missing out). And the obvious for any Django dev; select_related, prefetch_related, annotate, etc etc.
Yeah, I don’t get the issues here. I’ve led projects that served millions of requests a day, had dozens of apps and while there are always going to be pain points and bottlenecks, nothing about the framework itself is a hinderance to refactoring. If anything, Django plus good tests made me much braver about what I would try.
> And the obvious for any Django dev; select_related, prefetch_related, annotate
And sometimes not so obvious, I have been bitten by forgetting one select_related while inadvertedly joining 5 tables but using only 4 select_related: the tests work OK, but the real data has a number of records that cause a N+1. A request that used to take 100ms now issues "30 seconds timeout" from time to time.
Once we added the missing select_related we went back to sub-second request, but it was very easy to start blaming Django itself because the number of records to join was getting high.
The cases that we usually walk out of the Django path is for serializations and representations, trying to avoid the creation of intermediate objects when we only need the "values()" return.
> Then as a programmer, you have to find workarounds in Django instead of workarounds with programming.
The mental unlock here is: Django is only a convention, not strictly enforced. It’s just Python. You can change how it works.
See the Instagram playbook. They didn’t reach a point where Django stopped scaling and move away from Django. They started modifying Django because it’s pluggable.
As an example, if you’re dealing with complex background tasks, at some point you need something more architecturally robust, like a message bus feeding a pool of workers. One simple example could be, Django gets a request, you stick a message on Azure Service Bus (or AWS SQS, GCP PubSub, etc), and return HTTP 202 Accepted to the client with a URL they can poll for the result. Then you have a pool of workers in Azure Container Apps (or AWS/GCP thing that runs containers) that can scale to zero, and gets woken up when there’s a message on the service bus. Usually I’d implement the worker as a Django management command, so it can write back results to Django models.
Or if your background tasks have complex workflow dependencies then you need an orchestrator that can run DAGs (directed acyclic graph) like Airflow or Dagster or similar.
These are patterns you’d need to reach for regardless of tech stack, but Django makes it sane to do the plumbing.
The lesson from Instagram is that you don’t have to hit a wall and do a rewrite. You can just keep modifying Django until it’s almost unrecognizable as a Django project. Django just starts you with a good convention that (mostly) prevents you from doing things that you’ll regret later (except for untangling cross-app foreign keys, this part requires curse words and throwing things).
If you're doing simple CRUD apps, try https://iommi.rocks/ which we built because imo it's way way too slow and produces too much code to use standard Django to make CRUD stuff.
What does significantly complex mean though? You have to make sure you understand the queries made by the ORM, avoid pitfalls like SELECT N+1 queries and so on. If you don't do this, it'll be slow but it's not the ORM's fault - it's that of the programmer.