Comment by hinkley
To a first order approximation, Wikipedia serves everyone the same page. So the cost of pages in Wikipedia is proportional to the rate of edits, not the rate of page clicks and inbound links. Because once I hit Save, the page they display is pretty much the same one they'll show you when I send it to you to ask your opinion.
If I show each user of a website a highly customized, user- and workflow-specific page for the same url based on context and previous activity, then I have to generate it every time ("hand crafted, artisanal web pages") so the weight of the backend is now proportional to traffic.
Amazon.com tries to split the difference. You and I see the same bones of a page for an Apple Watch 10, but little bits load in and show you laundry detergent and me pickles. But Amazon makes more money every time I click on pickles and add to my cart, so there's an expectation that the fractional penny they pay to load the page fragment results in more sales. That doesn't work the same for SaaS applications, so you need to use even this trick sparingly, not build your whole product value statement around it.
For the control issues crack, the paying customer (not their users) is attracted by all the levers and dials, but cannot appreciate the cost using them exposes them to. The development cost can amortize over time, increasing your profit margins and letting you recoup the R&D costs, but the cost of keeping a cluster running cannot. And you've painted yourself into a requirements corner you can't get out of. Eventually their eye drifts to competitors with fewer high-cost, high-value features in favor of low-cost.
Thanks for explaining and I am sure you have more experience with this type of scale but wouldn’t you say that ChatGPT is an example of “hand crafted artisanal page”, where not only every person sees a different answer but every interaction with the page results a different response. Of course they have a ton of VC money to burn but could it be that technology optimization (especially on the backend and hosting provider) could be the solution to this cost issue as opposed to blaming product features?