Comment by timewizard 2 days ago
> This process involves looking up all of your followers, then inserting a new row into each of their Timeline tables in reverse chronological order with a reference to your post.
Seriously? Isn't this the nut of your problem right here?
You'd still have exactly the same hot write path, it'd just have maybe 50% of the load. That could be a legit optimization, but not having it hardly warrants an incredulous "seriously?" like the OP's.
(And the same for the inverse hybrid strategy of quarantining the writes of highly followed users and handling their fan-out at read time. A neat optimization, and maybe even absolutely once you have accounts with 100M followers. But the vast majority of posts would still be handled via the original strategy.)
What alternative design did you have in mind, given that a Twitter-like data model of individual follows is likely a strict product requirement?
There are obviously other ways of doing it (doing the timeline propagation in a batch job, fanning out the reads rather than the writes), but they've got their own problems. Probably worse ones.