Comment by mrkeen

Comment by mrkeen 4 days ago

Very often I find myself wanting to store item(s) using a key.

My items are not relations, and I don't see the point in transforming them to and from relational form. And if I did, each row would have like 5 columns set to NULL, in addition to a catch-all string 'data' column where I put the actual stuff I really need. Which is how you slow down an SQL database. So RDBMS is no good for me, and I'm no good for RDBMS.

RDBMS offers strong single-node consistency guarantees (which people leave off by default by using an isolation level of 'almost'!). But even without microservices, there are too many nodes: the DB, the backend, external partner integrations, the frontend, the customer's brain. You can't do if-this-then-that from the frontend, since 'this' will no longer be true when 'that' happens. So even if I happen to have a fully-ACID DB, I still lean into events & eventual consistency to manage state across the various nodes.

Given that I'm using more data than a naive CRUD/SQL app would (by storing events for state replication) and my data is stringy enough to kill my (and others') performance. So what's the solution? Make my read-writes completely independent from other read-writes - no joins, no foreign keys, etc.

The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it? For this reason I'd probably reach for Cassandra first. That said I haven't looked at the landscape in a while and there might be much better tools.

But it also wouldn't matter what I want to use instead of DynamoDB, because the DevOps team of wherever I work will just choose whatever's native&managed by their chosen cloud provider.

throwaway82452 4 days ago

> The thing that would put me off using DynamoDB is the same reason I wouldn't use any other tech - can I download it?

Amazon provides a downloadable version for development. I don't know how close it is to the real thing, but it makes it easier to do local dev.

Localstack also supports it in their paid version

Reply View 4 replies

dygd 4 days ago

The downloadable version is nowhere near ready for production. It's performance is also excruciatingly slow.

Reply View | 3 replies
- tempworkac 4 days ago
  
  It doesn't really make any sense to use it locally - the whole point is that it's managed. If you just want a clustered key value store you could use Cassandra, Garnet, etc.
  
  Reply View | 1 reply
  
  plandis 3 days ago
  
  I think people mostly use it for unit testing functionality since it’s generally a faster dev loop compared to running integration tests.
  
  Reply View | 0 replies
- snapcaster 3 days ago
  
  It's not supposed to be used for production, it's supposed to be used for development
  
  Reply View | 0 replies

Lapapapaja 4 days ago

> I still lean into events & eventual consistency to manage state across the various nodes.

You can get really far with a RDMS before event sourcing etc is needed, the benefit being both your dev and user experience are going to be much simpler and easier.

If you already know your problem domain and scaling concerns up front sure. But starting with a scalable pattern like this is a premature optimization otherwise and will just slow you down.

Reply View 4 replies

mrkeen 4 days ago

> You can get really far with a RDMS before event sourcing etc is needed
You can manage up to 0 partners easily. Once you go above that threshold, you're into "2-Generals" territory. At that point you're either inconsistent, eventually-consistent, or you're just bypassing your own database and using theirs directly.
> dev and user experience are going to be much simpler and easier.
I have objects, not relations. I'm not going to do the work of un-nesting a fat json transaction to store it in a single relation (or worse, normalise it into rows across multiple tables).

Reply View | 3 replies
- mulmen 3 days ago
  
  Yeah this is a much better initial dev experience but you still have a schema, even if you ignore it. When your objects are inconsistently shaped something has to fix them. That something is going to take the shape of custom code that would make even the Perl-iest DBA blush.
  
  Reply View | 2 replies
  
  mrkeen 3 days ago
  
  So we've shifted from:
  SQL now (for dev experience) && no-SQL later (for scaling)
  to:
  no-SQL initially (for *much better* dev experience) && no-SQL later (for scaling)
  I can get behind that.
  > When your objects are inconsistently shaped something has to fix them
  They have one schema (the class file) instead of two (the class file and the SQL migrations).
  
  Reply View | 1 reply
  
  mulmen 3 days ago
  
  > They have one schema (the class file) instead of two (the class file and the SQL migrations).
  But what happens when that schema defining class file needs to change? You put all your migration code there? How is that different from SQL migrations?
  
  Reply View | 0 replies

mike_hearn 4 days ago

Some RDBMS only offer single node consistency but others can scale write masters horizontally (e.g. Oracle).

Reply View 0 replies