Comment by lisper
Comment by lisper a year ago
I was responding to this:
> You can't get exactly once delivery using proposed solution in general case because you can not have infinite memory for dedupe.
That is a specific criticism of a specific and highly oversimplified algorithm for de-duplication that I described purely for illustrative purposes, not as a suggestion for how de-duplication should actually be implemented. Actual implementation of de-duplication is much more complicated, but a solved problem that I didn't think I needed to rehash. A real implementation doesn't require infinite memory.
(Actually, even the naive algorithm doesn't require infinite memory, just unbounded memory. Those are not the same.)
You certainly do need to rehash it or provide a link to it being solved without constraints on time window, msgs rates and msgs sizes. You also most certainly want to build it and provide it as service because no existing product claims this capability in unbounded case.