Comment by johng

Comment by johng 2 days ago

1 reply

How? They have an index from a previous date and nothing new will be allowed since that date? A whole copy of the internet? I don't think so.... I'm guessing, like others, it's based on the date the user/website/blog lists in the post. Which they can change at any time.

fragmede 2 days ago

Yes they do. It's called common crawl, and is available from your chosen hyperscaler vendor.