Comment by jsheard Comment by jsheard 2 days ago 3 replies Copy Link View on Hacker News Real search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.
Copy Link throwaway744678 2 days ago Collapse Comment - I suspect AI crawler would also (quickly learn to) respect it also? Reply View | 2 replies Copy Link jsheard 2 days ago Parent Next Collapse Comment - In that case, mission accomplished. Reply View | 0 replies Copy Link [removed] 2 days ago Parent Prev Collapse Comment - [deleted] Reply View | 0 replies
Copy Link jsheard 2 days ago Parent Next Collapse Comment - In that case, mission accomplished. Reply View | 0 replies
I suspect AI crawler would also (quickly learn to) respect it also?