Comment by jsheard Comment by jsheard 6 months ago 3 replies Copy Link View on Hacker News Real search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.
Copy Link throwaway744678 6 months ago Collapse Comment - I suspect AI crawler would also (quickly learn to) respect it also? Reply View | 2 replies Copy Link jsheard 6 months ago Parent Next Collapse Comment - In that case, mission accomplished. Reply View | 0 replies Copy Link [removed] 6 months ago Parent Prev Collapse Comment - [deleted] Reply View | 0 replies
Copy Link jsheard 6 months ago Parent Next Collapse Comment - In that case, mission accomplished. Reply View | 0 replies
I suspect AI crawler would also (quickly learn to) respect it also?