Comment by SR2Z

Comment by SR2Z 14 hours ago

4 replies

View on Hacker News

IA actually has technical and moral reasons to ignore robots.txt. Namely, they want to circumvent this stuff because their goal is to archive EVERYTHING.

prinny_ 12 hours ago

Isn’t this a weak argument? OpenAI could also say their goal is to learn everything, feed it to AI, advance humanity etc etc.

Reply View 2 replies
  • compootr 12 hours ago

    OAI is using others' work to resell it in models. IA uses it to presrrve the history of the web

    there is a case to be made about the value of the traffic you'll get from oai search though...

    Reply View 1 reply
    • [removed] 4 hours ago
      [deleted]
      Reply View 0 replies