Comment by SR2Z

Comment by SR2Z 3 months ago

5 replies

View on Hacker News

IA actually has technical and moral reasons to ignore robots.txt. Namely, they want to circumvent this stuff because their goal is to archive EVERYTHING.

prinny_ 3 months ago

Isn’t this a weak argument? OpenAI could also say their goal is to learn everything, feed it to AI, advance humanity etc etc.

Reply View 3 replies
  • compootr 3 months ago

    OAI is using others' work to resell it in models. IA uses it to presrrve the history of the web

    there is a case to be made about the value of the traffic you'll get from oai search though...

    Reply View 1 reply
    • [removed] 2 months ago
      [deleted]
      Reply View 0 replies
  • SR2Z 2 months ago

    It does depend a lot on how you feel about IA's integrity :P

    Reply View 0 replies