dotancohen a day ago

With no enforcement mechanism?

  • culi a day ago

    Some European laws give robots.txt files some legal weight. That's why you often see this in robots.txt that have content-signals

      # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.