• reddig33@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            ·
            edit-2
            3 months ago

            I also don’t know if a law that says search engines have to honor a robots.txt file. I guess we will see what happens if Bing or some other service decides to ignore it.

            • Evotech@lemmy.world
              link
              fedilink
              English
              arrow-up
              1
              ·
              3 months ago

              You can just require a log in to view content, or just flat out auto ban indexing robots.

      • helenslunch@feddit.nl
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        3 months ago

        How can they do that, logistically?

        Like I realize there’s a flag they can raise that asks not to be indexed but that’s not legally binding.

        • Evotech@lemmy.world
          link
          fedilink
          English
          arrow-up
          3
          ·
          3 months ago

          I guess they can make it hard to index by scraping by rate limiting or requiring login to view content etc and only provide Google the api to bypass the restrictions

          There’s probably a lot of ways to do it