• VonReposti@feddit.dk
    link
    fedilink
    English
    arrow-up
    0
    ·
    1 month ago

    They’re spamming all web logs too with an advertisement for their services in the user agent. I decided to ban them from all my websites because the logs took up too much space.

      • VonReposti@feddit.dk
        link
        fedilink
        English
        arrow-up
        0
        ·
        1 month ago

        They’re saying who they are, what they do, and are linking to their website and sometimes sends hundreds of requests in a minute. It might not say "For only €49.99 you can get your very own thing!”, but that does not mean they aren’t throwing their name up in every website owner’s arse whether they like it or not.

          • VonReposti@feddit.dk
            link
            fedilink
            English
            arrow-up
            0
            ·
            1 month ago

            That could have been done by just having a single entry called GoogleBot or BingBot, not an entire sentence explaining their product offering let alone hundreds of times a minute.