• Buddahriffic@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    1 year ago

    I mean, if you want to go around what the site wants you to do, you can just ignore robots.txt. Or use it to find the juicy stuff the site would rather you didn’t.