A Brazilian team used Discord’s API to scrape 10% of its open servers.

Researchers published a massive database of more than 2 billion Discord messages that they say they scraped using Discord’s public API. The data was pulled from 3,167 servers and covers posts made between 2015 and 2024, the entire time Discord has been active.

Though the researchers claim they’ve anonymized the data, it’s hard to imagine anyone is comfortable with almost a decade of their Discord messages sitting in a public JSON file online. Separately, a different programmer released a Discord tool called “Searchcord” based on a different data set that shows non-anonymized chat histories.

  • asbestos@lemmy.worldEnglish
    243·
    1 day ago

    Probably our only chance to find solutions to problems with open source software that uses Discord as their forum

    • boatswain@infosec.pubEnglish
      131·
      1 day ago

      Seriously. It’s beyond painful when some open source project only uses Discord for communication. You have to hope that you post your question at a time when the right people are online, and that there’s not a more interesting conversation going on, otherwise it just gets lost. Index that whole dataset.

      • Ulrich@feddit.orgEnglish
        1·
        2 hours ago

        Index that whole dataset

        I’ve seen a few projects doing just that with answeroverflow.com and they have come up in my web searches. Not really a solution but at least a stopgap.

        • The Quuuuuill@slrpnk.netEnglish
          50·
          1 day ago

          there’s a difference between using irc for livetime troubleshooting and not having a forum at all and directing everyone to your livechat discord. i’m sure some sicko out there has run an OSS project on only IRC, but their project likely got no traction because a history of problemsolving posts is important in open source. generally speaking, you need:

          • a wiki
          • a static indexable searchable forum
          • a live chat place for real time communication for novel problems

          too many projects these days only have that last one in the form of discord

        • AugustWest@lemm.eeEnglish
          9·
          22 hours ago

          For projects I am involved with all irc chats are archived and searchable. There is nothing private, no registration needed and searchable.

          Quite a bit different.

        • boatswain@infosec.pubEnglish
          12·
          1 day ago

          That would be equally annoying. Probably a better signal to noise ratio on IRC though; Discord descends into memes almost instantly.

    • Leon@pawb.socialEnglish
      17·
      1 day ago

      I spent nearly three hours today between discord and matrix trying to figure out how to get these two pieces of software to talk using a certain protocol.

      Imagine if there were online indexable platforms where people could publish this information so it’s easily accessible rather than having to scour through message logs hoping to find the right keywords. Such a technology surely doesn’t exist already, right?

      I hate discord.

      • dual_sport_dork 🐧🗡️@lemmy.worldEnglish
        35·
        24 hours ago

        I don’t hate Discord, I simply hate that so many projects and companies have unanimously decided to use it as the wrong tool for the wrong job.

        It’s fine for its intended use case, which is bickering with my friends about video games and fiction, and spamming each other with .gifs and meme images.

        • MBech@feddit.dkEnglish
          201·
          24 hours ago

          Discord is genuinely a great tool for what I used to use Skype for. Talking to my friends, and sharing dumb memes with them in a groupchat format. Companies need to learn that using it as a forum, a Q&A service, a wiki or any other information sharing purpose, is simply fucking retarded.

      • MDCCCLV@lemmy.caEnglish
        41·
        21 hours ago

        Yeah, but then you have something like when people protest deleted their history on reddit which is fine as a protest tactic but leaves a hole where your specific question came up but now there’s nothing there.

      • spiderhamster@lemmy.worldEnglish
        1·
        22 hours ago

        you get it to work? i didnt have time to get it working in both directions. matrix to discord worked fine but not the other way.

        • Leon@pawb.socialEnglish
          1·
          19 hours ago

          I’m not entirely sure what you’re asking here. I do not use any bridge between the two, but rather searched in separate communities for my answer. Would’ve been lovely if I could just use a search engine to search indexed forums or so, but since for some reason chat clients have taken the place of forums that’s just not doable.

          I’d like to move away from Discord but sadly a bunch of friends still use it. I haven’t read up enough about the bridge thing to figure out if it actually serves a purpose I’d be interested in or not.

    • nawa@lemmy.worldEnglish
      141·
      1 day ago

      Lol, I’ve read this headline and thought “thank fuck, probably the only option to have Discord’s content readable”, I like how universal this opinion is