EDIT, Update: We’ve been able to partially recover about 25% of the media uploaded to Pawb.Social.

Upon further investigation, data from pawb.social was priorly moved to the new server and was inadvertently deleted as part of the earlier incident affecting furry.engineer and pawb.fun’s cache while we were preparing the new server.

furry.engineer and pawb.fun media is unaffected, but all media directly uploaded to pawb.social is unrecoverable and has been lost. A miscommunication on my part OK’d the removal without fully checking the scope of data that was being removed; This was intended to only remove test data we were using to stress test the new hardware, but affected live data as well.

A large majority of posts are externally hosted and referenced (e.g. images from media.kbin.earth, d.furaffinity.net, etc.) are NOT affected. Only posts directly uploaded via the Create Post > Image field on pawb.social are affected.

We had previously transferred pawb.social to the new hardware as a preliminary trial and I had forgot this occurred thus failing to bring it up prior to the deletion going through.

I’m looking into options to be able to in-place reupload media on behalf of posters, but as the UUIDs are randomly generated by Pict-rs (the media service used by Lemmy), I’m unsure how to facilitate this. Any advice or information would be appreciated, and can be sent to network@pawb.social.

I understand this is a serious incident and reflects poorly on us, and I can only apologize for this loss and work to prevent this happening again in the future.

EDIT: I believe this will also impact profile pictures and community images (logos and banners), so please re-upload them at your earliest convenience.___

  • Crashdoom (he/him)@pawb.socialOPMEnglish
    11·
    5 months ago

    ℹ️ Thanks to the efforts of one of our moderators, @southernwolf@pawb.social, we’ve been able to recover some of the lost data from pawb.social that was still active in Cloudflare’s regional caches.

    Currently, this is about 25% of all of the known uploaded media files stored on pawb.social.

    We’ll be reviewing the final tally of recovered data in the morning and trying to work on getting what we can reimported into Pict-rs.

    • Southern Wolf@pawb.socialMEnglish
      6·
      5 months ago

      For those who see this before we release the final tally: As it stands right now, I’ve recovered ~1/3rd of media posts via Cloudflare caching. I’ll have a final total when I’m truly done, but honestly that’s pretty damn good to have been able to rescue all of that via cache recovery alone.

      • Southern Wolf@pawb.socialMEnglish
        6·
        5 months ago

        This is subject to change, but here is the preliminary tally:

        Recovery report generated:
2025-10-09 08:58:47,515 - INFO -   - Total entries in CSV: 6697
2025-10-09 08:58:47,515 - INFO -   - Images recovered: 2201
2025-10-09 08:58:47,515 - INFO -   - Images missing: 4496
2025-10-09 08:58:47,515 - INFO -   - Recovery rate: 32.87%
2025-10-09 08:58:47,515 - INFO -   - Total size: 1501.68 MB (1.47 GB)
        • Darren the Foxcoon@pawb.socialEnglish
          2·
          5 months ago

          Would the Pawb staff be open to using volunteers as secondary backups? A few GB seems like absolutely no problem for my own rig

          • natebluehooves@pawb.socialMEnglish
            4·
            5 months ago

            Amusingly we do have backup infrastructure on site. 30tb pawb nas, 80tb nas for backups, and lto tape backups periodically. my problem is getting it all set up. i’ve been battling depression and trying to catch up on housework.

            the issue here was doing a partial migration and allowing a couple months to pass before trying to finish, by which point I had forgotten pawb was moved.

            we plan on pawb to take a lot more storage as time goes on. we are at about 1.5tb plus ~450gb database size right now.

            edit: to make some sense of this: the 1.5tb includes cached media from other instances. our users don’t use a lot of storage yet.

  • KoboldCoterie@pawb.socialEnglish
    16·
    5 months ago

    No worries, mate. Thanks for the heads up that it happened; you’re providing a free service to us, we aren’t going to complain about the occasional hiccups, especially considering how infrequent there’ve been any.

  • Kovukono@pawb.socialEnglish
    14·
    5 months ago

    You guys are running a free server with 99.9% uptime and barely any notable issues in the time I’ve been here. Losing some pictures in a migration over a mistake is not as serious as you think it is. You guys do amazing work, and we love you for everything you’ve done.

  • Ethanol@pawb.socialEnglish
    8·
    5 months ago

    This is fine, you’re doing a great job. Mistakes happen and re-uploading my profile picture or banner is not a hassle at all. No need to beat yourself up. Thank you all for the work you put into this instance :3

  • LordAmplifier@pawb.socialEnglish
    7·
    5 months ago

    It’s a bit of a bummer to see some of the “history” of pawb disappear, but shit happens. You’re providing one hell of a good service completely for free, and that’s cool as heck. Thank you for the transparency and all the good work you do behind the scenes <3

  • Wahots@pawb.socialEnglish
    7·
    5 months ago

    Happens to the best of us! My work just nuked a ton of data in a similar incident, so you’re in good company, haha.

  • Leon@pawb.socialEnglish
    6·
    5 months ago

    Sometimes, shit happens. We’ve all been there, one way or another. Thanks for all your hard work.

  • Foxfire@pawb.socialEnglish
    5·
    5 months ago

    No worries, good to know the cause of the image issues. I’ll simply edit my posts to reupload the missing data since it was all my own art anyway.