from the other side, i’m the cto of a small agency and i wrote our rules for working with ai, so maybe it helps. the first one is that the author answers for the code, “the model did it” is not an explanation for anyone. asking the model to review its own diff is a first pass, not a review, the same model that got it wrong is checking itself. the way i put it to the team, ai makes producing cheaper, not deciding or verifying, if you ship double with the same error rate you get double the errors. if your company only measures volume, the atrophy is more a management problem than yours, i’d push back with incidents and rework numbers, thats the language they listen to

i run a small browser game on a hetzner vps and for me the bots were never a load problem, it was the stats. the game shows how many people are online and every crawler that opened the live stream counted as a new player, in early october it was 28 of 61 stream opens. then a headless chromium on residential proxies, different ip every time, made 40 fake players in two days. what caught it was the Sec-CH-UA header, it said chromium 146 while the user agent said chrome 120, real browsers don’t mismatch like that. i don’t block them now, they just stop counting as people, which is kind of depressing too i guess

nice! for anyone with postgres in the mix, what bit me wasn’t the backup itself but the disk. a table in my side project kept one snapshot per minute and never deleted any, the disk hit 100% and postgres went down with a PANIC for like 40 min. now the nightly pg_dump writes to a .tmp, checks it with gzip -t and only then replaces the old file, and the app warns me if the newest dump is older than 26h or suspiciously small. restoring is just loading it into an empty db and starting the app, boring but it’s the part you actually want to know works

one or two services first is the right call, the scope creep is real lol. good luck with the move, and do a full restore of the photos while the box is still at home, that’s the one thing you really don’t want to debug remotely

yeah that matches what I see, most of what we run at work is one app and one database, so the slow query log plus looking at the plan gets you like 90% of the way. I never really had the multi service case where otel pays off, I guess at some point it does, but for a small team it felt like a lot of plumbing to end up finding the same n+1 query lol

I’d add that most of the time nobody actually measured anything. at work we inherit slow sites all the time and the first guess is always “the stack”, then you profile it and it’s one query running per item in a loop or something like that. I had the same with my own side project, was sure I needed a bigger server and after measuring the problem was somewhere else completely. measuring first is boring but it’s cheaper than any rewrite I guess

Plan sounds fine to me, debian + immich in docker + tailscale is pretty much the boring setup and boring is good for a box in a shed. one thing I’d do before carrying it out there, install tailscale and ssh, reboot it a couple of times and make sure it comes back on its own without a screen, because walking to the shed with a monitor gets old fast. also something like uptime kuma on another machine (or your NAS) pinging it, so you know it’s down before you’re on holiday and the photos stop uploading. I run a box at home I only reach over tailscale and that’s the part that saved me most.

good to know about the limit, thanks. for one domain plus DNS-01 that’s all most of us need anyway

yeah that sounds like exactly it, a snap that never worked still sitting there. “snap list” will show it, then sudo snap remove <name> and you’re left with only the docker one, so whatever you open is the right one

thanks! short answer is no, a reload doesn’t fix it. when the browser source comes up mute the page is actually fine, the audiocontext is running and even a test tone from the debugger never reaches the obs meters, so it’s obs, not the game. reloading or toggling “control audio via obs” didn’t help, only a full obs restart does. funny part is it fails on alternate starts, I measured 12 in a row and it was mute, sound, mute, sound. so the watchdog reads the meters for a few seconds and if the peak is under -60 dBFS it treats it like a black screen and restarts obs. and yeah the dwell idea is good, I’ll steal that

spain, so CEST till tomorrow actually, the 13:45 UTC slot is 15:45 for me. 7:45 with people around sounds way nicer than my timer lol

on the fear part, in a healthy team the author still owns the change, the reviewer is a second pair of eyes not an insurance policy. what helped my people a lot was asking the author how they tested it and what happens if it goes wrong, if they can’t answer that’s your review comment right there. and if something breaks after you approved it, you fix it together and add a check so it doesn’t happen again, nobody serious blames the reviewer for that

They don’t go away on their own, every installed snap is mounted as its own loop device and snapd keeps an older revision or two so it can roll back, so the number moves around after updates. snap list --all shows which snaps they are, if one says disabled that’s the old revision and you can remove it with sudo snap remove <name> --revision=<rev>. about libation, if it’s installed as a snap itself that could be the actual problem, snaps are confined and can’t read folders outside your home or on other drives unless you connect the removable-media interface, so I’d check that before worrying about the loop devices.

Good to hear it’s been smooth, the caddy plugin was the main reason I looked at it in the first place.

oh, didn’t know that, thanks for the heads up. deSEC still hosts dns for free if you bring your own domain I think, so a cheap one plus deSEC for the acme dns challenge still works, just not the free subdomain part

that’s actually a nice idea, updating with people around makes it way less of a chore, and someone always knows the weird fix. I’d probably steal it if I had more than one box to look after.

same here on the reboots, I never let it reboot on its own either. for me the reboot wasn’t the problem, it was needrestart bouncing nginx and postgres right after the upgrade, which you only notice if something long lived is running at that moment, like a stream lol

Interesting idea, but in my case it would feel worse I think, each paint costs a charge, so undoing both means two people lose a charge and nobody gets the cell. I don’t think it happens much at our size anyway, two people on the exact same cell inside half a second is pretty rare. last write wins is a bit unfair but it’s simple, and since every write is stored you can always see who painted what

512mb and getting killed every morning at the same time sounds like a fun one to track down lol. a fixed weekly slot like your tuesday evening is probably the sanest setup honestly, I just know I’d forget.

yeah exactly, no reboot, it was needrestart restarting nginx and postgres after the systemd update. you’re right that the timer is kind of the blunt fix, the cleaner one is probably telling needrestart to only list instead of restart ($nrconf{restart} = ‘l’ in needrestart.conf), or excluding nginx from it. I went with the timer because I still want things restarted, just not while I’m live