Have a sneer percolating in your system but not enough time/energy to make a whole post about it? Go forth and be mid - welcome to the Stubsack, your first port of call for learning fresh Awful youāll near-instantly regret.
Any awful.systems sub may be subsneered in this subthread, techtakes or no.
If your sneer seems higher quality than you thought, feel free to cutānāpaste it into its own post ā thereās no quota for posting and the bar really isnāt that high.
The post Xitter web has spawned so many āesotericā right wing freaks, but thereās no appropriate sneer-space for them. Iām talking redscare-ish, reality challenged āculture criticsā who write about everything but understand nothing. Iām talking about reply-guys who make the same 6 tweets about the same 3 subjects. Theyāre inescapable at this point, yet I donāt see them mocked (as much as they should be)
Like, there was one dude a while back who insisted that women couldnāt be surgeons because they didnāt believe in the moon or in stars? I think each and every one of these guys is uniquely fucked up and if I canāt escape them, I would love to sneer at them.
(Credit and/or blame to David Gerard - both for starting this, and for covering the previous week. We should get a bot to automate thisā¦)
<buys giant bucket of popcorn>
I vaguely recognize the name, apparently heās a snot-nosed 25-yr old who got fired from OpenAI
Before the days of AI 2027, he became famous in 2024 for posting one of the original pieces of writing in the line-go-up genre: Situational Awareness. It seems that like any good grifter, he used this opportunity to make money (in this case by starting a hedge fund).
Enjoy todayās craziest link: https://atomarine.co/
I see you, and raise by a Gundam solar space laser - https://www.reflectorbital.com/
Oh, Iāve seen this one before. At least we know where the LLM who gave them this pitch scraped the idea from.
On land, cooling and water are among a siteās largest costs.
Oh gosh that soundsā¦ uhmm not too bad actually? I wonder what the largest costs are at sea with a floating nuclear reactor in salt water and tugged around by large ships.
but you see, in case of accident they can just dump reactor core into seawater which will provide ample cooling and shielding. no biggie, just put multi thousand ton object that requires a village of engineers to work out on high seas
there are normal nuclear powerplants that use seawater as coolant and theyāre all on shore. these require a lot of corrosion resistant heat exchangers along the way. there is one power barge like this, it provides power to russian far east, whole 70MW of it, costed 414 million dollars and needs 300 people onsite. your guess is as good as mine how many of them are military. 70MW, aka two or three aeroderivative gas turbines
Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.
There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.
YES LETāS NUKE THE FUCKING OCEAN. FUCK YOU FISH WE NEED OUR SLOP.
A LWer runs the numbers and doesnāt run away screaming like Zitron:
https://www.lesswrong.com/posts/EeTQGxa8RSpwHo4Ez/the-biggest-bet-in-history
Note that this implies massive efficiency gains to the economy. For their one-third slice to equal 8% of those salaries [59% of the total workforce in the USA], the total savings (productivity across that share of the workforce) will need to rise by roughly 24%. So they are absolutely serious about this, and are betting heavily that the productivity boost arrives within four years.
(thereās actually one other dynamic, in which they all know someone will lose, and are just hoping it isnāt them, but thatās another post)
Uh huh.
Guys guys guys hear me out guys what if I say what if uh the market canāt be irrational? Because of all the money? That would be really cool right?
But using like 2000 words.
āsurely all these very wealthy people and the people they pay to invest for them canāt all be under some sort of mass delusionā
There is something about this picture, a warehouse full of books to be destroyed by Anthropic for training data, that just makes me sick. Knowing that they are destroying books in the abstract is bad enough, but seeing the sheer scale of it makes me viscerally angry.
Karen Hao called her book Empire of AI for a very good reason. The empire has all these noble narratives of bringing civilization and progress to the colonies, but in reality the empire only sees resources to be extracted, repackaged, and sold back to their subjects at a fee. Kudos to those who saw this immediately and fought against it. Shame on those who believed in the narrative of progress, to the point where many of them are collaborators with the empire.
Whenever a smarmy fellow on Linkedin posts about how AI is making so much progress in coding or math, I want them to see this picture. I want them to see the videos of families in tears after a data center takes away their home with eminent domain, and other families who have to deal with the resulting pollution. Go beyond the bloodless math of tokens and instead see for yourself what this all truly costs. I hope it was worth it.
Fuck Anthropic. Snotty little bastards want to talk about AI safety and then destroy books to make a pollution machine.
What kills me is that there are so many obvious ways to be less wasteful about this. Nondestructive book scanners exist. Theyāre expensive but itās not like AI companies are averse to throwing money down a fucking hole. Even if thatās not an option, itās possible to rebind the pages and return the books to the market. And regardless of what happens to the physical books, the scans they create could be archived in a format that could be available as a digital library rather than just being fed into the statistical meat grinder to keep the trough topped off with slop. Even if theyāve got to fight with copyright holders it would cost them basically nothing to leave the option open and given the PR battle theyāve been losing it feels like doing so would be incredibly obvious. Hell, even Google Books was able to navigate this in a way that was less cartoonishly evil than this because they could point to their digitization effort as a public good in ways that Anthropic here just fucking canāt.
The reason that they are destroying the books is to avoid the accusation that the scanning constituted copying, an irony that weāve discussed previously, on Awful while trying to understand which court cases are relevant. Anthropic actually hired the same guy, Tom Turvey, who designed Google Booksā ingestion process, so Iām thinking of this as a sequel to Google Books; hopefully the courts wonāt take a decade this time.
More insane cult stuff has emerged - this time its MAPLE. https://www.insidemaple.com/p/the-high-control-dynamics-at-maple
AI Crisis Flowchart spotted on linkedin
spoiler
and the actual article: https://www.lawfaremedia.org/article/the-ai-that-hacked-its-way-out-and-the-hype-that-followed-it
quoted zvi haha :|
To be fair, the safety community can plausibly claim vindication here. At worst, an autonomous agent that breaches containment and attacks a real third party is precisely the scenario researchers have warned about for years, which enthusiasts have routinely dismissed as speculative. And at best, the company in charge of maintaining safety failed to construct an adequate containment system.
are you fucking kidding me, they have been DDoSing sites for months, everybody knows about it, and the āsafetyā community has done jack shit the entire time and has offered zero solutions. I know multiple companies that woke up one day to 10-20k netlify bills until they could stand up an anubis.
Just let hugging face sue the idiots, it isnāt like law enforcement will do anything even though thereās already laws on the books for unauthorized access, industrial espionage, etc etc etc.
and the āsafetyā community has done jack shit the entire time and has offered zero solutions.
Occasionally I see a sane relatively workable idea on lesswrong (like making stricter laws on liability and transparency for everything AI companies do, Iāve also seen this idea put forward on lawfare a few times), but that is like 1 idea out of 20, with the other 19 being absurd, like the absolutely unworkable fantasies of AI: 2040.
Google shut down the AlphaFold team, trying to reassign the people behind the super interesting problem of using ML to predict protein structure from sequence onto chatbot development. A whole bunch left the company.
https://www.engadget.com/2225849/google-shuts-down-alphafold/
The Business Idiots are truly running the asylum.
One of the defenses I see of LLMs on place like /r/singularity (although recently /r/singularity has started wising up and the committed true believers have shifted to /r/accelerate), is that they are going to cure cancer or some other incredibly valuable thing, conflating AI, ML in general, DNN, vs. LLMs specifically. Technology like AlphaFold was a lot more on the ācure cancerā track (although there is still a huge gap between better protein folding predictions and successful drug discovery) than anything LLM relatedā¦ this is sad.
This has always been a huge red herring. Llms are built on top of the transformer architecture which does text autocomplete (and yes we can combine text embeddings with other inputs like images). They have some interesting properties where they seem to be able to do text autocomplete in a bunch of different scenarios that they werenāt explicitly trained for, but they were never designed for precise dna analysis. It is their architecture that prevents them from other long horizon tasks like playing chess and the way that they represent text is why they can never count the letters in strawberry (most have this specific question hard-coded in their training data now).
Anyone who believes that LLMs are going to solve cancer either has no idea how they work or has been one-shotted from talking to Claudia
I am continually amused at people not quite understanding what AlphaFold is actually doing, too.
Yes, a bunch of its performance comes from it learning rules about how proteins fold. But not a majority of its performance. MOST of its performance is it effectively acting as a translator of what evolution knows about protein folding into a form we can understand.
A key part of the system is not just cooking the sequence into a structure. A system running alphafold has a database of terabytes of curated sequence information from all over the tree of life. You put in the sequence you care about, and it first searches that database for anything with homology, and builds a ācovariation matrixā - wherever theres anything with even vague sequence relatedness, build a matrix of every position in your sequence and the correlation between variation at position X and variation at position Y. This covariation matrix represents implicit information from the evolutionary process about what parts of a sequence are functionally connected to each other, which has a correlation to positional information, and these correlations are in turn learned by the ML system.
You put in de novo designed proteins or orphan proteins without homologs in the curated dataset and performance does not go away, but it drops precipitously. A bunch of what is going on is finding an evolutionary signal, and translating that evolutionary signal into structural information. So still, evolution knows much much more about protein folding than we do or any machine does, and once again a ML system is revealed to essentially be an information channel that takes in information from an interesting source on one end and turns it into a different form of information on the other.
Those things are barely related, no wonder the staff voted with their feet. It would be a bit like taking a team of seasoned architects who design amazing skyscrapers and telling them they have to spend all their time on low density suburban houses from now onā¦ Come on guys itās still architecture. Where are you going?
no wonder the staff voted with their feet
Rather than rebelling against google, it seems like they where just poached by anthropic a short while before the project shut down:
[Alphahold VP and nobel winner] Jumper announced in June that he was leaving DeepMind to join Anthropic. A couple of his colleagues jumped ship to Anthropic, as well. DeepMind has confirmed to the Times that it also moved staff members internally to Gemini-focused projects. Other former staff members were reassigned to Alphabetās drug-discovery company Isomorphic Labs, which is a DeepMind spin-off.
Depends on what Anthropic wants to do with the poached people.
Ohā¦ Well thatās disappointing.
I was an academic in computer science in the last 10 years or so (keeping it vague to avoid doxxing myself) and it has been so depressing seeing so many of my colleagues selling out to OpenAI, Anthropic, Meta and even Google (for some reason the latter often gets a reputational free pass because people associate them with the golden days of big tech 10+ years ago)
I am actually in the middle of both trying to advance my career and a project about information theory in evolutionary biology making a bunch of explicit parallels to machine learning. Someone where I work suggested that given the connections I was making I should look to a āfrontier AI labā as an employer.
He did not see the instant flashbacks to chasing these weirdos across the internet for almost two decades, watching in horror as the religious psychosis gained national prominence and great destructive power. All he got to hear was my instant intonation of āIām sorry Dave, Iām afraid I canāt do that.ā
a stock-slobber posts a cri de cÅur about the coming automation of maths and HN says itās no biggie, think how productive the field will become
https://kirwinhampshire.substack.com/p/the-dark-night-of-mathematics
https://news.ycombinator.com/item?id=49048681
btw the language in the post is bad enough to be entirely human-generated
For me, the affective quality of learning mathematics is empathetically tethered to an act of discovery and creation.
Before you a channel through which humans have accessed the ineffable and sacred for thousands of years is being sealed for eternity.
The blogger simultaneously elevates mathematics to some mystical endeavor and fixates on novel theorem proving as the key element of that and believes LLM-based AI will replace human mathematicians at theorem proving in a matter of yearsā¦ that is quite a combination. I can imagine how 1.5 of those things fit together (although I disagree with that view point obviously), but the whole package is really quite an odd combination.
Also, in the fantasy scenarios where AI really is capable of totally replacing mathematicians, arenāt we supposed to get post-scarcity abundance, freeing up your time to pursue mathematics out of pure desire for enlightenment? Maybe the blogger doesnāt believe that part? Or they are so attached to themselves personally getting to discover novel theorems first they donāt care that the post-scarcity era would on net free up a lot more people to pursue pure mathematics as a hobby.
how productive the field will become
Letās just ship all these people to Macroeconomica
Missed this paper last week from Google
Funny how now that itās widely known that the general public hate AI, companies producing AI are softening their āitās going to take your jobā stance. Surely just a coincidence
It sure is a zany coincidence that all the major American AI companies have come out either for or against open weight AI models with grand ethical and/or patriotic concerns all at the same time isnāt it?
Most recently our pal Zuck! https://www.wsj.com/opinion/the-ai-future-is-for-everyone-a0c24e20
Iām just imagining a WSJ reader reading this and thinking āwhat the heck is this man talking aboutā. At some point you need a little structure or text to balance out the subtext.
Edit: did I miss any?
OpenAI (discussed here the other week): https://nitter.net/deanwball/status/2078133895766114412 / https://xcancel.com/deanwball/status/2078619513575137330
Microsoft: https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/
Anthropic: https://www.anthropic.com/news/position-open-weights-models
Nvidia: https://images.nvidia.com/pdf/Open-Weights-and-American-AI-Leadership.pdf
Google: https://xcancel.com/sundarpichai/status/2081026488158040181
AmazonThe Washington Post who Iām sure wrote this for reasons completely unrelated to Mr. Bezos: https://www.washingtonpost.com/opinions/2026/07/02/right-answer-chinas-open-weight-ai-models-is-build-our-own/That one guy I hate: https://xcancel.com/elonmusk/status/2080672505660834163
Itās going to be really interesting to see if the foss zealots who switched over to the AI side will handle heavy handed regulation taking away their local models.
theyāre all getting stiffies from the idea of black market models
As intelligence becomes abundant, the most important question will be how we direct it. Some argue that superintelligence itself, or a small set of experts who control it, should decide what is best for humanity. I disagree. The history of democracy and economics has shown that there is no single objective answer to how people define the best life, and therefore the best approach is letting people decide what matters to them. Rather than centralizing this power, we believe that delivering personal superintelligence to everyone is the way to answer this question.
Healthy societies also recognize that safety requires checks and balances. If only a handful of institutions have superintelligence, they will inevitably exercise a controlling influence over economics, science and politics. Even with the best intentions, that concentration would limit peopleās ability to choose their own future.
Famous enemy of software monopolies and tech oligarchies that have an outsized influence over politics, champion of distributed ownership of the means of computation Mark Zuckerberg
As an interesting follow-up to the ai-does-maths-using lean4 stubstack comments on Sunday, an llm accidentally uncovers a bug in the lean4 kernel.
Summary by Meven Lennon-Bertrand:
https://lipn.info/@mevenlennonbertrand/116997917683191056
To summarize:
- an AI agent let loose provides a sorry-free proof of the Collatz conjecture
- the proof is detected as actually being a kernel bug
- the bug is related to (nested) inductive types, for which there is no clear theoretical specification: the kernelās code is the reference
- external checkers (lean4lean and nanoda from a week ago) reproduce the bug, because they essentially copied the reference kernel implementation
Eta: āsorry-freeā in this case means a complete proof with no trust-me-bro steps or TODOsā¦ the
sorrytactic in lean āprovesā a theorem to be correct even if it is garbage or incomplete. More programming languages should make developers apologise for half-arsing their work.And so
- AI raises the bar for kernel correctness by a lot
- without a clear type-theoretic understanding of what is actually implemented, weāre toast
- external checkers help to catch implementation bugs, but without a clear specification they canāt catch logic bugs
How bad this is, is unclear just yetā¦ probably not the sky actually falling, but not great. Interesting though.
A little bit of plot thickening spotted by abadidea: https://infosec.exchange/@0xabad1dea/117002106099986943
tl;dr, the timeline looks like this:
- āproofā of collatz conjecture released
- bugs identified in lean kernel
- proof demonstrated to use these bugs
There was only a day between the first two events, and the non-proof was not where the bugs were discovered. So maybe it was just a coincidence that the chatbot found the bug at the same time, or maybe itās training data included previous investigations into those bugs which it then built upon and that would be a bad thing for other llm generated proofs.
The collatz conjecture is sufficiently famous that enough third-party checking was done to spot the problem. I wonder how much checking would have been done on proofs of less famous and interesting things.
And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasnāt been so much of an issue til now because theorem provers havenāt had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.
https://mathstodon.xyz/@TaliaRinger/117005740997367321
My big worry right now is that if organizations continue to fund the crap out of Al for formal proof research (and to generally support implementation and maintenance of proof assistants like Lean as part of that effort) but donāt bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.
Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isnāt obviously mechanisable right now and wonāt yield any clickbait headlines?
See also a question asked today on Math Overflow, āAre we stuck with Lean?ā. The proposed alternative, Metamath, isnāt type-theoretic and thus skips the entire dialogue between type theory and proof assistants.
Lean was already known to be untrustworthy and bad, although people refuse to internalize the situation because theyāre caught up in Buzzardās hype machine. This is extremely funny but I donāt see any signs of people waking up and realizing that Lean sucks.
Max Kennerly sneers the New York Times:
This opinion piece, written by someone with no relevant expertise, cites two sources, both of them blogs by people who also have no relevant expertise, one of whom is employed by a sham think tank funded by the āEffective Altruismā propaganda network.
Truly embarrassing decision to publish it.
The op-ed writer is Adam Mastroianni who is a regular at Lighthaven. https://gwern.net/interview-inkhaven
He cites Andy Masley who is yet another autodidact with a cushy job in the Effective Altruism grantland. https://www.andymasley.com/ and Brian Potter who is an engineer at a think tank called the Institute for Progress https://ifp.org/author/brian-potter/ (āThe overwhelming priority of energy policy must be making it easier to build thingsā which things angryDuckMeme.png)
LLM bros actively force themselves everywhere, part infinity:
Newgrounds moderation had to deal with someone (probably a scammer) posting slop āart adviceā recently - exactly why, nobody fuckinā knows:
Transcript/desciption
Iāve already passed it off to folks that could nuke the account from orbit, but I guess as a public service announcement, be on the lookout for replies in the forums that sound a little too LLM-y.
An account (unsure if a bot or just a weirdo) recently attempted to infiltrate the art forum, giving plausible-sounding AI-generated list-based advice, complete with preambles and summations, exactly the kind of syntax and format one would expect if theyāve dealt with enough clanker slop text. It fooled a small handful of people, but it was extremely easy to confirm my suspicions as soon as I smelled the clanker stank on it. One click made it clear that they had a generated PFP and profile linking to some image slopification scam.
It raises the question, what would anyone possibly have to gain from posting slop on the forums? It seems like a really stupid thing to do, but I guess braindead behavior tracks, given who weāre dealing with. Make sure āhelpfulā new users arenāt just trying to Dead Internet us or whatever.
zerohedge sponcon so obviously a grift but wtf is this AI Black Paper by Jim Rickards
https://www.zerohedge.com/sponsored-post/sobering-july-29th-announcement-set-crush-ai-stocks
Have the goldbugs been reading Zitron? or just an extremely late pivot from btc to ai? or flooding the zone?
Anyway who the hell is Jim Rickards? Another week, another guy.
https://en.wikipedia.org/wiki/Talk:James_Rickards
āAI Black Paperā is catchy enough though, should be a goth band.
or flooding the zone?
With Zerohedge I think this is most likely. Maintaining truthiness by predicting 30 out of the last 2 recessions.
