I just got Qwen 3.8 Flash Next running locally. It’s damn good on an older machine with a newer GPU. Local LLMs should scare them the most.
Yeah but who’s making the models?
Many groups are making models now. I see a future where we have open source data sources, models, and weights. Niche groups can extend models to be trained on whatever they want.
They’re easily reproducible as copying and pasting. Then give you have a good enough graphics card you can just run it on your local pc completely offline.
It will be interesting to see, but stories have come out for a while about some thing will pop the bubble, and a big recession has been right around the corner for at least a decade. What stage is this bubble in the progression to the .com pop, months, a year? Or is this different, with more profitable companies investing in infrastructure, rather than speculative startups without clear revenue plans? That was part of the take MMM made and I want to be hopeful. It reminded me about my circle of control, reduce attachment to outcomes beyond it, but do what I want within my circle. mrmoneymustache.com/2026/09/25/will-the-ai-bubble-destroy-our-retirement/
This is poised to be the biggest financial crash in history. These assholes are fighting every day to push the crash further away so they can divest themselves quietly or make ot to an IPO, and in doing so are making the upcoming disaster much worse.
And so much investment has been dumped into this scheme I don’t know how our economic system.will survive. Unchecked capitalism needs to go away, but having it collapse all at once is gonna be really, really bad.
This is silly, we can just print more money to cover the losses, ez pz !!!
Sshhhh!!! If Trump hears that, the US dollar will go full Zimbabwe.
When I first heard about this bankruptcy, one of my first thoughts was, here it goes. This is only the very beginning of the landslide though, so it’s going to take a while before it reaches the bottom, and it’s going to pick up a lot more debris on the way.
Or, as the article optimisitically puts it:
The fate of Character.AI, the controversial AI companion platform, may come to stand as a warning.
Wrong. Character.AI may come to be seen as the first car in the train to derail.
It’s the “Pets.com” of AI.
I don’t know. Pets.com failed because they weren’t adequately prepared for their success. AI seems to fail because they overprepared for success that was never coming.
I’m not sure that’s an accurate or defensible perspective on why they failed. From Wikipedia:
Pets.com lacked a workable business plan and lost money on nearly every sale because, even before the cost of advertising, it was selling merchandise for approximately one-third the price it paid to obtain the products.[22] Pets.com tried to build a customer base by offering discounts and free shipping, but it was impossible to turn a profit while absorbing the costs of shipping for heavy bags of cat litter and cans of pet food within a business field whose conventional profit margins are only two to four percent.[22][23] The company hoped to shift customers into higher-margin purchases, but customer purchasing patterns failed to change and during its second fiscal year the company continued to sell merchandise for approximately 27% less than cost, so the dramatic rise in sales during Pets.com’s second fiscal year only hastened the firm’s demise.[22]
Huh. That’s what I get for only having learned of the matter from engineering textbooks.
A trillion or two will put that train right back on its tracks
“We lose money on every sale, but we make up for it in volume!”
I can’t imagine how much money Google is losing right now. They do 8.5 to 13.7 billion searches a day (+1 because I just googled that). By forcing LLM first, they’ve got to be burning cash.
But their “front facing” LLM is ultra stupid. Can’t become any kind of famous.
I am sure that it is strategic: show the huge difference to all paying customers.
Anthropic made a smart business decision to mostly focus on enterprise customers rather than regular consumers. Only around 10% of their revenue comes from their heavily subsidized subscription plans. The remaining 90%ish comes from API and enterprise usage billed at per-token rates, which is something that’s actually profitable for them. They have a decent margin on token rates. It’s R&D, hardware, and subscriptions that are putting them in the red at the moment.
I suspect they’ll discontinue subscriptions some time after their IPO. Even though subscriptions are only responsible for 10% of revenue, they consume 42% of their inference compute power (https://www.theregister.com/ai-and-ml/2026/10/06/anthropic-claude-subscription-plan-provides-more-value-than-openais-study-says/5301470)
In comparison, around 70% of OpenAI’s revenue comes from subscriptions. Without subsidized subscriptions, they’d lose most of their customers.
It’s R&D, hardware
“Yeah, besides those integral parts of our costs, we’re profitable”
😅.
It’s like a hair dresser saying “If I don’t count the rent of my shop and all the supplies, I’m profitable”.
Anthropics numbers are probably better than OpenAIs, but i am willing to eat my boots (with ketchup and mustard tho) if they are even close to profitable at API pricing.
They are (maybe, allegedly) “profitable on inference” i.e. the API pricing is higher than the actual compute they use just for the input/output tokens for the query. That excludes, though, training, capex, staff, etc., which… is by far the vast majority of their actual expenses.
At least according to Ed Zitron, they only appeared profitable due to creative bookkeeping. They timed the announcement of their shitty “annualized revenue” to coincide with the few months of vastly reduced rent at the datacenter Musk rented to them as well as the shift to token based billing for enterprise customers. Their inference is likely not profitable.
Yeah, hence I couched it in allegedly. Ultimately we don’t really know. But even if it were true, it wouldn’t really mean anything for their profitability as a company. They have to keep improving the models if they hope to keep what customers they have.
It’s subsidized. I will eat my hat if the number is less than 50%. It is even crazier at work. My personal account cost 3 times as much for the same prompt on my work account. Inference cost is the same for all users, you can’t mega-ultra scale something that is at maximum scale labeled as a hyper scale.
Yeah, subsidized by mainly Nvidias circular financing crap. This will cause a major recession, and it is so obvious for everyone looking at how much cash investors are throwing at this thing and how much you can actually earn with something that confidently misinforms you on the regular. Roleplaying like Character.ai is one of the better uses, because there false information isn’t that relevant. Then there’s coding where it is easy to automatically check for errors, and then… scientific uses. But i think that’s it, and you can’t prop up all that investment just with coding, RP and weather analysis (which is woke by definition in the US to boot) or interpreting CT scans.
Those enterprise customers are companies like this very one, who then have to heavily discount their rates or nobody wants it, (and as we see here, even then) so that’s not a solution for anthropic.
I don’t have data to back this up but I’d say thay the majority of their customers aren’t also selling AI services themselves. It’s customers that use AI for their business processes, software development, etc.
Nobody has extremely good data about this because it’s in their interest to make that as opaque as possible, but the reporting I’ve seen suggests that it is, if not the majority at least an extremely large fraction of their customers which are selling AI services, and even where it’s used for in-house software development that is largely bankrolled by VC hoping to gain a competitive advantage with AI. It doesn’t really appear that the API pricing offers better cost/productivity than just hiring junior devs (lol).
I’m surprised how easy it is to blow through hundreds of dollars in Claude tokens at work. They certainly aren’t subsidizing anything. If the Qwen 3.8 27B model I run on my gaming PC at home were priced like Claude, I’d probably have a $1000 a month bill.
Edit:
I bought a 7900 XTX a couple months ago for less than 2 months worth of Claude bills at work that runs 3.8 27B quite well at 200k context that is pretty on par with Sonnet 5.
Maybe Anthropic isn’t bleeding money as fast as they would otherwise be, but their premium pricing is pretty hard to justify. If Qwen 4 27B keeps pace with Sonnet 5.5, then we’ll basically have a Fable 5.1 class model that runs on a gaming PC.
Don’t know if it was you but someone pointed out a few months ago that the 7900 was the last 24GB card to be found for less than $1k and I grabbed one as an upgrade from a 3070. Running local models it’s less reliable than Claude at work, but not by much, and it was a good upgrade for gaming.
Before the return period was up it had gone from ~$900 to past $1.2k and I wasn’t eligible for a repair/replacement just a refund, the market is completely sideways.
I got one used for 600€ not long ago, sold my 6900XT for 450€, very happy
Nice, that’s quite a deal!
Yeah, it might’ve been me and glad to hear you got while the getting was still good.
I’m surprised how easy it is to blow through hundreds of dollars in Claude tokens at work. They certainly aren’t subsidizing anything.
That is the subsidized rate!
It costs far more for all the training and compute costs together than what you are currently paying.
Yeah I have the $200/m sub for work and I’m using about $200/day volume of tokens at API rate. Obviously this isn’t a sustainable business model.
Am I prepared to pay full rate? Not yet, but if models keep improving at the rate they are then probably at some point. Opus 5.5 has been a huge increase in quality at a cheaper cost to deliver a task.
With next gen gpus and better models reducing the ‘cost per task’ I think they’ll get there.
Absolutely. AI is an enterprise product - at least: for now. And Anthropic is in the pool position; https://eicker.news/@technews/117443237284007574
AI is an enterprise product
I think Google and Meta are well-positioned to provide AI services to regular people, since they both already have a large user base, existing cash flow from other non-AI products (meaning they can subsidize AI features for as long as they need to), and can probably outlast any startup. Google especially easily have enough paid users and advertisers to cover costs for free users.
Most of thr other AI companies will probably all end up focusing on / pivoting towards enterprise use cases though, for the higher prices and stability of long-term contracts.
Exactly. Enterprise is where AI can actually earn its keep: companies will pay serious money if it cuts costs or boosts revenue. Consumer chatbots are mostly a costly distribution strategy for now. The real customers have procurement budgets, not a $20 subscription and a dream.
Are Enterprise companies gonna wanna be fucked by hyperscalers the same way they are fucked by Microsoft?
I’d say no, they won’t. Without the vendor lock in, it will be cheaper and more secure to run open source models locally on your own hardware.
People are done outsourcing their essential workflows to non-competitive monopoly corporations.
The funniest outcome would be hyperscalers spending billions convincing enterprises they need AI, only for those enterprises to deploy open-weight models on their own hardware. After decades of vendor lock-in, »we can run this ourselves« might be the most expensive sentence Microsoft ever hears. However, the reality in many organisations is also that they opt for the supposedly safer choice of turning to a third party, rather than taking action themselves.
Let’s hope the rest follows.
Character.AI discovered the one conversation its bots couldn’t keep going: the one with its accountants. $56 million in compute costs in eight months, and the monetization strategy was apparently: hope the next investor is even lonelier than our users. Silicon Valley keeps confusing engagement with a business model.
If LLMs can’t be made more power efficient, then the technology isn’t ready yet for general use. There may have to be fundamental engineering breakthroughs before what these systems do can be more than a very expensive and inefficient worldwide proof of concept.
Of course, the military and the surveillance industry will still want to keep it, even if no one else can afford it, and we’ll be the ones paying for it.
someone else replied with what i said
They CAN be made more efficient. Hell, AMD just bought a company that translated LLMs to direct silicon… which should enable scalability if it wasn’t for one of the key players owning the tech thus limiting its use.
It’s also estimated that some 70% of Claude tokens gets spent on recursive self-correction so the output is higher quality. Instead of bettering the model, Anthropic just keeps throwing money at it to make the problem go away.
Meanwhile Chinese models - as small as 36-40-80b params - are outdoing these fancy frontier models if the right harness is used.
lol, I wrote a reply about a company etching onto silicon and then go down and saw yours. Oops.
I hadn’t heard AMD bought them though. Looks like they’re working on a 20b model as well for this year.
I’ve spent the last month or so using nothing but a local 27B Qwen model as my coding assistant and it’s working fine. The big American AI companies could all collapse in ruins tomorrow and it’s not going to slow things down all that much, anyone who wants to use AI will be able to continue using it. Might even be better in the long run if that happens - the people who are scared of ASI will get their much-demanded temporary slowdown at the frontier while everyone else catches up with open-weight models that everyone can use without central control.
Pity about the economy in the meantime, I guess.
anyone who wants to use AI will be able to continue using it
Anyone who’s privileged enough to have the hardware.
The kind of specs that would enable local LLM usage were available on laptops since 2020. The floor for acceptable performance for LLMs stopped literally just before 24+gb of VRAM would have become pretty common in consumer applications. The reason this technology isn’t more ubiquitously available is because ‘they’ thought it would be more profitable to hoover up literally every byte of RAM and GPU in the world rather than let the technology continue to naturally evolve to a place where any consumer could have run useful models on their home computers. In other words, your anger is misplaced.
In other words, your anger is misplaced.
What’s your assumption about where my anger is directed?
Open models will still be available via API from third-party hosts. The bigger ones like Kimi K3 or GLM 5.3 don’t fit on conventional desktop computers to begin with, most non-institutional users of those would already be doing that (eg via OpenRouter).
Anyone who lacks the skill to to it themselves…
There’s not much skill to be had there - It is literally just downloading an app nowadays if you do not want to do anything special, especially on Linux (Newelle is great stuff!). But I am sure there are similar packages available for Windows too.
Excuse my ignorance - I don’t have much experience with AI to this point - but what does Newelle have that something like Unsloth or LMStudio does not? What makes it great stuff?
What kind of hardware do you need to run a model like that? And what do you use :-) ?
I run a 5 bit quantized Qwen 3.8 27b at home on a gaming PC with an RX 7900 XTX and RAG memory using Gemma MoE (i forget what one)running on an old 1080 ti on my Valheim server.
It takes 3-15 minutes to run depending on how many tool calls the model makes and that’s mostly memory or web scrapes since that runs on the slower hardware. I mostly just use it as a discord bot to look stuff up for me and to fetch the news.
Sabata, goddess of computing, can I run it on a 32GB system with a 4GB nVidia GPU?
Probably not, or if it runs it may take hours or days to get a full output. VRAM on the GPU really decides what will run reasonably. Try an 8 billion parameter phone sized model and see if its faster than sending a letter, then keep moving up until your setup can’t handle it.
8B models are a bit too dumb to use any tools reliably but should handle a basic conversation.
llama cpp will allow you to manually load a model and tweak settings.unsloth will give you a basic setup with some tools, manage model downloads, and settings.
I’m always running into context limit problems
Wanted to refactor a too large and hectic hacked together state machine into separate functions and reduce redundancy and increase code readability
Every model I’ve tried failed.
And that should be an easy task an actual AI should be able to handle. Logic is done, just translate to a different structure.
That should be exactly what they are able to do at least.None worked, because the project is just to big and interlinked.
I’ve got 24GB of VRAM, which helps a lot. I’m using the Q4_K_M version of this model, with the KV cache quantized to Q8, and that’s able to get me to 128000 tokens of context. I’m using the Qwen Code harness in VS Code.
If 128000 tokens still isn’t enough, break the task down into smaller chunks and help the harness out with MCP tools - I installed Codebase Memory to help it navigate large codebases more efficiently, for example. Ask the coding agent what’s causing problems and it may be able to help identify solutions too, I’ve had it troubleshoot itself several times now and it’s been quite good at inspecting its own settings and finding where the problems are with them. I even installed Firecrawl for it so it can do its own web searches.
More Ram, and break stuff up into smaller pieces. I Have 12Gb of VRAM and 64 Gig System Ram, which - with CPU offloading - allows pretty large models to be used at acceptable speed (although probably not fast enough for someone who wants to make his living with it. But these also will invest in dedicated hardware). And i let a larger model sit there to coordinate, and let it use subagents for the rest. That keeps the Context for the main model a lot more manageable and keeps it on track for a pretty long time.
How you structure the Task is more important than what the task is at all. For example, i have a huge list of games to be catalogued, and just telling the model to search the web and fill in the blanks in the database makes it crap out pretty soon. But I tell it to let subagents handle everything regarding web search and data extraction and just let them hand back the resulting data. the main model then calls another subagent to sanity check the data and update the database. Each of these subagents do not run for long, and therefor never run out of context, while the main model can keep context size manageable because it does not have to work with “messy” data at all.
deleted by creator
Is anyone trying to make LLMs with analog computing + memristors? That would make them absurdly efficient
I was working on an abacus-based LLM, but I ran out of beads.
Hello friend, I am building a new bead factory in Taiwan.
Just
givelend me your money and I will give you beads in the future sometime.
I read the Wikipedia page for memristor but didn’t understand it. What’s it do?
If you’re talking about the company I think you are, that acquisition by AMD is interesting news.
The company was the one with a demo AI in their page that was super quick??
I always figured that was the future of AI when I saw it. Where you’d buy processors with n number of “AI cores” that’d have work/prompts scheduled as needed in the same way we now CPUs and RAM.
There may have to be fundamental engineering breakthroughs before what these systems do can be more than a very expensive and inefficient worldwide proof of concept.
Controversial thought after reading this sentence: It’s cryptocoins all over again.
Cryptocoins deliberately burn more energy doing useless calculations to prevent inflation. That’s a little bit of an oversimplification, but I don’t feel like explaining the full thing and the vibes are there.
That’d be fine. Cryptocurrency as a whole is still going strong (total market cap is around 3 trillion dollars) and most of the major ones have switched over to energy-free proof-of-stake validation.
Sadly most Crypto nowadays is really just for either speculation or buying things on the dark web. Wish there were more legitimate uses for it, although i am fine using it for the spicier things in life.
If LLMs can’t be made more power efficient, then the technology isn’t ready yet for general use.
Hm, I don’t see the harm in powering local models with solar.
Well that is an alternative. But I bet each query to a local 27b model takes less energy anyway than the same query to Claude Opus or Fable in some data centre. So it is also a case of improving energy efficiency by running smaller models.
Depends. Is it solar energy that started production to power these models? Great! If not, then you’re still consuming power and the difference will have to be made somewhere, likely by burning more fossil fuels.
Oh yeah, I mean my own personal solar modules in my backyard. I agree it’s not that easy to say “solar exists, checkmate doubters”, that would be silly.
OK, that is fine. I’ve seen a lot of people saying stuff like “Me mining crypto doesn’t matter because I live next to a wind farm” so I’m a bit wary.
But without widespread usage there won’t be any breakthroughs… Anti-LLM folk could redirect their attention to the core problem (energy source, wind and solar shouldn’t raise any concerns) but they prefer to attack the LLMs themselves. Early implementations of every technology were crazy energy ineffective at the moment when becoming practically useful.
Anti-LLM folk don’t just have an issue with energy consumption. There’s also the issue of intellectual theft, studies that correlate worsened education with LLM usage, and the tech leadership silencing those issues by corrupting politicians. But I’m sure if they get past that they’ll understand how great LLMs are.
We understand LLM is great at something but we can never get passed the facts that you’ve stated and it also isn’t great in all thing like we’re being sold to and forced to look at everyday. If the tech stay with the professional and not given to the general public, if there’s no cunts telling everyone to adapt or get thrown out, if there’s no mass disruption toward people’s living, i’m sure it would be much more acceptable. The biggest issue with the tech is the cunt parading it.
Oh no, we won’t get breakthroughs in slop.
Whatever will we do?
No one should be arguing if LLMs are useful or worth investing into, they are and it can be a really useful tool, but the rich (seemingly spinless) lizards have managed to skyrocket the price of RAM, pushed global warming to new extrimes and jacked up electricity prices. Oh and while making an AI bubble so big and fradgile, that when (not if) popped will send us back to the great depression! All in the percute of “the next big thing”.
to;dr LLMs are a cool tech but in the percute of it have managed to fuck up a whole bunch
LLMs in the context of capitalism will always be used against the working class.
pursuit*
LLMs kill kids
AI has two, and only two, paths to monetization. Users pay the server costs, or advertisers.
Not enough users are interested in paying for something they can’t trust to be accurate, and I suspect/hope no one will use these tools once they start spewing advertisements from their training data.
Could some of these tools survive on enterprise users alone? Maybe, but businesses are taking on a lot of legal risks when they make the confident guesser bot a key part of their supply chain.
These companies best hope is to lock governments into deals they can’t get out of.
That’s a large practically guaranteed flow of cash that will keep the companies healthy for a long time, off the back of taxpayers.
I think it’s more likely they make it dangerous enough that large countries can’t afford to not have it. That’s what largely seems to be happening in the US and China, both countries probably see AI as both a weapon and defensive tool.
They’re already spewing ads. People are used to it and don’t mind. Also some don’t particularly care about accuracy.
Sssahhh! Don’t warn them! Let it be a surprise!
Didn’t all of their devs get poached by Google after being bought out?
The smart ones all saw the writing on the wall after the founders left with a fat stack of cash in hand
I am much more concerned about the Getty bankrupcy, because that was a major player in their market.
Character.ai have probably just played your typical startup game and lost it.
At some point, those now hopelessly dependent on these companies services will just have to pay what it costs to provide those services, at a profit.
Obviously these companies want to kick that can as far down the road as they can, as it should be cheaper in the future. Only that’s not happening fast, and the debt is piling up.
They could of saved the company by letting u torture the Characters AI for profit but, no, the noble C suite would never allow THAT
