If you’re referring to genetic algorithms, those work by giving the computer some type of target to gun for that’s easy to measure and then letting the computer go loose with randomly changing bits from the original object. I guess in your mind, that’d be randomly evolving the codebase and then submitting the best version.
There’s a lot of problems with the idea of genetic codebase that I’m sure you can figure out on your own, but I’ll give you one for free: “better code” is a very hard thing for computers to measure.
That’s too bad I was hoping to be able to one day revive dead/abandoned mods using some form of ai, I was thinking of training it off of how the old mods funtioned in the last working version maybe it could find what needs to be changed
Ah I didn’t realize it was that big of an impossibility to get ai to update old minecraft mods no one else is interested in, no way I’ll ever be able to learn to code without going back to highschool
It’s alright! There’s a multibillion dollar advertising operation working to convince us that generative AI can do these sorts of things. Plus, it’s always tough to go through someone else’s work, much less mods from a decade ago that were written by ambitious amateurs. I couldn’t read my own code after a couple of months if I wasn’t such an absurd over-commenter.
This all bad news, guess I’ll never see an emulator written based on how the original game plays, but that would only matter if it was any more dmca proof than humans doing it and I know even less about that
zbyte64 gave a great answer. I visualize it like this:
Writing software that does a thing correctly within well defined time and space constraints is nothing like climbing a smooth gradient to a cozy global maximum.
On a good day, it’s like hopping on a pogo stick around a spiky, discontinuous, weirdly-connected n-dimensional manifold filled with landmines (for large values of n).
The landmines don’t just explode. Sometimes they have unpredictable comedic effects, such as ruining your weekend two months from now.
To elaborate on the other answers about alphaevolve. the LLM portion is only a component of alphaevolve, the LLM is the generator of random mutations in the evolutionary process. The LLM promoters like to emphasize the involvement of LLMs, but separate from the evolutionary algorithm guiding the process through repeated generations, LLM is as likely to write good code as a dose of radiation is likely to spontaneously mutate you to be able to breathe underwater.
And the evolutionary aspect requires a lot of compute, they don’t specify in their whitepaper how big their population is or the number of generations, but it might be hundreds or thousands of attempted solutions repeated for dozens or hundreds of generations, so that means you are running the LLM for thousands or tens of thousands of attempted solutions and testing that code against the evaluation function everytime to generate one piece of optimized code. This isn’t an approach that is remotely affordable or even feasible for software development, even if you reworked your entire software development process to something like test driven development on steroids in order to try to write enough tests to use them in the evaluation function (and you would probably get stuck on this step, because it outright isn’t possible for most practical real world software).
Alphaevolve’s successes are all very specific very well defined and constrained problems, finding specific algorithms as opposed to general software development
First, Microsoft isn’t using this yet but even if they were it doesn’t work in this context. What Google did was they wrote a fitness function to tune the Generative process. Why not have some rubric that scores the code as our fitness function? Because the function needs to be continuous for this to work well, no sudden cliffs. But also they didn’t address how this would work in a multi-objective space, this technique doesn’t let the LLM make reasonable trade offs between complexity and speed.
I forgot about alpha evolve, with all the flashy titles about I figured it wasn’t a big deal, I was more talking about the low level stuff I guess like “ai learns to play mario/walk” but I imagine it follows the same logic the other comment talks about
Is there a reason why that ai “evolution” thing don’t work for code? In theory shouldn’t it be decent at least
If you’re referring to genetic algorithms, those work by giving the computer some type of target to gun for that’s easy to measure and then letting the computer go loose with randomly changing bits from the original object. I guess in your mind, that’d be randomly evolving the codebase and then submitting the best version.
There’s a lot of problems with the idea of genetic codebase that I’m sure you can figure out on your own, but I’ll give you one for free: “better code” is a very hard thing for computers to measure.
For LLMs specifically? Code is not text, aside from the most clinical, dictionary definition of “text”.
But even then, it also fails at writing coherent short or longform, so even if code was “just text” it’d fail equally badly.
That’s too bad I was hoping to be able to one day revive dead/abandoned mods using some form of ai, I was thinking of training it off of how the old mods funtioned in the last working version maybe it could find what needs to be changed
…the fuck is this post
I think that what we’ve got here is a genuine victim of the hype.
you could be right
Ah I didn’t realize it was that big of an impossibility to get ai to update old minecraft mods no one else is interested in, no way I’ll ever be able to learn to code without going back to highschool
You can definitely learn how to code, I believe in you.
It’s alright! There’s a multibillion dollar advertising operation working to convince us that generative AI can do these sorts of things. Plus, it’s always tough to go through someone else’s work, much less mods from a decade ago that were written by ambitious amateurs. I couldn’t read my own code after a couple of months if I wasn’t such an absurd over-commenter.
If you want a chill intro to the real situation, I highly recommend this episode of On the Media that had Ed Zitron on. You could knock it out over a commute or two no problem: https://www.wnycstudios.org/podcasts/otm/articles/brooke-talks-ai-with-ed-zitron
A funnier and angrier thing that explains why this won’t work can be found here: https://ludic.mataroa.blog/blog/i-will-fucking-piledrive-you-if-you-mention-ai-again/
This all bad news, guess I’ll never see an emulator written based on how the original game plays, but that would only matter if it was any more dmca proof than humans doing it and I know even less about that
your posts keep just slinging words together and it’s just fucking weird
once again: the fuck is this post
I don’t understand
negative reactions? to your own shitty posts?
well fuck damn, I wonder what’s confusing
zbyte64 gave a great answer. I visualize it like this:
Writing software that does a thing correctly within well defined time and space constraints is nothing like climbing a smooth gradient to a cozy global maximum.
On a good day, it’s like hopping on a pogo stick around a spiky, discontinuous, weirdly-connected n-dimensional manifold filled with landmines (for large values of n).
The landmines don’t just explode. Sometimes they have unpredictable comedic effects, such as ruining your weekend two months from now.
Evolution is simply the wrong tool for the job.
To elaborate on the other answers about alphaevolve. the LLM portion is only a component of alphaevolve, the LLM is the generator of random mutations in the evolutionary process. The LLM promoters like to emphasize the involvement of LLMs, but separate from the evolutionary algorithm guiding the process through repeated generations, LLM is as likely to write good code as a dose of radiation is likely to spontaneously mutate you to be able to breathe underwater.
And the evolutionary aspect requires a lot of compute, they don’t specify in their whitepaper how big their population is or the number of generations, but it might be hundreds or thousands of attempted solutions repeated for dozens or hundreds of generations, so that means you are running the LLM for thousands or tens of thousands of attempted solutions and testing that code against the evaluation function everytime to generate one piece of optimized code. This isn’t an approach that is remotely affordable or even feasible for software development, even if you reworked your entire software development process to something like test driven development on steroids in order to try to write enough tests to use them in the evaluation function (and you would probably get stuck on this step, because it outright isn’t possible for most practical real world software).
Alphaevolve’s successes are all very specific very well defined and constrained problems, finding specific algorithms as opposed to general software development
Imagine if it was trying to build a feature for a client who needs to look at a demo every time.
Talking about Alpha Evolve https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ ?
First, Microsoft isn’t using this yet but even if they were it doesn’t work in this context. What Google did was they wrote a fitness function to tune the Generative process. Why not have some rubric that scores the code as our fitness function? Because the function needs to be continuous for this to work well, no sudden cliffs. But also they didn’t address how this would work in a multi-objective space, this technique doesn’t let the LLM make reasonable trade offs between complexity and speed.
I forgot about alpha evolve, with all the flashy titles about I figured it wasn’t a big deal, I was more talking about the low level stuff I guess like “ai learns to play mario/walk” but I imagine it follows the same logic the other comment talks about