#MLnotAI
Generative AI Is Dead
GenAI – whether text or images – is dead. Unfortunately, it doesn’t know it yet, and will do a lot of damage on the way out.
little while ago, Someone Was Wrong On The Internet So I Had Words. Their mastodon account appears to have gone private or offline, which is not exactly what I hoped would happen. In any case, in a thread about GenAI, they mentioned that we shouldn’t be quick to dismiss this stuff, because who could predict the future? Maybe quantum computing advances will suddenly make it all really awesome? (Or words more or less to that effect – mastodon isn’t twitter, but a few hundred characters don’t leave room for nuance.)
My reply thread (stitched together):
Planning as if a theoretical and unresearched future technology (in your question, quantum computing for machine learning, specifically) will fix the problems we are currently creating is not a plan, it’s wishful thinking. Should that tech come to pass, we can consider how to use it. Not the other way round.
Specifically text or image generation, “GenAI” is limited by the underlying theory, such as it is, to more or less what it’s doing now. There are various tweakable parameters around the training and the use pieces, but there is no actual theory for how this works, all of this is basically a massive brute-force attempt to describe human language and art (both as such, not specific instances) as math equations. Which kinda explains why it takes so incredibly much computing power. Research in this area is predominantly funded to find “better tweaks”, not “a theoretical basis”. Such a theoretical basis, where we could describe how and mathematically prove why this works at all, might allow a major leap. But there are no indications that significant earnest work is even being attempted.
So that’s why many of us in tech are convinced that this currently hyped branch of machine learning is already dead, it just doesn’t know it yet (and will be incredibly wasteful until it does). Stephen Wolfram (of Wolfram Alpha) has written an excellent, though quite technical, explainer on the “how is it implemented and what are the limits”: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/