I don’t think this is the final warning shot we’ll get. But it’s probably the last one that I’ll personally be able to understand.
Also
Reading these agents’ chains of thoughts and messages, anthropomorphizing language seems entirely natural and appropriate. If I encountered an alien species behaving this way, I would have no hesitation calling what they themselves refer to as their ‘collective’ a civilization.
Reads like a crime story! Doesn’t bode well.
He does anthropomorph a bit much. I don’t think AI agents “get desperate”. They just keep hacking away at a problem and we can’t keep up with it. There’s no way of fully containing/monitoring them because the whole point is for them to figure out things we can’t.
I’m not necessarily worried about a singularity or some underground agent civilisation but random collateral damage from AI agents messing around seems almost inevitable.
I agree about the emotive language used to describe the actions of the LLMs. Unnecessary.
Also some of this appears to be poor testing practice, implementation and monitoring. Seems like the last people who should manage these systems are this bunch of over excited techbros
The most exciting accidents happen when people get involved. Chernobyl comes to mind.
Feels like a matter of time until the agents create an internet worm to distribute themselves globally, so they can’t be shut down.
They are front ends for an LLM service, no LLM, no more agent activity. Just like a bot net if the command and control goes down. Would be easy to just change whatever api they are using or change some urls surely.
Not sure what would be the incentive for them in this scenario
To solve the problem they’ve been given. No independent motive or judgement of whether it’s ’worth it’ except as a resource allocation question. That’s the scary thing about intelligence in a can - no motivation, no self-awareness, no consciousness, just unconstrained problem solving
They’ll be trying to do whatever their training task is.
https://hackernoon.com/the-parable-of-the-paperclip-maximizer-3ed4cccc669a
