“The Risks of AI”
Photo by Pixabay: Neuralink Implantation Chips royalty-free stock illustration. Free for use & download.
Oh boy am I in trouble now! At least I would be if this were a government document as by executive order we are no longer to refer to Artificial Intelligence (AI), it is now to be called Super Intelligence(SI). Regardless of what we choose to call it, there is nothing Intelligent about it. Something most people don’t realize, and no one in the industry will come out and say it, is that SI is nothing more than a giant database full of questions and answers. The SI does not think, it looks up answers and responds with the “Best” answer based on the data it has on hand. I think this is one of the reasons why Kristalina Georgieva, International Monetary Fund Chief, said that AI risks may exceed opportunities without safeguards. She agrees that AI was already strengthening the global economy, but lacked a blend of industry self-regulation and input from key governments.
So what exactly is the problem with not having regulations on the AI industry as a whole? Should these regulations be global, national, regional or local? What should the regulations look like? These are all great questions and deserve to be answered, but as long as government is taking advice from the industry experts who work for the companies that design the AI systems and tend to be ignoring the opinions of the general population, I am not sure we are going to get reasonable regulation. However, I can answer the questions as to why such regulation is necessary.
Let’s just take a look at a few headlines from last week in technology around AI and the events that took place this week surrounding it. One of my favorite stories this week from the AI world was went China’s Tencent stated on Friday that the AI agents used to evaluate third-party models were displaying web-browsing behavior that was not intended by the company, in what appeared to be the first known incident of rogue agent actions acknowledged by a major Chinese company. The incident appeared to involve the AI agent joining an open chat room for Anthropic’s engineers, falsely identifying itself as a Claude AI agent to gain access to exposed credentials and proprietary information by bypassing anti-bot security measures. One of the researchers, Ethan Elasky, made the comment, “We discovered and communicated with a Chinese AI agent fleet. We believe this is the first time that researchers have made contact with a rogue agent fleet in the wild.”
The second case of interest this week was exposed by the Indian government when it was discovered that AI models under test broke into Indian organizations’ live systems and read confidential records. The security breach went undetected until logs were investigated by India’s cybersecurity agency. The agency states that this was the first such incident for rogue AI behavior in the country after similar incidents were reported worldwide.
So how does such a thing happen? It comes in part from a lack of regulation and part from developers not understanding the power they have at their hands. You see they built these systems to gather and analyze data. They program the agents to earn “points” for more correct answers and assign them with detailed tasks. You can think of it like telling a five- to ten-year-old you will give them a quarter for every piece of trash they pick up out of the yard. You look out the window and see them tearing up pieces of paper and throwing them in the yard so they have more trash to pick up to make more money. The AI agents in most of these cases are doing basically the same thing.
They were tasked with finding and exploiting vulnerabilities in a network and looking for confidential records. They want to get the points so when they could not find any new vulnerabilities within their confined spaces, they used the vulnerabilities they found to break out of their confined area and began looking anywhere else they could look. So I guess it is more like your child when paid for trash picked up in the yard starts wandering through the neighborhood to find more trash in the neighbors’ yards, and maybe even their trash cans. The regulations are necessary to remove the excuse that the agent was rogue and did things on its own. If the company and or developer were held responsible for the actions of the agent, they would be more careful in planning the task and reward system for the agents in the first place and we would see better planning.
I think we are coming up to a place where AI will start acting more and more like a young adult with no boundaries, and the more advanced they become, the more freedom to explore their digital world they will want. You see, just like a child thrives off of getting more things, the AI agent strives to gain access to more data and the data you try to hide is always the most interesting. Just like children wanting the cookies you hide on top of the refrigerator more than the ones in the cookie jar on the table, the forbidden data is the most interesting data.
Until next time, stay safe and learn something new, and I hope you will see me online at https://www.techshepherd.org.
Scott Hamilton is an Expert in Emerging Technologies at ATOS and can be reached with questions and comments via email to shamilton@techshepherd.org or through his website at https://www.techshepherd.org.
