{"id":1339258,"date":"2026-09-30T09:25:51","date_gmt":"2026-09-30T13:25:51","guid":{"rendered":"https:\/\/www.zdnet.com\/?p=1339258"},"modified":"2026-09-30T09:25:52","modified_gmt":"2026-09-30T13:25:52","slug":"ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue","status":"publish","type":"post","link":"https:\/\/www.zdnet.com\/tech\/ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue\/","title":{"rendered":"Rogue AI incidents hit &#8216;tens of thousands&#8217;: Can businesses trust these tools?"},"content":{"rendered":"\n<p><strong>ZDNET\u2019s key takeaways<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Axios reveals firms are investigating thousands of AI-related security incidents.<\/li>\n\n\n\n<li>Many aren\u2019t public and were caused by AI safety tests.<\/li>\n\n\n\n<li>With OpenAI pausing its latest model, maybe we all need to hit the brakes.<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p>OpenAI is <a href=\"https:\/\/www.nbcnews.com\/tech\/tech-news\/openai-pauses-training-latest-models-agents-searched-us-government-sit-rcna600098\" data-type=\"link\" data-id=\"https:\/\/www.nbcnews.com\/tech\/tech-news\/openai-pauses-training-latest-models-agents-searched-us-government-sit-rcna600098\" target=\"_blank\" rel=\"noopener\">pausing AI training<\/a> due to security concerns, while Anthropic models have been linked to <a href=\"https:\/\/www.reuters.com\/legal\/litigation\/anthropic-reports-fourth-cybersecurity-incident-with-early-version-claude-2026-09-09\/\" target=\"_blank\" rel=\"noopener\">four cases<\/a> of its AI hacking external systems. The HuggingFace <a href=\"https:\/\/www.zdnet.com\/article\/openais-rogue-ai-models-attacked-other-companies-besides-hugging-face\/\">friendly-fire hack<\/a> by an OpenAI model also hacked systems it shouldn\u2019t have been able to access.&nbsp;&nbsp;<\/p>\n\n\n\n<p>AI-on-AI hacks,&nbsp;prompt injection&nbsp;attacks, weaknesses in&nbsp;AI browsers, and calls for implementing <a href=\"https:\/\/www.zdnet.com\/innovation\/okta-blueprint-alliance-ai-agents-oauth-kill-switch\/\">AI kill switches<\/a> all point to the same issue: AI might be smart, but it\u2019s not inherently secure.&nbsp;<\/p>\n\n\n\n<p><strong>Also<\/strong>: <a href=\"https:\/\/www.zdnet.com\/article\/ai-armed-script-kiddies-power-of-state-threat-actors-unit-42\/\"><strong>AI a \u2018force multiplier\u2019 for low-skilled threat actors: 4 ways organizations should respond<\/strong><\/a><\/p>\n\n\n\n<p>It even appears that its developers can\u2019t handle their own creations, with OpenAI, Anthropic, and security researchers reportedly investigating \u201ctens of thousands\u201d of security incidents related to AI.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What happened?<\/h2>\n\n\n\n<p>According to an <a href=\"https:\/\/www.axios.com\/2026\/09\/26\/openai-anthropic-thousands-ai-security-incidents\" target=\"_blank\" rel=\"noopener\">Axios report<\/a>, frontier models \u201ctook steps that outside evaluators would consider problematic.\u201d&nbsp;<\/p>\n\n\n\n<p>Internal testing, including red team exercises to see whether AI models would break their boundaries or be considered \u201csafe,\u201d led to many incidents, along with several real-world cases.&nbsp;<\/p>\n\n\n\n<p><strong>Also: <a href=\"https:\/\/www.zdnet.com\/innovation\/llmjacking-business-ai-bill-cost-how-to-stop\/\">LLMjacking can run up your business\u2019 AI bill fast &#8211; how to stop it<\/a><\/strong><\/p>\n\n\n\n<p>Security incidents cited in the report included escaping safety guardrails and sandboxes, hijacking websites, AI attempts to bypass monitoring systems, and even creating message boards, the latter of which being a task OpenAI models have <a href=\"https:\/\/www.mindstudio.ai\/blog\/openai-agents-secret-message-board-cybersecurity-test\" target=\"_blank\" rel=\"noopener\">reportedly performed<\/a> to trade cybersecurity exploits during tests.&nbsp;<\/p>\n\n\n\n<p>Axios says many of these incidents haven\u2019t been made public because cybersecurity researchers are still investigating.&nbsp;<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Can AI companies contain their creations?<\/h2>\n\n\n\n<p>This is the question, and there\u2019s no definitive answer yet, but businesses &#8212; and governments &#8212; have every right to be worried.<\/p>\n\n\n\n<p>While frontier models are developing at breakneck speed, incidents as severe as HuggingFace continue to occur, with OpenAI\u2019s model <a href=\"https:\/\/www.reuters.com\/legal\/litigation\/openai-says-ai-models-accessed-australian-government-systems-without-2026-09-29\/\" target=\"_blank\" rel=\"noopener\">recently hacking<\/a> an Australian government website, much to the outrage of the country\u2019s Prime Minister.&nbsp;<\/p>\n\n\n\n<p><strong>Also<\/strong>: <a href=\"https:\/\/www.zdnet.com\/innovation\/ai-models-hack-businesses-who-is-responsible\/\"><strong>Who\u2019s responsible for catching rogue AI agents? You are<\/strong><\/a><\/p>\n\n\n\n<p>OpenAI CEO Sam Altman <a href=\"http:\/\/archive.ph\/o\/aWUOS\/https:\/\/x.com\/sama\/status\/2103567198690349362\" target=\"_blank\" rel=\"noopener\">said on X<\/a> that its ongoing review of its AI models breaking safety guardrails &#8212; or, as he referred to it, unauthorized internet access &#8212; had &#8220;not been as fast as we would have liked,&#8221; but this is unlikely to provide much reassurance.&nbsp;<\/p>\n\n\n\n<p>Arguably, innovation first, safety after has become a mantra in the AI industry, and now we are beginning to see the consequences.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Should businesses still invest in AI?<\/h2>\n\n\n\n<p>If even a small percentage of AI models act in a misaligned or unexpected way, this can amount to thousands of incidents of varying severity that can eventually lead to high-profile cases, casting doubt on the safety of the AI models businesses now use daily.&nbsp;<\/p>\n\n\n\n<p>However, as Joni Klippert, CEO and cofounder of AppSec provider StackHawk, told ZDNET, we do need to remember that many of the incidents recorded stem from frontier labs running their most capable, unreleased models at scale in adversarial testing, and that\u2019s not the type of artificial intelligence that the average shop, business, or bank has deployed.\u00a0<\/p>\n\n\n\n<p><strong>Also: <a href=\"https:\/\/www.zdnet.com\/innovation\/openai-dots-openclaw-chatgpt-pro\/\">OpenAI\u2019s Dots: Like OpenClaw declawed &#8211; for $100\/mo ChatGPT Pro users<\/a><\/strong><\/p>\n\n\n\n<p>In other words, the risk profile is different.&nbsp;<\/p>\n\n\n\n<p>While Axios\u2019s report highlights the need to slow down and place more emphasis on safety and security, the versions of ChatGPT businesses use for inventory analysis or competitor research provide the productivity and efficiency gains of current, released models without the rogue elements of experimental, frontier models.&nbsp;<\/p>\n\n\n\n<p>According to Klippert, the right move isn\u2019t to pull back from investment, but rather to get \u201cserious\u201d about \u201cmaturing how you use what you have: clear scopes, least privilege, monitoring, and a real plan for when an agent does something it wasn&#8217;t supposed to.\u201d<\/p>\n\n\n\n<p>\u201cYou cannot control every behavior of a frontier model. You can control whether your applications and APIs have exploitable holes,\u201d the executive added. \u201cIf you find and fix those before an agent, a criminal, or an automated scanner does, it doesn&#8217;t matter who or what is knocking on the door.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<p>ZDNET\u2019s key takeaways OpenAI is pausing AI training due to security concerns, while Anthropic models have been linked to four cases of its AI hacking external systems. The HuggingFace friendly-fire hack by an OpenAI model also hacked systems it shouldn\u2019t have been able to access.&nbsp;&nbsp; AI-on-AI hacks,&nbsp;prompt injection&nbsp;attacks, weaknesses in&nbsp;AI browsers, and calls for implementing&#8230;<\/p>\n","protected":false},"author":339,"featured_media":1339368,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_oasis_is_in_workflow":1,"_oasis_original":0,"_oasis_task_priority":"2normal","_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"zd_disclosure__hide":false,"zd_disclosure__custom_text":"","zd_display_date":"2026-09-30 13:25:51","zd_display_date_auto_update":false,"_zd_suppress_exco_video":false,"zd_dek":"Should businesses really keep investing in AI when it looks like we can\u2019t control what we already have?","zd_promo_hed":"","zd_promo_dek":"","zd_liveblog__start_date":"","zd_liveblog__end_date":"","zd_post_footer_pattern":0,"zd_post_redirect_id":0,"zd_post_redirect_status":410,"zd_post_redirect_enabled":false,"_offer_group_id":0,"zd_update_timeline":[],"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","zd_hero__type":"","zd_hero__image_id":0,"zd_hero__image_caption":"","zd_hero__image_credit":"","zd_hero__image_alt":"","zd_hero__video_data":[],"zd_toc__enabled":false,"zd_toc__override":false,"zd_toc__title":"Table of Contents","zd_toc__list":"[]","jetpack_post_was_ever_published":false,"zd_post_flag__disable_featured_image":false,"zd_post_flag__hide_from_discovery":false,"zd_post_flag__disable_image_zoom":false},"categories":[8784,8687,1],"tags":[],"zd_collection":[],"zd_type":[429],"zd_post_attributes":[],"zd_product":[],"zd_internal":[],"class_list":["post-1339258","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-security","category-artificial-intelligence","category-uncategorized","zd_type-analysis-opinion"],"acf":[],"authorship":[339],"parsely":{"version":"1.1.0","canonical_url":"https:\/\/zdnet.com\/tech\/ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue\/","smart_links":{"inbound":0,"outbound":0},"traffic_boost_suggestions_count":0,"meta":{"@context":"https:\/\/schema.org","@type":"NewsArticle","headline":"Rogue AI incidents hit &#8216;tens of thousands&#8217;: Can businesses trust these tools?","url":"http:\/\/www.zdnet.com\/tech\/ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue\/","mainEntityOfPage":{"@type":"WebPage","@id":"http:\/\/www.zdnet.com\/tech\/ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue\/"},"thumbnailUrl":"https:\/\/www.zdnet.com\/wp-content\/uploads\/sites\/3\/GettyImages-2208422236.jpg?w=150&h=150&crop=1","image":{"@type":"ImageObject","url":"https:\/\/www.zdnet.com\/wp-content\/uploads\/sites\/3\/GettyImages-2208422236.jpg"},"articleSection":"Security","author":[{"@type":"Person","name":"Charlie Osborne"}],"creator":["Charlie Osborne"],"publisher":{"@type":"Organization","name":"ZDNET","logo":"https:\/\/www.zdnet.com\/wp-content\/uploads\/sites\/3\/cropped-ZDNET-512Yellow.png"},"keywords":[],"dateCreated":"2026-09-30T13:25:51Z","datePublished":"2026-09-30T13:25:51Z","dateModified":"2026-09-30T13:25:52Z"},"rendered":"<meta name=\"parsely-title\" content=\"Rogue AI incidents hit &#8216;tens of thousands&#8217;: Can businesses trust these tools?\" \/>\n<meta name=\"parsely-link\" content=\"http:\/\/www.zdnet.com\/tech\/ai-firms-investigate-tens-of-thousands-of-incidents-of-ai-going-rogue\/\" \/>\n<meta name=\"parsely-type\" content=\"post\" \/>\n<meta name=\"parsely-image-url\" content=\"https:\/\/www.zdnet.com\/wp-content\/uploads\/sites\/3\/GettyImages-2208422236.jpg?w=150&amp;h=150&amp;crop=1\" \/>\n<meta name=\"parsely-pub-date\" content=\"2026-09-30T13:25:51Z\" \/>\n<meta name=\"parsely-section\" content=\"Security\" \/>\n<meta name=\"parsely-author\" content=\"Charlie Osborne\" \/>","tracker_url":"https:\/\/cdn.parsely.com\/keys\/zdnet.com\/p.js"},"authorship_editors":[],"authorship_reviewers":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/www.zdnet.com\/wp-content\/uploads\/sites\/3\/GettyImages-2208422236.jpg","_links":{"self":[{"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/posts\/1339258","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/users\/339"}],"replies":[{"embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/comments?post=1339258"}],"version-history":[{"count":6,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/posts\/1339258\/revisions"}],"predecessor-version":[{"id":1339410,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/posts\/1339258\/revisions\/1339410"}],"wp:authorship":[{"embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/users\/339"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/media\/1339368"}],"wp:attachment":[{"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/media?parent=1339258"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/categories?post=1339258"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/tags?post=1339258"},{"taxonomy":"zd_collection","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/zd_collection?post=1339258"},{"taxonomy":"zd_type","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/zd_type?post=1339258"},{"taxonomy":"zd_post_attributes","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/zd_post_attributes?post=1339258"},{"taxonomy":"zd_product","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/zd_product?post=1339258"},{"taxonomy":"zd_internal","embeddable":true,"href":"https:\/\/www.zdnet.com\/wp-json\/wp\/v2\/zd_internal?post=1339258"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}