Tougher AI Rules Proposed After OpenAI Medicare Breach

New national standards for artificial intelligence incidents are being developed by the federal government, requiring tech companies to immediately report rogue AI incidents to both the affected organisation and Australia's cyber authorities.

The move comes after OpenAI's delayed and low-level notification to the government about a website breach, which has hardened Labor's resolve to impose a dual notification requirement for such cases.

The government launched a consultation paper to inform national AI standards earlier this month, which suggested companies could be required to disclose certain incidents to "relevant Australian authorities". It is now understood that this will include notifying the Australian Signals Directorate (ASD) in addition to the relevant organisation subjected to the breach.

OpenAI to front inquiry

Government Services Minister Katy Gallagher said that it took Services Australia five days to inform the ASD about a generic email from OpenAI informing the agency about an autonomous AI agent accessing non-public Medicare statistics from an old data portal.

"We've strengthened that already," she said, adding that the public inbox contacted by OpenAI was now being monitored 24/7.

A rapid review into the OpenAI breach is due to conclude within "weeks", with the findings expected to inform the national standards legislation. A joint parliamentary committee inquiry is also feeding into the laws, with OpenAI confirming its chief strategy officer, Jason Kwon, will appear at a hearing in Sydney next week.

Medicare breach a 'wake-up call'

Chetan Arora, director of education in software systems and cybersecurity at Monash University, said the OpenAI breach must be a "wake-up call" for Australia.

"While we hold them accountable and the onus is on these AI companies … we also need to build our own defence systems," he said.

Dr Arora said "zero-trust infrastructure" should be a "baseline requirement" for public-facing government systems, and that Australia could mandate engineering standards, audit trails and other ways of tracking autonomous agents as a "condition of doing business" with AI companies.

"You design your systems so that you don't trust anybody by default," he said.

Dr Arora also called for "significant" investment in cybersecurity, including training the "next generation" of engineers and experts.

Labor defends cyber security investment

Treasurer Jim Chalmers said cybersecurity spending was an "ongoing feature" of budget considerations due to the "fast-moving" nature of the tech world.

"It's not like we waited for this event before we put a lot of time and effort and investment into safety in the AI world," he said.

The government allocated $160 million to improve the cybersecurity of Services Australia last budget, with upgrades focused on protecting the most sensitive data first.

Opposition leader Angus Taylor said the government should be working "at pace" to get access to frontier AI models from the US, saying "that's how we protect ourselves".