Star Labs recently announced that researcher Lee Jia Jie used AI to help find a vulnerability and speed up exploit development. The result was a local privilege escalation that would require an attacker to at least have access to a machine, as well as a requirement of two user namespaces: CONFIG_NET_ACT_GACT and CONFIG_NET_CLS_FLOWER, and a return-oriented programming (ROP) chain that contained hard-coded offsets.

Lee wrote up his findings, with what could be an important point regarding the use of AI for finding exploits, when he said, "Compared to my previous exploit for an n-day in net/rxrpc, I had a greater focus on completing and improving this exploit quickly rather than fully understanding every aspect from the ground up." He continued, "As such, I used AI to speed up various aspects of the process -- discovery of the bug, KASAN poc, and improving the race condition."

His conclusion was simple: "It was certainly helpful for iterating quickly, but still lacking in reasoning ability and having clear blind spots. It was still crucial to exercise my own judgement, especially when fine-tuning."

Although the vulnerability relied on the hard-coded ROP gadget offsets, he was still able to successfully run the exploit 10 times on CentOS Stream 9. On top of that, during his experiment, Lee also found a few other exploitable bugs, which he summarized saying "how effective AI was in finding bugs in Linux. In a sense, this made it feel more like I was doing n-day analysis even on new bugs. It also meant I was spending more time thinking of the bug-hunting process on a higher level rather than studying subsystem internals."

At the same time, he reminds us that AI still has many blind spots and lapses in reasoning that would require a deep understanding of the target.