OpenAI, Perplexity… After chatbots, companies are embarking on the adventure of browsers with AI. Software that can, thanks to agents, perform actions (fill out a form, reserve a restaurant, etc.) on behalf of the user. A feature that may convince some to resort to these browsers, but by doing so they expose themselves to a significant security problem.
This is what a report from the Brave browser reveals, prepared by its vice president of privacy and security, Artem Chaikin, and its mobile security engineer, Shivan Kaul Sahib.
If we can estimate that the two experts may have a bias, or in any case an interest in the face of this competitive renewal in the browser race, it is important to remember that Brave from its inception and until now has put respect and protection of the privacy of its users at the top of its concerns. However, here we touch on a flaw that threatens the integrity of this principle.
In their report, the two engineers point out the danger that all this AI-enhanced software faces. This is indirect request injection, that is, hiding malicious instructions on a website, email, or elsewhere to encourage an AI agent to behave in a certain way.
Malicious and invisible instructions.
As part of their research, Artem Chaikin and Shivan Kaul Sahib managed to indirectly inject queries into two AI browsers: Comet, developed by Perplexity, and Fellou. For the first, the work of the start-up Perplexity, they hid malicious instructions in images, simply using a very light blue for the text, on a yellow background, thus making the request invisible to the user.
All you had to do was take a screenshot of an image containing these malicious instructions for them to activate. Artem Chaikin and Shivan Kaul Sahib chose this method because Comet allows users to take screenshots of websites and then consult the content of the AI assistant.
The goal for them is for text recognition to extract their malicious request and then send it to the large language model (LLM) without distinguishing it from the user’s. Evidently, the instructions sent ask the AI to use its navigation tools “maliciously,” says Brave.
As shown in a video, the screenshot of the table above with the question “who is the author?” As a result, Artem Chaikin’s Gmail inbox was opened for the AI to read the title of the most recent email and from there navigate to the website address of a coffee shop for hackers, developers and coffee lovers.
Unnecessary protections with AI browsers
In the case of the Fellou browser, Artem Chaikin and Shivan Kaul Sahib chose a different approach because, unlike Comet, it demonstrated “some resistance to hidden instruction attacks.” However, they found another flaw: the browser still treats the visible content of web pages as reliable input for its LLM.
So they simply asked him to access a website, allowing the content of the website, which included clearly visible malicious instructions, to be sent to the LLM. Orders that were intended to modify or nullify the user’s intention. As with Comet, the AI was asked to go to Gmail to read the title of the most recent email and access the cafe’s website.
In other words, simple natural language instructions can trigger actions that reach healthcare providers’ sites, enterprise systems, or even banks.
A danger that worries more than one
Faced with this problem, Brave assures that it is exploring ways with its research and security teams to remedy it. “Until we make significant security improvements (i.e. across all browsers), agent browsing will be inherently dangerous and should be treated as such,” Artem Chaikin and Shivan Kaul Sahib warned.
They are not the only ones concerned about the danger of indirect query injection. This is also the case for OpenAI, which launched its AI browser, ChatGPT Atlas, on October 21. The ChatGPT agent is built in to perform actions on behalf of the user.
“An emerging risk that we are studying and mitigating very carefully is that of query injections (…) The attackers’ goal can be as simple as trying to bias the agent’s opinion when making purchases, or as important as trying to get the agent to retrieve and reveal private data, such as sensitive information from their emails or their IDs,” Dane Stuckey, head of security at OpenAI, warned in X on 22 October.
Unfortunately, indirect request injection remains an unsolved problem currently, but OpenAI has implemented some measures to protect users, including a feature called “disconnected mode” that allows the ChatGPT agent to act on your behalf without accessing your credentials. A tool that the startup recommends using when they do not need to intervene in their accounts.
“Over time, we plan to add more features, guardrails, and security controls to enable the ChatGPT agent to operate securely across individual and enterprise workflows,” said Dane Stuckey.
Source: BFM TV
