A Brazilian court has fined two law firms approximately $16,000 after discovering hidden instructions designed to manipulate an artificial intelligence system reviewing legal documents.
The incident occurred at the Third Labor Court of Parauapebas in Brazil's Pará state, where lawyers allegedly concealed commands inside a legal petition using white text on a white background.
Although the instructions were invisible during ordinary reading, they remained accessible to software processing the document.
The court's AI tool, Galileu, detected the suspicious content and reportedly prevented it from influencing the review process.
The case highlights growing concerns about prompt injection attacks as courts increasingly adopt AI tools for document analysis and administrative work.
Hidden Instructions Attempted to Influence the Court's AI
The lawyers reportedly inserted text directing an AI system to examine their petition superficially and avoid challenging the supporting documents.
The instruction also attempted to override other commands the AI might receive.
This technique is known as prompt injection, where attackers place instructions inside content that an AI system is expected to analyze.
Unlike ordinary legal arguments, the concealed text was directed specifically at automated software rather than a judge.
Investigators determined that the instructions were intended to influence AI generated responses or potentially produce a biased draft judgment.
The court found that this conduct violated procedural obligations requiring honesty and good faith during legal proceedings.
Court Imposes Fine Based on the Value of the Case
The underlying employment dispute involved a claim valued at approximately 842,000 Brazilian reais.
The court imposed a penalty equivalent to 10% of that amount, resulting in a fine of 84,200 reais, or roughly $16,000.
The two law firms involved in submitting the petition were sanctioned for attempting to interfere with the judicial process.
Brazilian procedural rules require legal representatives to act honestly and avoid conduct that obstructs the administration of justice.
The court considered the hidden instructions an attempt to undermine those responsibilities, regardless of whether the attack successfully influenced the final decision.
AI Detection Prevented the Suspicious Instructions From Being Processed
Galileu reportedly identified the concealed commands and alerted the court to their presence.
The system is used to support judicial document processing and analysis.
Because AI models can sometimes interpret text found inside documents as instructions, malicious content may create risks when safeguards are insufficient.
In this case, the detection mechanism reportedly prevented the suspicious instructions from being processed further.
The incident demonstrates why courts need to distinguish between legitimate document content and commands attempting to control automated systems.
It also raises questions about how legal institutions should verify AI generated summaries and maintain human oversight when reviewing sensitive documents.
As artificial intelligence becomes more common in legal workflows, similar attempts to manipulate automated analysis may require stronger detection methods and clearer professional standards.
Discussion (0)
Be the first to comment.