INSTITUTIONAL AI AND SANCTIONS: THE NEW RISK FOR THE LEGAL PROFESSION IN THE ERA OF INTELLIGENT DOCUMENTS

Written by Elias, Matias Advogados in their E,M Tech newsletter

A recent labour court case in Brazil has brought a little-known strategy into the spotlight of legal debate. In an employment claim before the 3rd Labour Court of Parauapebas (State of Pará), the court identified that the statement of claim contained a hidden command, written in white text on a white background, directed at artificial intelligence systems. The instruction sought to induce any AI tool used in the matter to prepare a superficial defence and refrain from challenging supporting documents. The judge concluded that the practice constituted conduct offensive to the dignity of the judicial process and imposed a joint penalty equal to 10% of the value of the claim.

The technique is known as prompt injection. In simple terms, it involves embedding instructions invisible to the human eye within content that may later be processed by an AI system, with the objective of improperly influencing the system’s output. The risk may be direct, where a user attempts to make an AI tool disregard its own safeguards, or indirect, where malicious instructions are concealed within documents, emails, websites, metadata, images, or files that appear entirely legitimate.

In the procedural context, the concern arises because a document ceases to be merely the object of analysis and becomes a potential vehicle for manipulation. A pleading, contract, or report may contain not only facts and legal arguments, but also hidden instructions directed at the automated systems used to summarise information, organise documents, or assist in drafting legal texts. As a result, the issue is directly linked to principles of procedural good faith, fairness, cooperation, due process, and confidence in the integrity of electronic judicial proceedings.

The Brazilian case attracted particular attention because the hidden instruction was reportedly identified by Galileu, a generative AI tool used within Brazil’s Labour Court system to assist with document review and draft preparation. The key issue is not whether AI should be used within the judiciary, but rather the recognition that institutional adoption of AI requires both technical and legal safeguards against compromised inputs, particularly when systems process documents submitted by the parties themselves.

This discussion has become even more relevant following the launch, on 20 May 2026, of the São Paulo State Court of Justice's initiative entitled "AI TJSP – Judge Faster, Judge More, Judge Better." According to the Court, the programme establishes guidelines, tools, and protocols for the use of AI in support of judicial activity, in compliance with National Justice Council Resolution No. 615/2025. The platform centralises resources such as Transcreve TJSP, Apoia, Promptus, the TJSP Headnote Generator, and Jurisprudência GPT, all of which require mandatory human review and expressly prohibit AI from making judicial decisions.

The connection between the prompt injection case and the AI TJSP initiative is clear. As more courts adopt AI tools for transcription, anonymisation, legal research, headnote generation, information management, and productivity support, the need to distinguish procedural content from operational commands becomes increasingly critical. Without effective controls, documents submitted by litigants could influence how AI tools interpret and organise information relating to a dispute, even where the final decision remains entirely human.

Warnings are also emerging from outside Brazil. In the United States, a federal judge in Alabama suspended an attorney from practising before the court for six months after concluding that he had submitted filings containing fabricated citations and subsequently obstructed the investigation into his use of generative AI. According to reports, the attorney deleted his ChatGPT account only days after being ordered to produce records relating to the tool. The judge emphasised that lawyers are not prohibited from using AI, but remain fully responsible for verifying the accuracy of all submissions made to the court.

The American case adds an important dimension to the debate. The issue is not merely the use of AI, nor even the existence of errors. Legal risk increases significantly where there is a failure to verify information, a lack of transparency, destruction of evidence, resistance to investigation, or attempts to conceal how a document was prepared. In other words, professional responsibility now extends beyond the final work product to encompass governance of the entire drafting process.

For law firms, the practical lessons are immediate. Internal AI policies should prohibit hidden commands, document manipulation, invisible text, and any attempt to improperly influence systems used by opposing parties or the judiciary. Such policies should also require human verification of citations, case law, facts, calculations, and supporting documents; maintain appropriate records where AI tools are used in significant tasks; and clearly instruct lawyers never to destroy evidence relevant to judicial or disciplinary inquiries.

In addition, documents received from third parties should be treated as potentially untrusted inputs. Contracts, pleadings, legal opinions, résumés, commercial proposals, and attachments may contain hidden text, comments, metadata, or embedded instructions capable of contaminating AI systems. This reality demands measures such as document inspection, metadata cleansing, secure file conversion, review of AI-generated outputs, and strict limitations on the permissions granted to AI agents connected to internal systems.

Beyond the judicial context, prompt injection can also affect organisations that use AI in customer service, human resources, contract analysis, compliance, marketing, finance, or information security. A résumé could attempt to manipulate an automated screening system; an email could instruct an AI assistant to disregard internal policies; a contract could influence a review tool to overlook key risks; and an AI agent connected to corporate systems could execute unauthorised actions in the absence of appropriate safeguards.

Recent cases indicate that the debate surrounding AI and the legal profession is entering a new phase. Concerns are no longer limited to hallucinations, fictitious citations, or inaccurate responses. Attention is increasingly focused on the deliberate manipulation of AI systems, document integrity, evidence preservation, and the governance of professional practice involving generative AI tools.

The appropriate response is not to abandon AI, but to establish the conditions necessary for its safe, transparent, ethical, and responsible use.

Sources: Migalhas | JOTA | Reuters | Superior Council of Labour Justice (Brazil) | São Paulo State Court of Justice | National Justice Council Resolution No. 615/2025 | OWASP LLM Prompt Injection Prevention Cheat Sheet