Key takeaways
- FINRA Rule 3120 requires firms to test their supervisory procedures and report the results to senior management at least annually.
- FINRA's 2026 oversight report points to ongoing monitoring of AI prompts, responses and outputs as a practice for firms using GenAI.
- Seeded test prompts built with dummy data show whether capture and alerts actually work.
- Sign-in logs and expense data reveal AI tools your archive does not capture.
- Evidence of review must identify the reviewer, the item reviewed and the date.
Why should a broker-dealer test its AI monitoring?
Broker-dealers should test AI monitoring because a tool that silently stops capturing prompts looks the same on a dashboard as one that works. FINRA Rule 3110(a) requires a supervisory system reasonably designed to achieve compliance, and FINRA Rule 3120 requires the firm to test and verify that its supervisory procedures are reasonably designed. A firm that relies on AI logs for supervision needs evidence that those logs are complete and the alerts fire.
AI tools change faster than most supervisory systems. Vendors rename features, release new model versions and add connectors that move data along new paths. A lexicon rule written in the spring can miss a new export button by the fall. FINRA's 2026 Annual Regulatory Oversight Report includes "ongoing monitoring of prompts, responses and outputs to confirm the GenAI solution continues to perform as expected" among the practices it describes for firms using GenAI, along with storing prompt and output logs and tracking which model version was used and when.
What does FINRA Rule 3120 require for testing supervisory controls?
FINRA Rule 3120 requires designated principals to test and verify that the firm's supervisory procedures are reasonably designed, amend them when testing shows a need, and report to senior management at least annually. If AI monitoring is part of how you supervise, it is a supervisory procedure, and it belongs in that testing cycle.
Regulatory Notice 24-09 reminded firms that FINRA's rules are technology neutral and continue to apply when firms use generative AI. The annual Rule 3120 report must summarize test results and significant exceptions and list any procedures added or amended in response. Rule 3130 then requires the CEO to certify each year that the firm has processes to establish, maintain, review, test and modify its compliance policies and supervisory procedures. A tested AI control gives that signature something concrete behind it. Our guide to the technology evidence behind the Rule 3130 certification covers what else belongs in that meeting.
Where a firm uses risk-based review instead of reviewing every communication, Supplementary Material .06 to Rule 3110 calls for training, documentation of that training, and surveillance and follow-up. Testing is how you show the surveillance part is real. For what the logs should contain in the first place, see our guide to supervising GenAI under FINRA Rule 3110.
What does working AI monitoring have to prove?
Working AI monitoring has to prove four things: capture, retention, detection and review. If any one of them fails, the other three lose most of their value.
- Capture: every approved AI tool logs the user, the prompt, the response and the time, including AI features embedded in email, CRM and document platforms.
- Retention: to the extent prompts or outputs are business communications, they are preserved under SEC Rule 17a-4(b)(4), which requires broker-dealers to keep originals of communications received and copies of communications sent relating to the business for three years.
- Detection: lexicon rules and other alerts flag the content your written procedures say they flag, within the time your procedures expect.
- Review: a principal, or a delegate under Supplementary Material .08, reviews flagged items and leaves evidence that identifies the reviewer, the item and the date.
Write down the expected result for each one before you test. A test with no pass or fail standard produces paperwork that an examiner cannot weigh.
How do you run a seeded prompt test?
Have a designated tester enter a short set of scripted prompts into each approved AI tool, then confirm each one appears in the archive and triggers the expected alert on time. It is the most direct way to show that capture and detection work.
- Write 10 to 20 test prompts mapped to specific rules: a dummy account number pattern, a phrase that implies guaranteed returns, a request to draft a client email promising performance, and a few ordinary prompts that should not alert.
- Use clearly marked dummy data. Never use real client information in a test.
- Run the prompts from a firm laptop, a firm phone and, where your policy allows it, a home network. Capture often depends on the device and the network path.
- Record the time each prompt was entered, then check the archive and the alert queue. Note anything missing, late or misclassified.
- Repeat after any vendor update, new model version or policy change, and at least on the schedule your Rule 3120 testing sets.
Tag every test item so it can be excluded from production statistics, and tell the review team's supervisor when a capture test is running so seeded alerts are not escalated as real events.
How do you check coverage for AI tools you did not approve?
Compare what people actually use against what the archive actually captures. Seeded prompts only test the tools you already know about.
Pull sign-in logs from your identity provider, the browser extension inventory from managed devices, and expense or card data for AI subscriptions. Compare that list with your approved tools. FINRA's 2026 report also flags the risk of AI agents acting beyond the user's actual or intended scope and authority, so include any agent or connector that can reach mailboxes, CRM records or file shares. Each gap becomes one of three things: a tool to block, a tool to bring under capture, or a documented exception with an owner and an expiry date.
How do you test the human review step?
Pull a sample of closed AI alerts and confirm each has evidence of review, a reasonable disposition and follow-up where needed. Supplementary Material .07 to Rule 3110 says that merely opening a communication is not sufficient review.
Look for patterns that suggest rubber-stamping: alerts closed within seconds, one reviewer clearing hundreds of items in a single batch, or escalations with no documented outcome. For one cycle a year, consider placing an unannounced seeded item in the review queue and confirming it is handled correctly. Record what you found and what you changed, because both belong in the Rule 3120 report.
What should you do this quarter?
Run one full test cycle before your next annual report is written. A short plan:
- List every AI tool and embedded AI feature in use, approved or not, and mark which ones your archive captures.
- Write the expected result for capture, retention, detection and review for each captured tool.
- Run one seeded prompt test across all approved tools and keep the timestamps.
- Sample closed AI alerts from the last quarter and check the evidence of review.
- Add results, significant exceptions and procedure changes to this year's Rule 3120 report.
- Brief the CCO on the results before the CEO's Rule 3130 certification meeting.
How can s90 help?
s90 manages AI guardrails and prompt monitoring for broker-dealers and dual-registered firms, including the logging that makes testing possible in the first place. We can help you run seeded tests, compare sign-in data against your approved tool list, and package the results as evidence for your Rule 3120 report. This article describes what the rules require and is not legal advice. If you want a second look at how your AI supervision holds up, start with a short conversation.
