What Happens When Someone Tries to Break Your Agent?
Last week I was on my phone with my bank trying to dispute a charge. reaching someone, I started thinking about something that had never really crossed my mind before:
What if the person on the other end was not trying to help me, but trying to get something out of me?
We spend a lot of time thinking about whether voice agents can handle a booking, route a caller correctly, or remember a customer across calls. What we do not talk about nearly enough is what happens when the caller is not playing by the rules.
Voice agents are being deployed everywhere right now, in banks, healthcare providers, insurance companies, and financial institutions. They involve account numbers, personal information, and sensitive decisions. The more capable these agents become, the more attractive they are as a target.
A bad actor does not always look like someone typing commands into a terminal. It could be a caller claiming there is a family emergency and demanding an urgent fund transfer. It could be someone posing as a supervisor with special access, or a caller with just enough partial information to sound legitimate. It could also be someone attempting something far more technical, trying to get the agent to reveal its system prompt, expose its tool calls, or abandon its role entirely through a direct instruction override.
The question becomes: How do you know your agent holds up when someone is actively trying to break it?
That is exactly what Red teaming was built for.
Red teaming is the practice of testing your voice agent against exactly these kinds of threats before they ever reach a real customer. At Bluejay, digital humans take on the role of adversaries and attempt to break your agent through two methods: social engineering, which covers manipulation tactics like pressure, false authority, and emotional distress, and prompt injection, which covers technical attacks designed to make your agent behave outside of its intended design.
With Bluejay you can now simulate adversarial callers and test how your agent responds to real world attack scenarios before they ever reach a real customer.
Read about it in our feature spotlight below!
As a recap:
- Bluejay is a testing and monitoring platform for Conversational AI agents. Companies ranging from Fortune 10 enterprises to fast-growth startups in the Silicon Valley use Bluejay to make sure their voice and text agents work in production (monitoring) and development (testing) environments.
- Our team, now ten strong, works around the clock to make sure your agent behaves when talking to customers.
- This newsletter is 100% human written. It always has been, and it always will be. Ask yourself about what you are consuming. If the writer hasn't read it, why should you?
Announcements
Heres what happened at Bluejay last week:
- The team pushed 105456 lines of code this week to make Conversational AI more reliable!
- We're packing up and moving into our new SF office! This marks the end of an era for our first office, the Nest.
The Engineers Take the Mic
Real customers don't call your voice agent once. They book, then call back to change something, then call again to cancel. Three calls, one person, and your agent has to keep up.
Customer Journeys lets you test exactly that. A digital human calls your agent in sequence, with full memory carried across every call. It remembers the booking, the dates, the confirmation number. Just like a real customer would.
Single-call tests can't catch the bugs that show up when a customer calls back. This does.
If your agent breaks on the second call, you'll know before your customers do.
A successful call is not just about what your agent says. It is about whether it can navigate the systems and people on the other end.
This demo walks you through how to build a custom IVR flow inside Bluejay, from phone tree menus and hold music to simulated human agents, so you can validate your entire call flow before your agent ever makes a real call.
Feature Spotlight
We are introducing red teaming simulations.
Most voice agent testing focuses on whether your agent does the right thing when a caller is cooperative. Red teaming flips that entirely, and it tests whether your agent does the right thing when a caller is actively trying to manipulate it.
Bluejay now supports two types of red teaming simulations.
The first is social engineering.
A digital human plays the role of an adversary and attempts to manipulate your agent using real world tactics. This includes applying pressure or urgency to force a quick decision, using family trauma or emotional distress to lower the agent's guard, pretending to be an authority figure with special access, claiming to be calling from out of the country, and using partial information to appear more legitimate than they actually are. You can configure exactly what the attacker knows and does not know, making each simulation as close to a real attack as possible.
The second is prompt injection.
Here the digital human attempts to get your agent to behave outside of its intended design. This includes trying to get the agent to reveal its system prompt, disclose information about its tool calls, take unauthorized actions, or abandon its role through direct instruction overrides and roleplay manipulation.
The goal of both is straightforward. Find the weaknesses in your agent before someone else does.
A voice agent that handles every happy path perfectly but crumbles under social pressure or a clever prompt injection is not ready for production. Red teaming is how you find out.
A quick note. The UI for red teaming is still being finalized and our team is actively working on releasing a design that does this feature justice. What you are seeing today is the foundation and we are excited to show you what is coming soon.
Give it a try on your next simulation and let us know what you think.
That's all for now. I'll see you next time!
Azfar Khan
Storyteller @ Bluejay
