1. Record the decision window
You need the audio the AMD heard: the first 2 to 5 seconds after answer, for every call, with its AMD verdict. VICIdial's call recordings usually start when an agent connects, which is too late. nomoVM keeps exactly this audio for you (Call logs, with playback). For Asterisk AMD(), record from the AMD extension with MixMonitor before the AMD line.
2. Sample fairly
- Take calls from whole days, not one hour: greetings and who answers change through the day.
- Sample both verdicts. If 70% of answered calls are machines, a random sample of 400 has about 120 humans, which is enough to measure.
- Write down the real share of each verdict that day, so you can weight your sample back to it.
3. Label blind
Listen to each call without seeing the AMD's verdict or the agent's disposition, and label it: human, machine, call screening, silence, or tone. Don't use dispositions as truth: agents mark voicemail as "no answer", and when we checked another AMD's "human" calls, about a quarter were machines. Listen for recorded hellos: the same voice and words on calls to different numbers are a recording.
4. Compute the two numbers
| Number | How | What it tells you |
|---|---|---|
| Live people kept | Humans the AMD called HUMAN ÷ all humans | How many conversations you lose. The one that costs revenue. |
| Agent calls that were people | Humans among calls the AMD called HUMAN ÷ all calls it called HUMAN | How much agent time goes to machines. |
Report both, weighted to the day's real mix. A vendor's single "99% accurate" can be true while it drops one live person in five, if most calls are machines and it calls almost everything a machine.
5. Compare on the same calls
The fairest test runs two AMDs on the same traffic. Run nomoVM on one campaign and your current AMD on a similar one for the same days, or label the same calls against both verdicts. Watch the agent-connect rate and the AA rate too.
Mistakes that make AMD look better than it is
- Using agent dispositions or another AMD's results as labels
- Testing on a clean hand-picked set of greetings
- Reporting one accuracy number without both error directions
- Ignoring calls that ended during the decision (hangups count)
Need help setting up a test? Contact us; the free plan's 30,000 calls a month is enough for a solid comparison.