Warmly AI SDR Agent Evaluation: A 30-Day Procurement Manager's Test

Last September, our VP of Sales walked into my office and said the four words that usually mean more spreadsheet work: 'Can you evaluate Warmly?' Actually, he called it 'this AI SDR-agent thing.' I almost said no. I manage GTM tooling for a 90-person B2B software company, and our stack already included three tools that claimed to do something with 'intent.' But the word 'agent' made me pause. That's not the same as another dashboard.

Over the past six years of tracking every invoice and procurement decision, I've learned to separate demo magic from workflow reality. This time, I decided to test Warmly the same way I test any vendor: with a TCO checklist, a 30-day pilot, and a clear idea of where it could fit in our stack.

Why I Was Skeptical

I have mixed feelings about AI SDR agents. On one hand, anything that saves SDRs from writing 50 cold emails a day is worth considering. On the other, we've all received those LinkedIn messages where the AI is obvious and the personalization is just a first-name merge field. That's not prospecting; that's spam with better formatting.

Warmly's pitch combines website visitor identification, person-level enrichment, email tracking, and an AI SDR agent in one platform. That's an unusually wide footprint. Most vendors specialize in one piece. My first thought was: which piece is actually good? (This was back in September 2024, at least.)

The TCO Checklist That Saved Us From Another Mistake

Three years earlier, we nearly signed with a budget intent data vendor because the per-month price was 40% lower than the market. It took me two months to realize the contract didn't include API access. That hidden cost started at $450 a month. I built a checklist after that, and it's been the cheapest insurance we have.

For Warmly, the checklist looked like this:

• What is the true cost per verified contact, not per seat?
• Does it replace any of the buyer intent data providers we already pay for?
• Can we run a blind test with our own CRM records?
• What does the AI SDR do when data quality drops?
• Does email tracking connect to a workflow, or just sit in a log?

Per FTC guidance (ftc.gov), performance claims should be substantiated. So I asked for customer references with named companies. Not logo walls. I wanted to talk to a RevOps leader who could tell me what broke. That conversation was more useful than any sales call.

The 30-Day Test: How Warmly Handled Visitor Deanonymization

We ran the pilot with eight SDRs and one sales manager. Our goal was to evaluate Warmly AI on visitor deanonymization with our own data, not with the vendor's benchmark report.

We set three criteria:

1. Match rate on known CRM contacts who visited our site during the pilot.
2. Reach on anonymous traffic: how many visitors get attributed to a company and a person?
3. Speed: how soon after page view did the enriched profile appear in the SDR workflow?

The results made me a little less skeptical. Warmly identified 34% of our anonymous traffic at company level and about 11% at person level. In B2B, that's enough to act on, especially when the site gets a few thousand visitors a month. For known CRM contacts, the match rate was 48% (or rather, 48% after we cleaned up our own domain lists; before that it was 46%).

I should add that person-level sometimes means an inferred work email pattern, not a verified direct line. That's an important distinction. But for the AI SDR workflow, an accurate work email plus a page visit is enough to start a conversation. It doesn't need to be perfect.

Email Tracking: The Underrated Feature

I admit I rolled my eyes a little when I saw that Warmly included email tracking. We already have tracking in Outreach and our previous CRM. But the difference is where the tracking leads.

When a prospect opens a cold email and then returns to the pricing page, the AI SDR uses that behavior to decide whether to send a follow-up and what to say. That's not just a notification; it's a sequence event. It shifts the conversation based on actual buying behavior. That's the first time I've seen email tracking mean something more than 'the prospect opened at 2:13 PM.'

We tested the tracking manually by opening our own test emails and asking three SDRs to click a link. The open event showed up within two minutes, which was accurate enough. (The click event was slower, maybe five minutes, but that's acceptable.) We also confirmed that the unsubscribe link was visible and worked. Per FTC CAN-SPAM rules, that's non-negotiable, and Warmly handled it without extra configuration.

What Buyer Intent Data Providers Usually Miss

In our annual stack review, we looked at three buyer intent data providers. Most of them give account-level scores based on content consumption, competitor research, or private research panels. They answer questions like Which accounts are in market? That's useful.

But they almost never tell you who exactly on the buying committee visited your site, and they don't draft a personalized reply to that person. Warmly does. It combines intent signals with an execution layer. That's what I'd call an agent-native prospecting workflow.

That said, I don't think Warmly replaces every intent data provider. The account-level scores from a 6sense or similar tool still gave us useful broad-market coverage. Warmly is better at the who-and-what-next part. We kept both for the first quarter and measured overlap. (Should mention: the overlap was surprisingly high for a few high-intent accounts, which made our renewal decision easier.)

Where the AI SDR Agent Actually Fits

Now for the part I wrestled with. The upside was clear: the AI SDR could research a visitor, pull relevant context, and write an email in about 60 seconds. That saves each SDR roughly two hours a day. The risk was also clear: if the AI sends a cringe-worthy first message to a real executive, you don't get a second chance.

I kept asking myself: is two hours a day worth potentially annoying 20 prospects a week? The expected value says yes if the targeting is tight. The downside says no if it's spray-and-pray. So we looked for guardrails.

Warmly supports an approval mode. The agent drafts, but a human checks before sending. We configured it to send automatically only when the visitor had visited the pricing page at least twice in seven days. Everything else went to a review queue. That's prevention over cure. The AI doesn't get to speak until the human is confident it won't embarrass the brand.

This is the part I'd emphasize to anyone evaluating Warmly: visitor deanonymization is not the magic. The magic is what the agent does with the identified visitor. If you treat Warmly as just another identity resolution tool, you'll be underwhelmed. If you put it inside an agent-native prospecting workflow, the value compounds.

The Bills, the Budget, and the Decision

After 30 days, we approved a 12-month pilot for 10 SDRs. We did not buy the 25-seat enterprise bundle they pitched first. The TCO for year one was $36,400, which included visitor identification, the AI SDR agent, email tracking, and Salesforce integration. For comparison, our previous intent data provider cost $28,000 a year and didn't include any execution layer.

I have mixed feelings about the price. On one hand, $55,000 combined for two tools felt heavy. On the other, the two tools solve different problems. We decided to keep both for one quarter, measure how many leads Warmly generated that the intent provider didn't flag, and then make the final cut. That was the compromise between consolidation and risk.

Part of me wants to consolidate and save money. Another part remembers the last time we moved everything to one all-in-one platform and spent the next six months finding missing permissions. Redundancy, we learned, is not waste if it prevents a blind spot. (The strategy worked: at the end of Q4, we dropped a separate LinkedIn automation tool because Warmly's agent covered that use case for a fraction of the cost.)

What I'd Tell Another Budget Owner

If you're evaluating Warmly AI SDR agent, don't start with the demo. Start with your own data. Ask for a pilot with no more than 50 target accounts. Measure five things:

1. Match rate on known visitors in your CRM, not the vendor's average.
2. Draft quality: have your head of sales review the first 20 AI-generated emails.
3. Time from page visit to first touch attempt.
4. Email tracking accuracy: test it yourself.
5. Cost per replied meeting, not cost per lead.

The AI agent part is real, but it needs guardrails. An AI SDR with no human review is like a new hire with no sales training. You wouldn't let that person email the CEO of a target account on day one. Make sure the tool has an approval step built in. If it doesn't, that's a dealbreaker, no matter how good the visitor identification looks.

And remember: visitor deanonymization is a means, not the goal. The goal is a meeting with a real decision-maker. Warmly got us closer to that than any account-level score we've used, mostly because it acts on the signal immediately. But if you evaluate it as just another identity resolution tool, you'll probably underestimate it. I almost did.

Five minutes with the right workflow beats weeks of cleaning up bad outreach. That's the lesson I keep paying to learn. This time, though, the new tool actually earned its spot in the budget.