A buyer's glossary for Trust & Safety vendor calls

I spent five years on the vendor side of T&S sales, and every sales call has its own dialect. Nothing in it is dishonest. Each phrase is true in a narrow sense and incomplete in a broader one, and the gap between the two is where buyers get surprised six weeks later.

This glossary translates six phrases you will probably hear on a vendor call. Each translation comes with the question that closes the gap.

"We're at 99% accuracy"

This is a number measured on a dataset, and the dataset is probably not yours. It also depends on how accuracy gets defined. On a platform where 97% of content is fine, a classifier that approves everything posts 97% accuracy and catches nothing.

In my experience, a well-performing classifier on real production data lands somewhere around 85% precision and recall. The number runs higher on mostly solved categories like profanity and lower on hard ones like grooming or coordinated harm. A vendor who gives you the honest 85 is not weaker than the one who says 99. The first vendor is just describing the same product more carefully.

The question to ask is: what are precision and recall per category, measured on a labeled sample of my content?

A classifier that approves everything can post 97% accuracy on a platform where 97% of the content is fine.

"Our taxonomy covers harassment"

This means the vendor has a category with that name. Whether the category matches what your policy means by the word is a separate question. Their harassment category may include bullying and exclude coordinated harm. Their adult content category may cover explicit imagery and miss the thematic content you care about.

The mismatch is nobody's fault, and it is the most common source of POC disappointment. The question to ask is: here are my policy definitions, so can you map each one to your categories and show me where the gaps are?

"Send us a sample and we'll tune it"

This means exactly what it says, and some buyers hear a trick in it. They worry that sharing labeled data lets the vendor tune to the test.

Calibrating a classifier against your labels is the work, and it is not cheating. A vendor that cannot adapt to your labels during a POC will not adapt to them after you sign. The question to ask is: what does tuning look like after the contract starts, and who does it?

"That one's an edge case"

This phrase cuts both ways. Every classifier misses individual cases, and a buyer who arrives at the review meeting with a list of "how did you miss this?" examples is spending the meeting on the wrong signal. Aggregate performance is what tells you something.

The phrase becomes a problem when it gets used to wave off a pattern. Three misses of the same kind are a category gap and not an edge case. The question to ask is: can you explain each of these decisions, and are the misses clustering around the same kind of content?

"Take as long as you need"

This is friendly, and it costs the vendor almost nothing to say. It is also how a two-week evaluation becomes a three-month one with no decision at the end. Enthusiasm fades, and the person who championed the test moves on to something else.

The fix is to agree on three things before anyone runs a single request: a budget (even a loose one), a timeline, and written success criteria that convert to an annual contract if the vendor meets them. The question to ask is: can we write down what a pass looks like, and what happens next if we hit it?

A good vendor will welcome that question. It tells them you are a real buyer.

"Our ML team will be all over this"

This is true, and you should take the attention. Close vendor involvement during a POC is how calibration gets done, and it is not a sign of desperation.

The follow-up matters more than the phrase itself. Attention that disappears at signature leaves you alone with the first confusing results, which is when a quick explanation is worth the most. The question to ask is: who is my contact at month three, and what does the relationship look like after we sign?

Every phrase in this glossary is true in a narrow sense. The skill is knowing which question widens it.

The pattern underneath

All six phrases share a shape. Each one gives a true statement and leaves out the context you need to use it. The remedy is a question, and a vendor's answer tells you more than the phrase did. The vendors who answer plainly, including the answers that are less flattering than the pitch, are the ones worth keeping.