Competitor content intelligence
Score competitor pages at scale to find the URLs where they're winning on keywords but losing on substance. Those are the pages where adding concrete specificity beats them — not on backlinks, not on age, on the one thing a fresh page can change.
We ran a competitor-mapping exercise at Clearscope across the top 100 keywords in the personal-finance niche in late 2024. About 40% of the top-three ranking URLs scored medium or high on slop_risk. The teams that won the next core update were the ones that picked off those specific URLs — not by writing 'better' content, but by writing content with the specific firsthand details (real ATM fees, real interest rates, real screenshots of the signup flow) the incumbents had skipped. A 'better' page is subjective. A more-specific page is measurable.
Business value
- Replaces the 'pick the highest-volume keywords' content strategy with 'pick the keywords where the SERP is weak.' Volume is a ceiling; weakness is an opening.
- Generates content briefs with specific competitor weaknesses to attack — not just keywords to target.
- Quantifies a niche's slop floor. Some industries (low-code, AI tools, supplements) are 80% slop on page one. That's a strategic signal for where to invest.
Agent job to be done
Be a competitive analyst with a budget. Score competitor URLs at scale, cluster the weak ones by what specifically they're missing, and turn the clusters into a prioritized brief list.
format: articleintended_use: otherdomain: competitor content intelligence / content gap analysis
Where to spend the slop-detection budget on a competitor crawl
Don't score every page on a competitor's site. Score the pages that rank top-three for your target keywords AND have a publish date older than 18 months. Old, ranking pages are the ones where complacency has set in — they're not getting refreshed because they're working. Those are the URLs most vulnerable to a fresh, specific challenger.
When to call VeracityAPI
Run during keyword-gap research and again 90 days after a competitor publishes a fresh page in your niche.
What text to submit
Competitor page main body, headings, comparison tables, recommendation/conclusion sections. Respect robots.txt; do not submit private or paywalled content unless permitted. Strip the boilerplate (their nav, footer, author bio) — score the unique body only.
Decision policy
- low competitor risk: this page is genuinely strong. Compete with a unique data angle, original research, or domain authority, not by writing the same thing better.
- medium competitor risk: targetable. Identify which sections scored worst (usually FAQ + comparison) and write a page that specifically out-specifies those.
- high competitor risk: priority gap if commercial intent is high. Build a brief that calls out the specific weaknesses by category.
- Store evidence spans as the 'why we can beat this URL' rationale in your content planning system.
Request template
The exact payload shape this use case sends. The sample below uses representative content for this workflow; substitute your own.
curl https://api.veracityapi.com/v1/analyze \
-H "Authorization: Bearer $VERACITY_API_KEY" \
-H "Content-Type: application/json" \
-d '{"type":"text","content":"Our travel safety guide gives you everything you need to know before your trip. Stay aware, keep belongings close, and choose trusted services for the best experience.","context":{"format":"article","intended_use":"other","domain":"competitor content intelligence / content gap analysis"},"store_content":false}'Automation recipe
- Crawler collects top 10 ranking URLs for each target keyword. Use a search API; don't scrape Google directly.
- Extractor isolates main content using a readability library (Mozilla's @mozilla/readability is the cheapest reliable option).
- Score each URL with intended_use=other (avoids the publish-grade threshold, which would over-flag dry factual content).
- Worker clusters high-slop URLs by missing-detail type using the evidence categories as features.
- Planner creates briefs that explicitly target each cluster's weaknesses (e.g., 'these top-3 pages all score high on missing_firsthand — win by adding hands-on testing screenshots').
Evidence spans agents should inspect
- 'thin_comparison' — comparison tables that don't help a buyer choose
- 'missing_firsthand' — listicles that read like aggregation, not testing
- 'unsupported_superlative' — best/cheapest/safest claims without a named benchmark
- 'paraphrase_summary' — page is recognizably a paraphrase of an earlier article on the same topic
Policy pseudocode
if (result.recommended_action === "allow") continueWorkflow(); if (result.recommended_action === "revise") rewriteWith(result.evidence, result.recommended_fixes); if (result.recommended_action === "human_review") queueForHumanReview(result); if (result.recommended_action === "reject") discardOrRebuild();
KPIs to track
- competitor URLs scored per dollar of API spend
- high-opportunity keywords identified per scoring run
- ratio of content-brief → published-page → rank gains within 6 months
- share of voice in target subniches over time
- editorial research hours saved per brief
What can go wrong
- A high slop score doesn't mean the page ranks poorly today. It means a fresh, specific page can outrank it — eventually. Rankings lag content quality by months.
- Respect terms of service and rate limits. Don't submit content you don't have the right to evaluate.
- Use as a prioritization signal alongside backlinks, search intent, business value, and competitive moat. Slop scoring is one input, not the whole strategy.
Cost and latency notes
Analyze only is $0.005 per 1,000 characters; Analyze + revise with auto_revise=true is $0.010 per 1,000 characters. Both round up to the nearest 1,000 characters. Short captions/emails usually cost $0.005; longer pages or chapters scale linearly by length. Current v0.1 latency is LLM-bound, so batch/concurrent orchestration is recommended for high-volume pipelines.
Agent evaluation checklist
- Does this workflow have a costly failure mode from generic or weak-provenance text?
- Can the agent map evidence spans back to editable source locations?
- Should this workflow fail open, fail closed, or queue human review if VeracityAPI is unavailable?
- Which field drives policy: recommended_action, risk_level, content_trust_score, specificity_risk, or provenance_weakness?
- What local rule should complement the API score?