Finding clients and getting a distribution channel was the first real step—not the website itself.
I spent years building sites from scratch without AI or Cursor. HTML, CSS, JavaScript, CMS integrations, the whole stack. The work was never the bottleneck. Finding people who would pay for the work was. Every project ended the same way: a decent deliverable, then silence until the next referral or lucky inbound message.
That problem is B2B lead generation wearing a freelancer costume. I tried Apollo-style databases, generic scrapers, and half a dozen “all-in-one” growth tools. They were either too expensive at volume, too generic for my niche, or wrong often enough that I still spent hours cleaning spreadsheets. I wanted something effective, customised, accurate, and cheap—almost free to run on a Mac between client calls.
This post is the full arc: the system I built, the stack underneath it, how we productised it as a dashboard SaaS, and where an agentic dashboard goes next. If you’ve read our B2B agencies AI agent breakdown, treat this as the origin story and product roadmap in one place.
The problem: great at building, bad at distribution
When you sell website development (or any project-based service), your funnel is not “build faster.” It’s:
- Find businesses that already want what you sell.
- Reach them on a channel they actually check.
- Trust enough to reply before they talk to three other agencies.
Off-the-shelf lead tools optimise for volume, not for your positioning. I needed leads who were asking for websites, local SEO, redesigns, or “who can build X”—not random company names with guessed emails.
That pushed me toward search-native scraping: go where intent shows up (Google, Maps, Reddit, X, LinkedIn), capture the raw results, then use AI only where it adds precision.
The system in five layers
Think of the pipeline as a straight line from “I need clients” to “here’s a row in your CRM with email, phone, and DM links.”
| Layer | What it does |
|---|---|
| 1. Scrape the web | Custom scrapers hit Google, Google Maps, Reddit, X, and LinkedIn search/results. |
| 2. AI creates queries | An LLM turns your problem statement into fresh search queries per platform. |
| 3. Run scrapers | Those queries feed the scrapers; raw HTML/JSON lands in a queue. |
| 4. Data enrichment | A second AI pass filters junk, finds emails, socials, WhatsApp hints, founder contacts. |
| 5. Store + dashboard | Clean rows go to Firebase/Supabase/Drizzle; the SaaS UI shows agents, tables, and cost. |
The enrichment layer is the moat. Scraping is commoditising; knowing which row is a real buyer with a reachable founder is what saves the back-and-forth.
Layer 1: Custom scrapers (almost free at the API layer)
Instead of paying retail for every platform wrapper, we consolidated scraping behind one backend pattern—documented in One HonoJS file for entire web scraping API. Hono keeps the surface area small: one deployable file, edge-friendly, easy to mount on Fly or Render.
Platforms we target first:
- Google — classic web search for intent phrases (“need website developer”, “looking for web agency”).
- Google Maps — local businesses with phone numbers and review context.
- Reddit — threads where people ask for recommendations (see also our AI Reddit trend scraper for how we think about subreddit monitoring).
- X — public posts and replies with buying signals.
- LinkedIn — posts and profiles where founders describe needs (respect ToS and rate limits in production).
If you’re comparing buy-vs-build on APIs, Firecrawl vs ScrapingBee vs ScrapingAPI vs ScrapeFast lays out when hosted scrapers still make sense—and when owning the Hono layer wins on cost.
Layer 2 & 3: The AI scraping agent loop (local on Mac)
I run the AI Scraping Agent locally on a Mac at 10 / 20 / 30 second intervals (tunable per agent). Each tick:
- Read the agent’s problem statement (e.g. “Find SMBs in the US asking for website development”).
- Generate new search queries tailored to Google, Maps, X, Reddit, and LinkedIn.
- Invoke the scrapers with those queries.
- Push raw payloads to enrichment.
This is not “one prompt, one scrape.” It’s a scheduled loop—closer to a tiny worker than a chatbot. For cloud scheduling and retries, we wire the same jobs through Trigger.dev so agents can run on Render/Fly without my laptop being open 24/7.
Model routing: OpenRouter, OpenAI, Anthropic, or Gemini depending on cost and context length. The query-generation step can use a smaller model; enrichment often needs a stronger one for parsing messy HTML snippets.
Layer 4: Data enrichment (where AI earns its keep)
Raw SERP rows are noise. Enrichment is where we:
- Drop duplicates, SEO spam, and listicles with no buyer.
- Extract email, phone, WhatsApp mentions tied to a business name.
- Resolve social accounts for DMs (X, LinkedIn, Instagram where visible).
- Prefer founder or decision-maker signals from LinkedIn posts or “posted by” metadata.
That last step is what killed manual work for me. Google might surface a phone number next to a brand name; LinkedIn might expose a founder email in a post; Maps gives call-ready numbers for local shops. The LLM’s job is to normalise those fragments into one trustworthy row—not to hallucinate a contact when none exists.
We’ve used similar enrichment ideas when scraping hundreds of links from email batches and when turning GitHub trends and scrapers into SaaS ideas. Same pattern: scrape wide, filter narrow.
Frontend stack (what clients actually log into)
The dashboard is a normal modern SaaS shell—nothing exotic:
- Next.js, Vite, or TanStack-based apps for the UI framework (pick one product surface; we’ve shipped more than one internally).
- Shadcn UI + Tailwind for tables, toggles, usage charts, and settings—same direction as our cool Shadcn libraries roundup.
- Firebase, Supabase, or Drizzle for auth, agent configs, and enriched lead tables—see Firebase vs Supabase vs Clerk if you’re choosing auth + DB together.
- Payments: Stripe, Lemon Squeezy, Dodo, or Polar depending on tax handling and MoR preferences—Lemon Squeezy with Next.js is the fastest path we’ve documented for indie billing.
- Hosting: Vercel or Cloudflare for the frontend; keep scraping workers off the edge if jobs are long-running.
Each client-facing dashboard shows:
- AI Scraping Agent card with status, start/stop, and last run time.
- Data table of enriched leads with export.
- Usage & cost — token/API spend with monthly rollups (critical when you pass through LLM costs).
Backend API stack
Behind the UI:
- Web Scraping API — Google, Maps, LinkedIn, Reddit, X (Hono routes above).
- AI Scraping Agent service — query generation + orchestration + enrichment.
- Database — Firebase, AWS RDS, Supabase, or Drizzle on Postgres.
- LLM providers — OpenRouter or direct OpenAI / Anthropic / Gemini keys.
- Worker hosting — Fly.io or Render for always-on or cron-style agents.
- Scheduling / queue — Trigger.dev for intervals, backoff, and failure alerts.
Keep scrapers and LLM calls metered per tenant from day one. You’ll thank yourself when an enterprise customer leaves an agent running over a holiday weekend.
Selling the dashboard as SaaS
Once the dashboard works for you, it becomes a product:
- Agencies buy their own agent + database partition.
- You sell setup + monthly for hosting, enrichment, and support.
- Each tenant gets isolated config: problem statement, platforms, interval, enrichment rules.
Monetisation features we’re shipping or already use with clients:
- Manual invoices today; in-dashboard pay tomorrow (Stripe/Lemon Squeezy checkout links per workspace).
- Wallet / credits — prepay for usage; decrement on each scrape + LLM enrichment run (great for clients who spike Q4 outbound).
- Team & RBAC — roles for “view leads”, “edit agents”, “billing admin”. Enterprise plans love this; it also raises ARPU without more scraper code.
This sits in the same family as 15 micro SaaS ideas actually making money in 2026 and building a directory listing that earns online—vertical data + a simple UI beats another generic AI wrapper.
What we learned running agents for real buyers
Intervals beat batch jobs for intent data. Search results change daily; a 10–30s loop with new queries finds fresher “I need a developer” posts than re-running yesterday’s keywords.
AI twice, not once. Cheap model for queries; stronger model for enrichment. Putting everything through GPT-4-class models burns margin fast—same lesson as our B2B agent cost post.
Compliance is part of the product. Rate limits, robots.txt, and platform ToS are not footnotes. Document what you scrape, why, and retention—especially if EU leads land in your database.
Distribution still matters. A perfect scraper with no ICP story doesn’t sell. Pair technical posts with distribution playbooks like free-to-paid traffic in 2026 so the SaaS has a marketing loop, not just a GitHub repo.
Future: the agentic dashboard
Today’s dashboard helps clients run agents and read enriched tables. The next step is bigger:
The user describes the data they need in plain language. The platform understands the requirement, creates a new AI Scraping Agent using our default scrapers, stores results in their database, and surfaces the agent in the UI—ready to start/stop.
That means:
- Prompt → agent config (platforms, query templates, enrichment schema).
- Many agents per workspace without filing a support ticket.
- Full customisation for verticals: recruiting, local services, SaaS partnerships, investor outreach—each with the same scraper primitives.
We’re already partway there with internal agents for Reddit trends, Maps local businesses, and programming-news monitoring. Productising “create agent from prompt” is mostly orchestration UX plus guardrails on cost and compliance—not a new scraping engine.
If you’re bootstrapping the SaaS shell itself, I built a SaaS starter kit so you don’t waste four weeks on setup covers auth, billing hooks, and deployment—the boring weeks you shouldn’t re-live for every scraping vertical.
Related reading on iHateReading
- One HonoJS file for entire web scraping API — the scraper API shape we reuse everywhere.
- AI automated Reddit trend scraper that saved us $1,000/month — Reddit-specific agent design.
- B2B agencies AI agent for Reddit, X, LinkedIn, and Google Maps (costs) — deployment and unit economics.
- How to build websites — the craft side, before distribution eats your calendar.
- 150 lines of AI web search agent — minimal agent loop if you want a tiny prototype first.
Bottom line: I didn’t solve “how to build websites.” I solved who to sell them to, with proof they already wanted help—then wrapped that pipeline in a dashboard other agencies can pay for. The scrapers are the commodity; query generation plus enrichment plus billing is the SaaS. The agentic dashboard is the step where customers stop asking you to configure agents and start describing outcomes instead.
If you’re building something similar, start local: one problem statement, one platform, one enrichment prompt. Ship the table before you ship the multi-tenant wallet. Distribution is still the first step—but at least now the machine can fill the top of your funnel while you sleep.