How AI Scrapers Crashed My Vercel App (And How I Saved It with DigitalOcean & Cloudflare)

If you run a public-facing website today, you might have noticed some strange, massive spikes in your traffic. You get excited, thinking your app finally went viral—only to realize it's just AI bots hammering your servers. I recently experienced this exact scenario with my project, Viconic.dev, a platform for finding and sharing icons. Here's the story of how AI crawlers almost took down my site—and how I regained control without losing AI search visibility. 🤖 The "Hug of Death" by AI Bots Initially, I hosted Viconic on Vercel. It was fast, easy to deploy, and worked perfectly for my needs. However, as the site grew, I started getting hit by massive waves of automated requests. Web crawlers from AI companies like Perplexity, OpenAI (ChatGPT), and others were aggressively scraping my site for training data and real-time search results. Within a short period, these bots completely exhausted my Vercel request limits. 👉 My site was choking. 👉 I was facing potential downtime. 👉 And unex