Banned From Yahoo

As with so many things in life, when everything is going well, something always comes along to bring you down. My site had been enjoying some of the best traffic months ever. One particular week, my site was featured in a radio DJ newsletter that had a one million subscriber base. You can imagine the traffic I received that week. It brought my server to its knees. My top day that week was just over 50,000 visitors (for ONE day!). My user registrations were climbing, my search engine rankings were climbing, all was going too well.

The day it happened was sometime late spring, early summer of 2005. I don't remember the exact date, but I do know that I started having a substantial traffic drop off about that time. At first I did not realize what had happened. I usually don't check up on my search engine placements that often. Maybe a few times a month for a few sets of keywords. I heard about a lot of sites getting banned/removed from the Yahoo index in the DigitalPoint forums. That's when I checked my logs and realized my Yahoo referrer traffic had plummeted to zero.

Yahoo happened to be my second largest search engine referrer next to Google. I immediately started reviewing all the webmaster forums looking for possible reasons as to why it happened. I learned Yahoo had done some house cleaning and removed sites they considered in violation of their webmaster guidelines. What was I doing wrong? I was finally feeling the pressure of owning an internet-only business that was dependent on other services, search engines. I was during this time I realized how vulnerable my business had become. The majority of my traffic came from search engines (50%). That's 50% of my business that I have no control over.

I started a complete analysis of my site in hopes to find any problems that Yahoo may have stumbled upon. I found a bunch of problems that related to how I had setup my domains many years before. In the early days, I had registered a number of domains, some related to my site, others not. Well I figured it wouldn't hurt to have them all point to my site, since they were just parked anyway. Why not get a few extra direct type in visitors? Well this was one of my biggest problems. Yahoo had been indexing each of these domains as its own site. Since I was simply pointing each of these domains to my site (straight IP pointer, not a 301 redirect), it appeared to Yahoo that I was running duplicate versions of my site. I had shot myself in the foot! I immediately turned off all pointers to my site from any domain other than my primary domain(s).

I found an online form on Yahoo's site where you could request re-inclusion into their index. I drafted an apologetic message and sent my request explaining my situation, and what I had done to fix it. The next day I received a response. My request was denied.

So I went back to the drawing board and re-reviewed my site. I found my sloppy use of subdomains (blog.mysite.com) was causing a similar problem! In the past (before I knew better), I had carelessly used subdomains as a way to make the URL to various sections of my site look better. For example, auto.mysite.com. However, it was just a cosmetic change, it took you to the same page as if you used www.mysite.com. So again, Yahoo viewed these as both duplicate content AND abuse of the subdomain functionality to stuff keywords in the URL.

I decided on trying a 301 redirect to start to clean up from this mess I had created. A 301 tells search engines that the site they were trying to get to is no longer the active site, and they should now only index the new destination site defined in the redirect.

I spent the next 6 months following up with Yahoo each month in an attempt to get back in. Constantly reviewing my site, finding little problems here and there and fixing them. Finally, after about 7 months of being out, I received a response from Yahoo stating my site may be re-included in the next few weeks and to check the search engine results. I had completely cleaned house on my site and server, and it paid off. A few months later I was back in their index, and my traffic levels started to climb back up again.

I had learned an invaluable lesson. I would not be so careless with my site configuration ever again. And perhaps the most important lesson: Keep updated on the major search engines webmaster guidelines. They are the bible to staying in good graces... and free traffic.

Next up... Managing Online Communities...