Enough Already - Crappy Keyword Based Ads

I was searching on Yahoo's Answers service for some advice on a coworker situation for a friend. I found lots of great relevant questions and answers from folks all around the country. Then I noticed the advertisements Yahoo was displaying next to the search results.

I had simply searched for the key term "coworker". Here are the ads Yahoo felt best matched my search and interest:

Come on Yahoo. Coworkers for less? Coworker ringtones? Spam ads like this are partly responsible for giving us legit web content publishers a bad rap. Imagine yourself a user on a site reading about coworkers and you see these ads next to the article. I would think the site was throwing spam at me, which would leave a bad taste in my mouth.

I know the contextual matching technology has a long way to go, but I think it is about time the search ad companies stop allowing advertisers to spam the search results like this. Irrelevant and annoying ads like this are only going to give our industry a bad rap, and hurt us all in the end.

PubCon Recap Day 3 - Spam Hunt

This being my first PubCon, I did not know what to expect from the much talked about dream panel of Matt Cutts (Google), Tim Mayer (Yahoo), Danny Sulllivan (SearchEngineWatch.com), Greg Boser (WebGuerrilla) and Todd Friesen (Range Online). They participated in a session called "Interactive Site Reviews and SERPS Quality Control". I discovered this was basically code for "spam hunt"...

Folks from the audience would submit business cards with their websites on them. The panel would then randomly select sites to review and provide feedback. Matt Cutts was most entertaining as he ran his toolkit of secret Google programs to uncover the dirt on any and all sites that got in his path. Site owners were given the microphone to introduce their site and what they are looking for help on. The panel in turn would offer suggestions and then, in most cases, expose hidden blackhat tactics on almost all the sites... It was all done in good fun and I found it by far the most enjoyable session.

Some learnings I took away from it:

1. Yahoo Penalty - If you have a penalty assigned to your site, Yahoo may not show the "cache" link of your pages, and the titles may appear in lower case. Both are a sign that you may want to consider evaluating your site for any problems.

2. Dashes in a URL are better than underscore. Not sure how important this is, but Matt Cutts did make this comment as a recommendation during one of the site reviews.

3. Yahoo Deep Crawl - Having an authoritive inbound link will encourage Yahoo to deep crawl your content.

4. Google Supplemental Results - One audience member asked about their pages showing up randomly in the results. Sometimes there, sometimes not. Matt Cutts theorized that they were most like on the fringe and jumping back and forth between supplemental and regular results. He mentioned the solve the problem, just get your pagerank up (more quality inbound links) and that should push you out of the supplemental.

Gotta run and catch my flight!

Banned From Yahoo

As with so many things in life, when everything is going well, something always comes along to bring you down. My site had been enjoying some of the best traffic months ever. One particular week, my site was featured in a radio DJ newsletter that had a one million subscriber base. You can imagine the traffic I received that week. It brought my server to its knees. My top day that week was just over 50,000 visitors (for ONE day!). My user registrations were climbing, my search engine rankings were climbing, all was going too well.

The day it happened was sometime late spring, early summer of 2005. I don't remember the exact date, but I do know that I started having a substantial traffic drop off about that time. At first I did not realize what had happened. I usually don't check up on my search engine placements that often. Maybe a few times a month for a few sets of keywords. I heard about a lot of sites getting banned/removed from the Yahoo index in the DigitalPoint forums. That's when I checked my logs and realized my Yahoo referrer traffic had plummeted to zero.

Yahoo happened to be my second largest search engine referrer next to Google. I immediately started reviewing all the webmaster forums looking for possible reasons as to why it happened. I learned Yahoo had done some house cleaning and removed sites they considered in violation of their webmaster guidelines. What was I doing wrong? I was finally feeling the pressure of owning an internet-only business that was dependent on other services, search engines. I was during this time I realized how vulnerable my business had become. The majority of my traffic came from search engines (50%). That's 50% of my business that I have no control over.

I started a complete analysis of my site in hopes to find any problems that Yahoo may have stumbled upon. I found a bunch of problems that related to how I had setup my domains many years before. In the early days, I had registered a number of domains, some related to my site, others not. Well I figured it wouldn't hurt to have them all point to my site, since they were just parked anyway. Why not get a few extra direct type in visitors? Well this was one of my biggest problems. Yahoo had been indexing each of these domains as its own site. Since I was simply pointing each of these domains to my site (straight IP pointer, not a 301 redirect), it appeared to Yahoo that I was running duplicate versions of my site. I had shot myself in the foot! I immediately turned off all pointers to my site from any domain other than my primary domain(s).

I found an online form on Yahoo's site where you could request re-inclusion into their index. I drafted an apologetic message and sent my request explaining my situation, and what I had done to fix it. The next day I received a response. My request was denied.

So I went back to the drawing board and re-reviewed my site. I found my sloppy use of subdomains (blog.mysite.com) was causing a similar problem! In the past (before I knew better), I had carelessly used subdomains as a way to make the URL to various sections of my site look better. For example, auto.mysite.com. However, it was just a cosmetic change, it took you to the same page as if you used www.mysite.com. So again, Yahoo viewed these as both duplicate content AND abuse of the subdomain functionality to stuff keywords in the URL.

I decided on trying a 301 redirect to start to clean up from this mess I had created. A 301 tells search engines that the site they were trying to get to is no longer the active site, and they should now only index the new destination site defined in the redirect.

I spent the next 6 months following up with Yahoo each month in an attempt to get back in. Constantly reviewing my site, finding little problems here and there and fixing them. Finally, after about 7 months of being out, I received a response from Yahoo stating my site may be re-included in the next few weeks and to check the search engine results. I had completely cleaned house on my site and server, and it paid off. A few months later I was back in their index, and my traffic levels started to climb back up again.

I had learned an invaluable lesson. I would not be so careless with my site configuration ever again. And perhaps the most important lesson: Keep updated on the major search engines webmaster guidelines. They are the bible to staying in good graces... and free traffic.

Next up... Managing Online Communities...