google algorithm after jagger update

Have you noticed anything different with Google lately? TheWebmaster community certainly has, and if recent talk on severalsearch engine optimization (SEO) forums is an indicator,Webmasters are very frustrated. For approximately two yearsGoogle has introduced a series of algorithm and filter changesthat have led to unpredictable search engine results, and manyclean (non-spam) websites have been dropped from the rankings.Google updates used to be monthly, and then quarterly. Now withso many servers, there seems to be several different searchengine results rolling through the servers at any time during aquarter. Part of this is the recent Big Daddy update, which is aGoogle infrastructure update as much as an algorithm update. Webelieve Big Daddy is using a 64 bit architecture. Pages seem togo from a first page ranking to a spot on the 100th page, orworse yet to the Supplemental index. Google algorithm changesstarted in November 2003 with the Florida update, which nowranks as a legendary event in the Webmaster community. Then cameupdates named Austin, Brandy, Bourbon, and Jagger. Now we aredealing with the BigDaddy!

The algorithm problems seem to fall into 4 categories. There arecanonical issues, duplicate content issues, the Sandbox, andsupplemental page issues.

1. Canonical Issues:
These occur when a search engine treats www.yourdomain.com, yourdomain.com, and yourdomain.com/index.htmlall as different websites. When Google does this, it then flagsthe different copies as duplicate content and penalizes them.Also, if the site not penalized is http://yourdomain.com, butall of the websites link to your website using www.yourdomain.com,then the version left in the index will have no ranking. Theseare basic issues that other major search engines, such as Yahooand MSN, have no problem dealing with. Google is possibly thegreatest search engine in the world (ranking themselves as a 10on a scale of 1 to 10). They provide tremendous results for awide range of topics, and yet they cannot get some basic indexingissues resolved.

2. The Sandbox:
This has become one of the legends of the search engine world. It appears that websites, or links to them, are "sandboxed" for a period before they are given full rank in theindex, kind of like a maturing time. Some even think it is onlyapplied to a set of competitive keywords, because they were theones being manipulated the most. The Sandbox existence isdebated, and Google has never officially confirmed it. Thehypothesis behind the Sandbox is that Google knows that someonecannot create a 100,000 page website overnight, so they haveimplemented a type of time penalty for new links and sitesbefore fully making the index.

3. Duplicate Content Issues:
These have become a major issue on the Internet. Because web pages drive search engine rankings,black hat SEOs (search engine optimizers) started duplicatingentire sites' content under their own domain name, therebyinstantly producing a ton of web pages (an example of this wouldbe to download an Encyclopedia onto your website). As a resultof this abuse, Google aggressively attacked duplicate contentabusers with their algorithm updates. But in the process theyknocked out many legitimate sites as collateral damage. Oneexample occurs when someone scrapes your website. Google seesboth sites and may determine the legitimate one to be theduplicate. About the only thing a Webmaster can do is track downthese sites as they are scraped, and submit a spam report toGoogle. Another big issue with duplicate content is that thereare a lot of legitimate uses of duplicate content. News feedsare the most obvious example. A news story is covered by manywebsites because it is content the viewers want. Any filter willinevitably catch some legitimate uses.

4. Supplemental Page Issues:
Webmasters fondly refer to this as Supplemental Hell. This issue has been reported on places likeWebmasterWorld for over a year, but a major shake up aroundFebruary 23rd has led to a huge outcry from the Webmastercommunity. This recent shakeup was part of the ongoing BigDaddyrollout that should finish this month. This issue is stillunclear, but here is what we know. Google has 2 indexes: theMain index that you get when you search, and the Supplementalindex that contains pages that are old, no longer active, havereceived errors, etc. The Supplemental index is a type ofgraveyard where web pages go when they are no longer deemedactive. No one disputes the need for a Supplemental index. Theproblem, though, is that active, recent, and clean pages havebeen showing up in the Supplemental index. Like a dungeon, oncethey go in, they rarely come out. This issue has been reportedwith a low noise level for over a year, but the recent Februaryupset has led to a lot of discussion around it. There is not alot we know about this issue, and no one can seem to find acommon cause leading to it.

Google updates were once fairly predictable, with monthlyupdates that Webmasters anticipated with both joy and angst.Google followed a well published algorithm that gave eachwebsite a Page Rank, which is a number given to each webpagebased on the number and rank of other web pages pointing to it.When someone searches on a term, all of the web pages deemedrelevant are then ordered by their Page Rank.

Google uses a number of factors such as keyword density, pagetitles, meta tags, and header tags to determine which pages arerelevant. This original algorithm favored incoming links and theanchor text of them. The more links you got with an anchor text,the better you ranked for that keyword. As Google gained thebulk of internet searches in the early part of the decade,ranking well in their engine became highly coveted. Add to thisthe release of Google's Adsense program, and it became verylucrative. If a website could rank high for a popular keyword,they could run Google ads under Adsense and split the revenuewith Google!

This combination led to an avalanche of SEO'ing like theWebmaster world had never seen. The whole nature of links betweenwebsites changed. Websites used to link to one another becauseit was good information for their visitors. But now that link toanother website could reduce your search engine rankings, and ifit is a link to a competitor, it might boost his. In Google'salgorithm, links coming into your website boost the site's PageRank (PR), while links from your web pages to other sites reduceyour PR. People started creating link farms, doing reciprocallink partnerships, and buying/selling links. Webmasters startedlinking to each other for mutual ranking help or money, insteadof quality content for their visitors. This also led to thewholesale scraping of websites. Black hat SEO's will take thewhole content of a website, put Google's ad on it, get a fewhigh powered incoming links, and the next thing you know theyare ranking high in Google and generating revenue from Google'sAdsense without providing any unique website content.

Worse yet, as Google tries to go after this duplicate content,they sometimes get the real company instead of the scraper. Thisis all part of the cat and mouse game that has become the Googlealgorithm. Once Google realized the manipulation that washappening, they decided to aggressively alter their algorithmsto prevent it. After all, their goal is to find the mostrelevant results for their searchers. At the same time, theyalso faced huge growth with the internet explosion. This has ledto a period of unstable updates, causing many top rankingwebsites to disappear while many spam and scraped websitesremain. In spite of Google's efforts, every change seems tocatch more quality websites. Many spam sites and websites thatviolate Google's guidelines are caught, but there is an endlesstide of more spam websites taking their place.

Some people might believe that this is not a problem. Google isthere to provide the best relevant listings for what people aresearching on, and for the most part the end user has not noticedan issue with Google's listings. If they only drop thousands oflistings out of millions, then the results are still very good.These problems may not be affecting Google's bottom line now,but having a search engine that cannot be evolved withoutproducing unintended results will hurt them over time in severalways.

First, as the competition from MSN and Yahoo grows, havingthe best results will no longer be a given, and these drops inquality listings will hurt them. Next, to stay competitiveGoogle will need to continue to change their algorithms. Thiswill be harder if they cannot make changes without producingunintended results. Finally, having the Webmaster community losefaith in them will make them vulnerable to competition.Webmasters provide Google with two things. They are the word ofmouth experts. Also, they run the websites that use Google'sAdsense program. Unlike other monopolies, it is easy to switchsearch engines. People might also criticize Webmasters forrelying on a business model that requires free search enginetraffic. Fluctuations in ranking are part of the internetbusiness, and most Webmasters realize this. Webmasters aresimply asking Google to fix bugs that cause unintended issueswith their sites.

Most Webmasters may blame ranking losses on Google and theirbugs. But the truth is that many Webmasters do violate some ofthe guidelines that Google lays out. Most consider it harmlessto bend the rules a little, and assume this is not the reasontheir websites have issues. In some cases, though, Google isright and has just tweaked its algorithm in the right direction.Here is an example: Google seems to be watching the incoming links to your site to make sure they don't have the same anchor text (this is the text used in the link on the website linkingto you). If too many links use the same anchor text, Googlediscounts these links. This was originally done by some peopleto inflate their rankings. Other people did it because oneanchor text usually makes sense. This is not really a black hatSEO trick, and it is not called out in Google's guidelines, butit has caused some websites to lose rank.

Webmasters realize that Google needs to fight spam and blackhat SEO manipulation. And to their credit, there is a GoogleEngineer named Matt Cutts who has a Blog site and participatesin SEO forums to assist Webmasters. But given the revenue impactthat Google rankings have on companies, Webmasters would like tosee even more communication around the known issues, and helpwith identifying future algorithm issues. No one expects Googleto reveal their algorithm or what changes they are making. Rumoron the forum boards speculates that Google is currently lookingat items like the age of the domain name, websites on the sameIP, and frequency of fresh content. It would be nice from aWebmaster standpoint to be able to report potential bugs toGoogle, and get a response. It is in Google's best interest tohave a bug free algorithm. This will in turn provide the bestsearch engine results for everyone.

By Rodney Ringler (c) 2006. Rodney Ringler is President of Advantage1 Web Services, Inc.,which owns a network of Web Hosting Informational Websitesincluding Hostchart.com (http://www.hostchart.com), Resellerconnection.com (http://www.resellerconnection.com), Foundhost.com (http://www.foundhost.com) and Resellerforums.com(http://www.resellerforums.com)