There's an easy way to find out which pages of your site that have been indexed by the big 3 Search Engines (Google, Yahoo and MSN). Search for site:www.yoursite.com. If the number of search results don't match your website's total number of pages, it means some pages may not be listed. Some pages are invisible to Search Engines, such as Flash-only, image-only, and complex database generated pages. More on those later.

Xenu, a piece of software or an alien galactic ruler?

This begs the question: how can I find out how many pages there are on my website? This is often hard to answer, especially if you have a blog with many entries. The simplest way to find out is to download Xenu and run it on your computer. Not only will it give your site's number of pages but will also list all your broken links. Over the years I found found this little piece of software incredibly useful. Thank you Tilman Hausherr!

On a side note, Xenu's creator is an anti-scientologist, which is the reason he called this software Xenu.

I read a post the other day on Slashdot (News for Geeks, Stuff that Matters), still the web's most popular blog as of March 2005. Slashdot is about technology news, and they had a discussion about whether Google was still clearly superior to Yahoo and MSN as a Search Engine.

Inspired by that thread I conducted my own research. I searched for why are rain clouds grey? (without the quotes) in the big 3 Search Engines and followed the first 3 links. Here are the results (note that these search results may very well change if you perform the same search since these engines constantly tweak their pages):

Google Search:

1. ANSWER
2. ANSWER
3. ANSWER

Yahoo Search:

1. ONLINE NOVEL
2. KID'S SITE
3. FANTASY SITE

MSN Search:

1. DAILY DIARY BLOG
2. RAP SONG
3. SCIENCE EXPERIMENT

Conclusion: I'll stick to Google for now.

grey clouds, courtesy of cohdra from morgueFile

And by the way, from Google's first link result, rain clouds are grey because

they are often large, grey clouds and appear dark because they are so large and full of water that sunlight cannot get through them. The heaviest rain falls from the deepest, darkest clouds which are high enough for the raindrops to develop properly.

You may have seen spam email or ads on the web promising to submit your site to hundreds, if not thousands of search engines. That may be very well, but 3 search engines have 96% of the market (as of December 2004): Google, Yahoo and MSN.

Search Engines: the Big Three

AOL, which comprised 9% of all searches, is absent from the pie chart because its results are fed by Google. So if you want to optimize your site for search engines, it makes sense to focus on these exclusively. Making efforts to rank highly on search engines such as Ask, Excite or LookSmart is not worth it.