19 common mistakes that prevent your Web site from showing up on search engines

Axandra.com 

Reason #13: Your Web pages require a full-fledged browser.

When search engines crawl the Web to find new Web pages, they use special software for it, called "spiders", "robots" or "crawlers".

These crawler programs don't have the functionality of full-fledged Web browsers such as Microsoft Internet Explorer or Netscape Navigator.

In fact, search engine robot programs look at your Web pages like a text browser does. They like text, text, and more text. They ignore information contained in graphic images but they can read <IMG ALT> text descriptions.

This means that search engine spider programs are not able to use Web browser technology to access your site. If your Web pages require Flash, DHTML, cookies, JavaScript, Java or passwords to access the page, then search engine spiders might not be able to index your Web site.

Therefore, it might be a good idea to test your Web pages with very old versions of Web browser applications or with the software program "Lynx", a text-only browser.

Lynx is available for download http://lynx.browser.org/. Here's an online version of Lynx that allows you test your Web pages with a text-only browser quickly and easily: http://www.delorie.com/web/lynxview.html

"Simulation of a search engine spider", see how search engines see your Web site:
http://www.delorie.com/web/ses.cgi

Reason #14: Search engines could not resolve your DNS name.

There's a mistake that novice users often do. They register a domain name (for example, www.my-great-site.com), and they immediately submit the Web site URL to the search engines.

Then they wonder why the search engines didn't index their site. The answer is, they weren't able to do it.

It takes approximately 2-4 days until a domain name becomes active. All Internet access providers must update their records (DNS tables) to reflect new site locations. The process of updating DNS tables is called propagation.

Search engines must also update their DNS tables and until then, the new domain name www.my-great-site.com doesn't work.

So when you register a new domain name, you must wait about 48-72 hours until you can submit the domain name to the search engines.

If you're interested in other factors that might prevent your Web site from getting top rankings on search engines, visit our newsletter archive:

http://www.Axandra.com/news/index.htm

Reason #15: Your Web site has a low link popularity.

Link popularity is becoming _the_ determining factor for a top search engine ranking.

Link popularity means the number of Web sites linking to your site. However, the quality of links is more important than the quantity of links. For instance, if the New York Times links to your site, their single link might count a lot more than 30 links from your friends' personal homepage.

By now, all top search engines use link popularity in their ranking formulas: AltaVista, Inktomi, MSN Search, HotBot. For Google, it's even the most important factor in ranking sites.

The idea behind link popularity is that other Web sites will link to your site only if you are a quality site offering quality resources. So if many Web sites link to your site, search engines come to the conclusion that your site must be very popular and deserves a high ranking.

As Google co-founder Sergey Brin said in an interview: "...a page that is pointed to by many other sites is important. In other words, external approval raises a page's ranking."

Link popularity can do a lot for your site. Not only will the most important search engines rank you higher, but links from other sites will also drive more traffic to you.

In addition, as more sites link to you, the odds increase that search engine spider programs will encounter your site more regularly so that it's less likely that they drop your site from their index.

What's the link popularity of your site?

The freeware Windows program Link Popularity Check tells you the link popularity of your Web site is and compares it with competitor sites.

Link Popularity Check is a classic freeware application: it has no nag screens, it doesn't change the system registry, it makes no unauthorized connections to the Internet and it comes with a hands-free uninstaller.

Download Link Popularity Check (freeware).

How to improve the link popularity of your Web site:

You can search the search engines for Web sites that are related to your business, find the webmaster's contact information and then solicit a reciprocal link. Do this every day and your link popularity will climb steadily but slowly. And it's very time-consuming.

That's why we created ARELIS, which automates most of the time-consuming tasks. Read the interesting comparison of the old-fashioned way of improving link popularity versus ARELIS, the new way.

Reason #16: Your Web page URL contains special characters.

Most search engines have problems indexing Web pages when their URLs contain special characters. The following special characters are known to be "search-engine-spider-stoppers":

• ampersand (&)
• dollar sign ($)
• equals sign (=)
• percent sign (%)
• question mark (?)

These characters are often found in dynamically generated Web pages. They signal the search engine crawler program that there could be an infinite loop of possibilities for that page. That's why they ignore Web page URLs with the above characters.

AltaVista and Lycos explain on their help pages why they cannot index such Web pages:

• http://insite.lycos.com/searchservices/frequently_asked_questions.asp?b=n#10
  

HotBot recommends that you submit your dynamic Web pages with all parameters added onto the URL (for example, "www.site.com/articles/query.asp?article=83").

Google and Inktomi utilize crawler programs that are able to index dynamically generated Web pages, even those that use question marks.

So what can you do if you have dynamically generated Web pages with special characters? If you use the Apache Server, ASP, CGI/Perl or ColdFusion, the following Web page provides some solutions:
http://spider-food.net/dynamic-page-optimization-b.html

Reason #17: Your Web site has a slow host server.

Search engine crawler programs that index Web pages don't have much time. There are approximately 2-4 billion Web pages all over the world and search engines want to index all of them.

So if the host server of your Web site has a slow connection to the Internet, you may experience that your Web site will not be indexed by the major search engines at all.

AltaVista and Google specifically mention the problem on their Web sites.

AltaVista: "If a site has a slow connection or the pages are very complex, it might time out before the crawler can index all the text."

Found at:
http://help.Altavista.com/adv_search/ast_haw_wellindexed

Google: "Your site may not have been reachable when we tried to crawl it because of network or hosting problems. When this happens, we retry multiple times, but if the site cannot be crawled, it will not be listed in our current index."

Found at:
http://www.Google.com/webmasters/2.html#A3

You may also want to limit the size of your homepage to less than 60K. It'd also benefit the still numerous users that connect to the Internet with a slow modem. For even the casual Internet user, the performance of a Web site can make the difference between pleasure and frustration.

How you can test the speed of your server:
http://web-hosting.candidinfo.com/site-response-speed.asp

Reason #18: Do you use the right keywords? It's important that you use the keywords on which you want to be found in the right places on your Web pages.

Imagine your Web site sells garden flowers. Your HTML source might look like this:

<HEAD>

<TITLE>Garden Flowers at Flower Garden Farm</TITLE>

<META NAME="keywords" CONTENT="garden flowers, flower arrangements, wildflower seeds, seed mixes">

<META NAME="description" CONTENT="Many different garden flowers available for your home, yard, garden and patio.">

</HEAD>

Some search engines (AltaVista, Inktomi) index the META Keywords tag, some (Google, AllTheWeb/FAST) don't.

For the search engines that index the META Keywords tag, it's important that you actually use the keywords of your META Keywords tag on your Web pages.

Some search engines nowadays consider it spam if the keywords of your META Keywords tag don't appear in the body text on your page.

Of course, you know that the most important thing for a good search engine listing is content on your Web site. Search engines need text, text and even more text to index your Web site.

For example, if you sell cars, have articles about cars on your Web site. If you offer package holidays, provide some tourist information about your destinations. Build this content and optimize the text for your target search terms.

In other words, make sure that the content on your Web pages contains the keywords that are important for your site.

And don't forget to use only those words in your META Keywords tag that appear on your Web page.

Reason #19: Have you waited long enough?

The Internet is growing rapidly. According to a study of the NEC Research Institute, most search engines need about 6 months to index a Web site.

Source: NUA Internet Surveys
http://www.nua.com/surveys/index.cgi?f=VS&art_id=905355013&rel=true

The data was collected when there were by far fewer Web sites than today. With todays number of Web sites, it might take even longer (although many search engines have also improved their equipment).

That means that your Web site submissions may need up to half a year until they appear in search engine listings.

The result of the study shows that it's important that you submit your Web pages the right way, right from the start. When search engines need six months to index your Web site, you should make sure that they'll index your pages correctly.

No one can guarantee a good search engine ranking. However, if you don't make the mistakes which we outlined in our 19-part series, a good ranking is much more likely.