Several tips on huge websites indexing

Today I want to talk about pushing big website`s pages to the Google index. I mean websites that are really huge.

Usually when you start your internet project with several pages and it is consistently growing for the first year to the thousand of pages, there are no any problems with indexing. But what if you need Google to index about 100000 or even a million pages of your website? That is really something else. In this case you should be much more careful and attentive creating the categorical structure and content of your website. There are several things you should avoid pushing big websites to the Google index.

1. You should avoid the duplicate content within your website. In this post you may find the example of its appearing even if you write all your text content by yourself and also there is described Wordpress based example of struggle with duplicate content.
2. High site growth rate. If you create about 50000 of pages for the first week of your website existence it could be considered as the automatic content generation techniques application. As a result you may get penalty from Google.
3. Don`t use session IDs into your urls structure. For user identification purposes better try cookies.
4. Avoid having duplicate TITLE and DESCRIPTION metatages on different pages of your website. Always remember, that the metatags should be unique (as far as the text content).

And now I`m going to list several things you should implement to speed up your big website indexing.

1. Create xml and html sitemaps. Actually you may use even several sitemaps (each for every big section of your website).
2. Make your website fast loaded for SE spider. Remove all the unnecessary html tags from the code. In this post I described services for checking site loading speed.
3. Make the page amount growth substantial. Add the equal amount of pages every day. Eventually you may increase the amount.
4. Create the strict categorical site structure to make it easier for SE spider to «understand» it.
5. Get backlinks to your main page, html sitemaps and main sections pages from frequently updated blogs, popular forums and social bookmarks.

Using my tips you`ll have more chances to make your big websites completely indexed by Google. But still you need to conduct your own experiments to get your «first million» into the Google index.

Tags: indexing

This entry was posted on Friday, February 12th, 2010 at 6:27 pm and is filed under Seo tips. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.