The general notion of indexing
Before the web- site to appear in search results , you should make sure that it has been indexed by search engines. Indexing web- site is that the crawler visits the created site , analyzes it , and then enters the information on the internet resource database in a particular search engine.
All pages that are included in the search index , on a particular search query are displayed in the search results. If a page is not indexed , then the user will not see it in the search results , as the search engine is not about her any information.
As search engine learns about the site?
Already at the stage of creating your own website there is a question , as the search engines learn about a new resource on the Internet. Report a new web- site search engine in two ways:
1. Adding the site address manually via a standard form search engine. In this web- site in the queue for its indexing by the search robot . When you manually add a site , be aware that you need only make your homepage Internet resource. The remaining pages crawler find the links to them.
2 . Automatically add site search robot . The robot to find the site yourself , you need to with indexed resources led to the new website external link . This method is best recognized by experts in particular, time-consuming . Received some inbound links , you can just wait for the robot visiting the site . If the resource is added manually, by indexing your site may take longer . Typically , the process takes up to two weeks (the time depends on the indexation of the search engine ) .
Friendliness web- site for search engines
A large number of web- sites successfully tested indexing by search engines. But to protect yourself from unpleasant surprises, worth knowing about certain things that take place during site creation .
At the stage of development of the site you need to make sure that he was friendly to search engine robots . To achieve this , you need to consider these criteria :
1. Each web- page should be accessible via links from the main page for a maximum of three transitions . If the structure does not allow a resource developed to follow this rule , then you need to create a web- site map .
2 . When using the site navigation, which is implemented through scripts , should all references to duplicate the traditional way.
3 . Particular attention should be paid to the amount of pages. Search engines only index of from 100 up to 200 KB of text on a web- page. If the pages have a larger volume , while search engines index the only first 200 KB . If you want the page to be indexed as a whole, stands to make sure that its volume does not exceed 100 KB.
Manage search robot
When indexing site on the behavior of search robots can be influenced . This is done in three ways :
- Explicit permission for indexing certain pages using the file robots.txt;
- A ban on indexation of certain web- site pages using robots.txt;
- Closing for indexing search robot parts resource pages with the tag «NOINDEX» ( method is suitable only for Russian search engines ) .
Database of all search engines are updated regularly , this leads to the fact that the number of pages indexed web- site may change from time to time. One of the reasons for the disappearance of the web- page of the index is often no access to it during the unavailability of the server. When the server operation is resumed , then the site and its pages reappear in the search engines index . At the same time , the more external references to a resource , the faster the web- site passes reindexing .
To independently track the process of indexing the Internet resource worth analyzing the log files on the server , where there is information about each visit crawlers systems.
