Indexing

Indexing — is a database search system that is going robotic spiders and contains information about all known Internet resources in the form of text and multimedia structure. The process of making a website into a database is called indexing.

Indexing is necessary for efficient search of information in the network.

There are two ways of influencing the process of indexing your site:

• Using external links from popular sites or internal links from indexed pages;
• Manually – by filling and sending a special form.

To prevent indexing of some pages, user can use the file robots.txt, the file will not allow the crawler indexing of some pages.