Robots txt generator

  • Categoría: Web Developer Tools
  • Licencia: Freeware
  • Tamaño: < 1 KB
  • Sistema: w2000 wXP vista
  • Idiomas:
Download

Are you fed up of the spiders that visit your website and consume bandwidth?

Spider robots are usually search engines that look in your website and link the content. But this has a negative side, as they consume a lot of your website`s bandwidth. The reason why a program like Robots txt generator exists is to control and restrict these spiders and their access to your website.

First and foremost Robots txt generator is a web utility, totally free, very small and simple to use (it doesn`t even need installing), that lets you limit the number of consultations a spider can do, per minute or hour, within your website. The program database comes equipped by default with a list of 300 spider bots, but you can add new ones if you want, and also specify different restrictions for each bot.

For example, in Robots txt generator you can let a Google Bot have access, but not a Google Image Bot.

Download

Robots txt generator related downloads

Website X5 Free 11.0.8.31

Download

Create your website just by following five simple steps.

WebSite X5 Evolution 13

Download

Build your website, blog or on-line shop quickly and simply.

PrestaShop 1.6.0.14

Download

Your virtual shop in a few small steps.

Website Access Manager 5.0

Download

A web utility to manage access rights for users of a specific website.