The robots.txt file tells search engines which parts of your website they can crawl and which they cannot. It goes in the root folder and it is the first thing Google's and other search engines' robots read when they visit your site.
With this generator you create it in a moment: choose whether robots are allowed, the crawl delay, the address of your sitemap and the folders you want to block. Then copy the result into a file called robots.txt and upload it to the root of your website.
