Include robots.txt by default - Mendix Forum

Include robots.txt by default


Upgrading to Mendix 8 I seem to have lost my robots.txt again. Thus getting errors in my logs.



I would like Mendix to place a robots.txt where it should be. Having these errors and searching for information how to solve them is a waste of every users time.

I also want Mendix to give documentation of where it is located and a few different use cases. My use case is that no robots should crawl anything. That is maybe not the standard one so some more examples are probably good.

Kind regards


2 answers

Nice. Thank you.


Hi Johan,

I would like to inform you that we decided to add the robots.txt to the `theme` folder. The file will allow all crawling by default. If you want to change this to disallow all crawling you should use the text:

User-agent: *
Disallow: /

The slash here will disallow any crawling starting from the root folder.


The robots.txt will be part of the Atlas module from v2.4.2 on (not released yet).