In the prod-log: 404 - file not found for file: robots.txt

in our logs, there is a message about “404 - file not found for file: robots.txt”. robots.txt is used for informing the crawlers what pages they can inspect. Found previous questions on this: and Where did the robots.txt go? They are no longer in the /theme-directory. Are they no longer needed? If so, how to prevent the error showing up in the log?
1 answers

You can create your own robots.txt file and drop it in the theme folder. Not sure why it’s not included by default anymore, but it should be simple to recreate, and you’ll get to set it according to your application’s needs.