Generate permissive or restrictive robots.txt in the config generator