git.squeep.com Git - akkoma/blob - docs/administration/CLI_tasks/robots_txt.md

   1 # Managing robots.txt
   2
   3 {! backend/administration/CLI_tasks/general_cli_task_info.include !}
   4
   5 ## Generate a new robots.txt file and add it to the static directory
   6
   7 The `robots.txt` that ships by default is permissive. It allows well-behaved search engines to index all of your instance's URIs.
   8
   9 If you want to generate a restrictive `robots.txt`, you can run the following mix task. The generated `robots.txt` will be written in your instance [static directory](../../../configuration/static_dir/).
  10
  11 ```elixir tab="OTP"
  12 ./bin/pleroma_ctl robots_txt disallow_all
  13 ```
  14
  15 ```elixir tab="From Source"
  16 mix pleroma.robots_txt disallow_all
  17 ```