git.squeep.com Git - akkoma/blob - docs/configuration/optimizing_beam.md

   1 # Optimizing the BEAM
   2
   3 Pleroma is built upon the Erlang/OTP VM known as BEAM. The BEAM VM is highly optimized for latency, but this has drawbacks in environments without dedicated hardware. One of the tricks used by the BEAM VM is [busy waiting](https://en.wikipedia.org/wiki/Busy_waiting). This allows the application to pretend to be busy working so the OS kernel does not pause the application process and switch to another process waiting for the CPU to execute its workload. It does this by spinning for a period of time which inflates the apparent CPU usage of the application so it is immediately ready to execute another task. This can be observed with utilities like **top(1)** which will show consistently high CPU usage for the process. Switching between procesess is a rather expensive operation and also clears CPU caches further affecting latency and performance. The goal of busy waiting is to avoid this penalty.
   4
   5 This strategy is very successful in making a performant and responsive application, but is not desirable on Virtual Machines or hardware with few CPU cores. Pleroma instances are often deployed on the same server as the required PostgreSQL database which can lead to situations where the Pleroma application is holding the CPU in a busy-wait loop and as a result the database cannot process requests in a timely manner. The fewer CPUs available, the more this problem is exacerbated. The latency is further amplified by the OS being installed on a Virtual Machine as the Hypervisor uses CPU time-slicing to pause the entire OS and switch between other tasks.
   6
   7 More adventurous admins can be creative with CPU affinity (e.g., *taskset* for Linux and *cpuset* on FreeBSD) to pin processes to specific CPUs and eliminate much of this contention. The most important advice is to run as few processes as possible on your server to achieve the best performance. Even idle background processes can occasionally create [software interrupts](https://en.wikipedia.org/wiki/Interrupt) and take attention away from the executing process creating latency spikes and invalidation of the CPU caches as they must be cleared when switching between processes for security.
   8
   9 ## VPS Provider Recommendations
  10
  11 ### Good
  12
  13 * ????
  14
  15 ### Bad
  16
  17 * AWS (known to use burst scheduling)
  18
  19
  20 ## Example configurations
  21
  22 Tuning the BEAM requires you provide a config file normally called [vm.args](http://erlang.org/doc/man/erl.html#emulator-flags). If you are using systemd to manage the service you can modify the unit file as such:
  23
  24 `ExecStart=/usr/bin/elixir --erl '-args_file /opt/pleroma/config/vm.args' -S /usr/bin/mix phx.server`
  25
  26 Check your OS documentation to adopt a similar strategy on other platforms.
  27
  28 ### Virtual Machine and/or few CPU cores
  29
  30 Disable the busy-waiting. This should generally only be done if you're on a platform that does burst scheduling, like AWS.
  31
  32 **vm.args:**
  33
  34 ```
  35 +sbwt none
  36 +sbwtdcpu none
  37 +sbwtdio none
  38 ```
  39
  40 ### Dedicated Hardware
  41
  42 Enable more busy waiting, increase the internal maximum limit of BEAM processes and ports
  43
  44 **vm.args:**
  45
  46 ```
  47 +P 16777216
  48 +Q 16777216
  49 +K true
  50 +A 128
  51 +sbt db
  52 +sbwt very_long
  53 +swt very_low
  54 +sub true
  55 +Mulmbcs 32767
  56 +Mumbcgs 1
  57 +Musmbcs 2047
  58 ```
  59
  60 ## Additional Reading
  61
  62 * [WhatsApp: Scaling to Millions of Simultaneous Connections](https://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf)
  63 * [Preemptive Scheduling and Spinlocks](https://www.uio.no/studier/emner/matnat/ifi/nedlagte-emner/INF3150/h03/annet/slides/preemptive.pdf)
  64 * [The Curious Case of BEAM CPU Usage](https://stressgrid.com/blog/beam_cpu_usage/)