what (not) to do with your Synology or Linux swap file - RAM issues or no!

The Swap File

Synology Aug 24, 2021

I can't stress this enough - you really, really shouldn't be messing with your swap file!

But I know that people will. And people don't always just want to restart their system to start with an empty swap file every X days or hours. So before you get all up in that, if you find you're running out of swap space and you really don't know why, below are some tips so that you can at least make sure you're not boldly going where everyone has gone before and wisely told you not to follow.

this isn't a definitive list of things to try, and I cannot stress this enough, follow points 5 and 6 at your own risk
  1. Increasing your RAM in DSM does not automatically increase your swap size, but the system now treats swap the same way as if you had increased the swap size. This can lead to overutilization of swap, and impact system performance. If you have increased your RAM since the first time you installed DSM, reinstall DSM with a Mode-2 reset so that the swap file can be re-sized to be appropriate for the new RAM amount (this does not remove your files, but it does reset users and networks, so some additional work will be required to get those settings back to how you had them)
  2. Check your swap and RAM usage by typing free -m (returns sizes in MB) or free -g (for GB). There is a distinction to be made between the 'used' column and the 'available' column. Although your 'free' RAM may be low, check 'used' plus the 'buff/cache'. Whatever is in 'buff/cache' can and will be released whenever necessary
  3. Check the processes which are using your swap file, and see if you can modify their settings to change how often they need to (though be warned, this could affect the performance of those apps and your system)
  4. If you think that swap is still being used too much, you can take a look at changing a parameter called vm.swappiness
  5. If you think your cached RAM is causing you problems, you can try clearing it. There are other methods than the one described on this page, but that's the one least likely to cause you problems
  6. At the very end, if nothing above has had any lasting impact, you can try dumping your swap to RAM which should, in theory, bring your swap down to zero again

Checking RAM and Swap usage

gdochvniqk.png
free -m used to check RAM and Swap usage

We find out our RAM stats in SSH by typing free -m. In our image above we can see that although our RAM (Mem:) is only reporting 3141MB used, you can see that only 347MB is free. That is because the buffer or cache is taking up the other >16GB.


Checking the processes using Swap the most

You can check the top 5 processes using your swap file by typing:

grep VmSwap /proc/*/status | sort -n -r --key=2.1 | head -5

which will give you an output similar to this (I recently cleared my swap file which is why the numbers are so low):

powershell_qyxhlxt0it.png

Breaking down the above, /proc refers to the process, and the numbers /21202 refer to the process ID (PID). This doesn't tell us very much on its own, so to check what processes we're seeing, we can type:

ps [PID]

so by inputting ps 21202 it returns the following:

powershell_zulbw7vd1o.png

and I immediately know that my docker package is using that swap.


Modifying when swap is used

My swap recently was constantly at 99%  even though RAM was less than 50%. I found out that this is because by default, the system is set to start swapping out memory once 40% of total RAM is used.

There is something called vm.swappiness which you can check by typing

sudo sysctl vm.swappiness

By default, this is set to 60, which tells the system to start using swap when 60% or less of RAM remains (another way of saying when 40% or more is in use)

To combat this, you can change this default amount using the following command:

sudo sysctl vm.swappiness=[value between 0 and 100]

where 0 means that no swapping will take place until 0% total RAM remains, and 100 means that swapping will begin immediately.

This however doesn't persist between reboots. To achieve this, follow the following steps:

  • Find the following file in your file system (normally in your user's default folder when you start an SSH session):
/etc/sysctl.conf
  • Nano or vim (edit) the file as root and change or add (if not there) the line:
vm.swappiness=

to the desired value.

For example, vm.swappiness=10 means that swap will not be used until only 10% RAM remains, or in other words, you have 90% utilization of your total RAM.

This does not guarantee that swap will never be used. In fact over time mine continues to creep up even though my RAM usage stays low. This may be related to specific apps/containers I'm using and the way they're set up/not releasing/caching. However I can say it's helped keep that usage a lot lower than it originally was

Clearing the Cache

After checking your usage first and determining that you need to clear your cache, follow the next steps.

We can actually clear cache in a number of ways, but clearing the 'PageCache' is probably the least likely to cause any negative impacts to your system.

You would do this with the following command, from root in SSH (to access root, type sudo su - followed by your password):

sync; echo 1 > /proc/sys/vm/drop_caches

Remember this image from above, before clearing the cache?

gdochvniqk.png
before clearing the cache

By running free -m again, we can see that that although our used RAM is the same, our free RAM has increased significantly, and our buffer/cache has reduced by the same amount. Running our swap script above will now work. You can create a new script schedule to reclaim cached RAM in the same way as well (for various reasons, I wouldn't suggest combining the scripts, rather run them as two separately scheduled commands to trigger within a few minutes of each other).

powershell_lc2tdk3y98.png
after clearing the cache

Clearing the swap file

The below comes with certain risks, and I've encountered broken systems in the past when doing this myself. Technically it should work, and I've had success, but it's not guaranteed.

Only attempt if you a) know what you're doing and b) are likely to be able to fix whatever may go wrong. On Synology Diskstations, restarting the device will likely be enough, but again, proceed at your own risk

In some cases you may want to completely clear your swap file. The following command will do this:

sudo swapoff -a && sudo swapon -a

You can write a script to do this periodically by creating the file

sudo nano /path/to/my/swap2ram.sh

and then pasting this code:

#!/bin/sh

mem=$(LC_ALL=C free  | awk '/Mem:/ {print $4}')
swap=$(LC_ALL=C free | awk '/Swap:/ {print $3}')

if [ $mem -lt $swap ]; then
    echo "ERROR: not enough RAM to write swap back, nothing done" >&2
    exit 1
fi

swapoff -a && swapon -a

To make the file executable, you type:

sudo chmod +x /path/to/my/swap2ram.sh

Then create a scheduled task to run this script at a certain time or event of your choosing.

The above script will only work if you have enough RAM to dump your swap file into. If you do not have enough free RAM, then nothing will happen

If your RAM is actually being used, there's not much you can do except for turning off/disabling processes, or restarting your system.

However, something else to check is cached RAM. Most Synology boxes will normally cache as much free RAM as possible to make the system more responsive, releasing it as it becomes required. This means that although your reported RAM use may be low, say 20%, the actual amount of free RAM may be nearly zero. When we look at our script above, even though we may only be using 2GB of RAM and have swap to dump less than 1GB, if your system is only reporting 300MB free due to cache, then the swapoff and swapon won't run.


PTS

PTS fell down the selfhosted rabbit hole after buying his first NAS in October 2020, only intending to use it as a Plex server. Find him on the Synology discord channel https://discord.gg/vgSq5pcT

Have some feedback or something to add? Comments are welcome!

Please note comments should be respectful, and may be moderated