Raising Inotify Limits

Syncovery’s real-time synchronization detects changes in the monitored folders as soon as they happen. On Linux and on Linux-based NAS devices, Syncovery 12 may use inotify for this, the file change notification system of the Linux kernel. inotify is used when Syncovery is not running as root and therefore can’t use the more advanced fanotify mechanism, which Syncovery also supports.

Each monitored folder requires one inotify “watch”, and the kernel only allows a limited number of watches. With large folder trees, the default limits are quickly exhausted, and changes in some folders are no longer detected. This page explains how to check the current limits, how to choose suitable values, and how to raise the limits permanently on popular Linux distributions and on Synology NAS devices.

On this page:

What Is inotify and Why Does Syncovery Need It?

The inotify interface is the part of the Linux kernel that informs applications about changes in the file system. When a Syncovery profile uses real-time synchronization, Syncovery asks the kernel to report every change in the monitored folders. This way, new and modified files can be copied within seconds, without scanning the complete folder tree again and again.

However, inotify cannot monitor a folder tree as a whole. One watch is needed for each individual folder, including all subfolders at every level. A folder tree with 150,000 subfolders therefore requires 150,000 watches. The number of files does not matter, because one watch covers all files directly inside its folder.

The kernel limits the inotify resources per user account, and all programs running under the same account share these limits. When a limit has been reached, additional folders simply cannot be monitored. Three kernel settings are involved:

  • fs.inotify.max_user_watches is the maximum number of watches, and therefore of monitored folders, per user. Older kernels use a default of 8,192. Since kernel version 5.11, the default is calculated from the amount of RAM and lies between 8,192 and 1,048,576. On top of that, distributions and some software packages set their own values.
  • fs.inotify.max_user_instances is the maximum number of inotify instances per user. Every program that monitors files opens at least one instance. The default is 128.
  • fs.inotify.max_queued_events is the maximum number of change events that can wait in the queue of an instance. If more changes occur at the same moment, the kernel discards events. The default of 16,384 is usually sufficient. If necessary, you can raise it in the same way as the other two settings.

Signs That the inotify Limit Is Too Low

The following signs indicate that the inotify limits on your system are too low:

  • A real-time profile does not react to changes in some folders, while changes in other folders are processed normally. Often, the affected folders are the ones that were added most recently or that are located deep inside a large folder tree.
  • Syncovery reports that the inotify limit has been reached or that a folder cannot be monitored in real time.
  • Error messages mention No space left on device, even though the disk has plenty of free space. This is how Linux reports that no more watches are available.
  • Error messages mention Too many open files. This can mean that the limit for inotify instances has been reached.
  • Other programs report problems too, because they share the same limits. For example, tail -f reports inotify resources exhausted, and development tools, file managers or cloud sync clients may warn that the file watch limit has been reached.

Checking the Current Limits

To display the current values, open a terminal window or an SSH session and enter:
sysctl fs.inotify

The output looks similar to this:
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 65536

If the system answers with command not found, which happens for regular users on Debian, enter /sbin/sysctl fs.inotify instead, or read the values directly from the kernel:
cat /proc/sys/fs/inotify/max_user_watches /proc/sys/fs/inotify/max_user_instances

Choosing Suitable Values

To find out how many watches Syncovery needs, count the folders that it monitors in real time, including all subfolders. Replace /path/to/folder with the monitored folder:
sudo find /path/to/folder -type d | wc -l

If Syncovery monitors several folders, you can specify all of them in one command:
sudo find /home/user/Documents /srv/data -type d | wc -l

Use the result to choose the new limits:

  • Allow for growth and for other programs running under the same user account. The watch limit should be at least twice the number of folders that you counted.
  • Each watch that is actually in use occupies roughly 1 KB of kernel memory. 100,000 monitored folders therefore need about 100 MB of RAM, and one million folders need about 1 GB. The limit itself does not use any memory.
  • If a current value is already higher than the value you intend to set, keep the current value. Lowering a limit can cause problems for other programs.

For most computers and servers, we recommend these values:

  • fs.inotify.max_user_watches = 1048576, which allows about one million watches. On devices with 2 GB of RAM or less, such as small NAS models or single-board computers, use 524288 instead.
  • fs.inotify.max_user_instances = 1024

Raising the Limits Temporarily

To test whether higher limits solve the problem, you can change them right away. Values set this way are lost when the system restarts:
sudo sysctl -w fs.inotify.max_user_watches=1048576
sudo sysctl -w fs.inotify.max_user_instances=1024

To keep the new values permanently, follow the instructions for your Linux distribution below.

Raising the Limits Permanently on Linux

Practically all current Linux distributions apply the settings from the files in the /etc/sysctl.d/ folder every time the system starts. The most reliable method is to create a separate file for Syncovery in this folder. Many older guides recommend adding the settings to /etc/sysctl.conf instead, but several current distributions no longer read that file at startup.

The files from all sysctl folders are applied in alphabetical order of their names, and a later file overrides the settings of an earlier one. The file name 90-syncovery-inotify.conf used below makes sure that Syncovery’s settings are applied after the usual defaults of the distribution and of installed packages.

Syncovery detects the new limits automatically and does not need to be restarted (see After Raising the Limits).

Ubuntu, Debian, Linux Mint, Raspberry Pi OS and Other Debian-Based Distributions

Create the configuration file with these two commands. The first one creates the file, and the second one adds the second line:
echo 'fs.inotify.max_user_watches = 1048576' | sudo tee /etc/sysctl.d/90-syncovery-inotify.conf
echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.d/90-syncovery-inotify.conf

Apply the settings without restarting:
sudo sysctl --system

Check the result:
sysctl fs.inotify

Notes:

  • Debian 13 (trixie) and newer, as well as Ubuntu 24.10 and newer, no longer apply /etc/sysctl.conf at startup. Settings stored in that file seem to work when you apply them manually, but they are gone after the next restart. The file in /etc/sysctl.d/ works with all versions.
  • On Ubuntu desktops, the GNOME desktop search (Tracker) installs a file such as /usr/lib/sysctl.d/30-tracker.conf, which sets the watch limit to 65,536. Syncovery’s file overrides this value because its name comes later in alphabetical order.
  • If sudo is not set up for your account on Debian, switch to the root account with su - and enter the commands without sudo. As a regular Debian user, check the values with /sbin/sysctl fs.inotify.
  • On a Raspberry Pi or another device with 2 GB of RAM or less, use 524288 for max_user_watches.

Fedora, Red Hat Enterprise Linux, CentOS Stream, Rocky Linux and AlmaLinux

Create the configuration file with these two commands:
echo 'fs.inotify.max_user_watches = 1048576' | sudo tee /etc/sysctl.d/90-syncovery-inotify.conf
echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.d/90-syncovery-inotify.conf

Apply the settings without restarting:
sudo sysctl --system

Check the result:
sysctl fs.inotify

Notes:

  • These steps work with all versions that use systemd, which includes Red Hat Enterprise Linux and CentOS 7 and newer. They also work on the Fedora Atomic Desktops such as Silverblue, because /etc remains writable there.
  • A file that is created directly in /etc/sysctl.d/, as shown above, automatically receives the correct SELinux label. If you create the file in a different folder and move it to /etc/sysctl.d/, it keeps its previous label, and SELinux may prevent it from being read at startup. In this case, restore the correct label:
    sudo restorecon -v /etc/sysctl.d/90-syncovery-inotify.conf

openSUSE and SUSE Linux Enterprise

Create the configuration file with these two commands:
echo 'fs.inotify.max_user_watches = 1048576' | sudo tee /etc/sysctl.d/90-syncovery-inotify.conf
echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.d/90-syncovery-inotify.conf

Apply the settings without restarting:
sudo sysctl --system

Check the result:
sysctl fs.inotify

Notes:

  • These steps apply to openSUSE Leap, openSUSE Tumbleweed and SUSE Linux Enterprise Server.
  • As on the other distributions, put your settings into a file in /etc/sysctl.d/ rather than into /etc/sysctl.conf.

Arch Linux, Manjaro, EndeavourOS and CachyOS

Create the configuration file with these two commands:
echo 'fs.inotify.max_user_watches = 1048576' | sudo tee /etc/sysctl.d/90-syncovery-inotify.conf
echo 'fs.inotify.max_user_instances = 1024' | sudo tee -a /etc/sysctl.d/90-syncovery-inotify.conf

Apply the settings without restarting:
sudo sysctl --system

Check the result:
sysctl fs.inotify

Notes:

  • Arch-based distributions do not read /etc/sysctl.conf at startup. Only files in sysctl folders such as /etc/sysctl.d/ and /usr/lib/sysctl.d/ are applied.
  • If sudo is not installed, log in as root and enter the commands without sudo.

Other Linux Distributions

  • Most other distributions, including those without systemd such as Gentoo with OpenRC or Void Linux, also apply the files in /etc/sysctl.d/ at startup. Create the same file as shown above.
  • If sysctl --system is not supported, for example because your system uses the BusyBox version of sysctl, apply the file directly:
    sudo sysctl -p /etc/sysctl.d/90-syncovery-inotify.conf
  • If your system does not read /etc/sysctl.d/ at startup, add the two lines to /etc/sysctl.conf instead and apply them with sudo sysctl -p.

Raising the inotify Limits on a Synology NAS

Synology’s DSM operating system is based on Linux, but the method described above does not work there. DSM sets the watch limit to its own default of 8,192 while it starts up, so values stored in /etc/sysctl.conf are overwritten at every restart, even though the file still contains them. In addition, DSM updates can replace system files. The solution that has become established in the Synology community is a triggered task in the DSM Task Scheduler that raises the limits again after every startup. Tasks created in the Task Scheduler are preserved when DSM is updated, and you don’t need SSH access to set them up.

Step 1: Create a Triggered Task

  1. Sign in to DSM with an administrator account and open Control Panel > Task Scheduler.
  2. Click Create and choose Triggered Task > User-defined script.
  3. On the General tab, enter a task name such as Raise inotify limits. DSM only accepts letters, numbers and spaces here. Select root as the user and Boot-up as the event, and make sure that Enabled is checked.
  4. Switch to the Task Settings tab and paste the following script into the User-defined script box under Run command:
    # Raise the inotify limits for Syncovery
    WATCHES=524288
    INSTANCES=1024
    echo $WATCHES > /proc/sys/fs/inotify/max_user_watches
    echo $INSTANCES > /proc/sys/fs/inotify/max_user_instances
    # DSM may reset the limits while it starts up, so set them again after 90 seconds
    (
    sleep 90
    echo $WATCHES > /proc/sys/fs/inotify/max_user_watches
    echo $INSTANCES > /proc/sys/fs/inotify/max_user_instances
    ) &
  5. Click OK to save the task. If DSM warns that the script will run with root privileges, confirm the message.

The first two echo commands raise the limits immediately. Because DSM may reset the watch limit to 8,192 while it is still starting up, the commands in parentheses set the limits once more after 90 seconds. They run in the background (the & character at the end), so DSM does not have to wait for them. This delayed approach is the method that has proven itself in the Synology community. The task must run as root, because other accounts, including administrator accounts, are not allowed to change these limits.

Each watch that is in use takes about 1 KB of RAM. The value 524288 used in the script allows about half a million watches, which is a good choice for most Synology models. If you monitor more folders and your NAS has plenty of memory, you can use 1048576 instead.

Step 2: Run the Task Once

You don’t need to restart the NAS to activate the new limits. Select the new task in the list and click Run. The limits are raised immediately. From now on, the task runs automatically every time the NAS starts. Syncovery detects the new limits automatically, so you don’t need to restart the Syncovery package (see After Raising the Limits).

Step 3: Check the New Values

If SSH is enabled on your NAS (Control Panel > Terminal & SNMP), sign in with an administrator account and enter:
cat /proc/sys/fs/inotify/max_user_watches /proc/sys/fs/inotify/max_user_instances

The output should show 524288 and 1024. Without SSH, you can check the values with a second task:

  1. In the Task Scheduler, click Settings, select Save output results, and choose a shared folder for the results.
  2. Click Create > Scheduled Task > User-defined script. Enter a name such as Show inotify limits, select root as the user, and enter the cat command shown above as the script. The schedule does not matter.
  3. Select the new task, click Run, and then choose Action > View Result.

To make sure that the task also works at startup, check the values again after the next restart of the NAS, about two minutes after it has started.

If the Values Are Reset Again

  • Check that the task is enabled and that root is selected as the user.
  • If the watch limit is back at 8,192 after a restart, DSM may need more time to start on your system. Increase the delay in the script, for example to sleep 180.
  • If the values are reset later while the NAS is running, create an additional task with Create > Scheduled Task > User-defined script that runs as root every hour and contains these two lines, with the same values as in the boot-up task:
    echo 524288 > /proc/sys/fs/inotify/max_user_watches
    echo 1024 > /proc/sys/fs/inotify/max_user_instances

Docker Containers and Other NAS Systems

Docker, Podman and LXC Containers

The inotify limits are settings of the Linux kernel, and all containers share the kernel of their host system. If Syncovery runs in a container, raise the limits on the host, for example on the Linux server, on the Proxmox server that runs an LXC container, or on the Synology NAS that runs Container Manager. The limits normally cannot be changed from inside a container: /proc/sys is read-only there, and Docker’s --sysctl option does not accept the fs.inotify settings.

Watches used inside a container count toward the limit of the corresponding user account on the host. Containers that run as root without user namespaces share their watch budget with all other programs running as root on the host.

Other NAS Systems

NAS systems from other vendors, such as QNAP, ASUSTOR or UGREEN, are also based on Linux but handle system files differently. After creating the file in /etc/sysctl.d/ as described for the Linux distributions, check after the next restart of the NAS whether the new values are still active. If they are not, apply the values at every startup with the same echo commands that the Synology script uses, via the startup script mechanism of your NAS, such as autorun.sh on QNAP devices.

After Raising the Limits

Syncovery detects automatically when the limits have been raised, even if it has already reached the old limit while setting up folder monitoring. You don’t need to restart Syncovery, its real-time profiles or the Syncovery package on your NAS.

Changes that were made while a folder was not being monitored are not reported afterwards. Run the affected profiles once manually to synchronize any changes that may have been missed. It is also a good idea to schedule a complete run of every real-time profile, for example once a day, as a safeguard. You will find more tips in our article on Real-Time File Synchronization with Syncovery.

Troubleshooting

The Old Values Return After a Restart

Most likely, another configuration file sets the same parameters and is applied after yours. To list all files that contain inotify settings, enter:
grep -rs inotify /etc/sysctl.conf /etc/sysctl.d /run/sysctl.d /usr/local/lib/sysctl.d /usr/lib/sysctl.d

The command sudo sysctl --system shows all files in the order in which they are applied, and the last value for a setting wins. If a file that is applied later sets lower values, remove the inotify lines from it if it is one of your own files, such as /etc/sysctl.conf. Otherwise, rename Syncovery’s file so that it comes later in alphabetical order, for example to 99-zz-syncovery-inotify.conf. Don’t edit files in /usr/lib/sysctl.d/, because package updates overwrite them.

Syncovery Still Cannot Monitor All Folders

  • Count the folders again as described in Choosing Suitable Values and compare the result with the limit.
  • Remember that the limits apply per user account. All programs running under the same account share the same watches. The following command shows which programs currently use the most watches, with the number of watches, the user account, the program name and the process ID:
    sudo sh -c 'for p in /proc/[0-9]*; do n=$(cat $p/fdinfo/* 2>/dev/null | grep -c "^inotify"); [ "$n" -gt 0 ] && echo "$n $(stat -c %U $p 2>/dev/null) $(cat $p/comm 2>/dev/null) (PID ${p#/proc/})"; done | sort -rn | head -n 20'
  • If Too many open files errors persist after raising max_user_instances, the process may have reached its limit for open files instead (see ulimit -n). For programs that run as a systemd service, this limit is set with the LimitNOFILE= option of the service.

Changes on Network Shares Are Not Detected

The inotify interface only reports changes that pass through the kernel of the computer on which Syncovery runs. If a monitored folder is located on a mounted network share (SMB/CIFS or NFS), changes made by other computers or directly on the server are not reported, regardless of the limits. In this case, run Syncovery on the computer or NAS where the files are stored, or use scheduled runs, for example every few minutes, instead of real-time monitoring.

Undoing the Changes

On Linux, delete Syncovery’s configuration file. The previous values apply again from the next system start:
sudo rm /etc/sysctl.d/90-syncovery-inotify.conf

On a Synology NAS, select the task in the Task Scheduler and choose Action > Delete. DSM uses its default values again from the next startup.

Post Category:Documentation