A Practical Use Case of Page Swapping
Recently, I had to deploy a React project on a Linode server. What I didn't know was that the server only has 1GB of RAM. When I tried npm run build I got the following error.
Node's Memory Problem
Node is notorious for having a lot of poorly written modules. If the module's writer does not know how Node leverages memory, then the modules consume a lot of RAM. So what looks like a simple application can in fact hog up all the memory during build time.
A bit of Googling shows that Node allows V8's GC heap to be controlled with the --max-old-space-size flag. So I set the heap memory using node --max-old-space-size=4096 $(which npm) run build. Sadly, I still got an out-of-memory error, so I ran sudo swapon --show to see what is the problem.
NAME TYPE SIZE USED PRIO /dev/sdb partition 512M 179.7M -2
Turns out that I only have 512GB of extra swap space, so setting the max RAM size to 4GB does nothing at all.
Fortunately, memory paging exists to deal with such issues. Memory paging retrieves blocks of memory called pages from secondary storage. The pages are then used as the main memory, which gives the illusion that the machine has more RAM.
First, I run sudo swapoff /dev/sdb to disable the the memory for paging.
I then run the following command to create the partition.
sudo dd if=/dev/zero of=/mnt/swap bs=1024 count=5242880
DD copies data from the /dev/zero file to /mnt/swap drive to be used as paged memory. Note that unlike /dev/null, /dev/zero can be used as a data source. bs specifies that the command writes in chunks of 1024 bytes at a time, for a total of 5242880 times. This effectively writes 5GB of data from the source to the destination.
Then, /mnt/swap needs to be set up as a swap area.
sudo mkswap /mnt/swap
Permissions for /mnt/swap need to be changed before using it as a page memory.
sudo chown root:root /mnt/swap
This changes the ownership of the /mnt/swap drive to the root user.
Finally, run sudo chmod 0600 /mnt/swap. This enables the drive to be read and written, but not executed. Ru ls -a /mnt/swap to verify that the changes have taken place.
-rw------- 1 root root 5368709120 Sep 22 16:08 /mnt/swap
Having done all that, sudo swapon --show gives me a 5GB of RAM to work with.
NAME TYPE SIZE USED PRIO /mnt/swap file 5G 121.4M -2
npm run build then proceeds smoothly.
File sizes after gzip: 77 kB build/static/js/main.be9bdc35.js 7.1 kB build/static/css/main.64e6a856.css The build folder is ready to be deployed.
Permanent Swap File
The changes are only valid for the current session, so upon reboot the swap settings would not be retained. This behavior can be changed by adding the swap file to the /etc/fstab file.
fstab is the Linux's filesystem's configuration table that makes mounting and unmounting files easier. Adding a file into this table allows the file to be automatically mounted upon boot.
echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab
Look within fstab to make sure that the changes have been applied.
# <file system> <mount point> <type> <options> <dump> <pass> /dev/sda / ext4 errors=remount-ro 0 1 /dev/sdb none swap sw 0 0 /mnt/swap swap swap defaults 0 0
The post examined a problem that I encountered when trying to build a project on a server with limited RAM. By using page swapping, additional memory can act as the RAM, which allows the machine to perform tasks that it otherwise cannot. Instead of using a more powerful machine, the available resources on a machine should always be used to the fullest.