Raspberry Pi and other single-board computers (SBCs) have always had minimal memory. At a maximum of 1GB in the Raspberry Pi 3, it was just sufficient to run lightweight programs and scripts. That trend held until the advent of the Raspberry Pi 4 with 2GB, 4GB and a few months later, 8GB models, opening doors to running databases and other memory intensive applications.

This shift is due in no small part to the explosive growth of the data industry, where previously the focus of SBCs was IoT and embedded applications and now the focus is slowly shifting towards data applications. Memory has also become cheaper over the years, allowing newer SBCs to have more memory while maintaining a similar price range.

My concerns

With the Raspberry Pi 3 trending towards obsolescence, I got worried about how my 28-core Kraken cluster can stay relevant and useful for at least another year.

On the other hand, I'm secretly looking forward to deprecating the Kraken cluster as I've run into countless awkward situations where the system runs out of memory and all SSH sessions lock up while the kernel struggles for a good 10 minutes before panicking and forcing a reboot.

In fact, that happens to me about once a week! If you've run a good number of applications on the Raspberry Pi 3 before, I'm sure you share my pain.

After a round of excavation in the depths of the Raspberry Pi forums, I found a glimmer of hope for the issues of impending obsolescence and restrictive memory capacity of the Raspberry Pi 3.

The solution is zram swap.  
StuartIanNaylor/zram-swap-config
Replacement for broken zram-config-0.5 package. Contribute to StuartIanNaylor/zram-swap-config development by creating an account on GitHub.

In this piece, I'll demonstrate how you can use zram to increase the available memory by about 50% with negligible performance impact.

What is zram?

zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk, but with on-the-fly "disk" compression. The block device created with zram can then be used for swap or as general-purpose RAM disk. (Wikipedia)

In English, the idea behind zram is basically setting aside a portion of your available memory, using it as a block device à la conventional disk storage and enforcing compression/decompression on all data being written/read from it.

Is there a performance hit?

With great power comes great responsibility

Although this widely-abused Spider-Man quote applies in most cases, this is an exception. Indeed, it is generally true that whenever compression is used, there would be increased computational requirements. But compression is a mature technology that has seen advancements over the years, that the additional compute load is minimal in modern-day processors, allowing for real-time applications such as in random-access memory.

In the case of SBCs such as the Raspberry Pi 3, you are more likely to be memory-bound rather than CPU-bound in most self-hosting scenarios as self-hosted apps tend to be relatively CPU-light and memory-heavy. With 4 cores available and only 1 GB of RAM, you're unlikely to sustain full utilization all 4 cores for extended periods of time without first running out of memory.

Given that there is always some CPU resource available, increasing the memory at the cost of increased CPU utilization is definitely not going to produce a noticeable performance impact.

What can you get out of it?

Memory bandwidth of Raspberry Pi devices from Zero thru 4 (Source)
Compression algorithm Ratio Compression Decompress.
zstd 1.3.4 -1 2.877 470 MB/s 1380 MB/s
zlib 1.2.11 -1 2.743 110 MB/s 400 MB/s
brotli 1.0.2 -0 2.701 410 MB/s 430 MB/s
quicklz 1.5.0 -1 2.238 550 MB/s 710 MB/s
lzo1x 2.09 -1 2.108 650 MB/s 830 MB/s
lz4 1.8.1 2.101 750 MB/s 3700 MB/s
snappy 1.1.4 2.091 530 MB/s 1800 MB/s
lzf 3.6 -1 2.077 400 MB/s 860 MB/s

In the table above (Github), we see the compression ratio and performances of each compression algorithm. With those performance numbers, we can easily see that majority of the compression algorithms qualify for real-time usage as their Compression/Decompression throughput exceeds or are close to the maximum memory Write/Read throughput of all Raspberry Pi models, respectively.

Even in the most conservative case, just using compression alone would yield an additional 108% of space, and even that estimate itself is conservative. This is because the tests assume a mixture of data types to be compressed, non-compressible binary data as well as text. In memory, most of the data can be safely assumed to be text so the real-world space yield is expected to be vastly larger.

Example

Assuming that 512MiB of RAM was set aside for zram swap, and assuming a conservative compression ratio of 2, then we'd stand to gain 500MB additional memory.

For an RPi 3, that brings the total memory to a relatively comfortable 1.5GiB!

Installing zram-swap-config

Enough idle talk, let's get down to business. There are just a handful of steps required to install zram-swap-config.

$ sudo apt-get install git
Ensure that git is installed
$ git clone https://github.com/StuartIanNaylor/zram-swap-config \
&& cd zram-swap-config
Clone the zram-swap-config repository
$ chmod +x install.sh && sudo ./install.sh
Install zram-swap-config
$ cd .. && rm -r zram-swap-config
Cleanup installation files

Configuring zram-swap-config

zram-swap-config comes installed with sane defaults. But if you're the kind that absolutely cannot sleep without tinkering with every knob and lever available, you're in luck. Here's how you can tweak it.

Edit /etc/zram-swap-config.conf and modify the following parameters to your liking.

Parameter Explanation
MEM_FACTOR Percentage of total system RAM to allocate to zram swap devices. Memory will then be allocated equally among all zram swap devices.
DRIVE_FACTOR Estimated compression ratio multiplied by 100.
COMP_ALG Compression algorithm to use.
SWAP_DEVICES Number of individual zram devices drives to provision. Each device supports multiple streams.

1 large drive is generally better.
SWAP_PRI Swap priority for each zram device.

Priority value should be well above other swap drives.
PAGE_CLUSTER (Expressed as the exponent of 2)
Number of page fetches to accumulate before performing an actual batched page fetch from the swap device.

E.g A value of 3 fetches in batches of 8. 0 forces single-page fetches, which can help reduce latency
SWAPPINESS How aggressive the kernel will swap memory pages. Higher values will increase aggressiveness, lower values decrease the amount of swapping.

E.g. A value of 100 treats zram swap as though it is free physical memory itself but runs the risk of accumulating load at high CPU usage. 80 is optimal.

Note: Numeric values in the configuration are integer only.

Generally, there is no need to concern yourself with any setting other than MEM_FACTOR, DRIVE_FACTOR and COMP_ALG.

Estimated compression ratios for configuring DRIVE_FACTOR (Source):

Compressor Ratio Compression Decompression
zstd 1.3.4 -1 2.877 470 MB/s 1380 MB/s
zlib 1.2.11 -1 2.743 110 MB/s 400 MB/s
brotli 1.0.2 -0 2.701 410 MB/s 430 MB/s
quicklz 1.5.0 -1 2.238 550 MB/s 710 MB/s
lzo1x 2.09 -1 2.108 650 MB/s 830 MB/s
lz4 1.8.1 2.101 750 MB/s 3700 MB/s
snappy 1.1.4 2.091 530 MB/s 1800 MB/s
lzf 3.6 -1 2.077 400 MB/s 860 MB/s

Caveat: Compression algorithm availability

Looking at the compression algorithms table above, one would easily surmise that zstd gives the best balance of performance and compression ratio. However, I'm afraid I would have to disappoint here, as zstd like many other compression algorithms listed, is not necessarily available in your Linux distribution.

To check the compression algorithms available for use, run the following command:

$ modprobe zram && cat /sys/block/zram0/comp_algorithm
lzo [lz4] deflate
Command and output showing available compression algorithms

The output would show available compression algorithms as well as the currently selected compression algorithm enclosed by the square braces.

My configuration for Raspbian on Raspberry Pi 3B

MEM_FACTOR=25
DRIVE_FACTOR=300
COMP_ALG=lz4
SWAP_DEVICES=1
SWAP_PRI=75
PAGE_CLUSTER=0
SWAPPINESS=80
My configuration file for zram-swap-config
  • MEM_FACTOR was set to 25, meaning 256MiB of memory will be set aside for zram swap.
  • DRIVE_FACTOR was set to 300, meaning that I'm estimating the net compression ratio to be 3.00 (still relatively conservative).
  • COMP_ALG was set to lz4 as it is the best among the available algorithms in Raspbian lzo, lz4 and deflate.

I found these configuration values to be optimal for my setup but if you're feeling adventurous or lucky, by all means, increase the DRIVE_FACTOR to 400.

If you're not using Raspbian, then there's also a good chance that you have other compression algorithms available at your disposal, so do check and tweak away!

While you're at it...

If you're installing zram-swap-config, you might want to disable swap on your Raspberry Pi.

Generally, regardless of whether you use zram swap, it's always advisable to disable the default swap installed by Raspbian due to a couple of reasons:

  1. It reduces the life expectancy of your SD card as a swap device is subject to heavy and frequent writes.
  2. It yields extremely poor performance given the abysmal random I/O performance of SD cards in general.
$ dphys-swapfile swapoff \
&& dphys-swapfile uninstall \
&& update-rc.d dphys-swapfile disable
Command to disable default swap device

With that you're all set for the next year or so with your Raspberry Pi 3Bs!

What's next?

Next up, I'll be introducing log2ram, a way to write logs to memory to reduce SD card wear and not to mention, it works in conjunction with zram-swap-config as well. See you around!