Memory usage tweaks
There are some imgproxy options that can help you optimize memory usage and decrease memory fragmentation.
This is an advanced part. Please make sure that you know what you're doing before changing anything.
IMGPROXY_FREE_MEMORY_INTERVAL​
Working with a large amount of data can result in allocating some memory that is generally not used. That's why imgproxy enforces Go's garbage collector to free as much memory as possible and return it to the OS. The default interval of this action is 10 seconds, but you can change it by setting IMGPROXY_FREE_MEMORY_INTERVAL. Decreasing the interval can smooth out the memory usage graph but it can also slow down imgproxy a little. Increasing it has the opposite effect.
MALLOC_ARENA_MAX​
libvips uses GLib for memory management, and it brings with it GLib memory fragmentation issues to heavily multi-threaded programs. And imgproxy is definitely one of these. First thing you can try if you noticed constantly growing RSS usage without Go's sys memory growth is set MALLOC_ARENA_MAX:
MALLOC_ARENA_MAX=2 imgproxy
This will reduce GLib memory appetites by reducing the number of malloc arenas that it can create. By default GLib creates one are per thread, and this would follow to a memory fragmentation.
Using alternative malloc implementations​
If setting MALLOC_ARENA_MAX doesn't show you satisfying results, it's time to try alternative malloc implementations.
jemalloc​
jemalloc is a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support.
Most Linux distributives provide their jemalloc packages. Using jemalloc doesn't require rebuilding imgproxy or it's dependencies and can be enabled by the LD_PRELOAD environment variable. See the example with Debian below. Note that jemalloc library path may vary on your system.
sudo apt-get install libjemalloc2
LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libjemalloc.so.2' imgproxy
Official imgproxy Docker images starting v3.17.0 have jemalloc preinstalled. Its usage can be enabled by setting the IMGPROXY_MALLOC environment variable to jemalloc.
TCMalloc​
TCMalloc is Google's customized implementation of C's malloc() and C++'s operator new used for memory allocation within our C and C++ code. TCMalloc is a fast, multi-threaded malloc implementation.
Most Linux distributives provide their TCMalloc packages. Using TCMalloc doesn't require rebuilding imgproxy or it's dependencies and can be enabled by the LD_PRELOAD environment variable. See the example with Debian below. Note that TCMalloc library path may vary on your system.
sudo apt-get install libtcmalloc-minimal4
LD_PRELOAD='/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4' imgproxy
Official imgproxy Docker images starting v3.17.0 have TCMalloc preinstalled. Its usage can be enabled by setting the IMGPROXY_MALLOC environment variable to tcmalloc.