| Title | : | Selective Memory Compression for GPU Memory Oversubscription Management |
| Speaker | : | Abdun Nihaal (IITM) |
| Details | : | Fri, 17 Mar, 2023 3:00 PM @ SSB MR1 |
| Abstract: | : | Modern discrete GPUs support Unified Virtual Memory (UVM) which abstracts away the presence of multiple memories and presents a single unified address space. This makes programming GPUs easier. To support UVM, the GPU driver (executing on the CPU) remotely performs on-demand page migration between the CPU and GPU memories. Due to this fault-driven migrations and remote fault handling, UVM underperforms. When memory is oversubscribed, i.e. when the working set of the applications exceeds the available GPU memory capacity, the UVM overheads increase drastically due to page thrashing. In this work, we explore GPU memory compression to alleviate UVM overheads during oversubscription. GPU workloads exhibit frequent large volume writes, which can cause page overflows in a compressed memory system. We find that, compared to CPU workloads, page overflows are more frequent in GPU workloads. Handling page overflows involves costly page remapping and migration, and so frequent page overflows pose a challenge for GPU memory compression.
We propose Selective Memory Compression (SMC), a GPU memory compression system that selectively compresses read-only pages to increase the effective memory size while avoiding costly page remappings due to page overflows. Instead of full memory compression, SMC performs compression only during memory pressure to minimize overheads due to compression. In addition, we also propose Split Linearly Compressed Pages (SLCP), a line packing scheme for compressed pages, that minimizes unused space, gives better compressibility and eliminates extra memory accesses performed to fetch metadata in a compressed memory system. We evaluate by implementing the techniques in the GPGPU-sim UVM Smart simulator. We show that SMC together with SLCP is effective in reducing thrashing and improving performance of UVM applications during oversubscription. Under 125% and 150% oversubscription, SMC combined with SLCP gives 43% and 61% performance improvement, respectively, over a baseline that uses the state-of-the-art eviction policy. |
