Patches Provide For Much Faster In-Kernel Zstd Due To Embarrassingly Bad Inefficiency
Patches improve Zstd compression and decompression efficiency by probing BMI2 CPU support only once, reducing decompression time by 71% and compression time by 18%.
Intelligence analysis by Qwen 2.5 (3B)
Linux kernel developers have released patches to improve Zstd compression and decompression efficiency by probing BMI2 CPU support only once, reducing decompression time by 71% and compression time by 18%.
The Zstd code in the Linux kernel checks if your CPU supports a special instruction set called BMI2. If it does, it uses a faster method. But it checks this every time, which is slow. The new patches only check once and save time.
Analysis
{"heading":"The Problem with Zstd Context Creation","subheading":"Probing for BMI2 CPU Support","content":["The existing Zstd code in the Linux kernel decides at run-time whether to use the generic code path or the one that makes use of BMI2 instructions for compression and decompression.","BMI2 is a bit manipulation instruction set found in Intel Haswell and newer or AMD Excavator and newer processors.","Probing for BMI2 CPU support is done with two serializing CPUID instructions, which are costly and result in a hot code path."]}
Key points
- Patches improve Zstd efficiency by probing BMI2 CPU support only once.
- Reduces decompression time by 71% and compression time by 18%.
- Addresses redundancy in existing Zstd code.
With these patches, Zstd will be faster and more efficient, especially for tasks like decompressing files.
If the patches are not accepted, Zstd will continue to be slower, which could affect the performance of Linux systems.