Argmax and XGBoost Training over Fully Homomorphic Encryption
Authors: Ramy Masalha (IBM Research and University of Haifa), Adi Akavia (University of Haifa), Allon Adir (IBM Research), Ehud Aharoni (IBM Research), Eyal Kushnir (IBM Research)
Volume: 2026
Issue: 1
Pages: 175–200
DOI: https://doi.org/10.56553/popets-2026-0010
Abstract: Fully Homomorphic Encryption (FHE) is a promising solution to enable privacy-preserving inference and training of machine learning models over encrypted data. Among the machine learning methods used in practice, Extreme Gradient Boosting (XGBoost) is one technique that shines in many applications. While previous works have tackled the problem of training tree-based models over FHE, these works either rely on interaction with the client, which adds the extra burden of communication, or consume a typically unreasonable amount of time to train a large model. In this work, we present an efficient system for a non-interactive XGBoost training over FHE that achieves up to 360 imes speedup compared to the state of the art.
The argmax operation is a basic building block invoked repeatedly during the XGBoost training as well as other machine learning algorithms, but computing it over FHE is time consuming. When utilizing the Single Instruction Multiple Data (SIMD) parallelism capability offered by most FHE schemes and using a configuration with s slots, the state of the art methods compute argmax on n <= s values using either O(log_2 n) SIMD-comparisons in tournament-style comparison or ceil{n^2 /s} SIMD-comparisons using all pairs comparison.
As a second contribution of this work, we propose an efficient argmax algorithm that is based on a novel technique to maximize SIMD-utilization, and computes the argmax of n <= s values using only O(log_2(log_2(n)) SIMD-comparisons. The method extends to n > s with complexity O(n/s) + log_2(log_2(s)), compared to O(n/s) + log_2(s) for state of the art methods. We conduct empirical experiments to compare our method with other existing argmax methods, and show that when using the HEaaN FHE scheme with a configuration of s=2^15 to compute the argmax of n=s values, our implementation is about 1.6 times faster than the state of the art.
Keywords: fully homomorphic encryption, privacy preserving machine learning, XGBoost, argmax
Copyright in PoPETs articles are held by their authors. This article is published under a Creative Commons Attribution 4.0 license.
