TY - GEN

T1 - Environment-Agnostic IRM via Unsupervised Clustering and Adaptive Penalty Scaling

AU - Miron, Bratenkov

AU - Bondarenko, Ivan

N1 - Conference code: 27

PY - 2026

Y1 - 2026

N2 - Generalization under data shifts remains a critical challenge in machine learning. The invariant risk minimization (IRM) paradigm enhances robustness by searching invariant features across environments, but its practical application is constrained by the need to predefine environment. To overcome this limitation, we propose a clustering-based method that enables IRM training without prior environment knowledge by treating clusters as environments. Our experiments show that this approach improves model robustness to data shifts compared to empirical risk minimization (ERM). Specifically, on a weather prediction task, the mean squared error (MSE) was reduced by 10%, while in a language modeling task involving long texts, perplexity improved by up to 75%. Additionally, we introduce an adaptive hyperparameter tuning strategy for the IRM penalty term, which stabilizes training and further enhances robustness. This adaptive IRM achieves an additional 10% MSE improvement for weather prediction and a 460% perplexity gain on long textual inputs compared to classical IRM. An analysis of linear dependence between input variables and targets reveals that adaptive IRM encourages learning more complex, nonlinear invariant features, which underpins its superior generalization under distributional shifts. These results demonstrate that combining environment discovery via clustering with adaptive IRM substantially improves model generalization under distributional shifts.

AB - Generalization under data shifts remains a critical challenge in machine learning. The invariant risk minimization (IRM) paradigm enhances robustness by searching invariant features across environments, but its practical application is constrained by the need to predefine environment. To overcome this limitation, we propose a clustering-based method that enables IRM training without prior environment knowledge by treating clusters as environments. Our experiments show that this approach improves model robustness to data shifts compared to empirical risk minimization (ERM). Specifically, on a weather prediction task, the mean squared error (MSE) was reduced by 10%, while in a language modeling task involving long texts, perplexity improved by up to 75%. Additionally, we introduce an adaptive hyperparameter tuning strategy for the IRM penalty term, which stabilizes training and further enhances robustness. This adaptive IRM achieves an additional 10% MSE improvement for weather prediction and a 460% perplexity gain on long textual inputs compared to classical IRM. An analysis of linear dependence between input variables and targets reveals that adaptive IRM encourages learning more complex, nonlinear invariant features, which underpins its superior generalization under distributional shifts. These results demonstrate that combining environment discovery via clustering with adaptive IRM substantially improves model generalization under distributional shifts.

KW - Domain shift

KW - ERM

KW - Empirical Risk Minimization

KW - IRM

KW - Invariant Risk Minimization

KW - Machine learning

KW - OOD

KW - Out of distribution

UR - https://www.scopus.com/pages/publications/105020042436

UR - https://www.mendeley.com/catalogue/bf16e3e5-aa7d-3de8-88e9-aa6ab49b3d9e/

U2 - 10.1007/978-3-032-07690-8_5

DO - 10.1007/978-3-032-07690-8_5

M3 - Conference contribution

SN - 978-3-032-07689-2

T3 - Studies in Computational Intelligence

SP - 47

EP - 63

BT - Advances in Neural Computation, Machine Learning, and Cognitive Research IX

A2 - Kryzhanovsky, Boris

A2 - Dunin-Barkowski, Witali

A2 - Redko, Vladimir

A2 - Tiumentsev, Yury

A2 - Klimov, Valentin V.

PB - Springer

T2 - XXVII International Conference on Neuroinformatics

Y2 - 20 October 2025 through 24 October 2025

ER -