Invariant Random Forest: Tree-Based Model Solution for OOD Generalization
Out-Of-Distribution (OOD) generalization is an essential topic in machine learning. However, recent research is only focusing on the corresponding methods for neural networks. This paper introduces a novel and effective solution for OOD generalization of decision tree models, named Invariant Decisio...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Out-Of-Distribution (OOD) generalization is an essential topic in machine
learning. However, recent research is only focusing on the corresponding
methods for neural networks. This paper introduces a novel and effective
solution for OOD generalization of decision tree models, named Invariant
Decision Tree (IDT). IDT enforces a penalty term with regard to the
unstable/varying behavior of a split across different environments during the
growth of the tree. Its ensemble version, the Invariant Random Forest (IRF), is
constructed. Our proposed method is motivated by a theoretical result under
mild conditions, and validated by numerical tests with both synthetic and real
datasets. The superior performance compared to non-OOD tree models implies that
considering OOD generalization for tree models is absolutely necessary and
should be given more attention. |
---|---|
DOI: | 10.48550/arxiv.2312.04273 |