A Region‐Based Analysis for the Feature Concatenation in Deep Forests

Deep forest is a tree‐based deep model made up of non‐differentiable modules that are trained without backpropagation. Despite the fact that deep forests have achieved considerable success in a variety of tasks, feature concatenation, as the ingredient for forest representation learning, still lacks...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Chinese Journal of Electronics 2022-11, Vol.31 (6), p.1072-1080
Hauptverfasser: LYU, Shen‐Huan, CHEN, Yi‐He, ZHOU, Zhi‐Hua
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Deep forest is a tree‐based deep model made up of non‐differentiable modules that are trained without backpropagation. Despite the fact that deep forests have achieved considerable success in a variety of tasks, feature concatenation, as the ingredient for forest representation learning, still lacks theoretical understanding. In this paper, we aim to understand the influence of feature concatenation on predictive performance. To enable such theoretical studies, we present the first mathematical formula of feature concatenation based on the two‐stage structure, which regards the splits along new features and raw features as a region selector and a region classifier respectively. Furthermore, we prove a region‐based generalization bound for feature concatenation, which reveals the trade‐off between Rademacher complexities of the two‐stage structure and the fraction of instances that are correctly classified in the selected region. As a consequence, we show that compared with the prediction‐based feature concatenation (PFC), the advantage of interaction‐based feature concatenation (IFC) is that it obtains more abundant regions through distributed representation and alleviates the overfitting risk in local regions. Experiments confirm the correctness of our theoretical results.
ISSN:1022-4653
2075-5597
DOI:10.1049/cje.2022.00.178