Learning Neural Eigenfunctions for Unsupervised Semantic Segmentation
Unsupervised semantic segmentation is a long-standing challenge in computer vision with great significance. Spectral clustering is a theoretically grounded solution to it where the spectral embeddings for pixels are computed to construct distinct clusters. Despite recent progress in enhancing spectr...
Gespeichert in:
Hauptverfasser: | , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | Unsupervised semantic segmentation is a long-standing challenge in computer
vision with great significance. Spectral clustering is a theoretically grounded
solution to it where the spectral embeddings for pixels are computed to
construct distinct clusters. Despite recent progress in enhancing spectral
clustering with powerful pre-trained models, current approaches still suffer
from inefficiencies in spectral decomposition and inflexibility in applying
them to the test data. This work addresses these issues by casting spectral
clustering as a parametric approach that employs neural network-based
eigenfunctions to produce spectral embeddings. The outputs of the neural
eigenfunctions are further restricted to discrete vectors that indicate
clustering assignments directly. As a result, an end-to-end NN-based paradigm
of spectral clustering emerges. In practice, the neural eigenfunctions are
lightweight and take the features from pre-trained models as inputs, improving
training efficiency and unleashing the potential of pre-trained models for
dense prediction. We conduct extensive empirical studies to validate the
effectiveness of our approach and observe significant performance gains over
competitive baselines on Pascal Context, Cityscapes, and ADE20K benchmarks. |
---|---|
DOI: | 10.48550/arxiv.2304.02841 |