A Robust Study of High-redshift Galaxies: Unsupervised Machine Learning for Characterizing Morphology with JWST up to z ∼ 8

Galaxy morphologies provide valuable insights into their formation processes, tracing the spatial distribution of ongoing star formation and encoding signatures of dynamical interactions. While such information has been extensively investigated at low redshift, it is crucial to develop a robust system for characterizing galaxy morphologies at earlier cosmic epochs. Relying solely on nomenclature established for low-redshift galaxies risks introducing biases that hinder our understanding of this new regime. In this paper, we employ variational autoencoders to perform feature extraction on galaxies at z > 2 using JWST/NIRCam data. Our sample comprises 6869 galaxies at z > 2, including 255 galaxies at z > 5, which have been detected in both the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey Hubble Space Telescope fields and the Cosmic Evolution Early Release Science Survey done with JWST, ensuring reliable measurements of redshift, mass, and star formation rates. To address potential biases, we eliminate galaxy orientation and background sources prior to encoding the galaxy features, thereby constructing a physically meaningful feature space. We identify 11 distinct morphological classes that exhibit clear separation in various structural parameters, such as the concentration, asymmetry, and smoothness (CAS) metric and M 20 , Sérsic indices, specific star formation rates, and axis ratios. We observe a decline in the presence of spheroidal-type galaxies with increasing redshift, indicating the dominance of disk-like galaxies in the early Universe. We demonstrate that conventional visual classification systems are inadequate for high-redshift morphology classification and advocate the need for a more detailed and refined classification scheme. Leveraging machine-extracted features, we propose a solution to this challenge and illustrate how our extracted clusters align with measured parameters, offering greater physical relevance compared to traditional methods.