Access unit and management segment memory operations

A system includes a first memory component having a particular access size associated with performance of memory operations, a second memory component to store a logical to physical data structure whose entries map management segments to respective physical locations in the memory component, wherein each management segment corresponds to an aggregated plurality of logical access units having the particular access size, and a processing device, operably coupled to the memory component. The processing device can perform memory management operations on a per management segment basis by: for each respective management segment, tracking access requests to constituent access units corresponding to the respective management segment, and determining whether to perform a particular memory management operation on the respective management segment based on the tracking.