Knowing the unknowns: uncertainties in simple estimators of galactic dynamical masses

Abstract The observed stellar kinematics of dispersion-supported galaxies are often used to measure dynamical masses. Recently, several analytical relationships between the stellar line-of-sight velocity dispersion, the projected (2D) or deprojected (3D) half-light radius and the total mass enclosed within the half-light radius, relying on the spherical Jeans equation, have been proposed. Here, we use the APOSTLE cosmological hydrodynamical simulations of the Local Group to test the validity and accuracy of such mass estimators for both dispersion and rotation-supported galaxies, for field and satellite galaxies, and for galaxies of varying masses, shapes and velocity dispersion anisotropies. We find that the mass estimators of Walker et al. and Wolf et al. are able to recover the masses of dispersion-dominated systems with little systematic bias, but with a 1σ scatter of 25 and 23 per cent, respectively. The error on the estimated mass is dominated by the impact of the 3D shape of the stellar mass distribution, which is difficult to constrain observationally. This intrinsic scatter becomes the dominant source of uncertainty in the masses estimated for galaxies like the dwarf spheroidal (dSph) satellites of the Milky Way, where the observational errors in their sizes and velocity dispersions are small. Such scatter may also affect the inner density slopes of dSphs derived from multiple stellar populations, relaxing the significance with which Navarro–Frenk–White profiles may be excluded, depending on the degree to which the relevant properties of the different stellar populations are correlated. Finally, we derive a new optimal mass estimator that removes the residual biases and achieves a statistically significant reduction in the scatter to 20 per cent overall for dispersion-dominated galaxies, allowing more precise and accurate mass estimates.