Calibration algorithm of Hall magnetometer in visible coordinate system

•Method for calibration of magnetometer in visible coordinate system was developed.•Algorithm allows reducing the calibration problem to the solution of linear system.•Single Hall sensor resolution of approximately 1 nT is achieved.•Accuracy in measuring of components of the magnetic field 100 nT was achieved. In this paper we describe a novel method for calibration of vector geomagnetometer in visible coordinate system. We use the edges of the guide frames of the magnetometer as visible axes of the coordinate system. Our method takes into account the sensitive axes nonorthogonality, different sensitivities and offsets of magnetic field transducers. Data from three-component accelerometer and three-component magnetometer is used in the method. We use information about the constancy of the scalar product between gravity vectors and magnetic field vector for finding the calibration parameters. The developed method avoids using of global extremum searching algorithms. To determine the calibration parameters, it is necessary to solve a system of linear equations. The relative error of magnetic field vector measuring in the experiments was less than 0.2%. The described method can be used for mapping of magnetic fields in terrain coordinates.