Calibration of fluxgate sensor using least square method and particle swarm optimization algorithm

•Compensation accuracy can be increased by using proposed method.•Using method ellipsoid fitting in the first step, limits the search space in second step.•The risk of falling into local minima is reduced by means of proposed method. In this paper, a two-step method is used to estimate the calibration parameters of a fluxgate sensor. To this end, at first, by dividing the sensor output data obtained in various situations into smaller sets and fitting those to an ellipse using least square method, elliptical parameters and consequently, calibration parameters are approximately calculated for each subset. Then, the mean of parameters obtained in the previous stage is calculated as the initial condition for optimization step. In the second step, by using particle swarm optimization (PSO) algorithm, the parameters are modified with a high precision. To limit the search space and avoiding to trap in a local minimum, known noise is added to pure data to estimate the worst deviation of parameters. Limiting the search space, in addition to reduce the risk of falling in local minima, improves computational speed. The results of simulations and practical implementations show the effectiveness of the proposed method.