Indicator designing for performance evaluation of collision avoidance algorithms programs on autonomous ships

This research aims at supporting performance evaluation on the collision avoidance algorithms programs that are to be implemented on autonomous ships. A comprehensive set of 13 indicators have been developed, comprising 3 indicators for mission efficiency, 3 indicators for compliance with the COLREGs (International Regulations for Preventing Collisions at Sea), and 3 indicators for chart challenges. The establishment of these indicators involves several preparatory steps, a pressure ratio formula is proposed, the ship maneuverability is considered, the entire trajectory is clipped into segments according to various encounter types, and the sheltered DCPA issue is repaired. To validate the effectiveness of the proposed indicators, simulation experiments are firstly conducted in 4 different types of chart environments: open waters, coastal areas, archipelagos, and crowded waters. After that, a set of 648 experiment cases on various collision avoidance algorithms are analyzed, Finally, additional experiments on 5 ship models with different lengths are carried out. The results demonstrate the feasibility and the generalizability of the proposed indicators. •13 indicators are developed to support evaluating performance of collision avoidance algorithm programs on autonomous ships.•Preparations include pressure curve, action ranking, maneuverability constraint, track clipping and sheltered DCPA.•Experiments are conducted in 4 types of maps, 3 different collision avoidance mechanisms and 5 simulated ship models.