Deep reinforcement learning for cognitive sonar

Current developments in cognitive sonar have leveraged explicit semantic representations such as ontologies to develop cognitive agents that enable adaptation of sonar settings to optimize behaviors such as waveform selection during active tracking. However, such cognitive systems based on explicit knowledge representations can be time-and-labor intensive to update in operation and limited in ultimate performance. In other applications, such as computer Go, breakthrough performance was achieved by going beyond learning from all prior games and actions of experts to allowing the algorithm to compete with itself to develop and learn from the outcome of new approaches. This hybrid approach of learning from experts and then learning via self-competition is vital for sonar applications such as active antisubmarine warfare (ASW) because ground-truthed performance data are sparse and may not display optimal solutions. Here, we discuss our application of reinforcement learning to active ASW in a simulated environment based on computational-acoustics models, including an assessment of the goal states and performance metrics we have used. The fidelity of the simulated environment is discussed in terms of the interaction with reinforcement learning and the impact on generalization from learning in simulated environments to application in real environments. [Work supported by ARiA IR&D.]