Real-Time Energy Management for Marine Applications Using Markov Approximation

All-electric and hybrid-electric ships have become the centerpiece for reducing greenhouse gas emissions and improving fuel efficiency in the maritime industry. Real-time power management of multiple power sources in both design and operation becomes critical due to the uncertainty and randomness of the vessel's power demand. Furthermore, the vessel operators may have additional requirements for operational flexibility, such as fully recharging the battery before the next trip and limiting engine switch on/off to reduce wear-and-tear, which lead to complex power management systems requiring domain-expert knowledge. In this paper, a hybrid solution for real-time power management is proposed. The model-based approach encapsulates the power system, and the data-driven approach (a machine learning technique called Markov approximation) handles the uncertainty. First, an offline deterministic optimization problem for power management is formulated. Second, the deterministic problem is transformed into a maximum weighted independent set (MWIS) problem. Next, the Markov approximation framework is applied to the transformed problem to utilize the machine learning techniques widely used in wireless communications and computing industries. Finally, a reinforcement learning-based real-time solution is proposed. Extensive simulations are performed on a ferry case study, and the results are within the proven theoretical bounds of machine learning techniques.