Do-Hyun Chun, "A Method for Collision Avoidance of a Ship Based on Reinforcement Learning in Complex Maritime Situations", Ph.D. Thesis, Seoul National University, 2024.02.29
Ph.D. Thesis
2025.06.05 11:22
Do-Hyun Chun, "A Method for Collision Avoidance of a Ship Based on Reinforcement Learning in Complex Maritime Situations", Ph.D. Thesis, Seoul National University, 2024.02.29
조회 수 777
| 첨부 '1' |
|---|
| Abstract | As shipping routes become increasingly complex, the number of human-caused ship accidents (such as by navigator inattention) is also increasing. Especially where many ships operate in small areas, there is a growing need for an autonomous navigation system for a maritime autonomous surface ship (MASS) that accurately predicts potential danger to own ship and performs safe navigation without collision with maritime obstacles such as target ships. The autonomous navigation system must accurately identify the position, speed, and direction of target ships in real-time, and assess the collision risk of each target ship to the own ship. Also, the autonomous navigation system must perform collision avoidance at the appropriate time based on the evaluation of the current situation around own ship. For collision risk assessment to quantify the potential danger between the own ship and the target ship, many studies have used the ship domain method and the closest point of approach (CPA) based method. The ship domain method is a method of creating a ship domain around the own ship based on the maneuvering performance. The ship domain is considered that own ship and target ship collide if the target ship invades the ship domain. Although it provides a clear standard for collisions, it has the disadvantage of not providing a quantitative value of collision risk. The CPA-based method is a method of assessing the value of collision risk based on various indicators calculated by the closest point of approach when the own ship and the target ship maintain their respective speeds and directions. Although it provides quantitative value, it has the disadvantage of not providing a clear standard for collision. Therefore, in this study, a collision risk assessment is proposed combining the ship domain method and the CPA-based method. However, the input data of the target ship that the own ship identifies may be subject to errors, such as false detection and noise of sensor data. As a result, it can be difficult to assess the accurate collision risk of the target ship. To solve this problem, this study proposes a collision risk assessment by reflecting the uncertainty of input data of the target ship based on multivariate normal distribution. The errors in the x, and y position, speed, and direction of the target ship are assumed to be multivariate normal distributions. The probability distribution of the area where the TS exists can be calculated by considering the uncertainty of input data. Then, the largest value of the collision risk assessed within the area is set as the representative value for each uncertainty. Meanwhile, all ships are required to perform collision avoidance by complying with the Convention on the International Regulations for Preventing Collision at Sea (CORREGs, 1972) to safely avoid collisions between ships when operating. In this study, a collision avoidance method is proposed that complies with COLREGs and can flexibly handle sudden changes in the situation. The proposed method is based on deep reinforcement learning, which receives increasing attention in the field of artificial intelligence and machine control, to derive the path and speed of own ship by considering multiple target ships simultaneously. The proposed collision avoidance method based on deep reinforcement learning recognizes multiple target ships around the own ship simultaneously and performs collision avoidance based on all target ships. In addition, the rudder angle and the propeller RPM of the own ship are controlled together to derive the most efficient path and speed. At this time, own ship receives a path following reward if the own ship follows the desired path and speed. If the own ship safely avoids target ships, own ship receives a collision avoidance reward. In this way, own ship is trained to follow an efficient and safe path to arrive at the destination. In this study, an autonomous navigation system is proposed that includes the three methods described above. To verify the proposed methods, the collision risk assessment and collision avoidance results for various ship encounter situations were confirmed. First, to verify the effectiveness of the collision risk assessment, the collision risk assessment was applied to various ship-to-ship encounter situations. Also, the collision risk assessment considering uncertainty was applied by generating an imaginary error that simulates the error of the actual sensor data. As a result, it was confirmed that the collision risk assessment considering uncertainty proposed in this study performs safe navigation considering potential threats even when the accurate collision risk cannot be calculated due to errors in target ship information. Next, to verify the effectiveness of the deep reinforcement learning-based collision avoidance method, we applied the proposed method to a benchmarking test problem for the collision avoidance of a ship. The results show that even when multiple target ships are operating around the own ship, the own ship avoids all target ships and arrives at the destination safely. Finally, the autonomous navigation system proposed in this study was applied when a large number of target ships were approaching the own ship and there were continuous changes and errors in the movement and information of target ships. As a result, it was confirmed that the proposed autonomous navigation system enables safe navigation even in an environment where the error of target ship information is large and many target ships are densely concentrated. |
|---|---|
| Publication Date | 2024-02-29 |
