From the magazine – Good teamwork is essential for safe and efficient maritime operations like ship navigation, port construction, or offshore maintenance. Miscommunication can lead to serious incidents. The challenge explored during my PhD is how to improve the sharing of perspectives and information among crew members. Augmented reality (AR) offers promising solutions to address this challenge.
This article originally appeared in SWZ|Maritime ‘s November 2025 issue. It was written by Floris van den Oever, Human Factors Specialist at Maritime Research Institute Netherlands (MARIN), f.v.d.oever@marin.nl.
AR overlays digital elements, like text, images, or 3D models, onto the physical world in real time, typically through headsets or screens. My research focused on how AR can support communication, team situation awareness, and team decision-making during ship navigation. This work combined insights from earlier studies, design methods, and experimental trials in both simulated and real maritime environments.
As shown in the literature review of this PhD project, AR has been developed for various maritime operations, like shipbuilding, harbour development, and maintenance and inspection. Similar technologies are also being developed for aviation, automotive, and surgical applications. These sectors may benefit from cross-disciplinary learning, given the shared challenges in spatial awareness, precision, and safety.
Also read: SWZ|Maritime’s November 2025 issue: Roaring twenties
Developing AR for ship navigation
A central component of the research involved the development and testing of an AR prototype in a field trial at sea. At the time of this study, there was no scientific literature on similar AR systems developed and tested at this “technology readiness level”. The prototype was developed over the course of three years, using an agile, iterative design process following the iterative design model.
Four design iterations were completed. In the first iteration, the initial prototype design was informed by interviews and use case workshops conducted on a platform supply vessel. In the second iteration, the first version of the prototype was designed and developed for individual coastal water navigation. In the third iteration, requirements for a multi-user version were identified through two interviews with end users and two workshops with users and designers. In the fourth iteration, the second version of the prototype was developed. It was designed to support collaborative navigation using two HoloLens 2s and is available on GitHub: https://github.com/thbc/sjoer/.
The prototype displays floating markers over nearby vessels, showing AIS data like name, position, speed, and CPA. It also displays markers above navigational aids, such as buoys, and lighthouses were also marked. This allows crew members to instantly identify and interpret their surroundings.
Testing AR at Sea
Trials were conducted aboard two research vessels operated by the Norwegian Institute of Marine Research, navigating through Norwegian fjords. On top of using radar and ECDIS, crew members wore Microsoft HoloLens 2 headsets running a prototype named Sjør. With the prototype, they could move freely, observe the environment directly, and access digital overlays hands-free.
The feedback was largely positive. Many participants described the experience as futuristic, but also emphasised the need for realistic expectations as the maritime world is conservative and slow to adopt new tech. One captain noted: ‘This could be the future, but it is also for the skippers of the future.’ Nevertheless, most participants expressed willingness to adopt AR once available.
The trials confirmed that AR can improve shared situation awareness and communication through several benefits:
- It can increase head-up time so that navigators look out the window more.
- It can support mental mapping from a first-person perspective because it gives information about heading and course of nearby vessels.
- It can support linking the outside view with ECDIS and radar because it shows labels on points of interest seen in the first-person perspective in a similar fashion that points of interest are labelled on ECDIS.
- It can help crew members quickly interpret the environment and communicate about it because points of interest are labelled and operators can use AR to point at things.
These are critical factors for safe and effective ship navigation. This shows AR could be useful during collaborative ship navigation, like watch changes or lookout-to-navigator interaction.
Also read: Integrating human factors from design to operation
Technical challenges and development needs
While the prototype showed promise, there are challenges to be overcome in order to introduce AR for ship navigation. Crew members saw the value, but were clear: ‘It must work flawlessly before it’s introduced in real operations.’ One major concern is precision: ensuring that digital markers consistently align with real-world objects. This is essential for trust and usability in highstakes environments.
However, it will be difficult to develop AR that is precise enough. The prototype of this PhD project was not connected to the ship systems due to regulatory restrictions. As such, it relied on web application programming interfaces (APIs) for AIS data and was not connected to the ship’s GPS systems. It can be connected to a ship’s systems for this information, which would greatly increase its precision. How precise that would make it and whether that would be enough remains to be seen. Other prototypes should be integrated with ship systems to test precision.
Another challenge is information overload. Navigators requested customisable filters and overlays, similar to ECDIS. Different roles require different data at different times, and giving users control over what they see helps prevent cognitive fatigue. Letting crew choose the amount of information gives them control and helps avoid mental overload.
For example, a lookout may use an overlay focused on situation awareness of the outside view, while a captain may want to include conning information like propellor rpms and pitches. Besides that, other information may be useful in harbours than at open seas.
Broader potential for maritime operations
Beyond bridge operations, AR holds potential in other maritime domains. For example, AR could support ship maintenance by overlaying technical schematics directly onto machinery, guiding technicians through complex procedures. In shipbuilding, AR could assist with assembly, inspection, and quality control by visualising design plans in real time on the physical structure.
More broadly, findings of this PhD may generalise to aviation, automotive, and surgical applications, given the shared challenges in spatial awareness, precision, and safety. These applications could improve efficiency, reduce errors, and enhance safety, especially in environments where precision and collaboration are critical.
A sustainable, safe, and secure use of the oceans
After my PhD, I have joined MARIN in Wageningen, where I focus on human factors in the development of the maritime sector of the future. My research continues to explore how technologies like AR can support both individual performance and team coordination.
Ultimately, the success of AR and similar innovations depends on their ability to support the professionals: the captain, the engineer, the nautical architect and the maritime suppliers. Technology must be designed to help them perform, individually and in teams. Together, we make the maritime sector smarter, cleaner, and safer. That way, we make it possible to tackle ever more complex tasks like fishing between floating wind farms and designing hybrid vessels with multiple propulsion systems.
Picture (top): A central component of the research involved the development and testing of an AR prototype in a field trial at sea.
Also read: Shipping company Jumbo uses augmented reality for transport job
