Oversight must be operational, not symbolic
A human-in-the-loop label does not guarantee meaningful control. The reviewer needs enough context, time, training, and authority to question the output. If an interface presents a recommendation as certain, hides uncertainty, or makes override difficult, the nominal reviewer may not be able to exercise sound judgment.
Define the system's intended use and prohibited uses. Specify who checks the result, what evidence they review, how they record an override, and who is notified when the system behaves unexpectedly. These arrangements should match the risk and operating environment.
Test failure and recovery
Evaluate incorrect recommendations, missing or stale data, connectivity loss, model updates, and unusual operating conditions. Set a safe fallback that does not depend on the AI tool. Review incidents and near misses so operational learning improves controls, not just model accuracy.
The IMO's MASS framework emphasizes risk assessment, robust design, cybersecurity, remote operations, and human oversight. For any maritime application, use applicable instruments and company processes as the governing context.