Robust State Estimation in Atomic Inertial Navigation Systems: A Comparative Study of CI-RKF and MKMC-EKF Filters

  • NAVIGATION: Journal of the Institute of Navigation
  • May 2026,
  • 73
  • navi.768;
  • DOI: https://doi.org/10.33012/navi.768

Abstract

Atomic inertial sensors offer exceptional long-term stability and ultra-low bias drift, but maintaining consistent navigation performance in dynamic environments remains challenging owing to non-Gaussian noise, sensor nonlinearities, and unmodeled dynamics. This study presents INS-ATOMIC, an autonomous inertial navigation system based solely on atomic gyroscopes and accelerometers, without the need for a global navigation satellite system or external corrections. Two filtering techniques are evaluated: (1) the chi-square increment robust Kalman filter (CI-RKF), which rejects inconsistent measurements, and (2) the multi-kernel maximum correntropy extended Kalman filter (MKMC-EKF), which mitigates non-Gaussian noise using adaptive kernel processing. Both methods were tested on real-world vehicular data sets under dynamic conditions. The MKMC-EKF outperformed the CI-RKF in accuracy and stability, achieving root-mean-square-error improvements from 37% to over 99% in velocity and up to 99% in acceleration and angular rates. These results demonstrate the effectiveness of correntropy-based filtering with atomic sensors for accurate, GNSS-free navigation in challenging environments.

Keywords

1 INTRODUCTION

Inertial Navigation Systems (INS) play a fundamental role in modern autonomous navigation by enabling continuous estimation of position, velocity, and attitude through the integration of accelerometer and gyroscope measurements. In recent years, atom-based inertial sensors, particularly atomic accelerometers and gyroscopes, have attracted considerable attention due to their ultra-low bias drift, high sensitivity, and remarkable long-term stability (Shettell & Dumke, 2023), (Travagnin, 2020), (Wang & Wang, 2025), (Templier et al., 2022). These characteristics make them highly promising for precision navigation, especially in long-duration missions and environments with limited access to external aiding sources, such as GNSS-challenged or high-interference scenarios (Wright et al., 2022), (Salducci et al., 2024), (Biswal & Jwo, 2024).

Despite these advantages, enhancing the accuracy and stability of INS in real-world and dynamic environments remains a significant challenge. Sensor noise, time-varying system dynamics, and model uncertainties can lead to error accumulation and degradation in estimation accuracy over prolonged operation. Conventional nonlinear filtering approaches, particularly the extended Kalman filter (EKF), are widely used due to their effectiveness under ideal conditions with Gaussian noise; however, their performance degrades when these assumptions are violated. In practice, navigation systems are often affected by non-Gaussian disturbances, heavy-tailed noise, sensor drifts, outliers, and strong nonlinearities, which can lead to instability in position, velocity, and attitude estimation (Zhang et al., 2025), (Fornasier et al., 2022), (Sun et al., 2023), (Jwo et al., 2024), (Hao et al., 2023), (Deng et al., 2025). Therefore, improving the robustness and adaptability of state estimation algorithms is essential for the effective utilization of high-grade atomic inertial sensors in practical navigation applications.

Although several studies have proposed robust extensions to Kalman filtering (such as outlier rejection and adaptive covariance tuning) to improve robustness in non-ideal conditions (Pagoti & Vemuri, 2022), few have applied correntropy-based robust filtering techniques, such as the multi-kernel maximum correntropy EKF (MKMC-EKF), in conjunction with atom-grade inertial measurement units (IMUs) under real-world operational conditions (Li et al., 2022; Biswal & Jwo, 2024). Moreover, existing validations are typically simulation-based and often lack experimental results obtained from highprecision atomic sensors under actual vehicular dynamics (Wright et al., 2022).

Therefore, a critical research gap exists in developing and experimentally validating an INS framework that not only leverages the inherent precision of atomic sensors, but also incorporates advanced, adaptive filtering strategies capable of mitigating non-Gaussian errors and maintaining stable navigation performance in complex motion scenarios (Biswal & Jwo, 2024; Li et al., 2021).

To improve estimation robustness in INSs, several advanced techniques have been developed that extend the EKF structure with mechanisms for handling uncertainty and outliers. One common strategy involves the integration of chi-square statistical tests within the EKF update process, which helps identify and suppress inconsistent measurements, thereby improving filter resilience in semi-structured environments (Li et al., 2023a). However, despite their effectiveness in reducing sudden errors, these approaches typically exhibit reduced performance when subjected to dynamic noise distributions or non-Gaussian disturbances. In parallel, adaptive filtering methods that use innovation re-weighting based on robust loss functions, such as Huber or correntropy-based criteria, have shown improved robustness to measurement anomalies (Peng et al., 2019). Nevertheless, the reliance on heuristic parameter tuning remains a key limitation, as suboptimal thresholds can compromise stability under varying operational conditions.

In another advancement, Wang et al. (2023) developed the maximum correntropy EKF (MCEKF), leveraging correntropy-based criteria within the EKF update step to effectively suppress measurement outliers and non-Gaussian disturbances. While this approach demonstrated improved performance in INS/ GNSS integrated systems under challenging conditions, its real-world validation using atomic iner-tial data and dynamic vehicle motion was not explored. Similarly, Zhang et al. (2024) introduced a robust MCEKF variant employing a Cauchy kernel-based correntropy cost function, validated on ground vehicle trajectories, yet this vari-ant was still dependent on GNSS for drift correction—highlighting the need for advances in GNSS-denied navigation scenarios.

A more innovative method proposed by Li et al. (2023b) introduced the gener-alized multi-kernel maximum correntropy Kalman filter (GMKMCKF), which applies different kernel bandwidths across multiple state channels to enhance robustness against heavy-tailed noise. While this filter exhibited stable performance in robotic disturbance estimation applications, its integration and validation using real-world inertial navigation data, especially from highprecision atomic sensors, remain to be explored.

Despite considerable progress in the design of advanced filtering techniques for inertial navigation, key challenges remain unresolved—particularly the depen-dency on external aiding sources, lack of validation using empirical motion data, and limited integration of high-precision atomic sensors within fully autonomous frameworks. Atom-interferometry-based sensors, characterized by ultra-low noise levels and exceptional long-term bias stability, present a compelling pathway toward GNSS-independent navigation systems (Wright et al., 2022; Saywell et al., 2023; Phillips et al., 2022). However, their practical integration with robust estima-tion algorithms under real-world dynamic conditions remains largely unexplored, revealing a significant gap in the development of fully independent, sensor-driven navigation solutions (Li et al., 2022).

To bridge this gap, we introduce a novel framework termed INS-ATOMIC. This system exclusively employs atomic accelerometers and gyroscopes, with the need for any external or hybrid data sources, to accurately estimate a vehicle's trajectory, velocity, acceleration, and rotation rate. Within this framework, two advanced filters (chi-square increment robust Kalman filter [CI-RKF] and MKMC-EKF) are implemented and evaluated based on real vehicular motion data. Filters grounded in the maximum correntropy criterion have shown remarkable efficacy in mitigating non-Gaussian and severe noise in other domains; however, their integration with atomic sensor data in an operational navigation system represents an unaddressed research void.

View this table:
TABLE 1 Technical Feature Comparison of Robust Kalman Filters: CI-RKF and MKMC-EKF

The primary innovation of this study lies in the practical application of the MKMC-EKF in conjunction with atomic sensors under real-world conditions. Owing to its adaptive nature and multi-kernel structure, this filter excels in eliminating outliers, non-Gaussian noise, and nonlinear fluctuations. Incorporating the MKMC-EKF within this framework significantly enhances system stability, markedly reduces position estimation drift, and improves the continuity and smoothness of velocity and angular rate estimations. Unlike conventional filters that gradually diverge over prolonged periods, the proposed system maintains estimation stability and data coherence along complex motion trajectories, effectively preventing unstable or oscillatory behaviors.

2 FUNDAMENTALS AND RECENT ADVANCES IN ATOM-BASED INERTIAL NAVIGATION

INSs have long been valued for their autonomy and reliability in Global Positioning System (GPS)-denied or signal-degraded environments such as military, underground, and aerospace operations. However, conventional INS architectures face persistent challenges, including cumulative drift, sensor noise, and long-term degradation due to reliance on mechanical or micro-electromechanical system (MEMS)-based inertial sensors. Recent advancements in quantum technologies, particularly atom-interferometry-based inertial sensors, offer a promising alternative by enabling ultra-low noise levels and enhanced long-term stability.

INSs estimate the position, velocity, and attitude (spatial orientation) of a moving object based solely on measurements from accelerometers and gyroscopes. Their operation is entirely independent of external infrastructure, making them indispensable in environments where GPS is unavailable, such as underground, underwater, or space environments.

However, INSs inherently suffer from accumulated errors due to sensor biases and noise, leading to long-term drift and degradation in accuracy over time. This issue is particularly pronounced in systems that utilize low-cost MEMS-based inertial sensors.

To address these challenges, researchers have turned to inertial sensors based on atom interferometry. These sensors exploit the wave nature of atoms and the principles of quantum interference to enable highly precise measurements of acceleration and angular velocity. In such systems, cold atoms are manipulated by sequences of laser pulses that coherently split the atomic wavefunction into two distinct paths, which are then recombined to produce an interference pattern.

To understand the working principle of inertial sensors based on atom interferometry, let us consider the typical configuration of an interferometer used in such systems (Figure 1). In this setup, cold atoms are subjected to a sequence of laser pulses that guide the atoms along two distinct paths, after which the atomic wavefunctions are recombined to form an interference pattern.

FIGURE 1

Implementation of a nuclear magnetic resonance gyroscope using a folded MEMS design (Noor et al., 2017)

The first pulse ( π/2) acts as a beam splitter, dividing the atomic wave packet into two separate trajectories. The second pulse (π) serves as a mirror, reversing the momenta of the atoms and redirecting them toward each other. Finally, the third pulse ( π/2) recombines the split wave packets, allowing them to interfere (Figure 2).

FIGURE 2

Configuration of a matter-wave interferometer based on Raman laser pulses

The phase shift generated between the atomic trajectories (caused by acceleration or rotation) forms the basis for measurement in these sensors. This interferometric phase, which depends on the differential path and timing of the atomic wave packets, is defined as follows:

ΔΦ=keffaT21

In this equation, ΔΦ represents the final phase difference between the two interferometric paths, keff is the effective wave vector resulting from the difference between the two laser beam vectors, a is the acceleration experienced by the atoms, and T is the time interval between the laser pulses.

The following equation expresses the phase shift ΔΦ caused by rotation (Sagnac effect), which depends on the angular rate Ω, interferometer area A, wavelength λ, and particle velocity v:

ΔΦ=4πAΩλv2

This equation serves as the fundamental basis for the operation of accelerometers and gyroscopes based on atom interferometry, in which the acceleration and angular velocity resulting from the object's motion are directly measured as output signals. By leveraging the principles of quantum mechanics, such sensors provide precision beyond that of classical inertial systems, playing a crucial role in high-accuracy navigation, particularly in GPS-denied environments.

In atom-interferometry-based sensors, the acceleration of a body is inferred by measuring the phase shift between two atomic trajectories. This phase shift, induced by acceleration, forms the basis for extracting motion-related parameters. Because the sensor output is discrete and typically available at low sampling rates (usually between 1 and 10 Hz ), the velocity and position must be estimated through time integration of the acceleration data. As a result, all motion parameters (such as velocity and position) are indirectly derived from phase analysis. In practice, to compensate for the low sampling rate and mitigate the impact of noise, estimation filters like the EKF or MKMC-EKF are employed. These filters help predict the dynamic state of the system between measurements, enabling continuous and accurate trajectory reconstruction. Therefore, the INS-ATOMIC system should integrate dynamic filtering methods to ensure reliable navigation performance, especially in GPS-denied environments.

3 PROPOSED FRAMEWORK: INS-ATOMIC

To transition from interferometric phase measurements to practical navigation, it is necessary to establish a dynamic model that links the atomic sensor outputs to the kinematic state of the carrier platform. In the proposed INS-ATOMIC framework, the classical strapdown INS is extended by incorporating measurements from atom interferometers. The platform state is represented by the position vector rn, velocity vector vn, and orientation matrix Cbn. These states evolve over time according to a set of first-order differential equations, which form the foundation of the system's dynamic model and allow integration of discrete atomic sensor outputs with continuous state estimation:

r˙n=vn3

v˙n=Cbnfibb(2ωien+ωenn)×vn+gn4

C˙bn=Cbn[ωibb]×5

In Equations (3)-(5), the platform state is described in the east-north-up (ENU) reference frame, where rn=[rE,rN,rU]T represents the position vector and vn=[vE,vN,vU]T denotes the velocity vector. The matrix Cbn is a direction cosine matrix that transforms vectors from the body frame to the ENU navigation frame, thereby capturing the orientation of the platform. The term fibb corresponds to the specific force measured by the atomic accelerometer in the body frame, whereas ωibb represents the angular rate measured by the body-mounted gyroscope. The platform's velocity is influenced by Coriolis and transport effects, expressed through the Earth's rotation rate ωien and the transport rate ωenn, respectively, as well as by the local gravity vector gn. Collectively, these equations establish a complete dynamic model for the translational and rotational motion of the platform, providing a foundation for integrating discrete atom interferometry measurements with continuous inertial navigation state estimation.

To directly connect atom interferometry measurements to the proposed navigation dynamics, the measured interferometric phase shift must be related to the physical motion of the sensor, particularly linear acceleration and rotational effects. In light-pulse atom interferometers, the accumulated phase encodes the inertial motion of the sensing platform over the interrogation time, providing a direct physical bridge between quantum measurements and classical navigation states. This relationship is expressed as follows:

ΔΦ=14keffaT2+12ω×vkeffT26

where ΔΦ denotes the measured interferometric phase shift, keff is the effective wave vector of the Raman laser beams, and T represents the interrogation time between light pulses. The vector a corresponds to the linear acceleration of the sensor, whereas ω denotes the angular velocity of the platform and v is the transverse velocity of the atoms relative to the interrogation axis. The first term reflects the direct contribution of linear acceleration to the phase accumulation, whereas the second term captures the rotational effect induced by Coriolis acceleration, which arises from the coupling between angular motion and transverse atomic velocity. This formulation establishes a clear and physically consistent link between interferometric phase measurements and the kinematic quantities employed in the INS-ATOMIC dynamic model.

Equation (6) establishes a direct and physically consistent link between the quantum sensing layer and the classical navigation dynamics described in Equations (3)-(5). Because the linear acceleration, velocity, and angular rate are either directly measured by the atomic sensors or estimated within the INS-ATOMIC framework, the model enables reliable reconstruction and prediction of the observed interferometric phase evolution.

Nevertheless, classical inertial dynamic models are subject to several practical limitations when deployed in real-world environments. These limitations include transient instabilities, rapid maneuver-induced variations, and dynamic uncertainties arising from environmental disturbances and unmodeled system complexities. Such effects are not adequately captured by standard mechanization equations and can lead to degraded modeling accuracy during highly dynamic or disturbed operating conditions. To address these challenges, the fundamental dynamic equations are extended in this work to explicitly account for transient behaviors and uncertainty-driven deviations, thereby providing a more realistic representation of the atomic navigation system dynamics and establishing a stronger foundation for advanced filtering and estimation techniques within the INS-ATOMIC framework.

Therefore, additional transient state variables are introduced to model short-duration, rapidly decaying disturbances affecting the system dynamics. These transient states are governed by first-order relaxation dynamics characterized by a time constant τ and are excited by an external input u(t), expressed by the following:

x˙trans=1τxtrans+u(t)7

fib,modified=fibb+xtrans,f8

ωib,modified=ωibb+xtrans,ω9

The resulting transient components are then incorporated into the inertial measurements by augmenting the nominal specific force and angular rate, where xtrans denotes the transient state vector, τ represents the relaxation time constant controlling the decay rate of transient effects, and u(t) models sudden excitation inputs associated with rapid maneuvers or external disturbances. The vectors xtrans,f and xtrans,ω correspond to transient contributions added to the specific force and angular rate, respectively. These modified inertial quantities allow the dynamic model to capture rapid variations, short-term instabilities, and non-ideal sensor behaviors that cannot be represented by conventional inertial navigation equations alone.

Equation (7) governs the temporal evolution of the transient states, while Equations (8) and (9) incorporate their effects into the specific force and angular rate, respectively. This formulation enables the dynamic model to account for short-term instabilities and rapidly varying disturbances in a unified manner.

To account for modeling uncertainties and disturbances induced by dynamic instabilities, the process noise is modeled as a stochastic component within the system dynamics. In the nominal case, this uncertainty is represented by zero-mean Gaussian white noise with a specified covariance, which captures regular modeling errors and small unmodeled dynamics. However, during severe instabilities, abrupt maneuvers, or highly dynamic operating conditions, Gaussian assumptions become inadequate. In such cases, non-Gaussian noise components are required to properly describe sudden state variations and rare extreme events. Accordingly, the system dynamics can be expressed in the general stochastic form as follows:

x˙(t)=f(x,t)+w(t)10

where x(t) denotes the system state vector, f(x,t) represents the nonlinear deterministic dynamics of the INS-ATOMIC framework, and w(t) is the process noise term accounting for modeling uncertainties and external disturbances. In nominal operating conditions, w(t) is commonly modeled as zero-mean Gaussian white noise with covariance matrix Q, denoted as N(0,Q)w(t), which captures regular unmodeled dynamics and stochastic perturbations affecting the system. This formulation provides a compact and flexible representation of the stochastic behavior of the navigation system and serves as the basis for subsequent extensions incorporating non-Gaussian noise components under highly dynamic or unstable conditions.

In highly dynamic operating conditions characterized by severe instabilities, abrupt maneuvers, or unpredictable nonlinear effects, the Gaussian noise assumption becomes inadequate to accurately represent the underlying system disturbances. To overcome this limitation, a non-Gaussian noise component is incorporated into the dynamic model, enabling the representation of impulsive disturbances and rare extreme events. Such disturbances may be modeled as jump processes or by means of heavy-tailed probability distributions, including Laplace or alpha-stable distributions, which are well suited for capturing sudden state deviations.

The extended stochastic dynamic model is expressed as follows:

x˙(t)=f(x,t)+wg(t)+wng(t)11

where wg(t) denotes zero-mean Gaussian white noise with covariance matrix Q representing nominal stochastic uncertainties and wng(t) represents the non-Gaussian noise component accounting for impulsive disturbances and heavy-tailed effects. The coexistence of these two noise terms enables the model to simultaneously capture both smooth continuous variations and abrupt extreme fluctuations, which is essential for robust navigation performance in high-noise and instability-prone atomic sensing environments.

The simultaneous inclusion of Gaussian and non-Gaussian noise components enables the dynamic model to represent both smooth stochastic variations and abrupt extreme disturbances, which is essential for analyzing critical operating conditions and high-noise scenarios in atomic navigation systems.

Building upon this stochastic formulation, the baseline INS-ATOMIC dynamic model is subsequently refined to incorporate the effects of transient phases, rapid instabilities, and instantaneous disturbances, resulting in a more realistic representation of the system's true dynamic behavior.

By incorporating transient-state dynamics and a combined Gaussian and non-Gaussian noise formulation into the classical inertial navigation equations, an enhanced INS-ATOMIC dynamic model is obtained.

In this refined formulation, transient components are injected into the specific force and angular rate, whereas stochastic disturbances are explicitly introduced into the position, velocity, and attitude dynamics. The resulting continuous-time navigation equations expressed in the ENU frame are given as follows:

r˙n=vn+wrg(t)+wrng(t)12

v˙n=Cbn(fibb+xtrans,f)(2ωien+ωenn)×vn+gn+wvg(t)+wvng(t)13

C˙bn=Cbn[ωibb+xtrans,ω]×+wCg(t)+wCng(t)14

Equations (12)-(14) describe the enhanced continuous-time INS-ATOMIC navigation dynamics in the local ENU frame. Equation (12) represents the kinematic relationship between position rn and velocity vn, augmented by Gaussian and non-Gaussian stochastic disturbances wrg(t) and wrng(t), respectively. Equation (13) governs the velocity dynamics, where Cbn denotes a direction cosine matrix transforming quantities from the body frame to the navigation frame, fibb is the measured specific force, and xtrans,f represents the transient acceleration component capturing short-term instabilities. The terms ωien and ωenn denote the Earth's rotation rate and the transport rate expressed in the navigation frame, respectively, while gn is the gravity vector. Finally, Equation (14) describes the attitude dynamics, where the angular rate ωibb is augmented by the transient rotational component xtrans,ω, and the skew-symmetric operator []×represents the cross-product matrix. The noise terms wvg(t),wvng(t),wCg(t), and wCng(t) account for Gaussian and non-Gaussian disturbances affecting the velocity and attitude states.

With the incorporation of a more realistic representation of process noise and motion disturbances, the proposed INS-ATOMIC dynamic model becomes capable of capturing rapid maneuvers, sudden instabilities, and nonlinear sensor effects. The complete state estimation framework is illustrated in Figure 3.

FIGURE 3

Integrated INS-ATOMIC mechanization and filtering structure for resilient navigation

The process starts with raw acceleration (a) and angular rate (ω) measurements acquired from atomic accelerometers and gyroscopes. These signals are processed within the INS-ATOMIC mechanization module to compute velocity, attitude, and position estimates. In parallel, a nonlinear dynamic model propagates the system states over time. The MKMC-EKF filter is employed in this study for continuous estimation of the system's state, including position, velocity, orientation, and sensor bias parameters. It is important to clarify that this filter inherently performs relative positioning. The primary inputs to the filter are raw data from IMUs, specifically the accelerometers and gyroscopes, which provide differential measurements.

According to the alignment process for initialization of all state variables, the initial state coordinates are given by a GNSS receiver or map. Therefore, the recursive outputs of the MKMC-EKF filter are presented in latitude, longitude, and altitude coordinates within the ENU local reference frame mechanization. In the Earth-tangent coordinate system, the body (X,Y,Z) axes are set to ENU axes to define Euler angles as ϕ: roll, θ: pitch, and ψ: yaw. Consequently, the filter's output coordinates represent the system's displacement and orientation changes relative to the real starting point with zero-velocity components, level roll and pitch angles from accelerometers, and compassed yaw angles. The initial sensor biases are set to zero. Therefore, the obtained trajectory map from the proposed MKMC-EKF is coincided upon a real map, such as that provided by a GNSS/GPS. As the approach of this paper constitutes an independent INS, it should be referred to as a dead reckoning mechanization. In this methodology, the position estimation is updated incrementally based on inertial sensor measurements, without reliance on external absolute references such as a GNSS. This clarification is essential for a precise understanding of the proposed method's capabilities and limitations.

For robust estimation, two advanced filtering strategies are evaluated: the CI-RKF and the MKMC-EKF. It is important to emphasize that no GNSS or external aiding is applied in the filtering process. All state updates are performed purely using atomic IMU data. During the field experiments, virtual interferometric aided navigation system (VITANS) reference measurements (with embedded GPS) are used only to generate ground-truth trajectories for quantitative performance eval-uation and error analysis.

By explicitly handling non-Gaussian noise, sensor nonlinearities, and unmodeled dynamics, this dual-filtering scheme significantly enhances navigation reliability and accuracy under highly dynamic and degraded conditions, reinforcing the viability of fully autonomous atomic IMU-based navigation in GNSS-denied environments. The symbols and parameters associated with the dynamic model, as outlined in Section 3 for the proposed INS-ATOMIC framework, are detailed in Table 2.

View this table:
TABLE 2 Summary of Symbols and Parameters Used in the Proposed INS-ATOMIC Framework

4 FILTERING TECHNIQUES IN INS-ATOMIC

To enable accurate state estimation within the INS-ATOMIC framework, a consistent and physically meaningful state-space representation is required. This representation forms the basis for implementing advanced filtering techniques by describing the temporal evolution of the navigation states (namely, position, velocity, orientation, and sensor biases) as driven by inertial measurements and stochastic disturbances.

The state vector employed in the INS-ATOMIC filtering framework is defined as follows:

x=[rvqbabg]T15

The transposed state vector xT consists of the three-dimensional position vector r expressed in the navigation frame, followed by the corresponding three-axis velocity vector v. The system attitude is then represented by the quaternion q, which is adopted to avoid the singularities associated with Euler angle parameterizations. Finally, the three-component vectors ba and bg denote the accelerometer and gyroscope bias states, respectively, which are included to compensate for long-term or quasi-static sensor errors and to mitigate the accumulation of angular rate drift.

This state vector structure provides a suitable foundation for implementing robust and adaptive filters within the framework of atomic inertial navigation.

To update the state estimate in the EKF, it is essential to define an accurate measurement model that maps the system's state vector to the sensor observations. In the INS-ATOMIC framework, atom-interferometry-based sensors provide outputs equivalent to the linear acceleration and angular velocity of the carrier. Therefore, the measurement model relates these observed quantities to the dynamic states estimated by the filter.

Within the EKF framework, the measurement model is expressed as follows:

zk=[ameasωmeas]=h(xk)+vk16

Here, the measurement vector zk includes the three-axis linear acceleration ameas and angular velocity ωmeas measured by the atom-interferometry-based sensors. The system state at time k, denoted by xk, contains the navigation variables, namely, the position r, velocity v, and attitude represented by the quaternion q, along with the accelerometer bias ba and gyroscope bias bg.

The nonlinear function h(xk) maps the state variables into the measurement space and describes how the sensor outputs are generated from the underlying system states. The term vk represents the measurement noise, which accounts for sensor imperfections and external disturbances and is assumed to be Gaussian with zero mean and known covariance. This measurement model provides a clear and consistent link between the atomic sensor outputs and the EKF state estimates, enabling reliable filter updates. The nonlinear measurement function in the INS-ATOMIC framework is defined as follows:

h(xk)=[fib,kb+ba,kωib,kb+bg,k]17

In this expression, fib,kb denotes the specific force measured by the atom interferometric accelerometer, expressed in the body frame. This quantity represents the gravity-compensated linear acceleration of the platform and is related to the true translational acceleration through the relation fib,kb=akCnb(qk)gn, where ak is the linear acceleration of the system, gn is the gravitational acceleration vector expressed in the navigation frame, and Cnb(qk) is the direction cosine matrix constructed from the attitude quaternion qk, which transforms vectors from the navigation frame to the body frame. As a result, the transformation term Cbn(qk)T(akgn), which commonly appears in conventional INS formulations, is inherently encapsulated in the definition of the specific force and therefore does not appear explicitly in the measurement equation.

The term ba,k represents the accelerometer bias, which accounts for slowly varying systematic errors in the atomic acceleration measurement and is modeled as part of the system state to mitigate long-term drift.

Furthermore, ωib,kb denotes the angular rate of the body frame with respect to the inertial frame, expressed in the body frame and measured by the atom interferometric gyroscope. The additive term bg,k corresponds to the gyroscope bias and is included in the measurement model to compensate for residual bias and low-frequency noise effects.

The proposed measurement function provides a compact and physically meaningful representation of the atomic inertial sensor outputs, ensuring consistency with inertial navigation principles while enabling effective state estimation within the EKF framework.

It should be noted that the true angular rate used in the error evaluation is derived from the VITANS reference navigation data, obtained by differentiating the reference Euler angles and transforming them into the body frame, rather than being directly measured by the gyroscopes.

4.1 Extended Kalman Filter (EKF)

In the INS-ATOMIC framework, owing to the use of atomic gyroscopes and accelerometers, the system's motion models are typically defined in a nonlinear form, making the EKF a suitable choice for implementing an effective state estimator.

Because the EKF operates in a discrete-time framework, the continuous-time dynamic model is naturally discretized to be applicable in the prediction and update steps of the filter. This relationship is generally expressed as follows:

xk+1=f(xk,uk)+wk18

Here, xk+1 represents the state vector at the next time step, xk is the state vector at the current time step, uk denotes the control input (derived from sensor measurements or system inputs), and wk is the process noise, which is assumed to have a zero mean with known covariance. The function f(xk,uk) represents the nonlinear state transition model, which governs the system's evolution over time.

Following the discretization of the dynamic model, the EKF proceeds with the prediction step, where the state estimate and error covariance matrix are propagated forward in time. The predicted state estimate is computed as follows:

x^k+1=f(x^k,uk)19

Pk+1=FkPkFk+Qk20

In this equation, x^k+1is the a priori predicted state estimate at time step k+1, x^k is the a priori estimate at time step k, and uk is the control input. The symbol x^k represents the a priori (pre-measurement) state estimate, which is propagated forward using the state transition model. In addition to predicting the state estimate, the error covariance matrix Pk is updated based on the nonlinear dynamics of the system.

In the prediction stage of the EKF, the propagation of the estimation uncertainty is governed by the linearized form of the nonlinear state transition model. Specifically, the predicted error covariance matrix at time step k+1, denoted by Pk+1, is obtained from the previous covariance matrix Pk through the state transition matrix Fk and the process noise covariance Qk, as shown in Equation (20). The matrix Fk represents the sensitivity of the nonlinear dynamic function f(xk,uk) with respect to the state vector and is formally defined as the Jacobian Fk=f(xk,uk)xk, evaluated at the current state estimate. This linearization enables the EKF to approximate the local behavior of the nonlinear INS-ATOMIC dynamics and to consistently propagate the state uncertainty in discrete time.

For the proposed INS-ATOMIC state vector, the resulting state transition matrix Fk is given explicitly in Equation (21):

Fk=[03×3I3×303×403×303×303×32C(q)T[ω]×(C(q)Ta)qC(q)T2C(q)T[v]×04×304×312Q(q)q(ωbg)04×312Q(q)03×303×303×403×303×303×303×303×403×303×3]21

The matrix Qk is the process noise covariance matrix, capturing the uncertainties and stochastic variations in the nonlinear dynamic model, including sensor noise and unmodeled dynamics:

Qk=[03×300000σv2I3×300000σq2I4×400000σba2I3×300000σbg2I3×3]22

This structure implies that the process noise affecting each state component is assumed to be statistically independent from the other components. Typical values for the process noise standard deviations can be set as σv=0.1m/s2, σq=0.01,σba=0.001m/s2,σbg=0.0001rad/s based on sensor noise characteristics and system behavior.

The prediction step provides a priori estimates of the state and its covariance based on the system dynamics. The update step incorporates new measurements to correct and refine the predicted state.

The update equations of the EKF corresponding to the nonlinear measurement model are as follows:

Kk=PkHk(HkPkHk+Rk)123

x^k=x^k+Kk(zkh(x^k))24

Pk=(IKkHk)Pk25

The matrix Hk is the Jacobian of the measurement function with respect to the state vector and is used in the update step of the filter. This matrix is calculated as follows:

Hk=h(x)x|x=x^k=[03×303×3Ha,qI30303×303×3Hω,q03I3]26

where Ha,qR3×4 and Hω,qR3×4 denote the partial derivatives of the predicted accelerometer and gyroscope outputs with respect to the quaternion components.

The EKF provides an effective iterative estimation of the system state by incorporating nonlinear dynamic and measurement models. However, the performance of the EKF is highly sensitive to the accuracy of these models and proper tuning of the noise covariance matrices. For scenarios in which the dynamic or measurement models include structural errors or unmodeled uncertainties, the EKF may produce unstable or inaccurate estimates. To address such limitations, techniques like the CI-RKF are employed. This method statistically evaluates the innovation (the difference between the actual and predicted measurements) to detect and reject inconsistent measurements, thereby enhancing the robustness and reliability of the filtering process.

4.2 Robust Kalman Filtering Based on Chi-Square Increment (CI-RKF)

In this section, a chi-square increment-based filter is introduced and applied within the INS-ATOMIC navigation framework as a statistical tool for monitoring filter consistency with observations and enhancing estimation stability.

Given the refined dynamic noise modeling and the structured prediction-correction framework described earlier, a solid foundation is now established for the application of advanced filtering strategies. Among these, the chi-square increment-based filtering approach provides a statistically grounded method for enhancing the reliability and consistency of navigation state estimation. The following section presents the design and implementation of a chi-square increment-based filter tailored to the INS-ATOMIC framework.

To implement the chi-square increment criterion within the EKF framework, the innovation and its associated covariance are employed to define a statistical index that evaluates filter consistency and identifies potential anomalies. The relevant equations are given as follows (Equations (27)-(29)):

yk=zkh(x^k)27

Sk=HkPkHk+Rk28

χk2=ykTSk1yk29

In this stage of the filtering process, a set of equations is employed to assess the consistency of the received measurement with the model prediction. First, the innovation vector yk is computed as the difference between the actual measurement zk and the predicted observation h(x^k). Then, the innovation covariance matrix Sk is calculated by incorporating the uncertainty of the predicted state and the measurement noise. The chi-square statistic χk2 is obtained by evaluating the Mahalanobis distance of the innovation. This statistic plays a critical role in verifying the validity of the measurement, such that if its value exceeds a predefined statistical threshold, it indicates a mismatch between the measurement and the model. This mechanism can be effectively used for detecting outliers, rejecting faulty data, or enhancing the filter's reliability. Consequently, this approach significantly improves the robustness and accuracy of the EKF-based navigation system in the presence of noisy or unreliable sensor data.

To detect sudden and unusual changes in the quality of measurement data, the incremental chi-square test is used, which calculates the change in the chi-square statistic between two consecutive time steps:

Δχk2=χk2χk1230

By analytically evaluating the measurement innovation and the predicted error covariance, the incremental chi-square statistic is formulated as follows:

Δχk2=(zkhk(x^k,k1))T(HkPk,k1Hk+Rk)1(zkhk(x^k,k1))31

This expression provides a rigorous statistical basis for outlier detection using prior estimates in nonlinear and noise-sensitive systems.

Under ideal conditions (where the system is affected solely by Gaussian noise and free of gross errors), the measurement innovation is assumed to follow a zero-mean normal distribution with a covariance matrix given by the sum of the measurement noise covariance and the predicted error covariance. This statistical assumption provides a foundation for the chi-square test, which is widely used for the automatic detection of inconsistencies and abrupt anomalies in measurement data:

zkhk(x^k,k1)N(0,HkPk,k1Hk+Rk)32

A significant deviation of the innovation from the expected normal distribution (such as an unusually large value) indicates the presence of a gross error or outlier. This forms the statistical basis of the chi-square test, which assesses the consistency of the innovation with its assumed distribution:

(zkhk(x^k,k1))T(HkPk,k1Hk+Rk)1(zkhk(x^k,k1))χ2(n)33

This statistic, computed from the innovation (the difference between the actual and predicted measurements) and the combined covariance of the predicted error and measurement noise, serves as the primary metric for hypothesis testing to detect outliers and inconsistencies in the data.

The hypothesis test for a single incremental chi-square quantity Δχk,i2 is defined as follows:

{H0:Δχk,i2χ2(1,0)H1:Δχk,i2χ2(1,λ)34

Here, H0 represents the normal, error-free case in which the test statistic follows a central chi-square distribution, whereas H1 corresponds to the presence of a gross error or outlier, in which case the statistic follows a non-central chi-square distribution with non-centrality parameter λ. This statistical framework effectively discriminates between valid and corrupted data, enhancing estimation accuracy in sensitive and noisy systems.

In dynamic positioning systems, a robust factor αi is employed to enhance the robustness of the filter against outliers and abrupt errors. This factor is computed based on the chi-square increment Δχk,i2 and is designed to reduce the influence of potentially erroneous measurements on the filter update.

The robust factor αi is defined as follows:

αi={1Δχk,i2χβ2(1)<c0Δχk,i2χβ2(1)c0<Δχk,i2χβ2(1)<c1Δχk,i2χβ2(1)Δχk,i2χβ2(1)>c135

where c0 and c1 are predetermined thresholds and χβ2(1) denotes the chi-square value corresponding to confidence level β with one degree of freedom.

This robust factor is incorporated into a diagonal matrix α, which is then used to adjust the observation noise covariance matrix as follows:

Rk=αRk

Subsequently, the updated covariance matrix R¯k is utilized in the robust Kalman filter update steps, allowing the measurement weights to be adaptively adjusted according to the likelihood of gross errors. This approach effectively enhances the accuracy and stability of the state estimation in the presence of abnormal measurement disturbances.

By integrating the robust factor into the classical Kalman update equations, as shown below, the proposed method ensures adaptive covariance correction in response to detected anomalies, thereby preserving the estimation integrity within the highly sensitive INS-ATOMIC environment:

Kk=Pk,k1Hk[HkPk,k1Hk+Rk]136

Pk=[IKkHk]Pk,k137

x^k=x^k,k1+Kk[zkhk(x^k,k1)]38

This formulation plays a critical role in maintaining robustness and convergence of the filter in the presence of transient disturbances or outlier measurements, which are particularly significant in the operation of high-precision systems such as INS-ATOMIC.

4.3 Multi-Kernel Maximum Correntropy EKF (MKMC-EKF)

In many navigation applications (particularly in systems such as INS-ATOMIC that are exposed to noisy, oscillatory, and occasionally abrupt outlier-prone measurements), the standard Kalman filter and even its robust variants often fail to deliver satisfactory performance. This limitation primarily stems from the assumption of a Gaussian distribution for both process and observation noise, an assumption that does not consistently hold in real-world scenarios.

To address this, the MKMC-EKF has been introduced as an advanced nonlinear approach with superior capability in suppressing the effects of non-Gaussian noise and outlier observations. The core idea behind this filter lies in replacing the conventional minimum mean square error criterion with the maximum correntropy criterion. Unlike traditional methods, the MKMC-EKF simultaneously utilizes multiple kernel functions with varying bandwidths, thereby enabling the filter to adaptively control sensitivity across different noise amplitudes. This feature significantly enhances the stability and accuracy of the estimation process in highly noisy and complex environments.

In the following section, the mathematical framework and main equations of the MKMC-EKF are presented.

In traditional filters such as the EKF or even the robust Kalman filter, the optimization criterion is typically based on minimizing the mean squared error, which is highly sensitive to non-Gaussian noise, sudden jumps, and outliers. In contrast, the correntropy criterion, which incorporates higher-order statistical information, is more suitable for non-ideal and noisy environments. By employing kernel functions (typically Gaussian), the correntropy assigns higher weights to small error values and significantly lower weights to large errors (e.g., those caused by outliers), thereby naturally suppressing the effect of anomalous data.

In the multi-kernel correntropy version, instead of using a single kernel function with a fixed bandwidth (σ), multiple kernels with different bandwidths are used simultaneously. This enables the filter to maintain adaptive sensitivity across a range of error magnitudes. Essentially, the multi-kernel correntropy represents a weighted combination of responses from multiple kernels, resulting in a flexible and adaptive objective function.

The multi-kernel correntropy between two vectors x and y is defined as follows:

VMKC(x,y)=Σi=1Lηiexp(xy22σi2)39

In the multi-kernel correntropy, the similarity between two vectors x and y is measured using a weighted combination of multiple kernel functions, each with a different bandwidth σi and associated weight ηi. Each kernel evaluates the squared Euclidean distance xy2 and assigns a weight accordingly. Smaller errors (i.e., smaller distances between predicted and true values) are given larger weights, while larger errors (often caused by heavy noise or outliers) are naturally suppressed. This multi-scale sensitivity enables the filter to effectively adapt to a wide range of noise levels, enhancing both the robustness and accuracy of estimation in non-ideal, noisy environments.

Based on the derivative of the multi-kernel correntropy function VMKC(x,y) with respect to the innovation error, the weighting of the error components is defined as follows. The weighting matrix Wk is constructed as a diagonal matrix:

wi=j=1Lηj1σj2exp(ek,i22σj2),Wk=diag(w1,w2,,wn)40

In the multi-Kernel correntropy-based weighting process for computing Wk, the key parameters include the innovation error vector ek=ykh(x^kk1), the kernel widths σj, and the mixing coefficients ηj for each of the L kernels. Each element ek,i is passed through the multi-kernel correntropy function, where the kernels' widths σj control the sensitivity to different error magnitudes and the weights ηj determine each kernel's contribution. The resulting weights form Wk, which downweights outliers and enhances robustness in the state and covariance updates.

In this work, the kernel widths σj and the mixing coefficients ηj are treated as tunable hyperparameters. The kernel widths are selected based on the empirical dispersion of the innovation sequence and are chosen to be proportional to the standard deviation of the residuals, allowing the kernels to adapt to the noise scale. The mixing coefficients are determined through offline validation by minimizing navigation error metrics such as the root mean square error (RMSE). This strategy enables practical tuning of the MKMC-EKF under different noise conditions.

With the incorporation of the multi-kernel correntropy criterion, the update step in the MKMC-EKF is reformulated to naturally suppress the influence of non-Gaussian noise and outliers, thereby enhancing the robustness and accuracy of the state estimation. In this context, the following equations represent the modified version of the classical EKF update, where a correntropy-based weighting matrix is integrated into the state and covariance update process:

Kk=Pk,k1HkSk141

Pk=(IWkKkHk)Pk,k142

x^k=x^k,k1+WkKkyk43

This structure enhances robustness against outliers and heavy-tailed noise.

The symbols and parameters associated with the filtering techniques, as described in Section 4 for the INS-ATOMIC framework, are summarized in Table 3.

View this table:
TABLE 3 Symbols and Parameters Used in the Filtering Techniques of INS-ATOMIC

5 EXPERIMENTAL SETUP AND EVALUATION

To evaluate the performance of the designed navigation filters under real-world conditions, an experimental framework based on the INS-ATOMIC system has been developed. This framework utilizes atom interferometric sensors to accurately capture linear acceleration and angular velocity data, enabling a robust assessment of filter performance in the presence of structured noise, sudden jumps, and outliers. The test routes, sensor mounting configurations, and data acquisition procedures have been carefully designed to provide a comprehensive analysis of the accuracy, stability, and robustness of the EKF, CI-RKF, and MKMC-EKF under realistic uncertainties. The following sections describe the hardware specifications, noise modeling strategies, and implementation details of this experimental setup.

In the proposed system, instead of relying on GNSS-aided or magnetometerassisted IMU architectures, all navigation updates are performed solely using the atomic accelerometer and gyroscope measurements. These sensors extract linear acceleration and angular velocity directly from atom-laser interaction dynamics. The resulting signals (derived from phase shifts between laser beams and atomic trajectories) are mapped into motion states through phase-to-displacement conversion and are used as input to the INS mechanization process. For field evaluation, reference trajectories are obtained from the VITANS integrated system, which uses a Garmin 35 GNSS receiver for high-precision positioning. However, no GNSS information is used to aid the proposed filters.

The measurement instruments are mounted on an aluminum frame rigidly fixed to the vehicle's chassis, eliminating relative motion and minimizing the influence of magnetic and vibrational noise. The output of the atomic sensors, along with a noise model that includes both Gaussian and non-Gaussian components, is fed into the prediction module of the navigation filter. State estimation is then performed using three different filtering strategies: EKF, CI-RKF, and MKMC-EKF.

Unlike conventional GNSS-aided or magnetometer-assisted systems, the proposed setup performs all navigation updates purely based on atomic IMU measurements, without using any external aiding. During field experiments, VITANS reference data are used only for ground-truth evaluation to quantify the system's accuracy under structured noise and outlier disturbances. A data logger records the processed signals at each stage, and the final analysis compares the performance of the three filtering approaches.

The experimental vehicle was equipped with a range of advanced sensors to facilitate precise data collection for navigation analysis. A Trimble BD982 GNSS receiver was mounted to provide accurate positioning data, while the MIT C-710 sensor was used to capture angular velocity measurements. Additionally, a ZED stereo camera was employed to obtain visual information from the surrounding environment (Figure 4). Inside the vehicle, a HUAWEI Honor V9 smartphone was utilized for real-time monitoring and timestamp logging during the tests. These components collectively supported the evaluation of the proposed navigation algorithms.

FIGURE 4

Sensor configuration of the test vehicle used in the experimental setup

The detailed technical characteristics and performance parameters of the sensors and auxiliary devices used in the experiments are summarized in Table 4.

View this table:
TABLE 4 Technical Specifications and Performance Parameters of the Sensors and Reference Devices Used in the INS-ATOMIC Experiments

In this study, a series of field experiments was conducted on the campus of Tabriz University to evaluate the performance of an INS based on atomic measurements (INS-ATOMIC). Test scenarios included acceleration, deceleration, turning maneuvers, and navigating paths with varying slopes, designed to assess the accuracy and stability of the state estimation algorithms under diverse and realistic conditions. The test vehicle was equipped with a high-precision IMU consisting of a three-axis atomic gyroscope and accelerometer, which directly provided angular velocity and linear acceleration data as the primary input for the INS. Additionally, a HUAWEI smartphone was used to log visual, temporal, and spatial data.

All equipment was firmly mounted on a rigid aluminum profile attached to the roof of the vehicle, minimizing vibrations and mechanical noise interference.

According to the trajectory diagram shown in Figure 5, the vehicle's motion followed a predefined path marked by points P1 through P6:

FIGURE 5

Trajectory of the vehicle during field experiments for evaluating the INSATOMIC navigation system

  • P1: The vehicle started in front of Building 14 of the Faculty of Mechanical Engineering.

  • P2: The vehicle proceeded to a location near the Department of Mechanical Engineering, marking the second turning point of the path.

  • P3: The vehicle then moved toward a point in front of the Central Library, where it performed a full turn to begin an ascent along a sloped section.

  • P4: After the ascent, the vehicle reached P4 at the far end of the slope, near the eastern boundary of the route.

  • P5: The vehicle then reversed along the same path, passing near the Central Library again, marking a symmetrical part of the route.

  • P6: Finally, the vehicle came to a stop near Building 14, returning close to the starting point.

The elevation profile of the route indicates an upward slope from the starting point to the Central Library, followed by a downward slope and repeated elevation patterns along the path.The trajectory and the start/stop points of the vehicle were also confirmed by analyzing the variations in the relative velocity vector components in the local coordinate frame.

These experiments primarily focused on evaluating trajectory and position estimation using atomic sensor outputs without any GNSS aiding in the filtering process.During the tests, GPS measurements were included only within the VITANS integrated reference system to provide ground-truth trajectories for validation and accuracy assessment.Accurate identification of the atomic sensor parameters (particularly the gyroscope bias and scale factor)significantly enhanced the performance of both estimation algorithms.Furthermore, the optimal Raman interrogation wavelength was tuned to improve the sensitivity of the atomic sensors.

To enhance estimation robustness and accuracy in the presence of non-Gaussian noise and outliers, two robust versions of the EKF were implemented: the CI-RKF, based on the chi-square innovation test, and the MKMC-EKF, incorporating the multi-kernel maximum correntropy criterion.Experimental results demonstrated the superior performance of these robust filters in suppressing disturbances and maintaining stable estimations.

To demonstrate the non-Gaussian characteristics of the measurement noise, statistical tests and graphical analysis were performed on the residuals of the raw inertial sensor data. The Shapiro-Wilk and Kolmogorov-Smirnov normality tests, along with metrics such as skewness and kurtosis, were employed to assess the deviation from Gaussianity.

  • Skewness and kurtosis: The gyroscope data exhibited a significant skewness of 2.64 and an excess kurtosis of 19.91, indicating a pronounced deviation from Gaussianity. These values are far from the expected values of 0 for skewness and 3 for kurtosis in a Gaussian distribution.

  • Shapiro-Wilk test: The Shapiro-Wilk test returned a p-value 0.05, confirming that the gyroscope data significantly deviate from a Gaussian distribution. This statistical result provides clear evidence that the measurement noise is non-Gaussian.

Histograms and Q-Q plots were generated for the residuals of both accelerometer and gyroscope data to visually inspect the distribution of the errors. The histograms revealed heavy-tailed behavior consistent with non-Gaussian noise, while the Q-Q plots confirmed significant deviations from linearity with respect to a normal distribution, as shown in Figure 6.

FIGURE 6

Statistical analysis of residuals from gyroscope and accelerometer data

Figure 6 shows the distribution characteristics of the inertial sensor noise residuals. Subfigures (a) and (c) depict the probability density function (PDF) for the non-Gaussian noise of the accelerometer and gyroscope, respectively. The heavy-tailed nature of these distributions clearly indicates a deviation from Gaussian behavior. Specifically, the gyroscope noise PDF in (c) visibly exhibits significant skewness and kurtosis.

Subfigures (b) and (d) present quantile-quantile (Q-Q) plots for the accelerometer and gyroscope noise. In these plots, the ordered noise data quantiles are compared against the theoretical quantiles of a normal distribution, with a straight red line representing an ideal normal distribution. A noticeable deviation of the data points from this straight line visually confirms the non-Gaussian nature of the noise. The parameters implicitly visualized in these Q-Q plots, particularly in (b) and (d), highlight key statistical properties of the noise. These properties include the skewness of the distribution, indicated by its asymmetry, and the presence of heavier tails compared with a normal distribution, visualized by points deviating outward from the line at the extremes. The pronounced nonlinearity in the gyroscope's Q-Q plot (d) strongly corroborates the high reported values of skewness (2.64) and excess kurtosis (19.91), signifying a distribution that is more peaked and exhibits a greater propensity for extreme values than a standard Gaussian distribution.

The residuals extracted using the corrected VITANS INS/GPS reference data exhibit clear non-Gaussian characteristics, directly influencing the resulting navigation performance.

6 RESULTS AND PERFORMANCE EVALUATION

6.1 Analysis of Filter Performance in Estimating Velocity, Acceleration, and Euler Angle Components

In this section, the results obtained from the analysis of the INS-ATOMIC navigation system data (based on atomic inertial sensors) are presented and discussed. All navigation outputs, including velocity, acceleration, and Euler angles, are ultimately expressed in the local navigation frame (ENU). Any internally used inertial axes (X,Y,Z) are transformed into ENU before performance evaluation, using the orientation matrix from the strapdown mechanization. The navigation performance presented in this section reflects the propagation of the non-Gaussian residuals obtained in Section 5 using the VITANS INS/GPS reference data.

The corresponding results of these quantities are generated to assess the performance of state estimation algorithms under real-world conditions and to compare the CI-RKF and MKMC-EKF.

This analysis is aimed at evaluating the stability of the estimators in the presence of Gaussian and non-Gaussian noise, environmental disturbances, outliers, and varying dynamic conditions. Furthermore, the drift behavior in angular velocity, linear velocity, and attitude outputs over time is examined to evaluate the long-term accuracy of the system. Finally, the impact of employing robust filter structures (particularly those utilizing statistical measures such as the CI-RKF and MKMC-EKF) is investigated in terms of enhancing estimation stability and reducing sensitivity to external disturbances.

Figure 7(a) shows relative velocity components estimated by the CI-RKF and MKMC-EKF against the VITANS reference data in the local navigation frame. The results demonstrate the superior accuracy and stability of the MKMC-EKF. In the horizontal directions, including northward and eastward motion, the MKMC-EKF exhibits closer alignment with the reference trajectory and significantly reduced fluctuations compared with the CI-RKF. Whereas the CI-RKF tends to show deviations and instability during dynamic maneuvers (particularly under acceleration or direction changes), the MKMC-EKF maintains consistent tracking and effectively suppresses measurement noise.

FIGURE 7

Comparison of tracking performance of the CI-RKF and MKMC-EKF with respect to the VITANS reference

In the vertical component, which is inherently more prone to oscillations and estimation noise, the performance gap between the two filters becomes even more pronounced. The CI-RKF produces outputs characterized by abrupt variations and noticeable noise, whereas the MKMC-EKF offers a smoother and more continuous trajectory that closely follows the reference signal. This behavior indicates the robustness of the MKMC-EKF in handling non-Gaussian disturbances and outliers more effectively than its counterpart.

The incorporation of the multi-kernel maximum correntropy criterion within the MKMC-EKF framework results in a substantial enhancement in tracking accuracy and noise resilience. The filter consistently delivers more reliable velocity estimates in all directions, with its performance particularly evident under dynamic scenarios and complex environmental perturbations. These findings underscore the practical value of MKMC-EKF in robust navigation and state estimation applications.

Figure 7(b) shows acceleration component estimates in the local coordinate system obtained using the CI-RKF and MKMC-EKF. The results demonstrate the superior accuracy and robustness of the MKMC-EKF, particularly under dynamic conditions and in the presence of non-Gaussian noise. Along the longitudinal axis, the MKMC-EKF successfully reconstructs the actual behavior with smoother and more stable estimates, whereas the CI-RKF exhibits noticeable instability owing to its higher sensitivity to disturbances and outliers.

In the lateral direction, the MKMC-EKF also provides more accurate tracking of the general trend and effectively suppresses noise compared with the CI-RKF. The difference between the filters becomes more evident in the vertical axis, where the MKMC-EKF yields a continuous and reference-consistent estimation, while the CI-RKF suffers from significant oscillations. These results highlight the enhanced resilience of the MKMC-EKF to complex and non-Gaussian noise, confirming its advantage in achieving stable and reliable state estimation.

Figure 7(c) shows angular rate component estimates in the local coordinate system obtained using the CI-RKF and MKMC-EKF. The results demonstrate the superior accuracy and robustness of the MKMC-EKF, particularly under dynamic conditions and in the presence of non-Gaussian noise. Along the longitudinal axis, the MKMC-EKF successfully reconstructs the actual behavior with smoother and more stable estimates, whereas the CI-RKF exhibits noticeable instability due to its higher sensitivity to disturbances and outliers. In the lateral direction, the MKMC-EKF also provides more accurate tracking of the general trend and more effectively suppresses noise compared with the CI-RKF. The difference between the filters becomes more evident in the vertical axis, where the MKMC-EKF yields a continuous and reference-consistent estimation, while the CI-RKF suffers from significant oscillations. These results highlight the enhanced resilience of the MKMC-EKF to complex and non-Gaussian noise, confirming its advantage in achieving stable and reliable state estimation.

Figure 7(d) shows a comparison of estimated Euler angles (ϕ, θ, and ψ) obtained using the CI-RKF and MKMC-EKF. The MKMC-EKF demonstrates a smoother response and closer alignment with the reference data compared with the CI-RKF. For the phi component, the MKMC-EKF effectively attenuates oscillations and provides estimates closely following the actual values. For the theta angle, the filter maintains minimal deviation and avoids abrupt fluctuations, showing strong agreement with the ground truth. The psi component, characterized by sudden discontinuities, is also more accurately reconstructed by the MKMC-EKF, which tracks sharp transitions with greater stability.

These observations highlight the superior capability of the MKMC-EKF in accurately monitoring rotational dynamics while maintaining robustness against complex noise patterns.

6.2 Evaluation of Estimation Errors and Navigation System Stability with an Emphasis on Drift Analysis

The following section analyzes the cumulative errors and drift behavior resulting from the filtered estimates along the three spatial axes. The evaluation focuses on the stability and noise sensitivity of each filter, which serve as key performance indicators in this analysis.

To quantitatively evaluate the estimation performance of the filters, the outputs are compared with the reference trajectory provided by the VITANS system. The estimation error is defined as the difference between the filter estimate and the reference value. Based on this error sequence, the RMSE, the mean error, and the standard deviation of the error are adopted as the principal measures of accuracy, bias, and error dispersion, respectively. These terms are formulated in Equations (44)-(46). In addition, the final drift denotes the accumulated error at the end of the data-recording interval:

RMSE=1Nk=1N(xkestxkref)244

Mean=1Nk=1N(xkestxkref)45

σ=1Nk=1N(eke¯)246

Here, xkest denotes the state estimated by the filter at the sampling instant k, and xkref represents the corresponding reference value provided by the VITANS system. N is the total number of samples within the evaluation interval. The index k indicates the discrete-time step. The difference (xkestxkref) defines the instantaneous estimation error, denoted as ek, used to compute the metrics in Equations (44) and (45). For the metric in Equation (46), ek represents the instantaneous error, and e¯ denotes the mean error calculated in Equation (45).

A smaller RMSE corresponds to a higher estimation accuracy, whereas a smaller mean value indicates a lower systematic bias. A smaller standard deviation (σ) signifies lower variability in the estimation errors, indicating more consistent and repeatable performance.

Figure 8(a) presents the velocity drift and error behavior of the CI-RKF and the proposed MKMC-EKF algorithms when compared against the reference VITANS data. Detailed quantitative results for these comparisons are presented in Table 5. These findings consistently demonstrate the superior performance of the MKMC-EKF across all velocity components.

FIGURE 8

Comparison of drift for the VITANS reference, CI-RKF, and MKMC-EKF

View this table:
TABLE 5 Comparative Drift Analysis Across Estimation Methods

As detailed in Table 5, the CI-RKF exhibits significant instability for the east velocity (Ve), characterized by a high final drift. In contrast, the MKMC-EKF maintains exceptional long-term stability, maintaining the accumulated error at a negligible level. Table 5 also shows that this enhanced stability is accompanied by improved accuracy, reflected by a notable reduction in the RMSE and mean error. These metrics collectively indicate a substantial mitigation of systematic bias by the MKMC-EKF.

In the north velocity component (Vn), a significant performance gap is evident from Table 5. The CI-RKF experiences moderate drift, while the MKMC-EKF achieves a dramatically lower final drift, approaching near-complete suppression of cumulative error by the experiment's end. As shown in Table 5, this improvement is mirrored in the RMSE and mean error, both of which are substantially reduced with the MKMC-EKF, confirming a significant decrease in estimation bias.

A similar and pronounced enhancement is observed for the vertical velocity component (Vu), as presented in Table 5. The CI-RKF shows considerable drift, whereas the MKMC-EKF effectively limits the drift to a very low level. Table 5 further indicates that the RMSE is clearly improved, demonstrating enhanced overall accuracy. Moreover, the mean error is dramatically reduced, indicating a strong mitigation of bias.

In summary, the MKMC-EKF demonstrates a substantial enhancement in navigation performance compared with the CI-RKF. The marked reduction in final drift across all axes highlights major improvements in long-term stability and effective suppression of cumulative errors, with specific values detailed in Table 5. The consistent decrease in RMSE confirms higher estimation accuracy, while the significantly smaller mean error values reveal a pronounced mitigation of systematic bias. These findings collectively verify that the proposed MKMC-EKF framework provides a more reliable and robust solution for sustained operation under realistic noisy and dynamic conditions, outperforming the CI-RKF across key performance indicators.

Figure 8(b) shows results for acceleration. For the ae acceleration component, the CI-RKF output is affected by noticeable noise and instability, leading to a significant final drift at the end of the experiment. When the MKMC-EKF is employed, this value drops to an almost negligible level, indicating an effective containment of error growth over time, as detailed in Table 5. At the same time, the RMSE is substantially reduced, and the mean error also decreases significantly. These results point to a smoother reconstruction of the signal and a strong attenuation of both random and systematic disturbances.

In the an direction, the CI-RKF shows a comparatively milder behavior, yet residual noise remains visible in the accumulated error, as shown in Table 5. For the MKMC-EKF, the final drift is dramatically reduced, demonstrating a much stronger convergence. The decrease in RMSE is substantial, and in addition, the mean error is significantly smaller, which indicates improved alignment with the reference and a reduced tendency toward gradual deviation.

The improvement becomes even more pronounced for the au component. The CI-RKF suffers from strong fluctuations and produces a noticeable drift. In contrast, the MKMC-EKF confines the accumulated error to a very low level. The RMSE shows a clear reduction, and the mean error approaches near-zero values. This behavior reflects a considerable enhancement in dynamic consistency and an efficient suppression of fast perturbations.

The acceleration results reveal that the MKMC-EKF does more than simply reduce numerical error levels; it changes the quality of the estimation process. The outputs become smoother, more convergent, and better matched to the reference trajectory. The simultaneous reductions in final drift, RMSE, and bias confirm the improved temporal stability and higher reliability obtained by the MKMC-EKF in realistic noisy environments. The RMSE reduction achieved by the MKMC-EKF is significant across the acceleration axes. A comparable decrease is also observed in the mean error, confirming a strong suppression of systematic bias in addition to random noise.

Figure 8(c) shows results for the angular rate. For the ωe component, the CI-RKF exhibits noticeable error accumulation, with a significant final drift. The proposed MKMC-EKF suppresses this value to a remarkably low level, indicating a strong containment of long-term divergence, as detailed in Table 5. The RMSE is sharply reduced, and likewise, the mean error decreases substantially. This behavior highlights a substantial improvement in both precision and bias regulation.

The ωn component exhibits a different pattern. While the CI-RKF remains relatively controlled in terms of drift, the MKMC-EKF pushes the final drift down to an extremely small magnitude. At the same time, the RMSE decreases significantly, and the mean error is also substantially reduced. These changes indicate a tighter tracking capability and improved estimation balance.

For the ωu component, the contrast between the two filters is particularly clear. The CI-RKF produces a noticeable accumulated drift, whereas the MKMC-EKF drives this value to almost zero. The reduction in RMSE confirms a significant gain in accuracy. Furthermore, the mean error decreases substantially, demonstrating a remarkable mitigation of systematic effects.

The angular rate results verify the effectiveness of the MKMC-EKF in controlling rotational error propagation. Drift growth is consistently restrained, estimation variance is lowered, and bias is minimized. The filter delivers cleaner and more dependable attitude-related information. Such improvements are essential for maintaining orientation integrity in precision navigation tasks. The MKMC-EKF achieves significant RMSE and mean error reductions across the rotational axes, confirming a strong attenuation of systematic bias in addition to random fluctuations.

Figure 8(d) shows results for the Euler angles. For the roll angle ϕ, the CI-RKF solution shows a slow but persistent divergence, resulting in a noticeable final drift. With the MKMC-EKF, the drift is confined to a much lower level, indicating a more stable attitude solution, as supported by the accompanying data in Table 5. The RMSE decreases, while the most notable change appears in the mean error, which drops significantly. This trend suggests a strong reduction of systematic offset in the roll estimation.

For the pitch angle θ, the CI-RKF exhibits a clear trend that gradually separates from the reference. The proposed filter substantially limits this behavior, dramatically reducing the final drift. A pronounced improvement is also observed in accuracy, as the RMSE decreases. Meanwhile, the mean error is significantly reduced. These results confirm better alignment and reduced long-term deviation.

The yaw angle ψ further illustrates the advantage of the MKMC-EKF. The CI-RKF output is influenced by strong fluctuations and yields a significant final drift. In comparison, the proposed method maintains a controlled drift level while simultaneously achieving a remarkable refinement in precision. The RMSE decreases substantially, and the mean error falls significantly. This combination indicates improved smoothness and more consistent heading information. The MKMC-EKF provides a substantial enhancement in attitude estimation. The final drift, RMSE, and mean error are all significantly reduced compared with the CI-RKF, confirming stronger stability, higher accuracy, and effective suppression of systematic bias.

The Euler angle evaluation demonstrates that the MKMC-EKF enhances attitude determination quality across all axes. The estimates remain closer to the reference, long-term divergence is restricted, and bias levels are markedly reduced. The filter therefore provides a more coherent and trustworthy orientation solution. This enhanced reliability is crucial for downstream navigation computations, as these subsequent processing steps, including accurate localization, path planning, and vehicle control, require dependable orientation data to function correctly.

The MKMC-EKF exhibits a remarkable ability to reduce drift in all Euler angles, reflecting its resilience to non-Gaussian noise and environmental disturbances. This performance underscores its suitability for high-precision navigation systems, particularly when leveraging atomic-grade inertial sensors.

Table 5 summarizes the relative drift values and the percentage of improvement achieved by the MKMC-EKF. These results clearly demonstrate the superior stability, higher robustness to non-Gaussian noise, and improved capability of the MKMC-EKF in capturing complex dynamic behaviors.

7 CONCLUSION

In this study, a comprehensive evaluation of robust EKF techniques was conducted to enhance the reliability and accuracy of INSs under real-world dynamic conditions. Leveraging high-precision atomic inertial sensors, including atomic gyroscopes and accelerometers, the proposed framework was designed to address the inherent challenges posed by non-Gaussian noise, dynamic maneuvers, and environmental uncertainties.

The experimental results highlight the challenges posed by outliers and unmodeled dynamics for standard filtering approaches. The CI-RKF offered a moderate improvement in estimation stability by incorporating statistical consistency tests to manage measurement anomalies. However, the MKMC-EKF consistently outperformed the CI-RKF in all evaluated domains (i.e., linear velocity, acceleration, angular rates, and Euler angles), demonstrating superior robustness and accuracy.

The superior performance of the MKMC-EKF is attributed to its ability to model heavy-tailed and impulsive noise distributions through a multi-kernel correntropy framework. This ability allowed for more accurate state tracking, reduced drift over time, and enhanced filter robustness, particularly during aggressive maneuvers, sharp rotations, and slope transitions. The filter maintained high tracking fidelity even in vertical components, which typically suffer from increased measurement uncertainty and dynamic variability.

The integration of atomic inertial sensing with advanced robust filtering not only improved localization accuracy but also ensured greater estimator resilience in noisy, real-time environments. These findings affirm the potential of the MKMC-EKF as a reliable core for next-generation INSs, especially for applications where GNSSs are unavailable or degraded.

HOW TO CITE THIS ARTICLE

Imani, F., & Keighobadi, J. (2026). Robust state estimation in atomic inertial navigation systems: A comparative study of CI-RKF and MKMC-EKF filters. NAVIGATION, 73. https://doi.org/10.33012/navi.768.

This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

References

  1. Biswal, A., & Jwo, D.-J. (2024). Maximum correntropy extended Kalman filtering with nonlinear regression technique for GPS navigation. Applied Sciences, 14, 7657. https://doi.org/10.3390/app14177657
  2. Deng, D., Yi, P., & Xiong, J. (2025). An adaptive robust event-triggered variational Bayesian filtering method with heavy-tailed noise. Sensors, 25, 3130. https://doi.org/10.3390/s25103130
  3. Fornasier, N. G., Mahony, R., & Weiss, S. (2022, May). Equivariant filter design for inertial navigation systems with input measurement biases. In International Conference on Robotics and Automation (ICRA) (pp. 43334339). https://doi.org/10.1109/ICRA46639.2022.9811778
  4. Hao, P., Karakuş, O., & Achim, A. (2023). Robust Kalman filters based on the sub-Gaussian α-stable distribution. ArXiv. https://doi.org/10.48550/arXiv.2305.07890
  5. Jwo, D.-J., Lai, C.-H., Biswal, A., & Cho, T.-S. (2024). An extended Kalman filter with mixture kernel maximum correntropy criterion for GPS navigation applications. Transactions of the Institute of Measurement and Control,48(4), 772780. https://doi.org/10.1177/01423312241287333
  6. Li, S., Li, L., Shi, D., Zou, W., Duan, P., & Shi, L. (2022). Multi-kernel maximum correntropy Kalman filter for orientation estimation. IEEE Robotics and Automation Letters, 7, 66936700. https://doi.org/10.1109/LRA.2022.3176798
  7. Li, S., Shi, D., Lou, Y., Zou, W., & Shi, L. (2023a). Generalized multikernel maximum correntropy Kalman filter for disturbance estimation. IEEE Transactions on Automatic Control, 69, 37323747. https://doi.org/10.1109/TAC.2023.3321368
  8. Li, S., Shi, D., Zou, W., & Shi, L. (2021). Multi-kernel maximum correntropy Kalman filter. IEEE Control Systems Letters, 6, 14901495. https://doi.org/10.1109/LCSYS.2021.3114137
  9. Li, Z., Chen, W., Sun, Y., & Chen, Z. (2023b). An improved multiple-outlier robust filter based on maximum correntropy criterion for integrated navigation. IEEE Sensors Journal, 23, 1745117461. https://doi.org/10.1109/JSEN.2023.3288033
  10. Noor, R. M., Gundeti, V., & Shkel, A. M. (2017, March). A status on components development for folded micro NMR gyro. In IEEE International Symposium on Inertial Sensors and Systems (INERTIAL) (pp. 156159). https://doi.org/10.1109/ISISS.2017.7935691
  11. Pagoti, S. K., & Vemuri, S. I. D. (2022). Development and performance evaluation of correntropy Kalman filter for improved accuracy of GPS position estimation. International Journal of Intelligent Networks, 3, 18, https://doi.org/10.1016/j.ijin.2022.01.002
  12. Peng, X., Zhang, B., & Rong, L. (2019). Robust unscented Kalman filter and its application in estimating dynamic positioning ship motion states. Journal of Marine Science and Technology, 24, 12651279. https://doi.org/10.1007/s00773-019-00624-5
  13. Phillips, A. M., Wright, M. J., Riou, I., Maddox, S., Maskell, S., & Ralph, J. F. (2022). Position fixing with cold atom gravity gradiometers. AVS Quantum Science, 4, 024404. https://doi.org/10.1116/5.0095677
  14. Salducci, C., Bidel, Y., Cadoret, M., Darmon, S., Zahzam, N., Bonnin, A., Schwartz, S., Blanchard, C., & Bresson, A. (2024). Quantum sensing of acceleration and rotation by interfering magnetically launched atoms. Science Advances, 10, 4498. https://doi.org/10.1126/sciadv.adq4498
  15. Saywell, J. C., Carey, M. S., Light, P. S., Szigeti, S. S., Milne, A. R., Gill, K. S., Goh, M. L., Perunicic, V. S., Wilson, N. M., Macrae, C. D., Rischka, A., Everitt, P. J., Robins, N. P., Anderson, R. P., Hush, M. R., & Biercuk, M. J. (2023). Enhancing the sensitivity of atom-interferometric inertial sensors using robust control. Nature Communications, 14, 7626. https://doi.org/10.1038/s41467-023-43374-0
  16. Shettell, N., & Dumke, R. (2023). Emulating an atomic gyroscope with multiple accelerometers. AVS Quantum Science, 5, 045003. https://doi.org/10.1116/5.0166281
  17. Sun, W., Zhang, X., Ding, W., Zhang, H., & Liu, A. (2023). Maximum correntropy-based robust square-root cubature Kalman filter for vehicular cooperative navigation. Scientific Reports, 13, 22961. https://doi.org/10.1038/s41598-023-50377-w
  18. Templier, S., Cheiney, P., d'Armagnac de Castanet, Q., Gouraud, B., Porte, H., Napolitano, F., Bouyer, P., Battelier, B., & Barrett, B. (2022). Tracking the vector acceleration with a hybrid quantum accelerometer triad. Science Advances, 8, 3854. https://doi.org/10.1126/sciadv.add3854
  19. Travagnin, M. (2020). Cold atom interferometry for inertial navigation sensors. Technology Assessment: Space and Defence Applications. https://doi.org/10.2760/237221
  20. Wang, B., & Wang, Z. (2025). Adaptive Kalman filter under minimum error entropy with fiducial points for strap-down inertial navigation system/ultra-short baseline integrated navigation systems. Journal of Marine Science and Engineering, 13, 990. https://doi.org/10.3390/jmse13050990
  21. Wang, D., Zhang, H., Huang, H., & Ge, B. (2023). A redundant measurement-based maximum correntropy extended Kalman filter for the noise covariance estimation in INS/GNSS integration. Remote Sensing, 15, 2430, https://doi.org/10.3390/rs15092430
  22. Wright, M. J., Anastassiou, L., Mishra, C., Davies, J. M., Phillips, A. M., Maskell, S., & Ralph, J. F. (2022). Cold atom inertial sensors for navigation applications. Frontiers in Physics, 10, 994459. https://doi.org/10.3389/fphy.2022.994459
  23. Zhang, J., Feng, K., Li, J., Zhang, C., & Wei, X. (2025). An adaptive unscented Kalman filter integrated navigation method based on the maximum versoria criterion for INS/GNSS systems. Sensors (Basel, Switzerland), 25, 3483. https://doi.org/10.3390/s25113483
  24. Zhang, L., Lou, Y., Song, W., Zhang, W., & Peng, Z. (2024). Performance enhancement of PPP/ SINS tightly coupled navigation based on improved robust maximum correntropy Kalman filtering. Advances in Space Research, 74, 20782091. https://doi.org/10.1016/j.asr.2024.05.072
Loading
Loading
Loading
Loading