Research ArticleRegular Papers
Open Access

Improving GNSS Positioning Correction Using Deep Reinforcement Learning with an Adaptive Reward Augmentation Method

Jianhao Tang, Zhenni Li, Kexian Hou, Peili Li, Haoli Zhao, Qianming Wang, Ming Liu, and Shengli Xie
NAVIGATION: Journal of the Institute of Navigation December 2024, 71 (4) navi.667; DOI: https://doi.org/10.33012/navi.667

Article Figures & Data

Figures

  • ALGORITHM 1
    ALGORITHM 1

    LSTMPPO-ARAM method for GNSS positioning correction

  • FIGURE 1
    FIGURE 1

    Framework of the training phase and testing phase of the proposed LSTMPPO-ARAM method

  • FIGURE 2
    FIGURE 2

    Example trajectories in semi-urban and urban environments of the target domain from the GSDC 2022 data set, with corresponding real-world images of each trajectory shown in each inset (a) Highway in San Francisco (SFO, semi-urban) (b) Highway in Mountain View (MTV, semi-urban) (c) Downtown in Mountain View (MTV, urban) (d) Road in Los Angeles (LAX, urban)

  • FIGURE 3
    FIGURE 3

    Example trajectories for training and evaluation from the Guangzhou GNSS measurement data set, with a corresponding real-world image of each trajectory shown in each inset (a) Example trajectory for training in Science City (b) Example trajectory for evaluation in Semi-urban-2 (Ersha Island)

  • FIGURE 4
    FIGURE 4

    Performance of the proposed LSTMPPO-ARAM for a range of η and learning rate lr values in semi-urban and urban environments (a) SFO (semi-urban) (b) MTV (semi-urban) (c) MTV (urban) (d) LAX (urban)

  • FIGURE 5
    FIGURE 5

    Performance of DNN-based methods for a range of learning rate values in semi-urban and urban environments (a) SFO (semi-urban) (b) MTV (semi-urban) (c) MTV (urban) (d) LAX (urban)

  • FIGURE 6
    FIGURE 6

    Convergence curves of cumulative rewards for training in semi-urban and urban target environments for A3C, LSTMPPO, and LSTMPPO-ARAM with corresponding optimal parameters (a) SFO (semi-urban) (b) MTV (semi-urban) (c) MTV (urban) (d) LAX (urban)

  • FIGURE 7
    FIGURE 7

    Convergence curves of total loss for training in semi-urban and urban target environments for A3C, LSTMPPO, and LSTMPPO-ARAM with corresponding optimal parameters (a) SFO (semi-urban) (b) MTV (semi-urban) (c) MTV (urban) (d) LAX (urban)

  • FIGURE 8
    FIGURE 8

    Reward augmentation (Δr) for training in semi-urban and urban source domain environments

  • FIGURE 9
    FIGURE 9

    Evaluation performance of the positioning distance error of trajectories collected by different types of mobile phones in different target semi-urban and urban environments (a) SFO (semi-urban) (b) MTV (semi-urban) (c) MTV (urban) (d) LAX (urban)

  • FIGURE 10
    FIGURE 10

    Positioning performance for one testing trajectory in MTV (urban)

    The left panel shows the overall trajectory, and the right panel provides a detailed view of the positioning trajectory and a corresponding real-world image.

  • FIGURE 11
    FIGURE 11

    Distribution of the number of visible satellites for Science City (target domain), a highway (source domain), and two environments for evaluation (Semi-urban-1 and Semi-urban-2) from the Guangzhou GNSS measurement data set (a) Science City (target domain) (b) Highway (source domain) (c) Semi-urban-1 (evaluation) (d) Semi-urban-2 (evaluation)

  • FIGURE 12
    FIGURE 12

    Distribution of the PODP for Science City (target domain), a highway (source domain), and two environments for evaluation (Semi-urban-1 and Semi-urban-2) from the Guangzhou GNSS measurement data set (a) Science City (target domain) (b) Highway (source domain) (c) Semi-urban-1 (evaluation) (d) Semi-urban-2 (evaluation)

    The mean and standard deviation of the PODP are shown at the top of each plot.

  • FIGURE 13
    FIGURE 13

    Trajectory plots of the positioning distance error in longitude and latitude for the RTK method and the proposed LSTMPPO-ARAM method in the Semi-urban-1 and Semi-urban2 environments for evaluation (a) Semi-urban-1 (b) Semi-urban-2

Tables

  • TABLE 1

    Experimental Details of Environments from the GSDC 2022 Data Set

    Target environmentTrajectory number for targetTrajectory number for sourceDescription
    San Francisco (SFO, semi-urban)730Highways and open roads in suburbs
    Mountain View (MTV, semi-urban)731Highways and open roads in urban fringes
    Mountain View (MTV, urban)325Downtown regions surrounded by buildings
    Los Angeles (LAX, urban)317Roads and overpasses surrounded by towns and forests
  • TABLE 2

    Evaluation Performance in Semi-Urban Environments: Average Positioning Errors (Mean±Standard Deviation) Along Different Axes and Total Distance in the ECEF Coordinate System for Different Methods with Corresponding Optimal Parameters

    SFO (Semi-urban)MTV (Semi-urban)
    X (m)Y (m)Z (m)Total distance (m)X (m)Y (m)Z (m)Total distance (m)
    WLS2.28±1.452.34±1.681.85±1.514.17±2.191.61±1.481.98±2.111.76±2.183.48±2.99
    KF (baseline)2.21±1.132.61±1.181.61±1.094.21±1.480.92±0.791.26±1.180.90±0.912.12±1.10
    KF+Deep sets (Zaheer et al., 2017)2.10±1.532.52±1.701.57±1.274.03±1.971.00±0.821.27±1.180.90±0.892.14±1.31
    KF+Set-transformer (Kanhere et al., 2021)2.09±1.572.52±1.701.57±1.284.04±1.960.92±0.791.26±1.180.90±0.912.09±1.32
    KF+GCNN (Mohanty & Gao, 2022)2.07±1.502.50±1.741.56±1.233.99±1.951.17±0.911.53±1.331.39±1.042.70±1.44
    KF+A3C (Zhang & Masoud, 2020)2.12±1.622.59±1.171.62±1.094.16±1.471.02±0.731.18±1.010.99±0.802.13±1.12
    KF+LSTMPPO (Zhao et al., 2023)2.09±1.142.43±1.171.57±1.083.96±1.450.95±0.721.06±0.960.87±0.781.91±1.10
    KF+LSTMPPO-ARAM2.08±1.192.12±1.241.61±1.133.77±1.530.86±0.711.07±0.950.91±0.791.87±1.03
  • TABLE 3

    Evaluation Performance in Urban Environments: Average Positioning Errors (Mean±Standard Deviation) Along Different Axes and Total Distance in the ECEF Coordinate System for Different Methods with Corresponding Optimal Parameters

    MTV (Urban)LAX (Urban)
    X (m)Y (m)Z (m)Total distance (m)X (m)Y (m)Z (m)Total distance (m)
    WLS2.28±2.033.51±2.562.65±2.455.61±3.222.72±1.913.83±2.663.76±2.316.73±2.77
    KF (baseline)1.25±0.942.45±1.071.26±0.773.40±1.122.29±1.373.25±1.863.20±1.495.63±1.73
    KF+Deep sets (Zaheer et al., 2017)1.36±0.982.06±1.361.74±1.423.34±1.682.50±1.792.75±1.952.96±1.835.40±1.97
    KF+Set-transformer (Kanhere et al., 2021)1.17±0.991.93±1.701.94±1.443.33±1.943.19±2.362.57±2.052.75±2.465.81±2.53
    KF+GCNN (Mohanty & Gao, 2022)1.16±0.892.19±1.201.23±0.993.09±1.162.44±1.852.79±1.832.75±2.075.29±2.09
    KF+A3C (Zhang & Masoud, 2020)1.26±0.952.03±1.091.46±0.733.14±0.972.44±1.722.85±1.693.10±1.725.39±1.84
    KF+LSTMPPO (Zhao et al., 2023)1.30±1.332.07±1.101.34±0.733.13±1.012.45±1.732.70±1.653.03±1.695.26±1.79
    KF+LSTMPPO-ARAM1.25±0.902.06±1.141.36±0.783.09±1.032.38±1.592.67±1.642.96±1.785.11±1.73
  • TABLE 4

    Performance for the Guangzhou GNSS Measurement Data Set: Distance Errors (Mean±Standard Deviation) in Longitude and Latitude for the RTK and LSTMPPO-ARAM Methods with Corresponding Optimal Parameters

    Science City (training)Semi-urban-1 (evaluation)Semi-urban-2 (evaluation)
    RTK (baseline)0.425±0.854 (m)0.782±0.461 (m)0.813±0.536 (m)
    RTK+LSTMPPO-ARAM0.398±1.067 (m)0.688±0.384 (m)0.726±0.435 (m)

Additional Files

  • Video Abstract
Back to top

In this issue

Print
Download PDF
Article Alerts
Email Article
Citation Tools
Improving GNSS Positioning Correction Using Deep Reinforcement Learning with an Adaptive Reward Augmentation Method
Jianhao Tang, Zhenni Li, Kexian Hou, Peili Li, Haoli Zhao, Qianming Wang, Ming Liu,, Shengli Xie
NAVIGATION: Journal of the Institute of Navigation Dec 2024, 71 (4) navi.667; DOI: 10.33012/navi.667
Twitter logo Facebook logo Mendeley logo
Bookmark this article

Jump to section

Keywords