Abstract
The past several years have seen a proliferation of software-defined radio (SDR) data collection systems and processing platforms designed for or applicable to satellite navigation (satnav) applications. These systems necessarily produce datasets in a wide range of different formats. To correctly interpret this SDR data, essential information such as the packed sample format and sampling rate is needed. Communicating this metadata between creators and users has historically been an ad-hoc, cumbersome, and error-prone process. To address this issue, the satnav SDR community developed a metadata standard and normative software library to automate this process, thus simplifying the exchange of datasets and promoting the interoperability of satnav SDR systems. The standard was ratified and formally accepted as an Institute of Navigation Standard in January 2020. This article describes the ION GNSS SDR metadata standard and associated open-source software project. All content associated with the standard is available on sdr.ion.org.
1 INTRODUCTION
Software-defined radio (SDR) receivers are a rapidly advancing area in satnav research and design. The past several years have seen tremendous growth in this field. Universities and other research organizations have developed and demonstrated advanced capabilities, particularly with respect to multi-constellation and multi-band satnav and satnav-plus-multi-sensor integrated navigation processing for GNSS-challenged environments. The recent commercial availability of multi-sensor data collection equipment, low-cost development platforms, and open-source software tools has catalyzed this rapid pace of innovation. Indeed, emerging commercial receivers are also embracing the SDR paradigm for software-based upgrades to support new signals and capabilities after units have been shipped. These innovations are motivated in no small part by today’s ongoing deployment of multiple global and regional satnav constellations with their diverse signal structures, the rapid advancements in low-power all-programmable systems-on-a-chip (SOCs), as well as the proliferation of inexpensive sensors.
Non-real-time SDR operational scenarios involve the storage and post-processing of samples. These sampled data files are also employed in radio frequency (RF) playback systems that are used for the satnav receiver test and evaluation in laboratory environments. Post-processing and/or playback requires several generic front-end parameters such as the RF and intermediate frequency (IF) center frequencies, sampling rate, file format, as well as satnav-specific information such as antenna location and type. We define this information as satnav SDR metadata. Traditionally, the manual transfer of metadata has been the default method – a process that is cumbersome and error-prone to say the least. No established method previously existed to exchange satnav SDR metadata automatically in a machine-readable format. During the ION GNSS+ 2013 conference, a group of attendees discussed the need for a formal standard to facilitate the automated exchange of satnav SDR metadata. This group identified the following benefits to the positioning, navigation, and timing (PNT) community for engaging in this activity:
It identifies and brings the international satnav SDR community together as a working group. This collaboration is critical in order to get broad acceptance and adoption of the standard.
Standardization will help to avoid technology segmentation issues while promoting the pace of innovation by employing standard practices and compliant tools.
The formal standard, if widely adopted, will help ensure compatibility and interoperability of future satnav SDRs. Specifically, front-end agnostic “plug-and-play” satnav SDRs were envisioned. These have the potential for revolutionizing PNT systems of the future.
Most contributors to satnav SDR technology and authors of significant publications in the field over the past two decades are active ION members and frequent technical meeting attendees. Working group meetings could therefore be scheduled to coincide with ION conferences. Hence, it was decided to pursue this standardization activity through ION sponsorship. During the January 2014 Council Meeting in San Diego, ION approved the process for establishing a formal standard (GNSS SDR Metadata Standard Working Group, 2014). The ION GNSS SDR Metadata Working Group (WG) was formed in April 2014. Membership represented academia, industry (including satnav SDR product vendors as well as traditional satnav equipment manufacturers), non-profit research entities, and government agencies spanning countries in Europe, America, Asia, and Australia. The working group developed the standard as well as associated normative software over a course of six years. The draft standard was adopted as a formal ION standard in January 2020. This article provides an overview of the standard and its development.
2 JUSTIFICATION FOR SATNAV SDR METADATA STANDARDIZATION
Figure 1 illustrates the satnav SDR metadata transfer scheme traditionally employed by the PNT community. The top row depicts data collection system (DCS) A, producing an SDR file with format A, and consumed by SDR processor A. This may represent an end-to-end solution provided by a commercial vendor or a system that was initially developed around a specific hardware platform. In any case, it may be assumed that the metadata for decoding format A is hard-coded into the SDR processor. Consequently, supporting other file formats involves extensive software revisions on a case-by-case basis. A group of researchers may want to share SDR files from DCS A with other groups using SDR processors X and Y for research collaboration. This involves conveying the file format and other relevant information accurately to these groups. Traditionally, this metadata transfer occurred in an ad-hoc way that is prone to interpretation errors.
Illustration of ad-hoc metadata exchange between SDR data creators and users
The second row depicts multi-stream DCS B that produces files with a more complicated format. SDR processors X and Y represent more flexible SDRs that are able to support multiple file formats. However, the data/metadata association requires manual intervention.
DCS C multiplexes other data (such as sensor data) along with multiple satnav sample streams in the same file. This type of multiplexed collection has the potential to become more common with emerging SDR-related data streaming standards such as VITA 49 (Cooklev et al., 2012). In this case, custom-designed processor C represents an SDR that fully supports multi-sensor integration capabilities. Because DCS C’s satnav stream parameters are open, SDR Y is also able to support satnav-only processing using an ad-hoc metadata transfer scheme. The sensor data parameters in format C may or may not be open.
As evident from Figure 1, this ad-hoc method of metadata exchange does not encourage interoperability and instead cultivates potential for technology segmentation (i.e., various PNT groups developing their own stove-piped solutions and technologies).
Figure 2 shows the same systems of Figure 1, but now they have adopted a metadata standard. As shown, each DCS produces a compliant metadata file along with the SDR file. The metadata file is read-in by the compliant SDR processor to correctly decode and process files seamlessly.
Illustration of seamless satnav SDR data exchange enabled by metadata standardization
Adoption of a metadata standard benefits DCS developers because their systems become applicable to a much wider group of users. Similarly, an SDR processor’s utility is extended when it is capable of supporting many file formats from multiple sources seamlessly. Thus, metadata standardization promotes interoperability of satnav SDR systems and greatly simplifies the exchange of files between groups.
It should be noted that prior to this ability for files from various SDRs to be interpreted and decoded seamlessly using the metadata standard, to facilitate sharing, some PNT research groups were known to “up-sample” their SDR data to a much higher bit depth and sample rate so as to be compatible with another group’s adopted format–a process that was incredibly inefficient.
Metadata standardization also benefits other use cases beyond post-processing satnav SDRs. For example, consider the use of the metadata specification to synthesize compliant SDR files for use in RF playback systems. Libraries of compliant SDR file sets containing various real-world scenarios could be used interchangeably in compliant RF playback simulators for repeatable and consistent testing of satnav receivers.
Many commercially available SDR devices collect and pack sampled signals in native machine types as part of their internal design or use of available software drivers. For example, two-bit samples are written as eight-bit signed words to disk. The ION SDR Metadata Standard, along with its normative software library (to be described later in this article), serves as a useful tool to compact such files for long-term archival as well as improves their share ability.
3 SDR DATA COLLECTION TOPOLOGIES
The ION Executive Committee stipulated that this standardization activity shall not create an unfair advantage or disadvantage to any entity. Specifically, the standard shall not require any existing system to undergo data format changes to achieve compliance. This do no harm stipulation implied that the standard needed to be designed such that it supports all current and future SDR file formats. This also meant that the WG “get this right the first time” since major revisions to the metadata standard would be undesirable to the goal of widespread adoption. Hence, the WG considered the entire space of possible satnav SDR data collection topologies to determine how metadata describing these can be represented in a clear and concise manner. Figure 3 illustrates these fundamental topologies.
Fundamental satnav SDR data collection topologies
Figure 3.a illustrates the simplest data collection topology that can exist. This is when a single contiguous region of RF spectrum (referenced henceforth as a “band”) is down-converted and sampled to produce a single data stream. The stream, which may be IF sampled (real samples) or baseband sampled (complex samples), is written to disk as a single file.
Figure 3.b represents a DCS that writes parts of a single data stream across multiple files. This may be done to reduce the sustained write performance requirement of storage drives. For example, a system may write odd data blocks (where a block comprises samples with in a set interval, such as 1 ms) to File 0 and even 1 ms blocks to File 1. This is similar to the striping mode in the redundant array of independent disk (RAID) systems. This also covers the case of a multi-band recording with a one-to-one correspondence between band and file thereby allowing a multi-band recording into separate files. The files can later be separated from each other, and for each sample file, an independent metadata description can be found.
Figure 3.c is similar to Figure 3.a, except that the data stream represents more than one RF band. An example of this topology is a direct RF sampling front-end architecture that intentionally aliases multiple bands to fall next to each other at baseband. Some bands may be spectrally inverted as a result of the digital down-conversion process.
A DCS may produce multiple data streams. Each stream may contain information from one of the many antenna elements (as shown in Figure 3.d, where each stream may also encompass multiple bands as in Figure 3.c). Alternatively, a stream may be sampling one of several bands received by a single wideband antenna, where the multiplexed streams written to file represent channelized samples. Combinations of the above are also possible.
Each stream may also be sampled at different rates and bit depths. For example, consider a civilian GPS L1, L2, L5 system. In this case, the L1 and L2 streams may be sampled at rate 𝑓𝑠0 and the L5 stream at 10𝑓𝑠0 (since the L5 signal’s first-null bandwidth is ten times wider than L1 C/A and L2C). In this example, 𝑓𝑠0 represents the base sample rate. Figure 3.d may also represent how these multiple streams are multiplexed into a single lane of packed binary data and written to a single file.
Similar to Figure 3.d, Figure 3.e illustrates a satnav data stream multiplexed with other data that is written to a single file. This non-GNSS SDR data may be from additional sensors (as shown) and may be encoded in a proprietary format that may or may not be known. The specification of metadata parameters for non-SDR data is outside the scope of this standardization effort. However, the standard supports necessary information to skip over these data bytes. Since the metadata schema is extensible, it can support the description of non-SDR data as needed by specific users.
It is important to note that although we use the term “satnav SDR data” in this article to generally refer to the type of sampled data for which metadata parameters are defined in the standard, the samples need not correspond to satnav frequency bands. For example, frequency bands containing RF signals of opportunity are supported as long as they can be represented by the standard’s defined metadata parameters. In this context, this standard’s usage is not only limited to satnav, but can serve as a powerful tool for many unforeseen SDR applications. This is especially true since the metadata definition is designed to be extensible.
Due to the typically high data rate of satnav SDRs, and sometimes due to the low write speed of storage devices, some DCSs write data as temporally split files, as illustrated in Figure 3.f. For example, long data collections of several hours may be written to disk as a series of short-duration files. This allows for efficient data management through multiple smaller files compared to a single large file. The metadata for each file specifies the previous and next file names in order to represent the ordered sequence. Note that the DCS block shown in Figure 3.f could represent any of those described in topologies a, c, d, and e.
Figure 3.g illustrates the spatial splitting of files. Here, the binary data lanes from two or more DCSs are written to separate files. These files may be written by different computer systems. In this case, a nonzero intersystem timing offset, Δ𝑡, may exist and is supported in the standard. Since Δ𝑡 may or may not be known at the time of collection, it represents an example metadata parameter that can be back-annotated into the metadata file after an initial data processing phase.
Multilane DCSs that write each lane to a separate file may also implement temporal file splitting according to Figure 3.f. We refer to this as “spatial-temporal splitting” of files, and it is illustrated in Figure 3.h.
4 METADATA CLASSES
The ION metadata standard enables the adopter to describe any known SDR file format and other information essential to satnav SDR processing. The standard employs a set of predefined objects, known as metadata classes, to specify these properties and their hierarchical association. This is conceptually illustrated in Figure 4.
Conceptual illustration of metadata classes and their hierarchical association used to form a complete metadata specification
In broad terms, the ION standard’s metadata classes can be categorized into the following types. The formal definition of each class and their detailed description is given in the standard document (ION GNSS SDR Standard Working Group, 2020).
4.1 Metadata associated with a given data collection activity
Information associated with a data collection activity includes the date, time, and duration of the activity, initial location of the system, the name of the campaign, and the person or organization that performed the activity (point of contact). This information is specified within the Session class. The Session class also refers to the data collection system(s) that it employs. The Session class represents the top level of the metadata class hierarchy.
4.2 Metadata associated with data collection systems
The data collection session employs one or more physical data collection units, each referred to herein as a data collection system. Each system takes, as input, signals produced by one or more antennas (or other originators of signals). These are here in referred to as sources. Each data collection system ultimately produces one or more digital streams of formatted data containing samples created from the sources. These data are then written to disk, creating one or more SDR files. In the standard, the formatted data streams are referred to as lanes. These lanes and the files produced by writing lanes to disk are here in referred to as sinks.
Each System class appears as a single object under the Session. The System class in turn refers to sources and sinks, as well as other information such as the base sampling rate and equipment details.
4.3 Metadata associated with data sources
For satnav applications, the relative positioning of antennas (sources) is critical. For example, the precise location and orientation of antenna element phase centers are essential for multielement processing. Groupings of antenna elements are referred to as a Cluster. The Cluster class specifies the relative locations of the sources in a specified coordinate frame. The Source class specifies information relevant to a single source (antenna element).
4.4 Metadata associated with formatted data
As described previously, a data collection system converts signals from one or more sources into one or more lanes of formatted digital data. This formatting can vary widely between data collection systems. In this standard, formatting is represented by a hierarchy of classes: Band(s), Stream(s), Lump(s), Chunk(s), and Block(s). A band is defined as a span of RF spectrum. The Band class describes how the center frequency of the band is translated to either baseband or IF. The Stream class describes the discrete time series output from a sampling device (such as sample rate, depth, and binary representation). The stream may contain multiple bands. The Lump class describes how samples from one or more streams that occur during the base sample interval (i.e., the largest integer sampling rate factor of all streams) is ordered. The Chunk class describes how one or more lumps are packed within one of the four standard one-, two-, four-, or eight-byte unsigned integer data types. Finally, the Block class describes a collection of chunks and optional header and/or footer bytes that may lead or trail these series of chunks. Together these classes describe SDR sampling and packed sample formatting. In fact, the sample decoding scheme can be determined algorithmically from this hierarchy of classes.
4.5 Metadata associated with data sinks
As described previously, the Lane class represents the formatted data that is written to disk. Multiple lanes are written to multiple files (spatial splitting), and this file writing process can be broken up into smaller sequentially ordered files (temporal splitting). Information needed to describe all of the files written during a session is contained in the FileSet class. Hence, Lane(s), File(s), and FileSet(s) represent sink classes.
5 METADATA FORMAT AND SCHEMA
The standard employs the extensible markup language (XML) to describe the core metadata classes (World Wide Web Consortium, n.d.). The formatting is enforced using an XML Schema Definition (XSD) (ION Metadata Standard Working Group, n.d.-b). Figure 5 shows an example of an ION standard-compliant metadata file (ION SDR Metadata Working Group, n.d.). This example shows header and footer data embedded within each block that contains non-satnav SDR sample data. As stated previously, an SDR processor that is unaware of the formatting for this nonstandard data simply skips over the relevant bytes. On the other hand, due to the extensible nature of XML, the formatting for this data can also be supported as a custom extension of the ION metadata standard.
ION metadata file example (ION SDR Metadata Working Group, n.d.)
6 NORMATIVE REFERENCE SOFTWARE
During the initial phases of development, the WG determined that it was essential to have open-source normative software as a reference implementation of the standard. The goal of this effort was to promote early and widespread adoption of the standard by making it straight forward for vendors and researchers to achieve standard compliance by integrating this library into their existing software. The WG was fortunate to receive voluntary participation for this task from individuals representing established commercial satnav SDR vendors.
The first contribution was a library that produces standard-compliant metadata files from an appropriately populated data structure in memory and a library capable of reading these files back to a data structure. The second contribution was in the form of a library capable of parsing and converting sample data based on the standard-compliant metadata description. When combined, these two libraries offer sufficient functionality to enable adoption of the metadata standard in SDRs, or to serve as a benchmark against which implementations of the standard can be validated.
The normative software is written in C++ and managed using CMake (Kitware Inc., n.d.). It is organized as two libraries: apilib implements the metadata interpreter; converterlib implements the binary data conversion. The libraries are accompanied by a selection of applications. These include a data-converter application and a simple format-agnostic SDR front-end application. All software associated with the standard is available through the Institute of Navigation’s GNSS Metadata Standard GitHub repository (ION SDR Metadata Working Group, n.d.).
The metadata-interpreter library includes a reader that can parse a metadata file and populate a corresponding metadata object that can then be queried through a selection of member-function calls. Similarly, a metadata object can be instantiated and configured through a selection of member functions, and subsequently, it can be instructed to write a standard-compliant metadata file.
The converter library can be used for parsing binary data files and interpreting the data according to a standard-compliant metadata file. The basic converter can be adapted to support processing of the converted data streams. Two such adaptations have been included in the reference software. The first is depicted in Figure 6, where the data converter is embedded within a file-converter application. The file converter configures the embedded data converter using a metadata-interpreter object and is capable of parsing a packed binary input file and producing one file per SDR data stream in a user-specified data type (int8, int16, float, double, etc.).
Use of ION metadata-compliant data converter embedded with in a file-converter application
The second functionality embeds the data converter as the front end of a satnav baseband processor, as depicted in Figure 7. This front end offers a means of loading short portions of the binary data file and converting them to a user-specified data type (int8, int16, float, double, etc.) while handling details such as sample alignment when different streams are sampled at different rates.
Use of ION metadata-compliant data converter as the front end of a satnav SDR baseband processor
The software suite includes a selection of example binary datasets and associated metadata files along with a simple MATLAB/Octave script to test the build against reference datasets. Several different file formats have been included in the repository including a wide range of front-end configurations and data-packing variations.
A number of working group members volunteered to perform “blind testing” of SDR data files against draft specifications. This involved exchanging SDR data files and associated metadata specifications among parties and verifying that the files can be fully decoded solely based on the metadata specification.
7 SDR DATA REPOSITORY
As with most standards and programming projects, intricacies are best understood through good examples. As part of this standard development effort, the WG created an SDR file repository to serve as the reference dataset for the standard (ION Metadata Standard Working Group, n.d.-a). This repository contains SDR data files and corresponding metadata files that represent a large part of the SDR data formats used by the PNT community. These formats cover many of the topologies covered in Figure 3.
All file sets within the repository have been tested to follow the standard and to be readable by the normative reference software. The binary files typically contain samples over a duration of more than 60 seconds. Recently, many of these files have been processed by well-known satnav SDRs, and the signal observables (pseudorange, carrierphase, and carrier-to-noise-density ratio) have been published on the site (Pany et al., 2019; CTTC, n.d.). Further details about processed observables can be found in Gunawardena et al. (2019). These observables serve to further aid the process of validating their own satnav SDR implementations.
8 EXAMPLES OF STANDARD ADOPTION BY ACADEMIA AND RESEARCH
Even during its development phase, several research and academic organizations adopted the standard and incorporated the normative software into their satnav SDR implementations and tools. Examples of this early adoption can be found in Favenza et al. (2016), Linty et al. (2018), Lucas-Sabola et al. (2016), Rügamer et al. (2017), and Zhu et al. (2015).
9 EXAMPLES OF INDUSTRY ADOPTION AND COMMERCIAL PRODUCTS USING STANDARD
The standard has seen early adoption by several commercially available products as listed below. Additional examples of adoption can be found in Rügamer (2018):
GTEC© GNSS Radio Frequency Front-End and MGSE Record and replay device from TeleOrbit GmbH (Tele-Orbit GmbH, n.d.)
SX3 GNSS Software Receiver from IFEN GmbH (IFEN GmbH, n.d.)
GIPSIE (GNSS multisystem performance simulation environment) GNSS Simulator from OHB Digital Solutions (OHB Digital Solutions, n.d.)
10 MAINTENANCE OF STANDARD
Formal development of the standard was concluded by the end of 2019. The standard was ratified and formally accepted as an Institute of Navigation Standard in January 2020. The core group of the original WG has pledged their commitment to maintaining the standard for the foreseeable future. This includes performing minor revisions to the standard and its documentation, updating and maintaining the example SDR file repository, updating and revising the normative software repository found on GitHub, as well as maintaining the permanent landing page at sdr.ion.org.
11 SUMMARY
There has been a proliferation of satnav SDR data collection systems in recent years. These systems necessarily produce datasets in a wide range of different formats. Correctly interpreting the metadata associated with these various datasets in order to correctly decode the embedded sample streams has historically been a cumbersome and error-prone process. To address this issue, the satnav SDR community developed the ION SDR Metadata Standard through consensus. The standard was ratified and formally accepted as an Institute of Navigation Standard in January 2020. This article provides an overview of the standard, its associated open-source normative software project, and examples of early adoption. All content associated with the standard can be accessed by visiting sdr.ion.org.
HOW TO CITE THIS ARTICLE
Gunawardena S, Pany T, Curran J. ION GNSS software-defined radio Metadata Standard. NAVIGATION.2021;68: 11–20. https://doi.org/10.1002/navi.407
ACKNOWLEDGMENTS
The ION GNSS SDR Metadata Working Group was comprised of 62 members representing academia, government, and industry. The members represented 12 countries in North America, Europe, Asia, and Australia. Throughout this effort, approximately 25 members were actively involved in the writing of the standard, the normative software development, and its testing and verification. Approximately 50% of members participated in meetings held in conjunction with ION technical conferences, provided invaluable comments, suggestions, and revisions of the various drafts and technical reports. The Working Group gratefully acknowledges the support of the ION staff during the development of the standard. Without your help, this work would not have been possible. ION leadership and Council are thanked for their continuous encouragement and support. We also acknowledge the work of many students, interns, and staff members in each of our organizations that worked on the many aspects of the standard. During its development, at various times, many members of the Working Group gave their time and energy and worked tirelessly to make this effort ultimately a success. Some of these members have left the PNT community or are no longer active. We sincerely appreciate all your hard work. Thank you!
The authors are sincerely grateful to the reviewers of this article for their invaluable comments and suggestions.
- Received August 6, 2020.
- Revision received November 11, 2020.
- © 2020 The Authors. NAVIGATION published by Wiley Periodicals LLC on behalf of Institute of Navigation.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.