Code Monkey home page Code Monkey logo

prx's Introduction

prx

prx reads a RINEX 3.05 observation file and outputs a CSV file of the following format

# {"approximate_receiver_ecef_position_m": [4627852.5264, 119640.514, 4372994.8358], "input_files": [{"name": "TLSE00FRA_R_20240011200_15M_01S_MO.rnx", "murmur3_hash": "f9fbd80567dac9c5f55586521385b067"}, {"name": "BRDC00IGS_R_20240010000_01D_MN.rnx", "murmur3_hash": "f7df97045419097c85cae76bb9324ccb"}], "prx_git_commit_id": "82aae81874c5dd81973720397c42d0305456a2e9"}
time_of_reception_in_receiver_time,clock_m,dclock_mps,x_m,y_m,z_m,dx_mps,dy_mps,dz_mps,ephemeris_hash,frequency_slot,relativistic_clock_effect_m,sagnac_effect_m,tropo_delay_m,group_delay_m,carrier_frequency_hz,code_iono_delay_klobuchar_m,elevation_deg,azimuth_deg,observation_code,C_obs,D_obs_hz,L_obs_cycles,S_obs_dBHz,constellation,prn
2024-01-01 12:00:00.000000,57153.224736,-0.018397,21838606.222023,36012172.441686,-1479022.231769,-3.184508,4.675864,-0.920381,2168809070289459030,1.000000,-0.668490,-39.902407,10.053189,0.029979,1561098000.000000,-15.044328,13.082414,115.281248,2I,40176280.391000,-14.730000,209208378.723000,35.400000,C,05
2024-01-01 12:00:00.000000,57153.224736,-0.018397,21838606.222023,36012172.441686,-1479022.231769,-3.184508,4.675864,-0.920381,2168809070289459030,1.000000,-0.668490,-39.902407,10.053189,0.000000,1268520000.000000,-22.784446,13.082414,115.281248,6I,40176273.273000,-11.969000,169998932.830000,36.900000,C,05
2024-01-01 12:00:00.000000,57153.224736,-0.018397,21838606.222023,36012172.441686,-1479022.231769,-3.184508,4.675864,-0.920381,2168809070289459030,1.000000,-0.668490,-39.902407,10.053189,-2.728111,1207140000.000000,-25.160417,13.082414,115.281248,7I,40176276.934000,-11.391000,161773185.396000,38.400000,C,05

...

In addition to the observation values extracted from the RINEX file, the rows of the CSV contain the corresponding satellite position and speed at the time of transmission as well as relativistic effects, broadcast group delays, broadcast ionosphere delays etc.

Running prx

From the prx repository root, run

poetry env use 3.12
poetry install
poetry shell
python src/prx/main.py --observation_file_path <path_to_rinex_file> 

You might have to add <path to prx root>/src/prx to your PYTHONPATH environment variable if you run into import errors.

How we manage python version and dependencies

We use poetry to make sure every developer and user of prx runs the same python version and the same set of dependencies such as numpy and pandas. Poetry reads pyproject.toml, resolves dependencies and writes the result to poetry.lock, which is the file used when running poetry install.

To install poetry see https://python-poetry.org/docs/#installation

To create the virtual environment, run poetry install in the prx repository root. This has to be run every time an update to the pyproject.toml and poetry.lock files is done (for example, after a git pull)

Jumping into the virtual environment

Run poetry shell to activate the virtual environment in a terminal.

Using the poetry virtual environment in PyCharm

Run poetry env info -p to find the path of the virtual environment, which we'll call <venv-path> Under File -> Settings -> Project -> Python Interpreter add <venv-path> as python interpreter. If you run into problems, try <venv-path>/bin/python on Linux or <venv-path>\Scripts\python.exe on Windows.

Add packages to poetry

Let's say you wrote some code that uses import new_package. To have new package added (what you otherwise would use pip or some other package manager for), run

poetry add new_package

Poetry will resolve dependencies - i.e. figure out which is the latest version of new_package that is compatible with our virtual environment - and add it to pyproject.toml and poetry.lock.

Testing

After poetry shell

Run pytest in the prx repository root to run all tests. Run pytest -x to stop after the first test that fails. Run pytest -k "my_test" to run a specific test Run pytest ---durations=10 to run all tests and have pytest list the 10 longest running tests.

Coding style

We use https://google.github.io/styleguide/pyguide.html as our python style guide.

Acronyms

See the Rinex 3.05 spec, page 5, for a list of most acronyms used in the code. Those not covered by the RINEX spec are listed below.

Acronym Long Form
PRX Preprocessed Rinex

prx's People

Contributors

janbolting avatar jtec avatar plutonheaven avatar shetty1103 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

prx's Issues

User-provided RNX NAV file

It may be interesting to let the user provide the NAV file together with the OBS file, e.g. to verify time to first fix in a constrained environment leading to difficulty to demodulate the Navigation Messages.

PRX currently download the NAV files from IGS servers, and therefore has always access to well-formatted files. Letting users provide their own files may break a few assumptions that we are making by using IGS files.

If we let other NAV files possible, careful check should be performed, for example by running additional options with gfzrnx.

Help to import lib/georinex

I have difficulties to use functions from the submodule georinex.

The file directory of my PyCharm project is

prx
|_ lib
    |_ georinex
|_ minimumLovablePrototype
    |_ processing_ephemerides.py

In processing_ephemerides.py, I imported the georinex module using import lib/georinex as gr
But when i call gr.load(path_to_rnx3), I get an error :

AttributeError: module 'lib.georinex' has no attribute 'load'

I was able to call gr.load() outside my PyCharm project, in a simple python3 session, by calling in Terminal

pip3 install georinex
python3
>>> import georinex as gr
>>> nav = gr.load('/home/paul/PycharmProjects/prx/datasets/TLSE_2022001/BRDC00IGS_R_20220010000_01D_MN.rnx')

I assume I don't do a correct import of the submodule. This is maybe because it is present in a parent folder, however, PyCharm seems to validate my call, and does not like when I tried from ..lib import georinex (as read from stackoverflow)

The code is in 82b42a5. Any suggestion @jtec?

Property-based testing

The unit tests we have today test specific sets of inputs such as "Compute position of G02 at time t and compare to sp3 precise position".

Property-based testing takes this approach a step further. It allows us to formulate tests of the type "For a test within the set of existing satellites, and for a time within t0 and t1, the error between broadcast and sp3 position is always lower than 10 meters".

The testing framework then tries to find test inputs within the set of admissible inputs that break the code.

See e.g. https://semaphoreci.com/blog/property-based-testing-python-hypothesis-pytest

Rename parameters computed by PRX

The name of the parameters computed by PRX should be

  • explicit (to be recognized by someone working on GNSS)
  • model-agnostic
  • contain the unit

For example, the parameter

  • x_m is not explicit enough
  • iono_delay_klobuchar_m is not model-agnostic

GFZRNX is not executed correctly on Windows

There is no executable of GFZRNX for Windows and the call to other binaries does not create an error (returncode = 0).

There is also the issue of the annoying pop-up when executing a command like gfzrnx_2.0-8219_lx64 in the Windows terminal.

One consequence is that parsing some NAV files result in some errors due to missing trailing spaces for some ephemerids (e.g. NAVIC)

Older NAV files from BKG do not contain IONO CORR in header

NAV files older than 2023-01-01 from the BKG http repo do not contain the iono correction in their header. We may want to find other repositories with such information, or be tolerant to missing header information.

Side note: the name for files older than 2023-01-01 change from BRDC00IGS_[...] to BRDC00WRD_[...].

Add dynamic unit test dataset

We need movement to unit-test the Doppler observation (by estimating antenna velocity) and carrier phase outputs (by estimating displacement between epochs with carrier phase time differences) of prx. Without antenna displacement, sign errors are not observable.

NAVIC ephemerides not tested

NAVIC ephemerides, being Kepler orbits and second-order clock models, might work out-of-the box, but we need a test for them. That includes testing

  • Position
  • Velocity
  • Clock offset
  • Clock offset rate
  • Group Delays, if any

Position-domain unit test

Let's add a unit test that

  • runs a RINEX observation file through prx
  • parses the generated CSV file
  • computes the LSQ PVT solution
  • compares to the expected solution
    • what would we compare the T to? Clock-steering receiver, so small?

Fancy version: use a station where >= 4 satellites of all constellations are visible (e.g. Japan), parameterize the test with a set of constellations to use and run for

  • all constellations
  • each constellation separately

Add Loss of Lock Indicator

This data is already present in the RINEX OBS file and just has to be written by prx.

It was done in #31, but the large refactoring lost the feature.

Consideration of the satellite health flag

There is a health flag that is transmitted in the broadcast navigation message.

We could add it as an additional column, or just discard the observations of the satellites with do not use status.

Add System Time Offset

Sytem Time Offset (RINEX 4) or Time System Corr (RINEX 3) are specified in RNX NAV files.

We could add this as an additional parameter, to ease the PVT computation by having to estimate only a single clock offset (usually RX time to GPST).

Observation files spanning several days

The ephemeris file discovery only consider a single ephemeris file, while sometimes the observations span several days.

Notably, this happens when dealing with an observation received at midnight (time of reception), which would have a corresponding time of emission on the day before.

This bug was addressed in #32, but the large refactoring effort lost this modification.

Code coverage

Measuring code coverage will allow us to spot untested parts of the code, and in the long run improve it's correctness, which is what prx is all about.

Many projects put in place a coverage threshold, I think this turns it from a useful tool into something "we need to do" though, and lowers the likelihood of people contributing. I see no problem with merging PRs that have low coverage and enhancing coverage, possibly across the whole code base, in later PRs.

Pipenv does not respect package version

e.g.

[packages]
numpy = "*"
pandas = "2.0.3"
pytest = "*"
georinex = "1.16.1"
gnss-lib-py = "0.1.8"

in Pipfile

and running pipenv install

leads to

 "gnss-lib-py": {
            "hashes": [
                "sha256:5fcd7681e44a2ea15a9697e7cc31c377f03b266186a499e8774311e394d377cd",
                "sha256:ad665fb083cefc777c94c354dcb5b254c68d2940b718fe556ab60ba8f9293e9a"
            ],
            "index": "pypi",
            "version": "==0.1.11"

in Pipfile.lock

Feature request: Parsing android raw measurements and navigation messages

I think it'd be great if prx can directly parse Android Raw measurements for pr/doppler/cp/cn0 logs, and Android navigation messages for ephemeris on supported phones.
Many new mobile phones now support nav messages too, which can be logged using GnssLogger App. Parsing this data directly from GnssLogger App .txt log is important, as GnssLogger miss some info when it logs rinex obs file - for example logging only dual band on triple band phones. Also, I found no lib until now that parses android navigation message from logs.

I would be opening similar issue in https://github.com/geospace-code/georinex too, but I think it's more related to prx than georinex

Running gfzrnx should not modify the input file

What if we copy the observation file in a new folder and add everything associated to the processing job: nav files, intermediate files, generated prx file, log files. For the folder name (job id) hash the file provided by the user.

No IODE and IODC in prx file

When time-differencing observations, and removing terms due to satellite motion and clock offset change it is important to know the IODE and IODC, since consecutive broadcast ephemerides are not continuous.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.