Comments (8)
@flytrex-vadim FYI added branch accommodate-RTCM-mixed-msg-stream
with some minor tweaks to your 'patch', including an RTCM protocol filter setting so users can select whether or not to see any RTCM3 data in the stream.
from pyubx2.
Hi @flytrex-vadim,
Feedback and contributions always welcome, but if it was your intention to submit a code change for pyubx2
, could I ask that you refer to the guidelines in CONTRIBUTING.md and submit a formal Pull Request rather than a code snippet in a .txt file?
That being said, I'm unclear what you mean by 'broken iteration' or your expectations were here?pyubx2
is intended to be a UBX protocol parser, not an RTCM protocol parser. The UBXReader
module does - as of v1.2.4 - also parse NMEA messages (via the companion pynmeagps
library), but it should simply ignore and discard any incoming data that is not a valid UBX or NMEA message, and either continue with the next valid UBX or NMEA message or terminate if there are no such messages remaining in the input stream.
Are you saying that this is not the case? What actual error message are you getting? Would you be able to provide a copy of the raw binary datalog you're attempting to process so that I can test it at our end?
from pyubx2.
Hi @semuadmin ,
- Submitting a pull request would require me figure out how to do that, a task I'm not sure I have the time for at the moment. So I chose to share the code immediately in a way I'm familiar with, in the hope that it would be useful for somebody :)
- Indeed the scenario you described is not the case in the vanilla 1.2.4. Unknown message types cause a few "Unknown protocol" messages at misc binary locations followed by stop of iteration. I had to make my changes so that it ignores the RTCM messages and continues parsing the UBX and NMEA until the end of the logfile.
- What's the best way to send you the logfile privately ?
from pyubx2.
Can I confirm what you're setting the optional quitonerror
keyword argument to? By default, this is set to 1 (= log errors to terminal and continue), the other options being 0 (= ignore any errors) or 2 (= re-raise errors and terminate).
If you have this set to 2 and your data stream contains (for example) RTCM data, then you will indeed get an error - this is expected behaviour. If you're using the default value of 1, then you will see an 'invalid protocol' error message written to the terminal but the iteration should not terminate at that point unless there happen to be no further UBX or NMEA messages in the data stream. If you want to suppress all 'invalid protocol' messages, you can set quitonerror
to 0. Can you confirm?
You can send a binary data log to my email address [email protected] but I recommend checking the above first. Cheers.
from pyubx2.
I can confirm using the default flags. this is also validated by the fact that I see multiple 'invalid protocol' messages as the reader goes over all the bytes of the unknown message, two at a time.
Then at some point it finds a pair that accidentally match the header, goes into the parsing code and fails quietly somewhere (couldn't trace exactly where).
This code seems to not be built for re-synchronization after encountering unknown bytes for several reasons:
- two bytes are read at each iteration, so after odd-sized unknown message the reader would not be positioned on header
- after accidentally going into parsing and failing due to incorrect CRC or other error, the unparsed bytes are not returned to the stream to be tried again from next valid header.
from pyubx2.
OK if you could send me a copy of your raw datalog I'll test it at our end and see exactly how and why it's failing. Thanks.
I had a quick look at your 'patch' and it appears to be attempting to parse RTCM data headers, which is (currently at least) beyond the scope of what pyubx2
is designed for. If there is indeed a bug in the UBXReader.read()
algorithm I'd ideally like to fix it for all possible non-UBX data streams, not just RTCM.
from pyubx2.
sent. I've trimmed the log to reasonable size while making sure the offending section is close to the beginning.
I'm afraid that any attempt to generically handle all possible binary data inclusion would be ad-hoc and error prone. According to the documentation, at least the F9 units support only UBX, NMEA and RTCM output format. So adding the RTCM header detection allows clean support of all ublox outputs regarding of the device configuration. At the cost of only a few extra lines of code.
from pyubx2.
Having seen the problem - thanks for the log and code suggestion - I'm inclined to agree. Accommodating UBX, NMEA and RTCM would seem sensible and proportionate. At some point we may get round to implementing a proper Python RTCM parser, but for now I'll endeavour to at least accommodate a mixed stream (but only actually parse UBX or NMEA). Surprised this hasn't come up before.
Leave it with me - thanks again for the shout.
from pyubx2.
Related Issues (20)
- Getting CFG-PRT to change UART speed HOT 6
- Value of "dyn" field in CFG-NAV5 HOT 1
- pyubx2 stop processing when there is a broken data in the ubx file. HOT 3
- I think this could become slightly more higher level. HOT 1
- Meta-data for "MON-RF.group.flags.jammingState" is incorrect. It is specified as U1 when it should be U2.
- my get stuck in SocketStream.read() HOT 10
- Typo in UBX-ESF-INS message
- UBX-MGA-GPS-EPH size mismatch
- Trying to set configuration from configuration file HOT 1
- Configuration Interface error CFG_UART1_BAUDRATE HOT 1
- UBXReader(stream) fails silently if stream.read(n) returns < n bytes HOT 10
- Correct is diffSoln, no difSoln HOT 1
- rinex output HOT 4
- Enabling RTCM Corrections on the ZED-F9K receivers via serial communication HOT 2
- Parsing .ubx files HOT 1
- quitonerror option doesn't work for nmeamessage.py _do_attributes errors HOT 2
- pynmeagps>=1.0.17 Not found HOT 4
- Getting ACK/NAK response from command HOT 4
- Serialization of messages into bytes sometimes gives non-hex values HOT 2
- Add sigId to RXM-RAWX message in ubxtypes_get.py
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyubx2.