Comments (7)
Yes, you're right @philshem
The problem is that we can never be sure which format are the timestamps. But, again, if we implement the fix from #92 then we can:
- reverse the order of parsers (
'%Y-%m-%d %H:%M:%S.%f
first). This will solve this issue. - change the existing DATE_FORMATS to be functions. Those function can do some basic sanity checks before parsing the timestamp the usual way. For example, if input string has no dots
.
then it makes no sense to continue parsing with'%Y-%m-%d %H:%M:%S.%f
. That way we can avoid multiple timestamp parsers in most cases.
from gpxpy.
Reversing the order is something I can do, but I guess you are referring to the DATE_FORMAT ;)
But as for the function for DATE_FORMAT, I'm more than happy to wait. At the moment I use a local dev version of gpxpy, but it's really custom and not good for a patch. Many thanks!
from gpxpy.
Up ?
from gpxpy.
Yes, I'm here but unfortunately I don't have the time to investigate. In this specific case I'm not 100% sure what you mean "got duplicate timestamps".
It would help a lot if you could provide me a code which fails (along with the gpx file). Ideally, it would be a test in tests.py
but a standalone script is also OK.
from gpxpy.
I'll do that
from gpxpy.
I'm not sure if this is the same issue, because I've not tested in Python 3, but I am seeing milliseconds truncated to seconds.
The reason may be because my GPX file starts with no (null) milliseconds value. Here are the first two timestamps of my file. The first one has only seconds, the second has 200 milliseconds.
- 2017-08-10T15:59:28Z
- 2017-08-10T15:59:28.200Z
My guess at the cause is the I think the issue is due to the DATE_FORMATS lookup, which may be only checked once per GPX file.
In gpxpy/gpxpy/gpxfield.py, the function parse_time() is called in the TimeConverter() class and contains a call to mod_gpx.DATE_FORMATS, which loops over potential datetime formats. I suspect that the first matching format is returned, which in my case is a timestamp without milliseconds.
From gpx.py
# GPX date format(s) used for parsing. The T between date and time and Z after
# time are allowed, too:
DATE_FORMATS = [
'%Y-%m-%d %H:%M:%S',
'%Y-%m-%d %H:%M:%S.%f',
#'%Y-%m-%d %H:%M:%S%z',
#'%Y-%m-%d %H:%M:%S.%f%z',
]
From gpxfield.py (parse_field function)
for date_format in mod_gpx.DATE_FORMATS:
try:
return mod_datetime.datetime.strptime(string, date_format)
except ValueError:
pass
Here's a sample code, using the pastebin GPX file linked above
import gpxpy
import gpxpy.gpx
with open('simple.gpx','rb') as gpx_file:
g = gpxpy.parse(gpx_file)
for track in g.tracks:
for segment in track.segments:
for point in segment.points:
print type(point.time), point.time
prints the second timestamp without milliseconds
<type 'datetime.datetime'> 2017-08-10 15:59:28
<type 'datetime.datetime'> 2017-08-10 15:59:28
I'm using Python 2.7.12 (64 bit) and gpxpy==1.1.2
from gpxpy.
One more thing, as I said, I'm pretty busy for any complex changes in gpxpy at the moment. But I think this one should be less than an hour of work (including tests). I think I could find some time this or next weekend. (of course, I'm accepting pull requests if somebody needs this sooner).
from gpxpy.
Related Issues (20)
- NameError: free variable 'from_start_to_end' referenced before assignment in enclosing scope HOT 1
- Setup.py method is being deprecated in python package
- Confusion!
- Wrong distance? 10.4 km instead of 7.6 km HOT 1
- Save nsmap of GPX in GPXTrack, GPXTrackSegment and GPXTrackPoint
- GPX track all messed up HOT 1
- Tag a release to ship improvements since 1.5.0 HOT 1
- TypeError in gpxfield since 1.6.0 release HOT 1
- Changes Backported To 1.5.0 HOT 2
- Typo in __repr__ of SimpleTZ HOT 1
- [newbie] Wrong return type on GPXTrackSegment.get_duration() HOT 1
- create CI testing scripts HOT 1
- feature request: include distance_from_start in NearestLocationsData class HOT 1
- Is Header malformed
- Check for GPX root node on parse HOT 1
- Custom attributes in `to_xml` not escaped properly HOT 1
- Wrong type annotation for radians in method elevation angle?
- to_xml removes metadata extensions without the name tag present HOT 1
- Implement support for adding extension section to a waypoint/track/...
- Links in XML not properly encoded
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gpxpy.