timezone / daylight saving not taken into account on exfat V1.2.3 about exfat HOT 24 CLOSED

A bit of historical info.

The exFAT implementation in Windows Vista (the one I used as a reference in 2008) supported timestamps only in local time. Later, in Windows XP and 7 they added timezone field: timestamps were still in local time but UTC time could be calculated by adding this offset. fuse-exfat does not support timezone fields (always writes 0 to them), i.e. acts like Windows Vista. This should not cause any problems while all your systems are in the same timezone.

Are you sure your Linux system has timezone set properly? Which distro is that?

The tz and time_offset options are not supported, thus they didn't make any change.

from exfat.

Dear relan,

thanks for the explanations.

I just double-checked the time. Time is taken automatically from internet using Ubuntu 16.04.

georg@netbook:~$ uname -a
Linux netbook 4.4.0-24-generic #43-Ubuntu SMP Wed Jun 8 19:27:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

georg@netbook:~$ date
So 26. Jun 21:51:57 CEST 2016

Strikingly I do see the +2:00 difference doing the same tests with a FAT formatted SD card. In my view exfat is missing either timezone or daylightsaving.

Since XP and Vista are running out, would it be possible to take into account the timezone fields. All cameras shall also be compatible to the Win7 and more recent versions thus leaving only linux to stand aside. Never tested android though ...

Greetings,

Georg

from exfat.

Since XP and Vista are running out, would it be possible to take into account the timezone fields.

Sure. But in your case the problem is somewhere else.

I couldn't reproduce this issue: a file created under Windows 7 has proper mtime under Fedora 23 (both in UTC+0300).

I need an FS image to analyze. Please

1. Erase you memory card:

   dd if=/dev/zero /dev/your_card_device bs=1M

2. Insert it into the camera, format and take a photo of something you can share (you can just cover the lens with your hand to shoot darkness).

3. Create a compressed dump of the FS:

   gzip -c /dev/your_card_device > exfat-dump.gz

4. Share this dump.

from exfat.

I can do this, but If a format the SDcard in the camera it will be FAT. For the tests both SDcards (FAT and exFAT) were formatted through Win7.
Shall I format them on the linux device this time? Or Win7 again?

from exfat.

In this case it does not matter where to format the memory card. For the clarity of the experiment let it be Windows again.

from exfat.

Hi relan,

here is the protocol/results:

SDcard formatted exFAT on Win7

SDcard inserted into camera: Filesystem is beeing rebuild
Camera is in CEST + Daylight savings

Picture taken at local time

Card taken out, inserted in netbook (CEST + Daylight savings).

georg@netbook:/media/georg/8E65-84B7/DCIM/100MSDCF$ identify -verbose DSC00958.JPG

..
Properties:
date:create: 2016-06-28T21:32:30+02:00
date:modify: 2016-06-28T21:32:30+02:00
..
exif:DateTime: 2016:06:28 20:32:31
exif:DateTimeDigitized: 2016:06:28 20:32:31
exif:DateTimeOriginal: 2016:06:28 20:32:31

georg@netbook:~$ sudo gzip -c /dev/sdb1 > exfat-dump.gz

SDcard removed & inserted in Win7

"touch" a file

SDcard removed and put back in netbook

georg@netbook:/media/georg/8E65-84B7$ ls --full-time
insgesamt 128
drwx------ 1 georg georg 32768 2016-06-28 21:32:03.000000000 +0200 AVF_INFO
drwx------ 1 georg georg 32768 2016-06-28 21:32:32.000000000 +0200 DCIM
-rwx------ 1 georg georg 0 2016-06-28 21:52:08.000000000 +0200 File_created_2052.txt
drwx------ 1 georg georg 32768 2016-06-28 21:32:32.000000000 +0200 MP_ROOT
drwx------ 1 georg georg 32768 2016-06-28 21:32:09.000000000 +0200 PRIVATE

again time difference of one hour

georg@netbook:~$ sudo gzip -c /dev/sdb1 > exfat-dump2.gz

georg@netbook:$ sudo dd if=/dev/zero of=/dev/sdb1 bs=1M
dd: Fehler beim Schreiben von '/dev/sdb1': Auf dem Gerät ist kein Speicherplatz mehr verfügbar
1876+0 Datensätze ein
1875+0 Datensätze aus
1967058432 bytes (2,0 GB, 1,8 GiB) copied, 223,428 s, 8,8 MB/s
georg@netbook:$

georg@netbook:/media/georg/8E65-84B7$ sudo mkfs.exfat /dev/sdc1
[sudo] Passwort für georg:
mkexfatfs 1.2.3
Creating... done.
Flushing... done.
File system created successfully.
georg@netbook:/media/georg/8E65-84B7$

Reinserted into camera, picture taken at local time 21:21, put back in netbook

(same behaviour as above with 1 hour difference)

georg@netbook:~$ sudo gzip -c /dev/sdc1 > exfat-dump3.gz

Hope this helps for finding the reasons. Thanks for your support,

Georg

exfat-dump.gz
exfat-dump2.gz
exfat-dump3.gz

from exfat.

Thanks for the dumps, I'll inspect them.

from exfat.

I've checked the dumps. I'm in MSK timezone (UTC+0300, no daylight saving).

First:

date:create: 2016-06-28T20:32:30+03:00
exif:DateTime: 2016:06:28 20:32:31

Second:

-rwxrwxrwx. 1 root root 0 2016-06-28 20:52:08.000000000 +0300 /tmp/t/File_created_2052.txt

Third:

date:create: 2016-06-28T21:21:20+03:00
exif:DateTime: 2016:06:28 21:21:20

All timestamps are in local time (both EXIF and exFAT) and they are correct. I think fuse-exfat misdetects your timezone. Could you compile and run this test:

#include <time.h>
#include <stdio.h>
#include <stdlib.h>

int main()
{
    time_t utc, local;

    tzset();

    utc = time(NULL);
    local = mktime(gmtime(&utc));
    printf("Timezone: %s, %ld, %d\n", getenv("TZ"), timezone, daylight);
    printf("UTC time:   %s", ctime(&utc));
    printf("Local time: %s", ctime(&local));
    printf("Offset: %ld - %ld = %ld\n", local, utc, local - utc);

    return 0;
}

What does it print?

from exfat.

Hi relan,

here we go:

georg@netbook:~/scripts/timezone$ ./a.out 
Timezone: (null), -3600, 1
UTC time:   Sat Jul  2 16:05:25 2016
Local time: Sat Jul  2 15:05:25 2016
Offset: 1467464725 - 1467468325 = -3600
georg@netbook:~/scripts/timezone$ date
Sa 2. Jul 16:05:51 CEST 2016
georg@netbook:~/scripts/timezone$ date -R
Sat, 02 Jul 2016 16:06:01 +0200
georg@netbook:~/scripts/timezone$ 

Uih, I do not completely understand the output. But at first sight UTC and local time are different from expectation.

Concerning your output:
I think it is reasonable, when the timestamps from the exFAT files in their local timezone upon creation are identical to the local time even if the computer is in a different time zone (as in your case). Thus as far as exif (and maybe also video) meta data is concerned create time is identical to meta data time stamps which are apparently currently always in local time without timezone.
This behaviour would be correct in all cases where timezones are identical across the used computers. In cases where different time zone are involved it is still a behaviour most users could agree with.

However in a more strict sense a exFAT file which crosses several timezones should show different local times (.. and identical UTC times). Maybe a new mounting option ?

Thanks for your help,

Georg

from exfat.

Seems like your timezone is wrong after all. Could you post the output of

readlink /etc/localtime
timedatectl

However in a more strict sense a exFAT file which crosses several timezones should show different local times (.. and identical UTC times). Maybe a new mounting option ?

Microsoft has added timezone fields exactly for this case. It'll be better to support them instead of making user specify a mount option.

from exfat.

georg@netbook:~$ readlink /etc/localtime
../usr/share/zoneinfo/Europe/Zurich
georg@netbook:~$ timedatectl
      Local time: So 2016-07-03 21:59:58 CEST
  Universal time: So 2016-07-03 19:59:58 UTC
        RTC time: So 2016-07-03 19:59:58
       Time zone: Europe/Zurich (CEST, +0200)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

from exfat.

I think I've found why fuse-exfat's timezone calculation code does not respect daylight saving time.

Could you compile and try the code from issue38 branch? It should fix this bug.

from exfat.

Hi relan,

unfortunately does not seem to work.

I installed with checkinstall. In synaptic a new version is seen for exfat (not for exfat-fuse nor for exfat-utils).

/dev/sdb1 on /media/georg/100RXM3 type fuseblk (rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,blksize=4096,uhelper=udisks2)

  Properties:
    date:create: 2016-07-03T12:53:34+02:00
    date:modify: 2016-07-03T12:53:34+02:00
    exif:BrightnessValue: 28304/2560
    exif:ColorSpace: 1
    exif:ComponentsConfiguration: 1, 2, 3, 0
    exif:CompressedBitsPerPixel: 2/1
    exif:Compression: 6
    exif:Contrast: 0
    exif:CustomRendered: 0
    exif:DateTime: 2016:07:03 11:53:35
    exif:DateTimeDigitized: 2016:07:03 11:53:35
    exif:DateTimeOriginal: 2016:07:03 11:53:35

from exfat.

Hmm... Could you please run this test and post the output:

#include <time.h>
#include <stdio.h>
#include <stdlib.h>

int main()
{
    time_t now;
    struct tm* utc;

    tzset();

    now = time(NULL);
    printf("Timezone: %s, %ld, %d\n", getenv("TZ"), timezone, daylight);
    printf("Local time: %s", asctime(localtime(&now)));
    utc = gmtime(&now);
    utc->tm_isdst = -1;
    printf("UTC time:   %s", asctime(utc));
    printf("Offset: %ld\n", mktime(utc) - now);

    return 0;
}

I'm trying to reproduce this issue by setting TZ variable:

$ TZ=Europe/Zurich ./a.out
Timezone: Europe/Zurich, -3600, 1
Local time: Wed Jul  6 10:51:37 2016
UTC time:   Wed Jul  6 08:51:37 2016
Offset: -7200

from exfat.

TZ is not set on my system:

georg@netbook:~/scripts/timezone$ ./a.out 
Timezone: (null), -3600, 1
Local time: Wed Jul  6 21:27:26 2016
UTC time:   Wed Jul  6 19:27:26 2016
Offset: -7200
georg@netbook:~/scripts/timezone$ date

georg@netbook:~/scripts/timezone$ TZ=Europe/Zurich ./a.out 
Timezone: Europe/Zurich, -3600, 1
Local time: Wed Jul  6 21:32:43 2016
UTC time:   Wed Jul  6 19:32:43 2016
Offset: -7200

UTC time:   Wed Jul  6 19:32:43 2016
Offset: -7200

TZ does not show up in printenv either. Hope this helps.

from exfat.

TZ is not set on my system

That's OK, /etc/localtime is used by default, while TZ variable just overrides it.

The output of this test looks correct. Please remove Debian's exfat-fuse package and make sure you are using fuse-exfat compiled from the issue38 branch.

from exfat.

Sorry relan I have problems getting the issue38 exfat-fuse installed. After removing with Synaptic installing issue38 via checkinstall and make install does not install exfat-fuse.

georg@netbook:~/source/exfat-issue38$ sudo make install
Making install in libexfat
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/libexfat“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/libexfat“ wird betreten
make[2]: Für das Ziel „install-exec-am“ ist nichts zu tun.
make[2]: Für das Ziel „install-data-am“ ist nichts zu tun.
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/libexfat“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/libexfat“ wird verlassen
Making install in dump
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/dump“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/dump“ wird betreten
 /bin/mkdir -p '/usr/local/sbin'
  /usr/bin/install -c dumpexfat '/usr/local/sbin'
 /bin/mkdir -p '/usr/local/share/man/man8'
 /usr/bin/install -c -m 644 dumpexfat.8 '/usr/local/share/man/man8'
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/dump“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/dump“ wird verlassen
Making install in fsck
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird betreten
 /bin/mkdir -p '/usr/local/sbin'
  /usr/bin/install -c exfatfsck '/usr/local/sbin'
make  install-exec-hook
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird betreten
ln -sf exfatfsck /usr/local/sbin/fsck.exfat
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird verlassen
 /bin/mkdir -p '/usr/local/share/man/man8'
 /usr/bin/install -c -m 644 exfatfsck.8 '/usr/local/share/man/man8'
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/fsck“ wird verlassen
Making install in fuse
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird betreten
 /bin/mkdir -p '/usr/local/sbin'
  /usr/bin/install -c mount.exfat-fuse '/usr/local/sbin'
make  install-exec-hook
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird betreten
ln -sf mount.exfat-fuse /usr/local/sbin/mount.exfat
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird verlassen
 /bin/mkdir -p '/usr/local/share/man/man8'
 /usr/bin/install -c -m 644 mount.exfat-fuse.8 '/usr/local/share/man/man8'
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/fuse“ wird verlassen
Making install in label
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/label“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/label“ wird betreten
 /bin/mkdir -p '/usr/local/sbin'
  /usr/bin/install -c exfatlabel '/usr/local/sbin'
 /bin/mkdir -p '/usr/local/share/man/man8'
 /usr/bin/install -c -m 644 exfatlabel.8 '/usr/local/share/man/man8'
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/label“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/label“ wird verlassen
Making install in mkfs
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird betreten
 /bin/mkdir -p '/usr/local/sbin'
  /usr/bin/install -c mkexfatfs '/usr/local/sbin'
make  install-exec-hook
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird betreten
ln -sf mkexfatfs /usr/local/sbin/mkfs.exfat
make[3]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird verlassen
 /bin/mkdir -p '/usr/local/share/man/man8'
 /usr/bin/install -c -m 644 mkexfatfs.8 '/usr/local/share/man/man8'
make[2]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38/mkfs“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38“ wird betreten
make[2]: Verzeichnis „/home/georg/source/exfat-issue38“ wird betreten
make[2]: Für das Ziel „install-exec-am“ ist nichts zu tun.
make[2]: Für das Ziel „install-data-am“ ist nichts zu tun.
make[2]: Verzeichnis „/home/georg/source/exfat-issue38“ wird verlassen
make[1]: Verzeichnis „/home/georg/source/exfat-issue38“ wird verlassen

from exfat.

make install does not install exfat-fuse

Why do you think so? From this log I can say that it has been sucessfully installed:

/usr/bin/install -c mount.exfat-fuse '/usr/local/sbin'

Automounting may fail because it expects mount.exfat in /usr/sbin, but you should be able to mount exFAT manually anyway.

If you want to make automounting work too:

sudo make uninstall # remove fuse-exfat files from /usr/local
./configure --prefix=/usr # set installation destination to /usr
make
sudo make install

from exfat.

Yes, automounting failed. I will try to mount manually or reinstall as you have suggested this evenning.

from exfat.

Hi relan,

good news:

 Properties:
    date:create: 2016-07-03T11:53:34+02:00
    date:modify: 2016-07-03T11:53:34+02:00
    exif:BrightnessValue: 28304/2560
    exif:ColorSpace: 1
    exif:ComponentsConfiguration: 1, 2, 3, 0
    exif:CompressedBitsPerPixel: 2/1
    exif:Compression: 6
    exif:Contrast: 0
    exif:CustomRendered: 0
    exif:DateTime: 2016:07:03 11:53:35
    exif:DateTimeDigitized: 2016:07:03 11:53:35
    exif:DateTimeOriginal: 2016:07:03 11:53:35

-rwxrwxrwx 1 root root    33 Jun 14 14:19 File created at local time 1419 on Win7.txt
-rwxrwxrwx 1 root root    64 Jun 14 14:21 File created at local time 1421 on Win7.txt
-rwxrwxrwx 1 root root     0 Jun 14 19:23 File created at local time 2023 on Ubuntu 16.04

There is a one hour difference at the linux file but this shall be due to writing this file at pre-issue38 conditions.

I did not manage to automount with the configure --prefix option. Manual mounting with mount -t exfat was unsuccessful as well but I got it with sudo mount.exfat-fuse /dev/sdb1 ./

I have another linux computer where I would need the modification. What is the best way of doing it. Or shall I wait for a new official release, ubuntu is currently at 1.2.3-1.

Thanks a lot in any case,

Georg

from exfat.

good news

Great!

There is a one hour difference at the linux file but this shall be due to writing this file at pre-issue38 conditions.

Yes, you are right. That's expected.

I did not manage to automount with the configure --prefix option.

Try

./configure --prefix=/

Maybe Ubuntu wants mount.exfat to be in /sbin.

I have another linux computer where I would need the modification. What is the best way of doing it. Or shall I wait for a new official release, ubuntu is currently at 1.2.3-1.

I'll definitely make v1.2.5 with this fix soon. You can, of course, wait for Ubuntu update, but this will be a long wait becuase it should first get into Debian.

from exfat.

./configure --prefix=/ that was the solution. It works now perfectly. To use checkinstall the fields provides and requires have to be filled with exfat-fuse,exfat-utils then the deb package is correct.
Thanks a lot for your help!
Georg

from exfat.

Awesome! Thanks for your patience.

from exfat.

Fix released in v1.2.5. Thanks for you help!

from exfat.