maxpat78 / fattools Goto Github PK
View Code? Open in Web Editor NEWFacilities to access (ex)FAT filesystems and disk images with Python 3
Facilities to access (ex)FAT filesystems and disk images with Python 3
I create new virtual machine in VirtualBox with VDI dynamic virtual hard disk.
I mount image of this hard disk with ImDisk in my host windows 8.1 (without creating any file system).
I read this disk in python:
VOLUME_PATH = r'\\.\D:'
with open(VOLUME_PATH, 'r+b') as disk:
disk.seek(0)
data=disk.read(512)
data
and I get b'<<< Oracle VM VirtualBox Disk Image >>>\n\x00\x00 ...
Then I unmount this image from host windows.
Then I write my own code (like nodos/nodos in your repository) to image with your library:
from FATtools.Volume import *
disk = vopen('VirtualBox VMs\\ddos\\ddos.vdi', 'r+b', 'disk')
disk.seek(0)
disk.write(b"\xb8\x03\x00\xcd\x10\xb4\x0a\xb0\x71\xbb\x00\x00\xb9\x02\x00\xcd\x10\xeb\xfe")
disk.close()
Then I again mount ddos.vdi with ImDisk to host OS and read it
and I get nothing changes in this vdi, I get again b'<<< Oracle VM VirtualBox Disk Image >>>\n\x00\x00 ...
(VirtualBox also print, that no bootable media, when I try to start it with this image)
Where is my wroted data?
The readme states: Manually install it under site-packages.
However, when I put the repo's root dir in my site-packages, e.g. py -m mkvdisk
tells me that C:\Program Files\Python39\python.exe: No module named mkvdisk
- unless I am in the folder of the package (but I assume this is not what should be required?).
But even then, I am unable to run samples\Reorder.pyw
:
Traceback (most recent call last):
File "C:\Program Files\Python39\Lib\site-packages\FATtools\samples\Reorder.pyw", line 17, in <module>
from FATtools.Volume import *
ModuleNotFoundError: No module named 'FATtools.Volume'
Any help/more hints would be highly appreciated. Cheers
Having some issues with sortby, so tried to just use example code, substituting just my drive letter:
from FATtools.Volume import *
# Assuming we have DirA, DirB, DirC in this disk order into X:
root = vopen('G:', 'r+b')
new_order = '''DirB
DirC
DirA'''
root._sortby.fix = new_order.split('\n') # uses built-in directory sort algorithm
root.sort(root._sortby) # user-defined order, in _sortby.fix list
root.sort() # default ordering (alphabetical)
Error returned is:
Traceback (most recent call last):
File "d:\data\labs\karaoke\testsort.py", line 11, in <module>
root.sort(root._sortby) # user-defined order, in _sortby.fix list
File "C:\Program Files (x86)\Python38-32\lib\site-packages\FATtools\FAT.py", line 1850, in sort
names.sort(key=functools.cmp_to_key(by_func)) # user order
File "C:\Program Files (x86)\Python38-32\lib\site-packages\FATtools\FAT.py", line 1818, in _sortby
return cmp(Dirtable._sortby.fix.index(a), Dirtable._sortby.fix.index(b))
NameError: name 'cmp' is not defined
Can you let me know what is wrong here?
Also, I was getting this same error with my own code, which is why I tried your sample.
In my code I am trying to sort files, not directories in a certain order. Is there a difference for this, or can I just feed it the filenames as you have supplied DirA, DirB, DirC, etc...
Also, I have these file names contained in a python list and reformatted them to a string with \n character to more closely fit with your sample. This resulted in the same errors as above, so I think there is more to it than my code.
However, is there a way to feed it a list directly without having to parse it to a string with defined separator?
thanks.
Hello,
I'm trying to create a FAT32 partition on my SD card. It's attached as removeable media with the ID 2 in diskpart
:
DISKPART> list disk
Datenträger ### Status Größe Frei Dyn GPT
--------------- ------------- ------- ------- --- ---
Datenträger 0 Online 100 GB 1024 KB
Datenträger 1 Kein Medium 0 B 0 B
Datenträger 2 Online 29 GB 0 B
DISKPART> select disk 2
DISKPART> clean
Now list partition
shows me a message that there are no partitions on the disk, so it should be wiped. To create the partition, I call the mkfat.py
script in an administrator cmd shell (I have cloned the entire repo in my working directory):
python mkfat.py \\.\PHYSICALDRIVE2 --fstype fat32
This take some time and then shows the following output:
Successfully applied FAT32 to a 29.81 GiB volume.
1952836 clusters of 16.0 KB.
29.80 GiB free in 1952835 clusters.
FAT #1 @0x4000, Data Region @0xEEA400, Root (cluster #2) @0xEEA400
The Windows Explorer shows a empty 32GB FAT32 partition and it seems to work, I could copy files on it. During my tests, I'd like to re-format it again, so my test case was to copy some files on the newly formatted card, run mkfat.py
again and I'd expect to have an empty partition again.
But the second call throws an access denied error:
Traceback (most recent call last):
File "c:\Users\Daniel\Downloads\FATtools-master\mkfat.py", line 102, in <module>
format(dsk, dsk.size, params=params)
File "c:\Users\Daniel\Downloads\FATtools-master\FATtools\mkfat.py", line 484, in fat32_mkfs
stream.write(bytearray(boot.cluster))
File "c:\Users\Daniel\Downloads\FATtools-master\FATtools\disk.py", line 338, in write
self._file.write(s[:full_blocks*512])
File "c:\Users\Daniel\Downloads\FATtools-master\FATtools\disk.py", line 113, in write
raise BaseException('WriteFile failed with code %d (%s)' % (GetLastError(), FormatError()))
BaseException: WriteFile failed with code 5 (Zugriff verweigert)
I guessed that I may need to remove the existing partition first. The disk management shows an error this request is not supported:
Using diskpart, neither the disk with the id 2, nor the volume (G:) is shown any more
DISKPART> list disk
Datenträger ### Status Größe Frei Dyn GPT
--------------- ------------- ------- ------- --- ---
Datenträger 0 Online 100 GB 1024 KB
Datenträger 1 Kein Medium 0 B 0 B
Datenträger 3 Kein Medium 0 B 0 B
Datenträger 4 Kein Medium 0 B 0 B
DISKPART> list volume
Volume ### Bst Bezeichnung DS Typ Größe Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 0 D VBox_GAs_6. CDFS CD 58 MB Fehlerfre
Volume 1 System-rese NTFS Partition 50 MB Fehlerfre System
Volume 2 C NTFS Partition 99 GB Fehlerfre Startpar
Volume 3 NTFS Partition 524 MB Fehlerfre Versteck
Volume 4 E Wechselmed 0 B Kein Medi
Volume 6 H Wechselmed 0 B Kein Medi
Volume 7 I Wechselmed 0 B Kein Medi
I had to close the shell window and start diskpart in a new one
DISKPART> list disk
Datenträger ### Status Größe Frei Dyn GPT
--------------- ------------- ------- ------- --- ---
Datenträger 0 Online 100 GB 1024 KB
Datenträger 1 Kein Medium 0 B 0 B
Datenträger 2 Online 29 GB 0 B
Datenträger 3 Kein Medium 0 B 0 B
Datenträger 4 Kein Medium 0 B 0 B
DISKPART> list volume
Volume ### Bst Bezeichnung DS Typ Größe Status Info
---------- --- ----------- ----- ---------- ------- --------- --------
Volume 0 D VBox_GAs_6. CDFS CD 58 MB Fehlerfre
Volume 1 System-rese NTFS Partition 50 MB Fehlerfre System
Volume 2 C NTFS Partition 99 GB Fehlerfre Startpar
Volume 3 NTFS Partition 524 MB Fehlerfre Versteck
Volume 4 E Wechselmed 0 B Kein Medi
Volume 5 G FAT32 Wechselmed 29 GB Fehlerfre
I tried to delete the partition using diskpart, but I cannot select it.
DISKPART> select disk 2
DISKPART> list partition
Partition ### Typ Größe Offset
------------- ---------------- ------- -------
* Partition 1 Primär 29 GB 0 B
When I try select partition 1
, I get There was no partition selected. Because of the asterix and articles like this, I also called select partition *
. It throws an error that the argument is invalid. delete partition
also doesn't work and says there was no partition selected.
I try to disconnect and re-connect the USB card reader, restarted the entire VM and also tried it on a Windows 10 host without virtualization. When restarting the VM, Windows did a disk check on the card. But none of them worked. By playing around with the readonly
attribute in diskpart
attributes disk clear readonly
it seems to get fixed one time. I tried to reproduce it for this ticket, doesn't work any more.
The event view of Windows shows multiple warning that the devices (2,3 and 4) have assigned a ID like one or more devices who are connected with the system. It seems that all of them are from the cardreader, which has multiple slots and creates a device for each of them. The entry refers to KB2983588, which refers to install MPIO on Windows Server. However, this is a Desktop (Windows 10 21H1 Build 19043) so Dism /online /enable-feature:MultipathIo
throws an error that this feature is unknown. I don't know if this warning is problematic.
Do you have any idea why FATtools cause this problems? It seems caused by using this lib, I have flashed multiple images to the card with the same card reader. And I also created 2 times a FAT32 partition with the graphical Windows tools, it worked without any issues.
I tried it with another card reader. A simple Micro-SD to USB converter. I don't get the warnings about device ids in the logs. But it's still not possible to delete the existing partition. The same error occurs when marking the partition as active. However, all other operations in the computer management GUI seems to work: Changing/deleting the drive letter as well as formatting the partition. Deleting is still not possible. Even after formatting the card with another file system like exFAT or NTFS.
It was possible to delete the partition with a strange workaround: Formatting it using the GUI without the checkbox for fast formatting, which is checked by default. Then right click on the running formatting in the computer management console and abort it. After that, the partition had changed to type RAW. I could delete it and re-create a FAT32 partition. Deleting seems now possible.
Now I've deleted all partitions (using the second card reader now) and re-run mkfat.py
with the same parameters. Just changed PHYSICALDRIVE2 to PHYSICALDRIVE1 since this has changed with the other card reader. The partition gets created, however I'm still unable to delete the partition like before 😢
I tried to make a custom Python script to break down the problem:
from FATtools import utils, partutils
from FATtools.Volume import vopen
from FATtools.mkfat import *
# Depending on the used cardreader, we have different device ids:
# 1 -> Simple USB-to-Micro-SD-Adapterm 2 -> UGREEN card reader with multiple slots
#dev = "\\\\.\\PHYSICALDRIVE1"
dev = "\\\\.\\PHYSICALDRIVE2"
disk = vopen(dev, 'r+b', 'disk')
print("Disk opened")
gpt = partutils.partition(disk)
# Throws KeyError: '\\\\.\\PHYSICALDRIVE2'
#disk.close()
print("Created partition")
# Also tried partition1 since "list partition" in diskpart shows the first partition as partition 1
# Throws the same error and gave me a 2048 GB GPT protection partition...
# Could not be removed in the computer management UI, but by re-running the script with partition0
part = vopen(dev, 'r+b', 'partition0')
print("Opened partition for formatting")
# Throws BaseException: WriteFile failed with code 5 (permission denied)
exfat_mkfs(part, (gpt.partitions[0].u64EndingLBA-gpt.partitions[0].u64StartingLBA+1)*512)
print("Formatting done")
This seems to work, I can delete the partition without any error. But I wondered that my partition is formatted, even when the exfat_mkfs
call throws an access denied error? I thought that my partition would be created but not formatted after this error.
It seems that something is wrong here. I copied some data on the card, deleted the partition in the UI and re-run the script again. Instead of an empty FAT32 card, it contains all the data before deleting the partition!
disk.close()
It took out that win32_disk.open_handles
in disk.py
is empty, since partutils.partition
already closes the disk. So it seems for me that this call in mkfat.py
is wrong. I just need to re-call vopen
to getch the first partition, which should exists after partutils.partition
was called successfully. But the formatting still doesn't work and throws code 5 access denied. The program runs as Administrator, so I can't understand this.
In reference to #2 I wanted to ask if it would be able to provide FATtools as package via PyPI. The standard way to distribute package is as a pip package in either sdist or wheel format. The benefit would be that this package could then be installed easily via pip. Generally speaking, simply providing a proper setup.py
, building the packages and then uploading them via twine would be the standard way of distribution.
I'd be happy to support, but due to legal limitations from my employer, the clearance for me doing that might take a while.
This is more of a question perhaps than an issue (though possibly a feature request??).
More info:
I have been using DriveSort (http://www.anerty.net/software/file/DriveSort/?lang=en) to do these kinds of operations against my USB stick for my car MP3 system. It works great and based on its description I was looking for a python alternative for a project I am looking to do for my daughter's Karaoke player which also needs MP3 sorted.
While I can continue to use DriveSort for this, I have to manipulate additional file data which I am thinking about using python to do (read JSON, modify, and dump back into .ini files for each song).
Since I need to do this in python I was hoping to skip having to use DriveSort and have python also sort the FAT table alphabetically.
thanks
So, I like FATtools
, It's a very useful package. But, I would like to see a proper documentation instead of some sample code
for example, a question I've been wondering around is, Is it possible to use a VHD .img file as an actual VHD. Instead of an img that just saves bytes. I.e like simulate a FS. I.e have a virtual C:
drive that I can write to read to and use as a regular HDD.
Because the sample examples do not explain it well, nor do they show the possibility, I've heard from other sources that it is possible.
But either you have to use 3rd party packages that are not in python or you have to make your own until I stumbled upon FATtools
now to simplify this whole message the questions I have are the following
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.