Atari file system support for Linux ?

[AtariZoll]

Maybe just need to stick to good, old Hatari, or Aranym - they will access any of partitioning system, byte-swapped or not ...
I don't know is direct access to attached Flash cards possible - there was some talk about it, but I never tried.

[frank71]

Hi guys, I've ported P.Putniks tool to QT 4.x (should also work on 5.x). Almost all functionality should work. Maybe there is a "delete file" function possible in future, but earnestly, I'm not a filesystem specialist. To deeply understand how P.P.'s tool is working inside, some work has to be done. The used code style is ... lets say slightly old fashioned. So code need to be reviewed and partly recoded.
I've removed all KDE components and used only QT components, so there is no need to install KDE framework. If there is anyone who can implement drag 'n drop features (of course I will try too) this would be a nice tool for managing ATARI partitions on PC.
Please test, but be careful. Working with real devices (flashcards) needs root access. I've removed choice of /dev/hda, /dev/hdb, dev/sda and /dev/sdb, but nevertheless there is enough potential for making huge desaster.

DrImg.tar.bz2 -> QTcreator project (source)
DrImgEx.tar.bz2 -> executable + desktop-icon (compiled on kubuntu 17.10 / QT 4.8.7)

some tests later: it seems (at least for ICDpro driven devices) GEM partitions will get damages during file transfer on it, and (these) devices are not bootable any more after trasferring files to BGM partitions on it. Okay, linux sources are V0.85, windows version is V1.35. However - something to do here...

[mikro]

Hi Frank, why not to put the project on github, that way it's easy to help you out with features / bugfixes / code review.

[frank71]

okay, done:
https://github.com/frankheisig71/drimg.git

[Badwolf]

okay, done:
https://github.com/frankheisig71/drimg.git

Thanks, Frank.

I've been doing a lot of image work recently, so will give this a go & let you know how I get on.

Cheers,

BW.

[frank71]

Hi Badwolf, I've just commit some changes and reworks. You should check out again, because I did many, many source changes just to get more structured code (by far not ready, but more or less understandable now). I've added a file / directory delete function and directory navigation within the file browser. I've tested it with several files, images and real SD drives - it seems to work.
Next steps are - enhance "AddFile" to handle whole directory trees (I will do), then implement drag'n'drop. Maybe it is possible to get the ST desktop look into the file browser - would be nice.
Furthermore reading and writing SD-Images is boring slow. Maybe you can improve it.

Frank

[Eero Tamminen]

Furthermore reading and writing SD-Images is boring slow. Maybe you can improve it.

Through what interface you're writing and on what device/media? And what's the block size you're using for the writing?

[frank71]

Hi Eero,
this tool reads/writes directly from/to e.g. /dev/mmcblk0. The reading speed is okay so far, but writing is about 5 times slower than under windows. Bigger block sizes (I use 1M) doesn't help. Asking Google shows me, I'm not the only one with this problem, but unfortunately is shows me no solution.
Even with dd writing half a Gig takes 10 minutes, under windows this tool needs 2 minutes. Some guys suggests to use open() with O_DIRECT flag instead of using fopen(), but this sounds weird to me. Some kernel caching and buffering cannot cause minutes of time loss during 512 writes.

[Eero Tamminen]

When you say that tool takes less time on Windows, is it to this same MMC card, and to same partition on it?

(MMCs may have some faster flash at beginning which is optimized for FAT file systems.)

And do you have same amount of free space in both cases before doing the test? Flash media does wear-leveling and if you've just freed a lot of disk data before doing the writes that are large than before you freed the space, it will need to erase the freed blocks before they can be written. Erasing might happen only after you've written enough new data that the space for previously remove data is needed again.

[frank71]

Hi Eero,
after my Windows takes me 1.5 days (evenings) to be usable after I didn't use it for 2 month, I've tested the currently checked in code under Windows and Linux. Same hardware, same SD card, same code: Windows needs 1 minute to write 500M, Linux needs 7 minutes. It is just a raw write to the device - regardless any data, partitions or file systems on it. I can change several times between the systems - always the same result. I'm aware of the internal structure and management of flash memory - that's not the problem here.

[frank71]

Maybe this is the reason:

Code: Select all

$ sudo hdparm -tT /dev/mmcblk0

/dev/mmcblk0:
 Timing cached reads:   1974 MB in  2.00 seconds = 987.80 MB/sec
 Timing buffered disk reads:  22 MB in  3.01 seconds =   7.31 MB/sec
 

But to be honest, I cannot believe that's the whole story...

[Eero Tamminen]

Writes cannot be cached like reads can. They could be buffered by kernel, but that would mean that writes haven't been pushed to card when the write command has finished. I.e. if you would eject the card, the data wouldn't be there, although file system might show it.

How do you do the "raw" write on Windows and Linux? Are you doing sync after it on both?

[frank71]

There is indeed a difference between Windows and Linux. To have a correct relation between progress bar and real written data on device, it is needed to do a fdatasync() from time to time under Linux - otherwise all the data is cached and the progress bar jumps to 99%, and it takes the whole rest of time to close the file - what is in fact a sync. Under Windows writing is obvious synced during the whole process.
Raw writing means, I open e.g. /dev/mmcblk0 with fopen(), write with fwrite() and so on...

[frank71]

During thinking about the slow writing speed under Linux I've finished the function I want to add: sorted view of directories, deleting files and dir's and adding whole directories from local drive. It's compilable for Windows too, by uncommenting a define in "gemddlg.h" (sure there are better ways...)
I compiled it with QT Creator 2.x - with higher version you will get masses of warnings (rightly, because there are still lots of global vars and dangerous casts and so on) QT-Version is 4.x - but it should work with 5.x too.
You may test it now, and comment.

[frank71]

Hi Eero, I've tested the tool with other hardware - everything seemed okay. Obviously the SD card reader in my notebook is supported weakly under linux. All in all this tool is working quiet good now.
So I will do some bugfixing (e.g. supporting some special characters in TOS file names, which are not allowed in other file systems...) next time and that's it.

[Eero Tamminen]

Hi Eero, I've tested the tool with other hardware - everything seemed okay. Obviously the SD card reader in my notebook is supported weakly under linux.

Ok, so the driver is using some really slow mode for accessing the card, or it's just written in very inefficient way. That can happen for Linux stuff that's not used on servers, and it's hard to know which of the consumer devices have good drivers (one rarely googles for specific card reader chips, unless one encounters problems, one doesn't do that kind of research for "fringe benefit HW" when looking for device that one might some day use as a Linux box)...

Anyway, great to hear that you got it working good on another HW!

[frank71]

After some bugfixing I've checked in a version which is quite usable now. Here a small overview of the new features of file transfer dialog:

- sorted list view
- navigation within list view
- deleting files and directories
- adding whole directories to Atari FS
- preserve all file and directory modification dates
- correct handling of special ST-charset (e.g. Atari symbol in file names) on local file names
- Linux: Changing owner and group of extracted files and directories to owner and group of destination folder.

If needed, I can place binaries here for Windows and Ubuntu Linux.

[wongck]

So with this tool I can attach my Falcon HDD to my Win10 PC and use this tool to image the GEM partition?
Also I am able to add/remove files & folders ?

If needed, I can place binaries here for Windows and Ubuntu Linux.

YES please !!!

[frank71]

Hi wongck,

So with this tool I can attach my Falcon HDD to my Win10 PC and use this tool to image the GEM partition?
Also I am able to add/remove files & folders ?

generally, yes. But I'm not sure about your 30GB HDD. I've tested this tool only with SD cards up to 4GB and max 512MB partitions. Can you give me some key data of your HDD? Partition size, cluster size, sector size...
Did you ever try this: http://atari.8bitchip.info/drimus.php? I've just taken the last available source code of it, recode most of it, and add the functions mentioned above. Main goal was to get a usable tool for Linux, but yes, most of the features above are not available with the last Windows-version of Peters tool, so why not do it for both platforms...
I will put here a Windows executable tomorrow - (un)fortunately I currently have no access to my Windows machine - but because I have no clue how you connect a Atari HDD to a Windows machine, you may test Peters tool for making a backup of your HDD. If this works, and if you can browse through the directories with it (File Transfer Dialog), it should work with my version of this tool too - plus a little more opportunities

[wongck]

I have used Ppera drive image nearly 10years ago with good success to recover my Falcon HDD, topic about that here.
Would be nice to have an updated version. Anyway, that particular version I have not used on my Win10, my guess is that I was using it on XP back then.

[frank71]

Here we go.
You will need the QT libraries mentioned in READ.ME. Usually they should be free to download, but it seems, there is only a installer with 1.2GB size. In fact you need only 27MB libraries, but I didn't find one site which provides working QT *.dlls.
You should try a binary copy of your HDD - there is a option to open image files instead of real devices - they are handled equal to real devices.

[frank71]

Okay, for Windows it's a little bit more complex - the right *.dll versions are depending on the build-system you use, I've used mingw32 and this executable needs the libraries compiled within that system. Most of the libraries you can freely download are built for MS Visual Studio development, which links QT against lots of other libraries and finally some procedure entry points are missing...
To shorten it:

[wongck]

Thanks, it beats downloading 1 GB of stuff for a 124KB program to work.
Will try it out.

[wongck]

Well, it did not run as it can't find the gcc lib used.
I can't capture the error dialog box as it goes on top of snipping tool.

[frank71]

Hi wongck,
this one?

libgcc_s_dw2-1.zip

[edit]
I checked with another Windows machine just copying dlls, unfortunately it seems to be not that easy. I will try to find the root cause tomorrow.
[/edit]