How to get GEMDOS to use FAT16 instead of FAT12

Troubles with your machine? Just want to speak about the latest improvements? This is the place!

Moderators: Mug UK, Zorro 2, Greenious, spiny, Moderator Team

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Wed Sep 17, 2014 2:02 am

I've been away from my ST for about 20 years and brought it back to life back in July (had to pull out a socket and resolder the 68K back in) and started working on a little cartridge interface for "fast" RS-232 to my PC. I put a Microchip 18F2620 along with a FTDI USB to TTL cable running at 921600 buad. I have a little tuning left to do but currently I can transfer files back and forth at 50K bytes/second (max rate at that baud rate would be 92K bytes/second).

Now that I can move files back and forth I figured the next step was to implement a serial disk. I've coded up a few RAM disks back in the day so I just hacked my code which uses rwabs (trap 13 #4) to move sectors back and forth to the PC which has some code looking for read sector/write sector and allocates a memory array of 32MBytes.

So far so good as it works faster than a floppy (about 5x) so I can store my compiler and tools on the PC and copy them to the RAM disk in about 20 seconds and I'm good to go.

I wrote a little tool to take the sector dump on convert it to files and directories using my knowledge of Microsoft's FAT file system and noticed that the File Allocation Table in the sector dump is using 12 bit sectors (FAT12). I thought this was because I was using a small disk size at first but when I ramped up the size to where there are more than 4096 (2^12) clusters that it still used FAT12.

My website of projects is here:

https://www.sites.google.com/site/tedrossin

You can find my $15 logic analyzer along with my more powerful one used to fix my ST here:
https://www.sites.google.com/site/tedro ... ronics/pic

I'll publish the schematic and code when I have it all tidy.

Any insights or the proper place to ask this question would help me out.

Also, to pass the 32 Mbyte limit (as rwabs only uses a 16 bit sector) do I have to implement my own open, read, write, lseek .. or is there another trick. I've seen rumors about big sectors but no details.

Thanks
Ted.

Here is the bpb that I expose in my serial disk to GEMDOS:
recsize=512 // Bytes per sector
clsiz=2 // Cluster size in sectors
clsizb=1024 // Size of cluster in bytes (cluster is the allocation unit size in the file allocation table)
rdlen=8 // Root directory size in sectors
fsiz=127 // FAT size in sectors
fatrec=128 // backup FAT sector offset
datrec=263 // sector number of cluster 2 (first two clusters do not exist because FAT entries have magic info)
numcl=32507 // Number of clusters (32507*1K= about 32Mbytes)
bflags = 0

I can't find info on the bflags so maybe that is where I can tell it to use FAT16. In the short term, I'm changing over to a cluster size of 8Kbytes so that the number of clusters is less than 4096.
Last edited by trossin on Tue Dec 09, 2014 8:08 pm, edited 1 time in total.

joska
Hardware Guru
Hardware Guru
Posts: 4381
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby joska » Wed Sep 17, 2014 6:33 am

I'm no expert, but IIRC you can use 32 bit sector index with rwabs if your driver meets a certain AHDI standard. Take a look at the AHDI 3.00 release notes.

Edit: Set bit 0 of bflags to use FAT16.
Jo Even

VanillaMiNT - Firebee - Falcon060 - Milan060 - Falcon040 - MIST - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby AtariZoll » Wed Sep 17, 2014 8:03 am

Yes, bpb flags (bflags) is where you need to set bit 0=1 for FAT16. And of course other parameters properly. Some things are described in Atari ProfiBuch, which is in German.
You don't need to deal with filesystem functions like fopen, fread etc. if make it like usual hard disk drivers. But then must keep limitations, as max partition sizes, etc.
What means that this should go rather in coding section of forum, btw.
As I see, you make directory and file transfer from PC, and actually from floppy image files - according to having FAT12. So, you basically create an image file on PC, fill it with desired content, and then read it on Atari ? Right ?
That way allows you usage of much bigger images, up to 512MB, what is partition size limit on TOS 1.04-2,06, 3.06 . Then need to use logical sectors of 8KB.
And it will act like hard disk, in fact. Simpler is to limit it to 32MB, then can use standard sectors of 512 bytes. In above cases you don't need to care about fopen, fread, etc. functions.

Other way would be filesystem wise, something like GEMDOS type hard disk emulation in Atari emulators. Then you don't need to care about FAT, because working on file level. But then must write own filesystem functions - so all them as fopen, fread, fwrite - only if want to write to PC, fseek, etc . + code for creating virtual directories on virtual logical drive, with content of DIR on PC, what you just opened. That is little harder, I think, than disk image way. I dealt with both ways.

About big sectors: Atari uses so called large, big sectors instead way used by MSDOS in case of partitions bigger than 32MB. While DOS uses always sectors of 512 bytes, but clusters with up to 64 sectors (32KB, for 2GB partitions) , TOS uses always clusters with 2 logical sectors ( in versions 1.00-1.62) , and logical sectors up to 8 KB. Therefore is 512MB limit - 8KB x 2power16 = 512MB . As Joska mentioned, there is 32-bit addressing possible in rwabs (BIOS 4), but fact is that it is not used in TOS, at least not in versions 1.00-2.06, and as I know not in TT or Falcon too. That was for future versions, I think. Used in Mint for sure. Atari just kept 16-bit code in FAT16 filesystem handler.
So, your driver need to set proper BPB in case of big sectors - including sector size, cluster size (=2x sector size) + FAT, ROOT DIR sizes, in logical sectors.
All those parameters are placed normally in bootsector of partition.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Wed Sep 17, 2014 4:38 pm

See next one. Bad refresh.
Last edited by trossin on Wed Sep 17, 2014 7:46 pm, edited 1 time in total.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Wed Sep 17, 2014 4:38 pm

Thanks for all the info and especially the bflag = 1 secret. I'll give it a try tonight.

I attached the schematic for my interface so that I can leave this post in hardware. I have not soldered up the SD connector yet but that is next on my list of fun.
You do not have the required permissions to view the files attached to this post.

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby AtariZoll » Thu Sep 18, 2014 8:47 am

Of course it is not "secret" . You can find description of BPB and lot of other things in Compendium:
http://dev-docs.atariforge.org/files/Th ... endium.pdf
In BIOS functions section.

This cartridge port IF is one way - only read possible from SD card ? From schematic not much is visible. There is some address shown, in purpose of writing, but not clear is it only for commands, or may write sector contents too.
It would be good if you provide some description about this project - features, estimated speed, etc ..
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.

User avatar
Anima
Atari Super Hero
Atari Super Hero
Posts: 667
Joined: Fri Mar 06, 2009 9:43 am
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby Anima » Thu Sep 18, 2014 3:29 pm

Interesting project.

I have developed a similar program named "Serial Disk". ;) It's also based on a RAM disk source and the PC program runs on Linux, MacOS and Windows. The sector size of the virtual disk depends on the total size of the disk so you can choose which one works best for your TOS version.

Well, it's not finished either because I want to add a GUI to it for a better user experience.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Thu Sep 18, 2014 3:52 pm

Setting bit 0 of bflags did the trick to get FAT16. Thanks for the help. The ST Internals book from 20+ years back that I have had no information on that.

I'm able to read and "write" to the cartridge port using the trick to pass the data on the address bus. Read data is returned from the PIC using the 74LS244 to the data bus. Writes are done via the 74LS374 using the address bus into the PIC. I use the capture/compare edge detectors in the PIC to detect when a read or write was performed by the ST so it knows to either interpret the command/do something with the data or that the read return data or status has been accepted. The PIC is much faster than the 8 MHz 68K but I did have to play games to get it to be quick enough to capture in coming serial port data at the same time as ST commands.

If you look on my web site under the PIC projects, I already have coded up a FAT32 with long filenames file system to read and write SD cards using the same PIC. This project has way more features than are needed to just do sector games but I plan to allow the PIC to use the FAT32 file system to access fake sectors in a 32 Mbyte file per partition. The SD card can then be read on a PC to dump the partitions and I already have the code to read the GEMDOS FAT12 or FAT16 image and create directories of files on a PC. I just need to go the other direction which should not be that hard.

Later, I will attempt to override all the read/write/lseek/open .. commands to let the PIC do the work of managing the file system. This way the SD card can be read and written on a PC without any special translator like I have to use now to decode the 32 Mbyte image. I just need to get rid of the long file name support to save some code space. This should allow a 64 Gbyte drive with a file size limit of 4 Gbytes as lseek seems to use an 32-bit variable and that is the limit of FAT32. I have tested my SD file system code with 16 Gbyte SD cards so at least that size should work. Not that I plan on having anywhere near that much data for my 1985 vintage 1040ST. I think 32 Mbytes will be more than enough for me but maybe I might need a few partitions.

To write a byte to the PIC I just OR the write data with the address 0xfbf000 and issue a read. I use MWC on the ST and am able to get it to generate pretty good code with these MACROS.

Code: Select all

#define CART_WRITE_SETUP() \
    register unsigned char *CartWrPtr=(unsigned char *)0xfbf000L; \
    unsigned char CartPoo2
#define CART_WRITE_FAST(Value) (CartPoo2 = *(CartWrPtr+(Value)))
    /* Add a pause at end to give PIC more time to respond */
#define CART_WRITE(Value) (CartPoo2 = *(CartWrPtr+(Value))); \
    CartPoo2 = CartPoo2 + 3


With CART_WRITE_FAST I can do back-to-back writes are 2.5us apart (400 Kbytes/s) with constants (too fast for the PIC) or when in a loop to transfer sector data I am able to get about 100 Kbytes per second. I don't have my notes handy but I think I was able to get 200 Kbytes/sec on reads in a loop to transfer sector data for a read.

So, yes. I expect this to be slow but my plan is to use it as a backing store and work from a RAM disk. I may implement a little read only cache so I don't have to bother with the copy and if a program crashes I won't lose data. If I feel lucky, I'll do a write back cache like I did in the PIC SD file system but maybe add the scrubber to periodically push back dirty sectors.

Below is the rest of the code to send bursts of data to the PC via the PIC. Once I finish verifying proper operation and tuning this up, I'll post all the code.

Code: Select all

SendRS232Packet(Count,Data)
unsigned int Count;
unsigned char *Data;
{
    CART_READ_SETUP();
    CART_WRITE_SETUP();
    unsigned char Num;
    register unsigned int NumToSend;
    register unsigned int i;
    register unsigned char *ptr=Data;

    while(Count){
       CART_WRITE(A_CMD_READ_RS232_WR_FREE_COUNT);
   Num = CART_READ();
   if(Num>=90 || (Count<90 && Num)){
       NumToSend = (Num>Count) ? Count : Num;
       CART_WRITE(A_CMD_WRITE_RS232_BLOCK);
       CART_WRITE(NumToSend);
           for(i=0;i<NumToSend;i++){
      CART_WRITE_FAST(*ptr++);
           }
       Count-=NumToSend;
   }
    }
}

GetRS232Packet(Count,Data)
unsigned int Count;
unsigned char *Data;
{
    CART_READ_SETUP();
    CART_WRITE_SETUP();
    unsigned char Num;
    register unsigned int NumToRead;
    register unsigned int i;
    register unsigned char *ptr=Data;
    unsigned int TotalToRead = Count;

    while(TotalToRead){
       CART_WRITE(A_CMD_READ_RS232_RD_COUNT);
   Num = CART_READ();
   if(Num>0){
       if(Num>TotalToRead) Num = TotalToRead;
           CART_WRITE(A_CMD_READ_RS232_BLOCK);
           CART_WRITE(Num);
           NumToRead = Num;
       i = NumToRead;
       do{
           *ptr++ = CART_READ();
      i--;
           } while(i);
       TotalToRead -= NumToRead;
   }
    }
}

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby AtariZoll » Thu Sep 18, 2014 5:09 pm

This is interesting project, and I'm sure that working on it is good fun.
But I must say that Atari people has already some very usable mass storage solutions, using Flash cards, and TOS/DOS compatible partitioning, so file transfer on relation Atari-PC with DOS, Windows, Linux or even MAC is pretty simple, and no need for any special SW for that. Speed can be very high with good USB card reader - I had 15 MB/sec with Sandisk CF for instance.
Speeds on Atari are in range from 150 KB/sec (Satandisk, older design) , 800-1300 KB/sec (UltraSatan, Gigafile) . And even more with my not manufactured yet cartridge adapter . Then, Atari people can use CF cards on IDE adapters - speed about 1400 KB/sec with good cards. Important part of this are proper hard disk drivers. So, no need for RAMdisks. Partition sizes are up to 512MB, what is enough for Atari SW .
Ah, and there was cart port IDE adapter called Paskud. Overriding write limitation of ST cart. port is something used in diverse projects, so in Paskud. My first ST HW project was EPROM programmer for cart port, and of course it was writable. Speed was not problem in that case. But I can do with proper SW some 280 KB/sec in write, using some logic and latching. Or even 2 MB/sec with very special tricks, where SW part is probably most interesting - only with CF cards.

"This should allow a 64 Gbyte drive with a file size limit of 4 Gbytes as lseek seems to use an 32-bit variable and that is the limit of FAT32. "
So, you worry about 4 GB file size limit in this project :D While most of Windows SW has such limit, because still using 32-bit variables for files.
4 GB card is just enough for all ST SW what you can use, unless you run Mint, but then can use FAT32 .
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.

joska
Hardware Guru
Hardware Guru
Posts: 4381
Joined: Tue Oct 30, 2007 2:55 pm
Location: Florø, Norway
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby joska » Fri Sep 19, 2014 7:31 am

AtariZoll wrote:But I must say that Atari people has already some very usable mass storage solutions


I agree, I don't think this is the best option for plain mass storage. But with some modifications on the software-side this could be a fast, simple and reliable way of sharing files with a modern computer. I can imagine using something like this on my Falcon instead of ethernet. Swapping media is fine in many cases, but not when you need instant access to files edited/created on another computer (e.g. when testing software).
Jo Even

VanillaMiNT - Firebee - Falcon060 - Milan060 - Falcon040 - MIST - Mega ST - STM - STE - Amiga 600 - Sharp MZ700 - MSX - Amstrad CPC - C64

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby AtariZoll » Fri Sep 19, 2014 7:52 am

Well, I'm the one who did/does lot of SW for Ataris on PC, and then tests it on ST, Mega STE, Falcon, TT . And did not ever felt that I need instant file transfers, without turning off Atari. Because normally I need to think some time when it works not as wanted. So, while I move flash card to PC, I thinking about how to correct problem. Additionally,, I would need solutions for CF cards too, not only SD cards.
In any case, it is useful to have option for instant file transfer, via USB. We can achieve similar speeds with current solutions as PARCP, btw . All depends from price of this - if is planned for sale. And of course, lot of time is needed to develop SW - what is normally more than time needed for developing HW part, including firmware.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Fri Sep 19, 2014 3:56 pm

This is all freeware stuff. It can be built using a Douglas Electronics protoboard $10, a $6.50 PIC, $2 for 3 TTL parts, and $15 for the FTDI USB to TTL cable. If you don't have a PIC programmer, that will cost another $35 or I can program up a part for the cost of shipping. Same goes for anyone who wants to build the cheap logic analyzer.

I'm amazed that Douglas Electronics still has the same boards that I ordered 20 over years ago:

http://www.douglas.com/hardware/pcbs/br ... atari.html

So, I currently have two little PC programs. One to send/receive to/from the ST and one to pretend to be a disk drive. I don't run both at the same time but it is possible to switch between them as long as the disk image is saved and reloaded and the ST is not accessing the serial disk.

Last night I was able to copy the 10 floppy disk compressed image I made with Diamond Back 2 from May 1993 to the serial disk, dump the image and then convert the image to 14.4 MBytes of files on the PC. I now have access to my old Atari file system on the my PC. I no longer have that SCSI drive that flaked out on my shortly after that date so it is nice to see the old files which was the main point of messing around with this little interface.

I forgot to mention that you can skip the nice FTDI cable and use one of those cheap $4 USB to TTL 3.3V boards if you use your own extension cable. You also need to add some resistors to convert 5V down to 3.3V for transmit and a resistor and a diode to convert 3.3V up to 5V for recieve (the PIC has a schmidt trigger input with a higher threshold so you really need to get the voltage up to work well).

AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby AtariZoll » Fri Sep 19, 2014 4:18 pm

10 $ for experimental board, fitting in ST cart port ? It's not bad, but not cheap too. Especially if must ship to oversea.
In any case, I must tell that Jookie's Satandisks are open source too, but people builds not them, because most can not do it. Same stays for IDE adapters, which are much simpler.
"10 floppy disk compressed image I made with Diamond Back 2 from May 1993 to the serial disk, " - you mean transferred 10 floppies to PC ?
But there is way to read them on PC directly - OK, only on those with internal floppy drive, if format is 800K or 1600 . It looks that they are 1440 KB, so you could copy/image them even with USB floppy drive.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Tue Oct 07, 2014 1:52 am

I finished the first pass of the cartridge based RS-232 interface along with code to implement a serial disk drive and ftp server for windows and ftp client for TOS. The current code gets me near 50 KBytes/s transfer rates. The max rate I can hope for is about 90KBytes/s so I still have a few more things to look into.

Details about this as well as my old TOS2.06 upgrade, 4 MByte RAM upgrate and some simple programs can be found here:

https://www.sites.google.com/site/tedro ... cs/atarist
You do not have the required permissions to view the files attached to this post.

trossin
Atarian
Atarian
Posts: 9
Joined: Mon Jul 14, 2014 7:17 pm

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby trossin » Tue Dec 09, 2014 8:09 pm

My web site moved to Google sites so I updated the links in my previous posts. The new link is included below:

https://www.sites.google.com/site/tedro ... cs/atarist

User avatar
1st1
Atari Super Hero
Atari Super Hero
Posts: 836
Joined: Mon May 07, 2012 11:48 am

Re: How to get GEMDOS to use FAT16 instead of FAT12

Postby 1st1 » Tue Dec 23, 2014 1:50 pm

Do you know, what would be pretty cool? A Windows version of the Midicom network driver which is using such a cheap USB-to-Midi adapter. On ST/TT/Falcon MidiCom runs pretty fine.
Power without the Price. It's not a bug. It's a feature. _/|\_ATARI

1040STFM in PC-Tower (PAK68/2, OvrScn, 4 MB, 1GB SCSI, CD-ROM...) * 2x Falcon 030 32GB/14MB+ScrnBlstrIII * 2x TT030 73GB/20MB+Nova * 520/1040STFM * 520/1040STE * 260/520ST/+ * some Mega ST * 2x Mega STE 500MB/4MB+M.CoCo * Stacy * STBook * SLM605 * SLM804 * SLM605 * SMM804 * SH 204/205 * Megafile 30/44/60 * SF314 * SF354 * 5x Pofo * PC3


Social Media

     

Return to “Hardware”

Who is online

Users browsing this forum: No registered users and 4 guests