PASTI/STX File Format

[DrCoolZic]

To all interested in Pasti format.
I am currently updating the documentation, preparing code to publish, and therefore doing tests!

I have found an interesting case: Maupiti Island. As you know this games uses tracks with 1 huge sector.

In fact there are 0 real sector and the pseudo-sector is read with a read track command. The track starts with for example A1 A1 A1 FE 07 00 07 B7 07 66 - followed by 07 and by all the bytes of the pseudo sector.
As you can see the begging looks very similar to an Address field and indeed the STX file store this information in the address field of the sector descriptor !!!!
So in that case we have Track=07, Head=00, SectNum = 07, SIZE=B7 (=183), CRC=6607

As the STX format does not provide a size for the sector in the Sector Descriptor your best guess for the size of the sector is to use (128 << size) but in that case (128 << 183) = 1073741824 !!!
In this case the descriptor data offset is 0 and therefore the sector data = the track data so it is easy to fix the problem in the code.
However the Pasti lib must be doing something special ... for this kind of track.

This is limit to be a bug? On a real Atari using the WD1772 doing a read address on this track will return an ID of 07 with a CRC error and a read sector 7 command will return sector not found.
So I would have prefer to have 0 sector information in STX file !

Feedback is welcome
I will update the code and the documentation to work with this case

As far as I know "single sector track" uses either 0x07 as escape character but they can also use 0x0F

[AtariZoll]

I would not call it huge sector. It is just track data, track dump. That data segment is just similar to sector address field. Pasti detects it, and stores in image file, but it is not used/read. I don't think that Pasti lib does something special. It just needs to copy track data in RAM when track read command is given. Other is on game's decoding SW - to filter out proper final data. If you'd use read sector or read address in emulator will have similar errors. And I looked track data - there is only track dump, no any sector data.
Surely would be better that there is shown 0 sector count in such tracks, but I guess that it is just not done, since image works fine as is
P.S. : maybe it is best as is - as is more similar to real HW behaviour.
"As far as I know "single sector track" uses either 0x07 as escape character but they can also use 0x0F'" - this is not a must. Probably some other methods are possible.

[DrCoolZic]

My mistake. In fact the Pasti file contains the right information
The behavior of wd1772 is: read address return 1 sector but read sector return no sector found and this is what is stored in Pasti file.
In this case the RNF flagis correctly turned on in the FDC status

Please find attached a new and hopefully better version of the documentation.

Pasti documentation-v03.rar

For info: people dealing with timing - with newer revision the timing record can be used to provide timing information for protection other than speedlock

Also attached is a small program that allow to display the structure of a Pasti file

PastiRead.rar

This one is in C# and I have an older version in C++
I will publish source (not very interesting but ....) soon

[DrCoolZic]

I have been testing more Pasti file and I have found an interesting case not covered in the documentation.*

As you know the fuzzy record in the file can be "dispatched" to several sectors. For example you have one fuzzy record of 1024 bytes -> dispatched to two 512 bytes sector ...

I have found that the same can happen with Timing records.
So if you have two 512 bytes sectors with timing info the lenght of the timing record will be (512/16 + 512/16) * 2 + 4 = 132 bytes (64 ushort + 2 ushort)
The 64 entries needs to be dispatched to the two sectors

So again a new version of the doc

Pasti documentation-v04.rar

And a new version of the program

PastiRead.rar

The program now do assertion check for "holes" in the pasti file. If you get a debug assert please report
Thanks

For info Aufit alpaha release available here viewtopic.php?f=102&t=25906

[npomarede]

I have been testing more Pasti file and I have found an interesting case not covered in the documentation.*

As you know the fuzzy record in the file can be "dispatched" to several sectors. For example you have one fuzzy record of 1024 bytes -> dispatched to two 512 bytes sector ...

I have found that the same can happen with Timing records.

Hello

could you tell which STX game you saw with this behaviour ? It could help later when I will need to check my STX support covers all cases in Hatari

Nicolas

[DrCoolZic]

Theme Park Mystery Track 02.0 02.1
But I do not understand why not Track 00.0 -> 04.0 & 00.1 -> 04.1
all track looks the same they have a no flux area at the end and adding timing info in NFA has no meaning ?????

[DrCoolZic]

I found another strange thing for track with one "data sector" like Maupiti Island
Here we have a track descriptor that have TrackFlag=0x61 => Track data with only track image size (versus flag 0xE1 sync off + track size).
- First of all I am surprise to see 0x61 because this Pasti latest version 3.2 and I would have expected 0xE1 that provides more information ??? Therefore I do not understand when/why 0x61 versus 0xE1 ???
- But the problem is that when I add up all the fields as described in the doc I am ONE BYTE SHORT to where next track descriptor point ??? If I look at the data with an hex editor it seems that indeed the first two bytes in the Track Data records is the track lenght but I have an orphan byte at the end. I cant find why? is it to be on an even boundary ???
I am investigating and will let you know.

But bottom line always follow this rule: to go from one track descriptor to the next only rely on adding the track record lenght to the strat of track and ignore hole at end of track record if any.
Since I added the timing record, on all images tested this is now the only case left with a hole at the end of the track record?

PS: On the dump program you have probably notice that the position in the file given between parenthesis for example (0-16) should be (0-15). The last number is in fact the position of the Next byte. This is corrected in the new version ....

[IFW]

Theme Park Mystery Track 02.0 02.1
But I do not understand why not Track 00.0 -> 04.0 & 00.1 -> 04.1
all track looks the same they have a no flux area at the end and adding timing info in NFA has no meaning ?????

You may want check that it actually works... Many games using NFA have non-trivial protections, that pass even if the protection fails, but the game e.g. becomes unplayable, like Theme Park Mystery does. In this case, no matter what you do, you won't be able to get on the train between the game zones.

[DrCoolZic]

Theme Park Mystery Track 02.0 02.1
But I do not understand why not Track 00.0 -> 04.0 & 00.1 -> 04.1
all track looks the same they have a no flux area at the end and adding timing info in NFA has no meaning ?????

You may want check that it actually works... Many games using NFA have non-trivial protections, that pass even if the protection fails, but the game e.g. becomes unplayable, like Theme Park Mystery does. In this case, no matter what you do, you won't be able to get on the train between the game zones.

haha you are looking at STX format .... exciting ...

Coding NFA for stx format is really easy just save the 2 sectors that allow to read data and clock ...
These sectors also have fuzzy bits and reads with CRC error.
Adding timing information is probably a "glitch" in the imager software. It does not harm but it is useless!
And by the way I think I now understand how NFA is actually created by tools like KryoFlux! Pretty easy in fact Unfortunately I do not have the HW to check it

But I think that what I mentioned above is strange and a semi-bug. If you move from track descriptor to track descriptor using the track record length you are safe but why this unused byte?
The track first sync information is not really useful as the first sync can be for anything... The more interesting information is the position of each sector in the sector descriptor. I wonder if the first sync info is (or was) intended to be the position of the first sync preceding the sector placed just after the track splice (hardto detect with a WD1772)?
Contrary to what I thought it seems that this information is present in version 3.0 but not present in 3.2 (not a hard rule need to be confirmed) indicating that the info was considered as useless?

[AtariZoll]

I found another strange thing for track with one "data sector" like Maupiti Island
Here we have a track descriptor that have TrackFlag=0x61 => Track data with only track image size (versus flag 0xE1 sync off + track size).
- First of all I am surprise to see 0x61 because this Pasti latest version 3.2 and I would have expected 0xE1 that provides more information ??? Therefore I do not understand when/why 0x61 versus 0xE1 ???
- But the problem is that when I add up all the fields as described in the doc I am ONE BYTE SHORT to where next track descriptor point ??? If I look at the data with an hex editor it seems that indeed the first two bytes in the Track Data records is the track lenght but I have an orphan byte at the end. I cant find why? is it to be on an even boundary ??? ....
I am investigating and will let you know..

If it is OK without sync offset, and as in this case track read is used .. I guess that no need for $E1 type.
Pasti images are made on Atari, so no wonder if track data must be on even boundary. Did you ever see track data starting on odd address ?

[DrCoolZic]

I have found another bug(?) in the files written by Pasti.

In the Track Descriptor you have a field called trackLength defined as: length of the track in number of bytes
In case you have a Track Image record (bit 6 of TrackFlags set to 1) it is preceded by a Track Image size field. You would expect to have the same value in both places.
Unfortunately this is not always true! In most cases both values are equal but in some cases they differs.
When you read the track data you should always use the track image size given in front of the data and ignore the trackLength given in the Track Descriptor.

I have only found this problem on images generated with DCartridge Tool = 0xCC

And remember to use the recordSize to move from track to track as the Track image might be shorter by one byte!

For information my sources are released under GNU license at https://github.com/DrCoolzic/AIR
You will find C++ and C# sources. The C++ sources are not maintained. Unfortunately C# is not good at reading writing binary file as has very limited casting capabilities
I have completed about 50% of the writer and I will update the sources at this time.

Meanwhile a new better version of the reader (the writer write incomplete file and is useless).

PastiRead.rar

[DrCoolZic]

Pasti images are made on Atari, so no wonder if track data must be on even boundary. Did you ever see track data starting on odd address ?

I am not sure that there is a good reason to be on an even boundary. As the first byte of the track descriptor is a 4 bytes value may be it need to aligned on a double word? I am not too much familiar with alignment with 68000 and I do not know if the code was written in assembly or not.
I will a check in my code to see if the track descriptor is always double word aligned.

EDIT:
I have added an assertion check to check the alignment and on all tested file the value is word aligned but not double word align.
All this information is going in the next version of the doc

[AtariZoll]

Word align is what is relevant for 68000 CPU. Because then you can use instructions like move.w (transfers 2 bytes) and move.l (transfers 4 bytes). In case of move.l address may be on not double word align (CPU will do it in 2 steps, as bus is 16-bit) . Condition is even address.
Of course, you can do everything by data read/write bytewise, but it is much slower.
I'm sure that Pasti imager is done in ASM, at least most of code.
Btw. even DMA chip works always with even base address, because it accessing RAM 16-bit wise. If you give odd base address, bit 0 is ignored (just don't try this in Steem, as is not correct emulated) .

[DrCoolZic]

Thanks for the info. This is probably the reason why it is always even
Remember here we are just reading/writing an image file so no DMA involved ...

[DrCoolZic]

OK It looks like the true story is that Track Data Image content is always padded to an even boundary and therefore the complete Track record is always even.
If you are just reading you should not have to care. Just follow the given pointers but in practice the imager can (should) add one byte at the end of the Track Image Content record to pad...
I have updated the spec to reflect this. In this new revision I have also indicated what value to use for macrodos/speedlock. Therefore if you have a file with revision 0 and bit 0 of the fdcFlags set use the provided values otherwise read the timing records from the file.

I have completed the Pasti writer
If reading Pasti files is a pain writing them is a nightmare
I have added few advices in the documentation like padding for people that want to write Pasti file.

Attached is my Pasti Reader/Writer. It allows to read a Pasti file and indicates the location in the file of all records this is great when debugging the reading and writing of files...
Click Pasti file to select the file to read and click Write file to rewrite the file read - You can't write if you do not read
The written file should behave the same but the content can be somewhat different:
- It seems like the Ijor imaging tools uses Sector descriptors in cases where it is not useful (so my writer do not write them!)
- The file is always written using format 3.2. latest format even if the input is 3.0
- Currently the FirstSyncOffset value is not written (bit 7 of trackFlags = 0). This value seems useless and apparently not present in new revision of the files (If need to be added I just need to flip of a flag in my soft).
- When reading 3.0 file with speedlock flag set the file is rewritten in 3.2 and the timing records are automatically added. So if you do not support 3.0 speedlock you can use the soft to convert the file.

I have only tested on a limited number of test cases. The program is full of assertion check to verify consistency of the read/written files so it should report inconsistency. In cases like this you can hit continue or abort.
I have run under Steem several files rewritten with my prog and using tough format (Maupiti, Theme park mistery, D50, Dungeon master, ...) but only basic verification has been done.

Please try it and let me know if you find problems. I will probably not spend time to do a nice GUI but I will be doing bug fixes.

PastiRead05.rar

Pasti documentation-v05.rar

[Steven Seagal]

Btw. even DMA chip works always with even base address, because it accessing RAM 16-bit wise. If you give odd base address, bit 0 is ignored (just don't try this in Steem, as is not correct emulated) .

I'll change that.

[DrCoolZic]

Looking at more STX files.
It seems like recent version of imaging tool is very conservative ...
It seems to generate Track Image Data record and Sector Descriptor almost for all tracks (used not to do this is early versions).
My guess is that file size is less of a concern and therefore adding more information is probably less critical.
Another thing that seems to have changed is that it seems like the sector timing information in the sector descriptor is always given even on a standard sector (used to be 0).
Again adding this information does not hurt.

If you need to write STX file you can take the same approach always write information even if not 100% required.
Finding if you need Sector Descriptor is relatively easy, but deciding on track image is tough (for example It is especially difficult to find hidden data into gap). It adds about 1MB but it is safe.
Optimizing sharing of data between Track Data and Sector Data allow to reduce the added size
I will probably take this approach in Aufit.

[npomarede]

Hello
looking at the different structures, I wonder how we can compute easily the start of the Timing Record (for variable bit rate sectors in revision 2) ?
Timing Record is after the Track Data ; Track Data's start is easy to compute, Optional Sectors Image is also easy to compute.

From there, Timing Record can be seen as :
- either after the last sector in the Optional Sectors Image block, but we don't know directly how many sectors are stored there, nor their size
- either after the Track Data block, but here also we don't know the size of this Track Data block

I can think of a way by looking at all the sectors' DataOffset to compute the max position in the TrackData for all the sectors ; adding the size of the corresponding sector to this max position would give the end of the TrackData block.
Do we really need to scan all the sectors of a track to compute the start of the Timing Record, or is there a more straightforward way to do it ?

Nicolas

[DrCoolZic]

Hello
looking at the different structures, I wonder how we can compute easily the start of the Timing Record (for variable bit rate sectors in revision 2) ?
Timing Record is after the Track Data ; Track Data's start is easy to compute, Optional Sectors Image is also easy to compute.

From there, Timing Record can be seen as :
- either after the last sector in the Optional Sectors Image block, but we don't know directly how many sectors are stored there, nor their size
- either after the Track Data block, but here also we don't know the size of this Track Data block

I can think of a way by looking at all the sectors' DataOffset to compute the max position in the TrackData for all the sectors ; adding the size of the corresponding sector to this max position would give the end of the TrackData block.
Do we really need to scan all the sectors of a track to compute the start of the Timing Record, or is there a more straightforward way to do it ?

Nicolas

I have asked myself the same questions and I have not found an easy way to compute the start of the timing data. What I am doing is to store the maximum value for the offset while reading the track data and the sector data. You cant just take the last offset value. I have run in some cases where the last sector was inside the track data for optimization!
If you really want to compute before reading the data I guess it is possible: you have the offset in the sector descriptor from track data start, and you have the size given in sector size of the address field (128 << size). So I guess you can scan all the sector descriptors and compute the max value from there

[npomarede]

Hello
looking at the different structures, I wonder how we can compute easily the start of the Timing Record (for variable bit rate sectors in revision 2) ?
Timing Record is after the Track Data ; Track Data's start is easy to compute, Optional Sectors Image is also easy to compute.

From there, Timing Record can be seen as :
- either after the last sector in the Optional Sectors Image block, but we don't know directly how many sectors are stored there, nor their size
- either after the Track Data block, but here also we don't know the size of this Track Data block

I can think of a way by looking at all the sectors' DataOffset to compute the max position in the TrackData for all the sectors ; adding the size of the corresponding sector to this max position would give the end of the TrackData block.
Do we really need to scan all the sectors of a track to compute the start of the Timing Record, or is there a more straightforward way to do it ?

Nicolas

I have asked myself the same questions and I have not found an easy way to compute the start of the timing data. What I am doing is to store the maximum value for the offset while reading the track data and the sector data. You cant just take the last offset value. I have run in some cases where the last sector was inside the track data for optimization!
If you really want to compute before reading the data I guess it is possible: you have the offset in the sector descriptor from track data start, and you have the size given in sector size of the address field (128 << size). So I guess you can scan all the sector descriptors and compute the max value from there

OK, that's the same method I wanted to use ; that's where we can see that the STX format could need a little polishing, having a higher version and/or revision with an offset to the timing record would have been much cleaner

[DrCoolZic]

Yes I agree and as I said I started to define a cleaner STX but the problem is that it would probably break the Pasti.dll so I guess the best is to live with that.
But as I mentioned we can provide advice on image witter to best practice. For example it does not hurt to always ask for sector descriptor (unless one sector track), and advice to always provide timing info, etc.

[DrCoolZic]

New version of my Pasti reader/writer v0.6

This new version not only allow to see the Pasti sector/track metadat but this new version allow to display the data as well including fuzzy bits and timing data
Also fixed abug you may have encontered to write Pasti file (te file had to exist!).

To read the data enter the track side in textbox and click Track/ Sectors content

[DrCoolZic]

Pasti Reader/Writer information is now available in my web site
http://info-coach.fr/atari/software/FD-Images.php#STX

The latest source code for v0.7 is now available in https://github.com/DrCoolzic/AIR
This include the source for the reader as well as the writer.
Note that only the C# source files have been updated. The C++ sources will need to be updated as well but there is no plan for immediate future.

Latest version 0.7

PastiRead.rar

Documentation for developpers
http://info-coach.fr/atari/software/Ima ... index.html

[DrCoolZic]

IMPORTANT MODIFICATION
Currently Bit 5 of TrackFlags is marked as not used. But in fact it is used!
Sarnau describe it as: 5 - with PASTI 0.4b always set, if Bit 0 is set (= track protected)
Files that I have generated with Aufit did not have this bit set and the resulting file was not working.

I have looked at many stx files and here what I believe is the meaning of bit 5:

- If bit 5 is set (bit 5=1) then this is the new format that implies that the readTime from Sector Descriptor HAS to be used (i.e. this time must not be equal to zero) unless the sector has the RNF flag set in which case readTime=0
For example this is the case in Maupiti Island where you have
Track 00.1 6293 bytes 1 sect FuzBytes=No Flag=61 TImage SDesc RecSize=6328 (11102-11117)
Sector T=7 H=0 SN=7 S=183 CRC=6607 bitPos=440 Time=0 Flags=18 Off=0 (11118-11133)

- if bit 5 is not set (bit 5=0) this is old format and in that case he readTime behave as explained in the doc: value of 0 means normal sector, value !=0 means you have to use the provided value

This need to be confirmed before I modify the Pasti Doc
Any feedback is welcome

[IFW]

Just wondering... why don't you ask the author to open-source the imager...?
Then you can replicate the process exactly via emulating the WD drive.