Power Drift and Pasti & patch

[DrCoolZic]

Do we really need to define a new format?
- Pasti has limitations but it does the job in almost all cases even if it is sometimes ugly
- IPF can do anything you can imagine. The problem is that currently it is only generated by SPS people. Getting IPF files from stream files simply does not happen (I have now be waiting almost three years to gt nothing)
so supporting it in emulator is good but it is not an option to end user that want to save their game (unless Aufit write IPF )
- Flux level is an option. It p^rovides information at a level of details even lower than IPF. Is it useful? I do not know

[AtariZoll]

However unless AtariZool find a games with a track that has no sector and byte variation it is not usefull. The most common on a track is to use a shorter/longer time for bytes that results in a track that has more or less bytes.
.....That's why in another thread I wrote that STX format was IMO limited as it can not store intra gap variations for example (which IPF format can)......
Is this used somewhere? do you mean gap bytes with varying lenght?
As far a s I remember IPF file does not store any timing information directly. It just refers to a density format (e.g. short, long, speedlock, ...) and the library is responsible of converting this to timing values, so yes anything is possible.

I dis not see or hear, read about protection where part of track dump was with different density. It is possible in theory of course, but maybe is troublesome for duplicating HW to change density in middle of track, without changing sector.

"For example, on a 9 sectors track, you have approx 600 bytes of GAP (usually value $4e) at the end of the track.
One could imagine a protection where those bytes would not be written at 32 ns, but 30 or 34 ns. In that case, you could check that when you read the track you have 128 consecutives bytes $4e that last 128*30=3840 ns instead of 128*32=4096 ns (similar to macrodos, but at the track level)
This is just theoretical, maybe no game was protected this way, but having a generic format where each byte is stored with its duration would be better than just handling the special case of the sector."

Yes, it is possible, but as said writing such GAP could be troublesome. Additionally code for testing it would be complicated too. Don't see need for it. Much used sector with different density is good enough - not possible to copy without special equipment.

[npomarede]

Yes, it is possible, but as said writing such GAP could be troublesome. Additionally code for testing it would be complicated too. Don't see need for it. Much used sector with different density is good enough - not possible to copy without special equipment.

I don't think duplicating machines such as the 'Trace' have any knowledge of the inner layout of the track, they just write bytes at a specified duration, so changing duration during the GAP would be no problem.
I don't say it would be better than at sector level, but the goal of a protection is also to disturb the potential cracker by using not yet known method, to try to stop crackers or at least to slow down cracked release as much as possible.
The code for testing it would not be that complicated too.

[AtariZoll]

Do we really need to define a new format?
- Pasti has limitations but it does the job in almost all cases even if it is sometimes ugly
- IPF can do anything you can imagine. The problem is that currently it is only generated by SPS people. Getting IPF files from stream files simply does not happen (I have now be waiting almost three years to gt nothing)
so supporting it in emulator is good but it is not an option to end user that want to save their game (unless Aufit write IPF )
- Flux level is an option. It p^rovides information at a level of details even lower than IPF. Is it useful? I do not know

Well, I already talked about something what would be almost perfect and not much space hungry: ST - like tracks data for unprotected tracks, but better STT like for unprotected and simpler protections (like non-standard sectors and headers), and flux level for better protected tracks. It is fine for most of floppies where 99% of sectors is standard - will have smaller images than public tool made Pasti ones. But in many cases it would have larger images, in range of 10 MB, where whole floppy is with custom tracks - track dumps, high-density (Space Ace serial ...) . When SPC will be enough mature it should solve all protections, I guess. 10 MB is not much for today storage.

[AtariZoll]

I don't think duplicating machines such as the 'Trace' have any knowledge of the inner layout of the track, they just write bytes at a specified duration, so changing duration during the GAP would be no problem.
I don't say it would be better than at sector level, but the goal of a protection is also to disturb the potential cracker by using not yet known method, to try to stop crackers or at least to slow down cracked release as much as possible.
The code for testing it would not be that complicated too.

Crackers don't need to know much about floppies and all this fuzzy, weak bits, density etc. things. They need to know 68000 code, little about WD1772 code. Point in cracking is not to understand 100% how protection works, but to deactivate, fool it. So, you focus on some comparisons, writing of some key values - in code part which checks protection(s) and then replace it with writing constant if there is key in protection, or just put bra instead beq at comparison. What slowed cracking most is well protected copy protection check code self - so well hidden, encrypted, hard to trace, and like.

[npomarede]

Do we really need to define a new format?
- Pasti has limitations but it does the job in almost all cases even if it is sometimes ugly
- IPF can do anything you can imagine. The problem is that currently it is only generated by SPS people. Getting IPF files from stream files simply does not happen (I have now be waiting almost three years to gt nothing)
so supporting it in emulator is good but it is not an option to end user that want to save their game (unless Aufit write IPF )
- Flux level is an option. It p^rovides information at a level of details even lower than IPF. Is it useful? I do not know

Well, I already talked about something what would be almost perfect and not much space hungry: ST - like tracks data for unprotected tracks, but better STT like for unprotected and simpler protections (like non-standard sectors and headers), and flux level for better protected tracks. It is fine for most of floppies where 99% of sectors is standard - will have smaller images than public tool made Pasti ones. But in many cases it would have larger images, in range of 10 MB, where whole floppy is with custom tracks - track dumps, high-density (Space Ace serial ...) . When SPC will be enough mature it should solve all protections, I guess. 10 MB is not much for today storage.

The "problem" of such an hybrid format is "how do you determine which level of detail should be used to store a specific sector/track" ? If you know the protection used, you can have some scripts to store detailed informations only for the protected track, but what if the track is slightly damaged and gives CRC error when reading it ? Is this damages, or is the bad CRC part of the protection ?

For me, the problem is not only the storage format, it's the analysis required to choose which informations matter and which don't, and this is exactly the way IPF files are produced (and why they take so long)
So, I agree storing raw flux data solves this problem, as you store the highest level of details possible, but at the expense of a bigger file size (I know size is not that much a problem, but coders like to optimize things, why store 2-3 MB when it could be squeezed ).
And even at flux level, you sometimes need an expertise to watch the graph of the disk dump, see if data are scattered or not, which could indicate a damaged floppy that need to be cleaned and dumped again.

STX is nice, it can be used by anyone without any technical knowledge, but in the end, if the input floppy is damaged in a certain way, the user will have no idea the dump is bad.
A suitable format should come with tools to validate the data, maybe some parts of the analysis can be automated, but some tricky cases can remain.

Nicolas

[npomarede]

I don't think duplicating machines such as the 'Trace' have any knowledge of the inner layout of the track, they just write bytes at a specified duration, so changing duration during the GAP would be no problem.
I don't say it would be better than at sector level, but the goal of a protection is also to disturb the potential cracker by using not yet known method, to try to stop crackers or at least to slow down cracked release as much as possible.
The code for testing it would not be that complicated too.

Crackers don't need to know much about floppies and all this fuzzy, weak bits, density etc. things. They need to know 68000 code, little about WD1772 code. Point in cracking is not to understand 100% how protection works, but to deactivate, fool it. So, you focus on some comparisons, writing of some key values - in code part which checks protection(s) and then replace it with writing constant if there is key in protection, or just put bra instead beq at comparison. What slowed cracking most is well protected copy protection check code self - so well hidden, encrypted, hard to trace, and like.

I agree, but the protection should also fool copy programs ; I know variable sector length are not copiable on Atari, so it's not better to do variable duration at the gap level, but at least it can confuse copy programs (and for that case, it would completely confuse Pasti.prg, letting you believe you made a correct dump while it's not the case)

[DrCoolZic]

For me, the problem is not only the storage format, it's the analysis required to choose which information matter and which don't, and this is exactly the way IPF files are produced (and why they take so long)

IPF files relies on the idea that all floppy disks must match a limited number of known "format".
So what really takes time is to find all "formats" used on a platform and when a new format is discovered it needs to be described using the CTA language and potentially the library need to be modified if a new "type" of protection is discovered.
I have discovered with Aufit that about 30% of the stream files proveded to SPS are bad!
So making good images is not that easy and is probably getting more and more difficult as time pass ...

[IFW]

For me, the problem is not only the storage format, it's the analysis required to choose which information matter and which don't, and this is exactly the way IPF files are produced (and why they take so long)

IPF files relies on the idea that all floppy disks must match a limited number of known "format".
So what really takes time is to find all "formats" used on a platform and when a new format is discovered it needs to be described using the CTA language and potentially the library need to be modified if a new "type" of protection is discovered.
I have discovered with Aufit that about 30% of the stream files proveded to SPS are bad!
So making good images is not that easy and is probably getting more and more difficult as time pass ...

Both you and Nic are right.
It's also worth noting that mastering the disk in the first place relied on describing the format exactly - so somebody somewhere have thousands of FreeForm (the language used by Trace to describe disk formats) scripts on their harddrive that could be converted for CTA use... but tbh it's very unlikely anyone would ever submit those, if nothing else for NDA and IP reasons.
IPF follows this "tradition" and remastering a disk takes time, sometimes with very diminishing returns, e.g. if a format was just a one-off...
As for user-generated IPFs... they will happen.

[Hippy Dave]

Most of the time. And depending on your drive's capabilities.
No-flux-areas (strong-bits, just to adapt your wording) have been discarded even though they have the perfect feature of not being copyable the analogue way. The reason for that is that they didn't work as expected with every drive model that has been used in a particular computer type.

Very interesting can you provide more information. From what I understand the NFA is obtained by providing a high bit rate data write signal to the floppy drive. When reading the data signal is shifted beyond normal boundary resulting in no transition. From what I have heard it is difficult (and drive dependent) to find the right frequency for the write signal.
No sure you are right on this one or I may not understand your point?

Code: Select all

Typically, the read-write gap length is chosen to be in the range from
one third to one fourth of the minimum recording wavelength, and the
throat height is chosen to be less than a few tens of a micrometer.

Highest frequency High-Density erasing writes are performed with:
(Double-Density might be at half the frequency)

Minimum 200 ns /Write_Data Low
Minimum 500 ns /Write_Data High

___     _________     __
   \___/         \___/
   200ns  500ns

The write head toggles on the high to low transition.

Thus:

Frequency = 1 / 2*(200ns + 500ns) = 714,286 Hz (1429 Kbits/s)

This frequency is more than sufficient, as 250,000 Hz (500 Kbits/s) is
normal. 1MHz erasing may be possible with (200ns + 300ns) or (250ns + 250ns)
if using a good cable and going beyond specifications.

Jim Drew noted that his hardware sees least values of 1.5us (666 Kbits/s).

[Hippy Dave]

Raw flux files are the way to go because the data can be processed before compressing.
I guess that most compressed raw 720kb DD floppy data will be less than 3 megabytes,
without any exceeding 5 megabytes.

[DrCoolZic]

Raw flux files are the way to go because the data can be processed before compressing.
I guess that most compressed raw 1.44 Mb floppy data will be less than 3 megabytes,
without any exceeding 5 megabytes.

Ironically the track that do not compress well are the unformatted tracks that contains no information

[AtariZoll]

The "problem" of such an hybrid format is "how do you determine which level of detail should be used to store a specific sector/track" ? If you know the protection used, you can have some scripts to store detailed informations only for the protected track, but what if the track is slightly damaged and gives CRC error when reading it ? Is this damages, or is the bad CRC part of the protection ?
For me, the problem is not only the storage format, it's the analysis required to choose which informations matter and which don't, and this is exactly the way IPF files are produced (and why they take so long)
So, I agree storing raw flux data solves this problem, as you store the highest level of details possible, but at the expense of a bigger file size (I know size is not that much a problem, but coders like to optimize things, why store 2-3 MB when it could be squeezed ).
And even at flux level, you sometimes need an expertise to watch the graph of the disk dump, see if data are scattered or not, which could indicate a damaged floppy that need to be cleaned and dumped again.

STX is nice, it can be used by anyone without any technical knowledge, but in the end, if the input floppy is damaged in a certain way, the user will have no idea the dump is bad.
A suitable format should come with tools to validate the data, maybe some parts of the analysis can be automated, but some tricky cases can remain.
Nicolas

You are right. But there is lot of images where everything is OK, only there is 1 slower sector with fake CRC. And it is called Copylock
I give it just as example of many used protection, present in hundreds of titles. I already shrinked many of such Pasti images - so removed all track dumps.
Some things can be automated, some harder. Some people can trace protections in emulators. All it can make things better, but if it works not from some reason, you can go on full flux image - will not harm much

[AtariZoll]

...
I agree, but the protection should also fool copy programs ; I know variable sector length are not copiable on Atari, so it's not better to do variable duration at the gap level, but at least it can confuse copy programs (and for that case, it would completely confuse Pasti.prg, letting you believe you made a correct dump while it's not the case)

Well, I think that you forgot something: there was no Pasti when copy protected SW for Ataris was made It is little late now to thinking about how to confuse Pasti or crackers

[JimDrew]

Do we really need to define a new format?
- Pasti has limitations but it does the job in almost all cases even if it is sometimes ugly
- IPF can do anything you can imagine. The problem is that currently it is only generated by SPS people. Getting IPF files from stream files simply does not happen (I have now be waiting almost three years to gt nothing)
so supporting it in emulator is good but it is not an option to end user that want to save their game (unless Aufit write IPF )
- Flux level is an option. It p^rovides information at a level of details even lower than IPF. Is it useful? I do not know

Flux is useful because it ALWAYS works, and it works right now. There are still protections that ipf can not handle. You can always use Aufit to check an image file for integrity... no need to wait for someone(s) to provide an ipf to you.

[JimDrew]

Raw flux files are the way to go because the data can be processed before compressing.
I guess that most compressed raw 720kb DD floppy data will be less than 3 megabytes,
without any exceeding 5 megabytes.

A .scp image of a double-sided Atari ST disk is ~7MB to ~14MB (~10MB is average) in size (non-compressed). Depending on the exact data, the images can be compressed by as much as 80%. It's not uncommon to see a 10MB image around 3.5MB when compressed with 7zip.