Copy Protection Information: Weak Bits, Bit-rate ...

[DrCoolZic]

Greetings,

I would like to start a new thread to discuss Fuzzy/weak bits!
I have selected this forum as this is related to preservation … but I might be wrong …

Motivation: I am trying to understand and document the copy protection mechanisms used for the Atari ST diskettes.
For that matter I have already collected and analyzed, for the last few months, lots of documents from the Net related to floppy disks: diskettes, drives, controllers, protections, etc… (see below for few relevant references). I have identified about 20 possible protection mechanisms and most of them are pretty straightforward. The only two challenging ones are the Fuzzy/Weak bits and the bit rate variation (bit density).
I have also conducted experiments to confirm some findings. I use several Atari ST computers with various peripherals (HD, external FD, Blitz cable …) but most importantly I own a Discovery Cartridge for which I have developed a “dump” program that allow me to visualize the precise content of the floppies with precise timing at the flux level.

I will start to give some information on what I have analyzed (to set the context and avoid unnecessary discussion) and I will end up with questions for matters I do not understand…

Drive background information
A 3-1/2” floppy drive uses a ferrite/ceramic head which is a U-shaped iron core wrapped with electrical windings to create the read/write head (almost a classical electromagnet, but very small). When writing, the current in the coil creates a polarized magnetic field in the gap between the poles of the core, which magnetizes the surface of the platter where the head is located. When the direction of the current is reversed, the opposite polarity magnetic field is created. For reading, the process is reversed: the head is passed over the magnetic fields and a current of one direction or another is induced in the windings, depending on the polarity of the magnetic field. Since the signals read from the disk are very weak, special circuits are required to read the low-voltage signals coming from the drive heads, amplify them and interpret them, to decide if each signal read is a one or a zero.
The floppy disk also uses two erase heads in addition to the read/write head. These are called tunnel-erase heads. They are positioned behind and to each side of the read/write head. Their function is to erase any stray magnetic information that the read/write head might have recorded outside the defined track it is writing.
First of all it is important to note that while the signals flowing through the drive’s head are analog signals, the floppy drive interface signals are digitals. This is obviously quite different from an analog recording device like a VCR where the interface signals are analog.
Remarks on so called “analog copier”: The usage of the term analog copier (e.g. blitz copier) for copying floppy disks is somewhat misleading, if not wrong, as this kind of devices do not copy analog signals. However, if clearly specified and understood, the term analog is acceptable to indicate the fact that the signal produced by the head of the reading drive is not processed by a digital circuit (the FDC) but directly sent to the head of the recording drive. This implies that doing copies through an analog copier do not degrade the strength of the signal (as it would do with a VCR) yet multiple copies (and sometimes even single copy) may fail but for totally different reasons (mainly due to rotation speed and/or synchronization of the two drives).

From the above information we conclude that it is not possible to directly write "weak signals" and it is not possible to directly know that we have read "weak signals". But of course this does not mean that it is not possible to have (or create) "weak signals" on a media.
We therefore need to clearly describe how it is possible to create and detect weak signals…

Terminology
Different people use different names: weak bits, fuzzy bits, flaky bits, flakey bits, phantom bits ...
Weak bits is the most commonly used term, however I prefer fuzzy bits as it indicates the resulting effect without suggesting any underlying causes.

Definition
Fuzzy bits on a floppy diskette are bits which will appear different in subsequent read operations. In other words each time you read a sector with weak bits you get different data (i.e. random data values).

Causes
It seems that they are two possible causes for Fuzzy bits:
(1) bits recorded on floppy media with a weak (or inexistent) magnetization of the bitcell. I will call them weak bits
(2) bits coming after a long series of missing clock and data bits. I will call them flaky bits
In the first case reading weak bits result in a weak flux transition that can be interpreted differently at different time by the drive due to environment (e.g. random noise).
In the later case internal the PLL of the FDC gets out of synch and the next received bit may be interpreted differently based on small fluctuation of the drive rotation speed.

Creation
We need to make a distinction between the Weak bits and the Flaky bits:
Weak bits can be the result of a physical defect on the floppy media either for "natural" reasons (i.e. defects on media see http://www.grc.com/srphysics.htm, or bit rot http://www.softpres.org/?id=glossary:bit_rot), or "artificial" reasons (using techniques ranging from disk scratching to evaporation of surface layer by laser). We are not interested by this category of physical defects has we do not have control over them. But weak bits can also be created on a media with no physical defects [followings are suggestions]:
- ??? By formatting only a portion of a track???
- ??? By writing information totally out of synch???
- ??? By ???
Flaky bits are created by simply following their definition: that is write bits following a long period of time (enough for the PLL to get desynchronized) without any clock or data transition.

Detection
Detection of fuzzy bits is quite easy: read a sector several times and check that it returns random data.
Note that:
- If the fuzzy bits are located inside a DATA block the content of the data block changes and the FDC return a CRC error.
- If the fuzzy bits are placed in an ID block the corresponding sector will be randomly existent or not (due to the CRC error).
The detection mechanism should also test that the random values returned are not due to usage of tricks like duplicate sectors.

Example
Dungeon master: Track 0, sector 7

References
Here is a short list of references I have used:
Copy Protected Disk http://www.atariage.com/forums/index.php?showtopic=51523&mode=linear, Atari Protected Disk Image Format http://members.chello.nl/taf.offenga/app/software/atp/ATP16.htm#_Toc69480551 , Introduction into Copy Protection, Micro Floppy Disk Drive Spec: Teac http://www.teac.de/dspd/downloads/datasheets/dl_fd05hf8630.pdf & Citizen http://www.citizen.co.uk/pdf/x1de00a.pdf , SpinRight Technical note http://www.grc.com/files/technote.pdf , Flakey bits http://www.softpres.org/?id=glossary:flakey_bits ...

Now the Questions!
Of course my questions are related to the “creation” of weak bits (and not flaky bits).

Keeping in mind the fact that (see SpinRight doc for more info):
• While we provide perfect pulse to the drive’s head, the write and read cycle returns a significantly rounded and much broader signal with a small overshoot at the end.
• Closely spaced pulses of opposite polarity overlap each other and partially cancel each other out.
A first idea is:
If you format a complete track on a “virgin” media and that during portion of a specific sector you “disable” the formatting this should result in weak transitions (in fact no transition) on the media?
However this is not a general solution as most of the diskettes are already formatted or preformatted! We therefore need a more generic mechanism that can explain for example how a Discovery Cartridge performs a copy of a weak sector on an already formatted diskette…

I can only think of one way to create weak bits on preformatted diskette: By writing “uncorrelated” bits on the media. In other words by writing sequences of data which are not related (synchronized) with the pre-existing data on the media. This should result in random combination of new and existing bits and therefore result in random weak transition on the media?!?
The how is another story but for example using the DC it should be possible to disable the write gate signal while writing data …

Et Voila … Sorry for this long post ...

Jean (aka DrCoolZic)

[ijor]

For reading, the process is reversed: the head is passed over the magnetic fields and a current of one direction or another is induced in the windings, depending on the polarity of the magnetic field.

Not exactly. The head doesn’t read the magnetic field polarity, it only detects changes. As in most electromagnetic applications, it is the variation of the magnetic field that produces the current. So you cannot know the absolute polarity, but only when there is a change in the polarity.

Remarks on so called “analog copier”: ... This implies that doing copies through an analog copier do not degrade the strength of the signal (as it would do with a VCR) yet multiple copies (and sometimes even single copy) may fail but for totally different reasons (mainly due to rotation speed and/or synchronization of the two drives).

This is not the main problem of analog copier. This affects only the capability of them to copy some time of protections. But the drawbacks of analog copiers are much more far reaching. Search for analog copier in this forum. Some time ago I elaborated extensively in the issue.

But weak bits can also be created on a media with no physical defects [followings are suggestions]:
- ??? By formatting only a portion of a track???
- ??? By writing information totally out of synch???
- ??? By ???

There is no way you could “create” a demagnetized area (not without a special drive or special equipment at the drive side). You can of course, using timing, leave some area of the disk surface virgin. And you can also create all sort of weird effects in a small overlapping area (at the point where writing just ended).

If you format a complete track on a “virgin” media and that during portion of a specific sector you “disable” the formatting this should result in weak transitions (in fact no transition) on the media?

Yes (and no). It means you'll get random pulses.

We therefore need a more generic mechanism that can explain for example how a Discovery Cartridge performs a copy of a weak sector on an already formatted diskette…

You can’t directly, but you can indirectly by stopping and restarting the writing. You can also deselect the drive for a small period of time. None of this would be very useful.

The DC software is pretty dumb. It just copies the flux transitions. This might not recreate the exact signal as in the original media (obviously it won’t if it was virgin). But this doesn’t matter. In first place the protections usually aren’t too strict about detecting weak bits. As a matter of fact some of them can be fooled by using double sectors (something that Procopy and Acopy try to do).

But even when they are stricter, they can’t detect the exact magnetic signal. You must realize that the main principle of a protection is that it should be detected reliable on end user hardware. The protection software runs in a stock computer using a stock FDC.

I can only think of one way to create weak bits on preformatted diskette: By writing “uncorrelated” bits on the media. In other words by writing sequences of data which are not related (synchronized) with the pre-existing data on the media. This should result in random combination of new and existing bits and therefore result in random weak transition on the media?!?

Sorry, I don’t really understand what you mean in this paragraph.

[DrCoolZic]

Some quick answers to your comments

ijor wrote:

DrCoolZic wrote:
For reading, the process is reversed: the head is passed over the magnetic fields and a current of one direction or another is induced in the windings, depending on the polarity of the magnetic field.

Not exactly. The head doesn’t read the magnetic field polarity, it only detects changes. As in most electromagnetic applications, it is the variation of the magnetic field that produces the current. So you cannot know the absolute polarity, but only when there is a change in the polarity.

Yes and no: Of course this is an oversimplified description and yes the reading is only possible because of flux variation. However the recording is made of flux reversal on the magnetic surface and it is possible to know the polarity of the flux read but this has no interest as we are only interested by transitions.
If I try to depict the signal coming from the head it looks like:
---^--v--^--v (hope this make sense)

This is not the main problem of analog copier. This affects only the capability of them to copy some time of protections. But the drawbacks of analog copiers are much more far reaching. Search for analog copier in this forum. Some time ago I elaborated extensively in the issue

Again you are right this is oversimplified and the problems are far more complex as you explain very well in http://www.atari-forum.com/viewtopic.php?t=3583&highlight=analog+copier
But this was not my point. I just wanted to reinforce the fact that data signals coming in and going out of a floppy drive are digital signals and therefore there is no signal degradation during copy like it would happen with analog recorder (e.g. a VCR).

...
Sorry, I don’t really understand what you mean in this paragraph.

The problem is how do we create a weak analog transition on the magnetic surface with only a digital interface??? Of course we have not the choice of providing a weak digital bit!!!
Therefore I am not convince that there is a way to create weak transitions from the digital interface at least without being very creative?!?

So my creative idea was the following: As we know that adjacent bitcells interfere (you can look at the SpinRight site for a good figure http://www.grc.com/srphysics.htm ) my idea was that may be if we write bits that are not related to pre-written adjacent bits then they would interfere randomly and produce weak transitions. Actually you just mentioned some means of doing that (stop/start writing, deselection of the drive …). This is what I meant by by writing “uncorrelated” bits on the media
NOTE - After relecture this does not seems too realistic

You mention the DC and the fact that it copy the flux transition.
This where I do not understand how the DC would copy weak bits due to weak transitions.
Lets take an example:
I just made a copy of dungeon master track 0 and as you may know the sector 7 contains weak bits.
I took the original diskette analysed it 5 times in DC flux mode and the content of sector 7 always return different values for each reading after about 32 bits. This is what is expected from weak bits.
After that I took a copy made with the DC and sure enough each reading of sector 7 in flux mode was returning different values at about the same spot. Therefore it seems like the DC succeeded in copying the weak bits!

Great but how?
One idea is that they read the track several time and find that there are bits changing. But unfortunately this is not the case and it is possible to dump the content of a floppy on a hard disk and use this dump to write the “copy”. You May argue that they use some indication about weak bits in the dump. Well here again the answer is no if you look at the content of the dump file in flux mode only the flux transitions are recorded and nothing more.
Lets recap: Knowing that there is now way at the digital interface signal to know that we have read a weak bit, knowing that the DC do the copy in one pass, knowing that the DC do not have indicator for weak bits … How is it possible to copy weak bits … Well may be weak bits = Timing trick.

Looking at the flux it does not seems like we have any timing violation as we would find for example with Rob Nothern protection. Therefore it must be a more subtle timing trick (if any - remember that this is only a guess at this time).
I still need to experiment but it looks like: if you place a data bit just in the middle of normal spots (i.e. 4, 6, 8 µs) let say at 5µs then it will be read sometime as 0 sometime as a 1
Actually I looked at the flux transition on this sector and I found that it seems like on different reading the bits are shifted by 1:

Code: Select all

The two readings both start the same (here the first three bytes):
= DATA 515 bytes l=16454.47us @121.368ms *BAD CRC=7300
  0ed4 4.00  fb 07 50  .|.|.|.|.|...|.| ..|.|.|.|..|.|.| ...|...|..|.|.|.

But after a while first reading:
  0efe 4.00  07 07 07  ..|.|...|..|.|.| ..|.|...|..|.|.| ..|.|...|..|.|.|
And second reading:
  0f2e 4.00  d0 d0 d1  .|.|...|..|.|.|. .|.|...|..|.|.|. .|.|...|..|.|..|

As you can see the flux is exactly the same but shifted by one position

.

This is the where I am right now and I must experiment more...

But the more I think about it the more I am convince that weak bits used for protection (not to be confused with weak bits due to defects) are probably based on timings rather than on weak transitions ?!?

Jean

[ijor]

(analog+copier)But this was not my point. I just wanted to reinforce the fact that data signals coming in and going out of a floppy drive are digital signals and therefore there is no signal degradation during copy like it would happen with analog recorder (e.g. a VCR).

Analog copiers make a severe degradation of the signal. That is their main problem. It is not correct to say that the signal coming from the drive is completely digital, it is only partially digital. The interface is TTL, so indeed the voltage levels are digital. However the timing between pulses is, obviously, not digital. And hence the waveform of the signal can’t be considered digital.

The timing between pulses is subject to minor variations mostly to mechanical imprecision, and those variations would be amplified (instead of being equalized) by the analog copier. Furthermore, the read signal is not exactly what is recorded in the disk surface. Nor what is recorded is exactly the write signal.

The problem is how do we create a weak analog transition ...
This where I do not understand how the DC would copy weak bits due to weak transitions ...
But the more I think about it the more I am convince that weak bits used for protection (not to be confused with weak bits due to defects) are probably based on timings rather than on weak transitions ?!?

There is no such thing as “weak transitions”, at least not that they were intentionally and professionally created. I never seen or heard any protection that is based on weak transitions. That would be too complicated to create and too unreliable.

To actually create weak transitions you would need to record using a weaker magnetic flux. Or otherwise partially erase the area to obtain an attenuated signal. This is not reliable at all because the disk magnetic layer is manufactured for a saturated (digital) magnetization.

If you check some damaged disks, you could sometimes see there are weak transitions. In most of these cases they won’t produce a weak bit effect. Because the typical result is that the pulse would be read always, or not at all depending on the drive.

There are certainly no weak transitions in Dungeon Master. The weak bit effect in Dungeon Master is (as it seems you find out yourself) just a consequence of some pulses falling outside of the MFM decoding window.

But not all weak bit protections are like this.

[DrCoolZic]

Ijor

Great information. You 100% right on analog copier. I did not thought too much about the timing problems induced by the "analog" read/write cycle and only concentrated on the strength of the signal. This perfectly makes sense: the width and position of the pulses should degrade. I’ll update my information thanks again.

There is no such thing as “weak transitions”, at least not that they were intentionally and professionally created. I never seen or heard any protection that is based on weak transitions. That would be too complicated to create and too unreliable.

This again make sense and is good news. Actually I could not make sense of these weak transitions due to low or neutral magnetic polarity. I was looking at them because I found the information on the great SPS site (look at http://www.softpres.org/?id=glossary:flakey_bits )
where they say:

Flakey bits: These are particular bit cells on a disk that indicate neither a 0 or a 1 clearly. They are indeterminate because they have neutral magnetic polarity. You can think of them as unformatted, and when read, they just return “noise”, i.e. random data...

Ok so one way of making fuzzy bits is too place them into uncertain decoding windows (which seems to be the case for DM). If we do that without creating MFM violations this imply to place bits around 3µs, 5µs, and 7µs central position (which fall in the middle of normal 1,3 RLL coding). How much apart from this central position?

But you also mention that not all weak bits are like that … So question is how other weak bits are…?

As I mention in my original post I can think of long period without transition (this obviously violate the MFM encoding) that would desynchronized the PLL and trigger random read values. ???
Please can you detail the different techniques you know to create fuzzy/weak bits?

While this is another topic that I would like to discuss later, I now remember looking at the timing of DrT musical program which uses protection based on bit rate variation. It uses a pattern quite different from Rob Northern coding (which uses a change of less than 3-5% of the bit rate equivalent to drive spinning slowly - as you know the subject has been discussed several times on this forum). With this DrT diskette the pattern looks totally random and alternate between pulses less than 4µs apart (short timing violation sometimes down to 2µs) and up to ~20µs (long timing violation) apart!

For example:
DrT D50 Editor Track 0 sector 10

Code: Select all

Beginning of sector is ok (.=not transition |=transition scale 2µsec):
= DATA 515 bytes 16542.31us @ 0.000ms clk=4.01 *BAD CRC=314e
  0000 4.00  fb 01 ff  .|.|.|.|.|...|.| ..|.|.|.|.|.|..| .|.|.|.|.|.|.|.|
  0003 4.00  ff ff ff  .|.|.|.|.|.|.|.| .|.|.|.|.|.|.|.| .|.|.|.|.|.|.|.|

But after a while:
  00a5 4.00  83 84 34  ||..|.|......||| .|..|......||... |.|..|.|..|||.|.
  00a8 4.00  70 72 e2  |..|.|.|....|... |.||.|.|..|..||. .|.|.|..|.|..|..
  00ab 4.00  4c 45 07  ...||.|..|.|.... |.||.......|...| ....|......|.|.|
  00ae 4.00  24 d0 10  ..|.||.....|..|. ||.|...|....|... .......|..|...|.
  00b1 4.00  8f c2 7f  .||.|....|.|.|.| .|.|..|.|.|..||. ..||.|.|.|.|.|||


I did not pay attention to that but I bet it should result in fuzzy/weak bits (i.e. random values)? Would that be another form of weak bits?

One last point you mention in your original reply:

But this doesn’t matter. In first place the protections usually aren’t too strict about detecting weak bits.

Actually I am not at all interested in protection mechanism per se and how to crack or full them. I am actually interested in diskette's preservation and therfore in ways to make perfect duplication with the DC.

Thanks Jean

[ijor]

But you also mention that not all weak bits are like that … So question is how other weak bits are…?

As I mention in my original post I can think of long period without transition (this obviously violate the MFM encoding) that would desynchronized the PLL and trigger random read values. ???

Yes, transitions too far apart will make the PLL loose sync and produce a weak bit effect.

A virgin unformatted area, that could be done by stopping the recording with timing is another way.

With this DrT diskette the pattern looks totally random and alternate between pulses less than 4µs apart (short timing violation sometimes down to 2µs) and up to ~20µs (long timing violation) apart!

I did not pay attention to that but I bet it should result in fuzzy/weak bits (i.e. random values)? Would that be another form of weak bits?

I haven't seem, or at least don't recall, the exact details of Dr T. So I can comment too much. But a demagnetized area will produce random pulses, which might be what you are seeing.

Actually I am not at all interested in protection mechanism per se and how to crack or full them. I am actually interested in diskette's preservation and therfore in ways to make perfect duplication with the DC.

Preservation and perfect duplication are two different things. For the purpose of preservation you only need an image. The fact that you are unable to write back the image exactly as the original, or even not at all, doesn't affect preservation.

Take for example protections based on physical alteration to the media (such as laser burnt holes). You can preserve them with an image, but you can't write them back (not without a laser equipment).

[DrCoolZic]

Lets me first comment the end of your post

Preservation and perfect duplication are two different things. For the purpose of preservation you only need an image. The fact that you are unable to write back the image exactly as the original, or even not at all, doesn't affect preservation.

You are right that they definitively differ and duplication is a subset of preservation . I do not want to open a debate here but for me preservation is to be able to keep something as close as possible with the original. Yes it is true that it is better to be able to run a preserved image of a game on an emulator than nothing! But for me nothing replace running a game on an real old Atari with it’s mouse that have a very very special touch, the clunky and noisy disk drive, the small screen, … they all contribute to the pleasure ... but lets not get too nostalgic here . If I take an analogy emulation reminds me the technically amazing work done on augmented reality where you can for example see part of now destroyed cathedral superposed with vestiges: nice but not exactly the same …

Take for example protections based on physical alteration to the media (such as laser burnt holes). You can preserve them with an image, but you can't write them back (not without a laser equipment).

Just for anecdote I had a friend who was expert in reproducing with a needle laser burnt holes (but to be honest this was on 5-1/4” diskette! ).

Ok now let’s get back to technical stuffs.

So if I understand correctly for fuzzy bits we need to think about any possible way that would confuse randomly the PLL, data separator and shift register chain inside the FDC. I will summarize this as weak bit are created by writing bits in “uncertain areas”.

For now I keep aside the “virgin unformatted area” which cannot be easily reproduced. However this may need more investigation as apparently you mentioned that something similar was used in games like Turrican that uses a strong erasure of area of disk…?

Now I have a question about the way PASTI detect and handle weak bits? While it is quite easy to detect misplaced bits using the DC I can’t think of a way to find this information (i.e. bit timing position) through the Atari FDC. You can certainly detect presence of weak bits by reading several times the sector and checking the CRC but probably not at the level of bit position? Is there a flag in the STX file that indicate weak sector so that when emulator reads this sector some "randomness" is returned? If this is the case do you record where this "randomness" starts? As a matter of fact I have found that usually the beginning of weak sectors seems to always start correctly up to a point where the weak bits start.

Would it possible to get a description of the PASTI image format. I remember that you said in a post that you intended to make it public at some point in the future. I fully understand that the format might not be final and may not be well documented, but I would be interested. The reason is that as I mentioned in my original post I have written a program that mimic part of a FDC: a PLL (not yet working very well), a detector of strings of zero in gap (for clock/data disambiguation – not really necessary but allow to anticipate the sync marks), an address mark detector (really a sync byte detector in MFM) and a data shift register. This allows me to decode MFM flux as provided from the DC and to display them in different format (hex, ASCII, flux, timing). It even has the so called sync byte bug in read track mode (which is not really a bug in the FDC but a poor choice of the $C1 sync mark)! The goal is to be able to visualize and analyse DC dumps in order to have a precise vision of the bit transitions on a track. I am now writing a program that will allow writing back a manually edited version of the DC dump. This should allow to experiment playing with bits position in order to answer interesting open questions … I would therefore be interested in writing a program that would convert the content of an STX file into DC dump format (which I know reasonably well). If this is possible you can leave me a pm.

By the way I should normally be on your list of your beta tester for creating images from STX files (as I replied sometime ago). For info I have a limited list of original diskettes around 30, but some might be not so common (e.g. over 10 diskettes from Dr T) most from US (I was living in California at this time), and I have a long list (over 350) of duplicated diskettes (of course not good for preservation but ok for some experimentation) which I got as part of deal on eBay…

Thanks
Jean

[ijor]

Yes it is true that it is better to be able to run a preserved image of a game on an emulator than nothing! But for me nothing replace running a game on an real old Atari with it’s mouse that have a very very special touch, the clunky and noisy disk drive, the small screen, …

Please do not misunderstand me. Write back is a great, nice, good feature. And I consider it as important as you do. But again, this has nothing (or not too much) to do with preservation.

Preservation means that the item is not going to be lost. Running an image under emulation, or writing it back to physical disk are not directly related to preservation.

Some Pasti images cannot be run currently under any emulator. This doesn't mean at all they are not preserved. They are preserved, and once emulation would improve, you would be able to run them.

I do not want to open a debate here but for me preservation is to be able to keep something as close as possible with the original.

I agree. But again, one thing is to preserve it as accurately as possible, another thing is to write it back (or emulate it) as accurately as possible.

games like Turrican that uses a strong erasure of area of disk…?

That protection has no relation at all with weak bits.

Is there a flag in the STX file that indicate weak sector so that when emulator reads this sector some "randomness" is returned? If this is the case do you record where this "randomness" starts?

Yes and yes.

As a matter of fact I have found that usually the beginning of weak sectors seems to always start correctly up to a point where the weak bits start.

Indeed. This was done intentionally for making more difficult to copy sectors with weak bits.

Would it possible to get a description of the PASTI image format. I remember that you said in a post that you intended to make it public at some point in the future.

Please bear with me a little more. I am introducing some significant modifications for the next version.

Nevertheless, if you are in a hurry of getting a DC format dump of a specific game or protection, let me know by PM.

[DrCoolZic]

Please bear with me a little more. I am introducing some significant modifications for the next version.

No problem I can wait …

Meanwhile I would like to continue to bother you (or others?) on disk protection mechanisms. There are still two mechanisms I would like to discuss further: one is the bit rate variation and the other one is data under the index “hole".

Let’s start with some background information taken from the http://www.atari-forum.com/viewtopic.ph ... ob+northen thread on Rob Northen CopyLock protection.

CopyLock protection has a portion of one sector with a different bit-rate than the nominal one. A different bit-rate means that it will take a different amount of time to read. The code reads the "protected" sector measuring the time it takes, does the same with a "normal" sector; and then compares the time. If the difference is at leas the expected one, the code assumes the disk is original.
In all disks I've seen the "protected" sector is #6 on track 0. The protected sector also has a key that it is used to decrypt the software…
Some disks have a few additional disk checks. And there is an earlier CopyLock protection that is much simpler and can be copied with a software copier.

The ST floppy drive reads the track at the same speed as normal, but the data either comes into the floppy controller slower or quicker than normal depending on whether the drive motor was speeded up or slowed down during writing… If you speeded up the drive then the disc would rotate quicker meaning the time for a single rotation was reduced... The FDC would write data at the normal rate so the track would be shorter (less bytes) than normal. Conversely, if you slowed the drive speed down then you could write more data during a single rotation. When reading the track back in at normal speed the bytes of the "long track" would be shorter and coming in quicker than normal, with a "short" track the bytes would be longer and coming into the FDC slower.

In a “normal" ST system the FDC writes at a bit-rate of 250 Kbps (one bit every 4 microseconds)… Within certain limits, the reading FDC will adapt itself to the bit-rate coming from the disk drive. This is thanks to an FDC component called “PLL" (Phase-Lock Loop)… The important point for our purpose is that the FDC will deliver data to the computer at the rate sent by the drive. So if the disk was written at a faster bit-rate, the FDC will take less time to read a sector…
It is interesting to note that measuring the sector time in the ST is not so easy. This is because the DMA chip seats in the middle between the CPU and the FDC (you can disable DMA if you want, but it won't help). Not so difficult for CopyLock, but harder for other one…
There is another ST protection, more advanced than CopyLock if you want, that has a variable bit-rate in the very same sector. The sector is divided in sections and each section has a different bit-rate. The average is the normal/nominal one! Actually CopyLock has a variable bit-rate in the protected sector as well. But the protection doesn't check this. It just checks if the average rate is slower enough than other sector.
I don't know the exact range for data bit-rate variation, but it is quite small for reading without errors. Something in the order of 4-5% offs the nominal rate. It’s not symmetric. It seems the FDC is more tolerant for slower than faster bitrates. You can extend the range (probably to around 10%) if you don't mind getting some errors (could be useful for a protection). The exact numbers will be different for a different FDC (it mostly depends on the PLL).

From memory, I think the 'slow' sector had to take at least 15% longer to read than the other sectors or it failed the protection test...

Seems that his memory is not too good. It is just ~3% slower, not 15%.

First one short comment on the ~3%: I do not know the value used by RNC but the 15% number that Rob gives is the same value that can be found in CopyLockDecoder's Guide. Are they both wrong or did the CopyLockDecoder guys took this number from Rob? (I’ll try to find a diskette protected by RNC to make actual measurements).
FYI about the capture range of the PLL: per IBM specification the PLL has to be able to cope with 4% from central frequency. But in practice it usually handles a 100% drift in FM and over 10% drift in MFM with a reliable correct data read. Therefore 15% is marginally over the but still close enough to return the right data in most cases…

Now to the bit-rate variations.
From the above quotes as well as from other searches I think that the bit rate variation protection technique can be sub-classified into three classes:
1. Slow/Fast sectors (e.g. RNC): Basically for a portion of a sector (usually beginning is done at standard rate) you either shorten or lengthen the bitcell length within the capture range of the FDC's PLL and therefore you read all the data in the sector correctly. Therefore the only way to detect this protection is by measuring the time it takes to read this sector compared its neighbourhoods.
2. Variable bit-rate sectors: Here the bitcell density differs in different sub section of the same sector and potentially end up with a “normal length" sector. It is therefore difficult to detect as the overall sector reading time is the same as other sectors.
3. Random bit rate variation. As I already mentioned I have seen this protection in Dr T software. The variation looks random, largely violates the MFM encoding rules, and may go beyond PLL capture range. However, here also, it seems that the “randomness" does not start from the beginning of the sector but after so many bytes.

Questions for (1): How do you measure the time to read a sector? As you mentioned the DMA is sitting between the CPU and FDC and it certainly does not help. Are you using the timers? If the difference is really 3% the measure has to be relatively precise. Actually a code snippet or some pseudo code would be wonderful!

Questions for (2): Do you know if the bit-rate variations are kept within PLL capture range (that would allow correct reading of the sector) or can they go beyond? If measuring timing for a sector is not easy how can you measure timing for section of a sector??? Actually this is a pretty smart protection as, if the bit-rate variation is kept within PLL data capture range, software would read the sector correctly and sector time would look normal! Can you give name of games that use this protection?

Questions for (3): First question is how do we detect such sector? Actually do we try to do anything with this sector or is it there just to make any software copy attempt fail? The problem with this king of random variation is that it is almost impossible to predict how the PLL will react. As a matter of fact PLLs are design to cope with data jitter and so not to be disturbed by Instantaneous Speed Variation. This means that ISVs are normally "filtered" by an integrator in the PLL and that only variations within a certain range are considered and clock is adjusted progressively (does my explanation make sense?).
I have tried all sorts of tricks in my "software PLL" to "mimic" the behavior of the real PLL on this kind of sector but did not succeed in decoding bytes the same way! But I think it does not really matter as the data returned by the FDC are highly random and for that matter fall into the "weak sector" category as we have already discussed…
To summarize I think that in the case of wide random bit-rate variations, we can verify that we can read predictably the first few bytes of the sector then that we have random values returned for the remainder of sector (like weak sector) and sector CRC error. Also having this kind of bit-rate variation cannot be copied by software copiers.
Any title that use this protection ?

And now for the last protection mechanism I want to cover: the data "under index hole" (which is not a hole with 3-1/2 diskette).

Here is my understanding (also from DC documentation): normally all sectors should end up before the index hole. But we can format a track with a total data length slightly more that the track can hold (Actually I believe this is not possible with the WD1772 FDC as it will stop writing when it detects the index and therefore it requires special HW).
There is small area at the beginning of track (post index GAP), which is not used by sector data (it is used by index address mark in IBM format). This area can be overwritten by last sector on track (sort of wrap around), but if the overlap is too much important, sector 1 will be erased.
It should be possible to detect this kind of sector by the fact that the reading of the sector is interrupted but I do not know what king of error is returned (not documented in FDC). May be we can also find that number of transfers done by DMA is not what expected?
Here again if you have knowledge on how to detect this kind of sector it would be nice to have some pseudo code …
Do you know of any games that uses this protection???

Voila for this subject … Thanks again for your patience and time … Hope this bring interesting information for the all community...
Jean

[ijor]

First one short comment on the ~3%: I do not know the value used by RNC but the 15% number that Rob gives is the same value that can be found in CopyLockDecoder's Guide. Are they both wrong or did the CopyLockDecoder guys took this number from Rob?

Yes, they are both wrong (at least as far as the ST, it might be right for other platforms).

Questions for (1): How do you measure the time to read a sector?

There are several ways to do it. You can easily check how protections do it by using Steem's debugger.

Actually this is a pretty smart protection as, if the bit-rate variation is kept within PLL data capture range, software would read the sector correctly and sector time would look normal!

There are some protections with variable bit-rate where the average is normal. But I never seen anyone that needs the data to be 100% correct all over the sector. That wouldn't be reliable.

Questions for (3): First question is how do we detect such sector? Actually do we try to do anything with this sector or is it there just to make any software copy attempt fail?

Sorry, I don't understand this question.

Also having this kind of bit-rate variation cannot be copied by software copiers. Any title that use this protection ?

No kind of bit-rate variation can be reproduced by software copiers.

And now for the last protection mechanism I want to cover: the data "under index hole" (which is not a hole with 3-1/2 diskette)...
but I do not know what king of error is returned (not documented in FDC) ... Do you know of any games that uses this protection???

It doesn't give any error (why it should?). It is usually detected by the protection software by using the read track command. Several earlier US releases use this protection (F-15 I, Silent Service).

[DrCoolZic]

Questions for (1): How do you measure the time to read a sector?

There are several ways to do it. You can easily check how protections do it by using Steem's debugger.

Any hint ? Using timers? using FDC/DMA interupt? polling the FDC?

There are some protections with variable bit-rate where the average is normal. But I never seen anyone that needs the data to be 100% correct all over the sector. That wouldn't be reliable.

So How do you check the fact that the bit-rate is "modulated" on a sector? By measuring timing? But overall track as you mentioned can have normal timing? By measuring timing on segment of sectors (should be even more difficult!)?

Questions for (3): First question is how do we detect such sector? Actually do we try to do anything with this sector or is it there just to make any software copy attempt fail?

Sorry, I don't understand this question.

For this kind of random bit-variation I think that it is only possible to find out that we read the beginnig of the sector correctly and that we get a CRC error at the end. And of course, as you already mentioned, to rely on the fact that this kind of track cant be copied by software.

And now for the last protection mechanism I want to cover: the data "under index hole" (which is not a hole with 3-1/2 diskette)...
but I do not know what king of error is returned (not documented in FDC) ... Do you know of any games that uses this protection???

It doesn't give any error (why it should?). It is usually detected by the protection software by using the read track command. Several earlier US releases use this protection (F-15 I, Silent Service).

The WD1772 doc indicates: The write track command starts with the leading edge of the index pulse and continues until the next index pulse and generates an interupt. If I interpret this correclty it is not possible with the FDC to format a diskette with a sector that span under the index hole. I have extrapolated and assumed, apparently wrongly, that reading would also be terminated by index pulse!
So you are saying that it should be possible to read normally such a sector with a read command, and to verify that this sector wrap around by performing a read track command.

Thanks again
Jean

[DrCoolZic]

Here is my first attempt to document the copy protection mechanisms used with the Atari ST.

This document is based on lots of information (mostly from the web) that I have processed and of course on the information provided by Ijor in this thread and others (see the reference section of the document).

This is still a preliminary revision and there are probably a lot of mistakes and missing information.

Any feedbacks / suggestions / comments … are welcome.

[DrCoolZic]

V0.2
Found another protection mechanism:
Sector with Wrong Track Number in ID field

Added to the document as #21

[ppera]

V0.2
Found another protection mechanism:
Sector with Wrong Track Number in ID field

Added to the document as #21

Hi,

Game Virus uses uses such protection too. I succeed to copy it with PC and run on real Atari ST.

http://www.ppest.org/atari/floimgd.php

Some things what can not be done with ordinary Atari can be done with PC...

I can give you more details about...
I need some cooperation in making program floimg better. Ideas about disk image formats, what records/fields in them, how to present weak sectors and similar.
Just don't tell me that why to do it or similar. I don't want to make replacement for Pasti on PC, what is not possible btw. I want to achieve max possible on PC.
Pasti and floimg can complement each other nice - some things aren't possible with Pasti, but possible with floimg and vice-versa.

PS. download not works.

[Zorro 2]

V0.2
Found another protection mechanism:
Sector with Wrong Track Number in ID field

Added to the document as #21

Very excellent job mister DrCoolZic !

[ppera]

I don't know has it been discussed, but it is also very preservation related:

Looking to my small collection, I have more games without floppy copy protection, but with 'manual protection'. Preserving them should mean - image with some imaging program (most will go with most of programs )
and scan manual.
Is someone worked on it? Having manual with game is nice thing.
Main problem here is legal stuff, and I think that it will not work well until someone not start legalization, similar as at WOS (World Of Spectrum) .

[Mug UK]

Talk to Goldrunner re: preservation of manuals etc.

In theory, almost any game protected with a manual - that is, that can be installed on a h/drive, should be image-able with any program. Bog standard MSA file should be easily creatable.

However, Gremlin Graphics especially tended to do both. On-disk protection as well as manual protection.

[ijor]

Looking to my small collection, I have more games without floppy copy protection, but with 'manual protection'. Preserving them should mean - image with some imaging program
Is someone worked on it?

Preserving software involves much more than just imaging. Please see:

http://www.atari-forum.com/viewtopic.php?t=7430

[ppera]

Understood. My point was on digitazing of manual, and legal stuff.

I'm 100% sure that my 'manual protected' floppies have not copy protection.
By many stays something like this in manual: ' Don't use original disk. Make copy immediately'.
And I have 3 peaces of Sublogic floppies. No any protection. Real Plug & play

[ppera]

About copy protection document:

Very good source.

I can only add something for section standard floppies:

There is in fact 2 level of 'standard floppy' for Atari ST serie.
First is 9 sector/track, and it is what ROM format routine produces.

Second level is very flexible. It can be described for instance: any format, where Atari can read and write files with TOS, without extra program.
With other words, it is format where parameters are stored in boot sector of floppy on appropriate fields.
In praxis that means that floppies with 1 to 22 sector/track are usable. It is checked, not theory.

One thing more: developers at Atari or DR made strange solution by floppy format - they choosed FAT size of 5 sectors, instead 3, what would ensure PC compability by 720K floppies. I have no idea what may be reason.

[ijor]

I'm 100% sure that my 'manual protected' floppies have not copy protection.

Very likely. Most programs have either on-disk copy protection, or the other one (manual, code-wheel, dongle, etc). Only a small minority have both of them (Dragon Flight, some Us releases, some Gremlin and Firebird ones spring to mind).

And there is a fair number without any protection whatsoever (as most Atari US releases).

[ijor]

There is in fact 2 level of 'standard floppy' for Atari ST serie. First is 9 sector/track, and it is what ROM format routine produces.
...
Second level is very flexible. ... that means that floppies with 1 to 22 sector/track are usable. It is checked, not theory.

There is virtually nothing in TOS that constrains the number of sectors per track. Specifically, the Xbios call for formatting tracks is not hardwired at all for 9 sectors. Only the Desktop utility that formats the whole disk, and the Xbios call that creates a boot sector prototype have that limitation.

That's why you can use HD disks under any TOS.

One thing more: developers at Atari or DR made strange solution by floppy format - they choosed FAT size of 5 sectors, instead 3

That's indeed a curiosity. And it was probably just an oversight.

[ppera]

Of course that format is limited by in ROM built desktop format routine.
It is not usual to put format in ROM, and I personally could find some more useful things for there (or more frequently needed) as timeset for instance.

[ijor]

Btw, Pera, regarding the different FAT size, you might now know better than anybody else. Which kind of incompatiblity it actually represents?

[ppera]

Btw, Pera, regarding the different FAT size, you might now know better than anybody else. Which kind of incompatiblity it actually represents?

Quting myself: '...3, what would ensure PC compability by 720K floppies.'

But it is well known. I hope that you thought on something else...