Floppy Disk Copy protection

[DrCoolZic]

Weak bits in 4.11.3 are presented for reference as they are described in patent "US Patent 4,849,836 *** Copy Protection for Computer Disks - Used in Dungeon Master, Chaos Strike Back ...." *** http://info-coach.fr/atari/documents/ge ... 849836.pdf found in page http://info-coach.fr/atari/software/pro ... references
As described in the pattent these are real "Weak bits" (as they produce weak signal) that results in fuzzy bits.

I also have doubts about this patent being for Dungeon Master. What they do seemed to be different when I read it, but maybe I misunderstood, can't explain the differences from memory, just that I removed the reference in Steem doc.

I am almost 100% sure this patent describes DM CSB This patent was granted to Software Heaven Inc the maker of the games DM CSB and fig 1 & 2 describes exactly what is used for protection (fuzzy bytes) in these games.
Refer to detail explanation from Gothmog on Dungeon Encyclopedia Web site http://dmweb.free.fr/ specially http://dmweb.free.fr/?q=node/210

I do not know why they describe weak bits with these strange techniques presented in fig 3,4,5?
As far as I know these techniques have probably never been used because it would require very specific mastering machine with very very special floppy drive.
But all of them would result in weak bits in a different way that what we are used to. These techniques would result in Weak ANALOG signals (due for example to head misalignment) and the conversion from analog to digital would result in different values returned because the analog signal would be close to the detection threshold.

[DrCoolZic]

"Fuzzy bits" is not exactly the same thing as REAL weakbits. This is what I have been trying to convey for awhile. That Dungeon Master protection relies on the fact that valid bitcell durations are written that cross the boundary of what the FDC accepts as valid data for each of the possible (3) different timing windows. Because the disk is rotating, the bitcells are deliberately not the same duration between groups, and the FDC's clock is much faster than the rotational speed of the disk, each time data is read it will change - but the value of ALL of the bitcells times fall within the 4us-8us window. REAL weakbits, where the bitcell time is random every single time you read it, will have values that dip below and above the data separator's clocking window - which is a single window that encompasses much more time. When the data separator's clocking window is exceeded, whatever time is in the clocking timer is output. This is conjunction with the data separator turning up the AGC to try to fix the issue adds to the randomness, but since we have N/S poles it is balanced.

REAL weakbits can in fact be determined using a single revolution. This is accomplished by looking at consecutive bitcells. There is an interesting byproduct of the how the data separator works on floppy drives - if you add up all of the bitcell times of the weakbits they will equal the same amount of time that you would have for normal data. So, you can scan through and see which bits are outside of the FDC's normal 10% clocking window and then draw a comparison to the surrounding bitcells and mark anything outside of the range as a weakbit, and typically (especially with C64 protections) you can tie together any bitcells that appear to be valid (within the total amount of space allowed for bitcells - ie 4us-8us) that are within a series of invalid bitcells. Remember, we are dealing with N/S poles with flux too so weakbits typically come as pairs.

Because the bitcells times all add up to be the same length as normal data, the track length doesn't fluctuate much (if at all), and that is why we can write back a weakbit protection using just a single revolution of capture. If an emulation simulates the rotation of the disk, the length of the flux data likely will not come out perfectly so that track repeats the exact same data. WinUAE's data separator emulation just clocks in the data at the correct sampling rate, using the raw flux data and since the flux data was captured on a drive and has the same drive speed "wobble" that a real drive would, that jitter is automatically introduced. WinUAE can load anything with weakbit protections without having to do anything special, just clocking in the data, and only using an image that contains a single revolution.

As I already mentioned many times I prefer to use the term fuzzy bits in analogy with fuzzy logic https://en.wikipedia.org/wiki/Fuzzy_logic (results somewhat unpredictable). Because Weak bit terminology somehow implies weak signal!

On Atari there are basically two/three ways to get weak bits: “border bits”, “out of band bits”, and “unformatted area”.
The “border bits” were almost exclusively used by Software Heaven Inc. because they were protected by patent 4,849,836. But some “variations” of this technique was used on program like the one from Dr.T. In this example you can see that the beginning of the sector uses a kind a fishbone distribution of the flux that results in fuzzy bits, followed by a more conventional unformatted area.

fuzzy sector.png

“Out of band” and “unformatted” are similar in the sense that they result in flux transitions in most cases not in standard range. On Atari most of this kind of fuzzy sectors is created by placing an unformatted area inside a sector but in some rare cases they are created by deliberately write few transitions out of range. The reason why unformatted area return fuzzy bits is well known: no transitions for a period of time pushes the ACG in the analog channel to its maximum and therefore any noise picked by the head result in a transition. As this noise depends on unpredictable environment variables each time you read an unformatted area you get different results and therefore by definition you have fuzzy bits/bytes.

unformatted-sector.png

I have also noticed that in most cases fuzzy areas do not change much the length of a fuzzy sector. This is actually not surprising as “border bits” (either “wobulated” or fishbone) by definition have same amount of transitions above and below normal. And for unformatted areas the transitions are random and therefore there are almost as many transitions above as they are below (but usually the average is slightly above which make sense). You can for example look at the histogram of an unformatted track to get a feeling of the flux distribution

unformatted track.png

[Steven Seagal]

I do not know why they describe weak bits with these strange techniques presented in fig 3,4,5?
As far as I know these techniques have probably never been used because it would require very specific mastering machine with very very special floppy drive.
But all of them would result in weak bits in a different way that what we are used to. These techniques would result in Weak ANALOG signals (due for example to head misalignment) and the conversion from analog to digital would result in different values returned because the analog signal would be close to the detection threshold.

This is certainly why I was confused, this had nothing to do with the inspection window.

[Steven Seagal]

WinUAE's data separator emulation just clocks in the data at the correct sampling rate, using the raw flux data and since the flux data was captured on a drive and has the same drive speed "wobble" that a real drive would, that jitter is automatically introduced. WinUAE can load anything with weakbit protections without having to do anything special, just clocking in the data, and only using an image that contains a single revolution.

Does it also work with Dungeon Master?
I can't find in the source how that drive "wobble" is done.

[JimDrew]

As I already mentioned many times I prefer to use the term fuzzy bits in analogy with fuzzy logic https://en.wikipedia.org/wiki/Fuzzy_logic (results somewhat unpredictable). Because Weak bit terminology somehow implies weak signal!

THAT is exactly what a true weakbit is, and that is why we call them that! We have been calling them that since the late 70's.

[JimDrew]

WinUAE's data separator emulation just clocks in the data at the correct sampling rate, using the raw flux data and since the flux data was captured on a drive and has the same drive speed "wobble" that a real drive would, that jitter is automatically introduced. WinUAE can load anything with weakbit protections without having to do anything special, just clocking in the data, and only using an image that contains a single revolution.

Does it also work with Dungeon Master?
I can't find in the source how that drive "wobble" is done.

Yes, Dungeon Master works - and nothing special needs to be done to make it work. No patches, no hacks, no wobble or added jitter - nothing else is required. The flux data is spooled in. Strong bits (what you call NFA) also works the same.

[Steven Seagal]

In Steem I must add some randomness in flux times, which is realistic, I wonder how it can work without.

[JimDrew]

Because the rotation of the track (a circular buffer) is random each time due to the fact that they are an uneven number of bitcells-->MFM, and by virtue that the image data itself already has drive speed variations that are part of the capture. When I was first doing SCP, I thought I needed to read 5 or so revolutions and then average every single bitcell (overlaying all 5 revolutions). Then I realized that this won't work for emulations because the resulting data won't be like a real drive.

[ijor]

I just want to add my two cents about the terminology history. I don't think there was an official, single, formal name for weak or fuzzy bits back at the day. Weak bits was, perhaps, the most common term, and that's the one I became used. But several other terms were used by authors, experts, publishers, duplicators, etc. Some I remember are: fuzzy, phantom, wondering, changing, etc. I don't remember hearing flakey bits back then.

The first weak bits protection I've seen (in the Atari 8-bit) were using an unformatted area. That was circa 84 IIRC.

Hi Jean. Keep up the good work!

[DrCoolZic]

Hi Ijor glad to hear from you

[DrCoolZic]

Dungeon Master fuzzy bits are particularly difficult for the DPLL
Here is an extremely zoomed picture of DM fuzzy transitions

dm-anot.jpg

As you can see we can have two "border transitions" one after the other.
Many DPLL algorithm do not use transition's drift history for phase correction and tuning the parameters can be difficult ... I am still working on this in Aufit

[JimDrew]

I just want to add my two cents about the terminology history. I don't think there was an official, single, formal name for weak or fuzzy bits back at the day.

"Fuzzy bits" (ala Dungeon Master) and "weak bits" are NOT the same thing. The protection used in Dungeon Master does not contain any weak bits (except perhaps the write splice as a bi-product of turning off the write). Instead, it contains bits that fit within the normal 4us-8us bitcell domain, but traverse into regions that are outside of the DPLL's window of the WD1772 (and other similar) FDC chips. You can see from the zoomed in example that there are data bits in the 5us range, which is clearly outside of the typical 4us/6us/8us clocking window. If this were true weak bits, you would find bitcells well below 4us and well above 8us. The Dungeon Master protection just uses the 4us-8us range.

[ijor]

In Steem I must add some randomness in flux times, which is realistic, I wonder how it can work without.

Some randomization is required. And it is not only for the purpose of emulating weak bits. Furthermore, it is required not only for floppy disk copy protection. That was a surprise I found out rather early.

Some titles with manual (documentation, chart, etc) protection would always ask the same question without some introduced randomization. Most titles wait for some human interaction before the protection, press any key, or click. The human timing provides randomization. Otherwise (at least under emulation, see below), everything is constant, the randomizer is not seeded, and the protection question is always the same. In other platforms the real time clock is used as a seed, but we normally don't have one on the ST. So the only natural randomization is the human, or the floppy.

I never tested what would happen on real hardware if you remove the floppy randomization (say, using a solid state floppy emulator). The ST, due to having multiple independent clock crystals (master clock, MFP timers clock, etc), it's not fully deterministic. Just curious.

[ijor]

I just want to add my two cents about the terminology history. I don't think there was an official, single, formal name for weak or fuzzy bits back at the day.

"Fuzzy bits" (ala Dungeon Master) and "weak bits" are NOT the same thing...

I think you misunderstood me (probably my fault). Trust me, I know exactly the difference between the protection in Dungeon Master and the one (you call) weak bits. What I said, or wanted to say, is that back at the day, some people used other terms for both what you call weak bits, and what you call fuzzy bits. And by some people I include some experts on the field, not just average users. I actually recall even Doug Bell (Software Heaven) himself using the term weak bits for the DM protection.

I think it is expected. After all, and certainly on the early days, there was no place where formal and standard terminology could develop on the subject. You couldn't study copy protection at the university, and it was almost unheard that experts would publish papers, except for the very few patents, they were very secretive.

I am not arguing about the technical difference, I am not even arguing about what terms should be more appropriate. Just describing how they were used in the industry, as I heard then back at the day.

[JimDrew]

I used the term weakbits in my era, back in the late 70's. I also coined the term strongbits in the late 80's with the Amiga. Strongbits is what DrCoolZic calls a NFA (no flux area). The reasoning for using these terms was quite simple. To create a weakbit, you write a 000000 pattern. To create a strongbit, you write a 111111 pattern. 0 is weak, 1 is strong. When the Amiga reads weakbits, the data is random (to be expected). When the Amiga reads strongbits, it reads them as all 00's. To write them, you need to write all 11's. So, strongbit protection was rather easy to duplicate, just you have to parse the track buffer looking for 00's and replace them with 11's.

[ijor]

I used the term weakbits in my era, back in the late 70's

Weak bits at the late 70's ??? On which platform? On the context of creating them by design for the purpose of copy protection? Or just as an unintended effect or defect?

Weak bits protection on the platforms I were familiar, appeared at 1984 (1983 at the earliest). Protection at the late 70's on these platforms was very primitive.

[JimDrew]

In 1979 Seiko mini-main frames had 8" floppy drives and used an interesting copy protection for their inventory control system. The last sector of each track had just a header with unformatted data. The data block was decoded and the checksum always failed, but the checksum had to change due to the weakbits in order for the protection to pass. It was next to impossible to write just the header. However, I did make working copies because these were hard sectored disks, so you could figure out where the last sector was and just scratch the disk or poke a hole in it in the data block area. It's a primitive weakbit protection, but it worked reliably. In the 80's weakbit protection evolved to the point of putting valid data around weakbit data and extracting both to check the protection routine.

[DrCoolZic]

As I already explained I have used the term “fuzzy bits” because I was looking for a terminology that would indicate the “randomness of bits read” without inferring the method used to create this randomness. The term Fuzzy is used here to indicate that there is a certain probability for a bit to belong to the 0 state or the 1 state (see https://en.wikipedia.org/wiki/Fuzzy_set) ensemble when read.

If you look at the Dungeon master patent (see http://info-coach.fr/atari/documents/ge ... 849836.pdf) we have the following description (Col2-Line60): “Thus, if a pattern of weak or otherwise ambiguous bit is written on an original disc, successive reads of that pattern will produce inconsistencies which can be detected by comparing successive reads with one another”. The terminology used here refer to method used to create fuzzy bits: either weak transition bits (Fig 3, Fig 4) or ambiguous bits Fig 2.
As explained in the patent creating “weak transition bit” requires “sophisticated track follower circuits …” (Col3-Line63) and therefore to my knowledge this has never been used, at least for the Atari.

The two known way of creating fuzzy bits for Atari FD are the “ambiguous bits” (as in DM) and the “out of range bits” (usually unformatted areas). But of course having ambiguous bits or Out of range bits do not imply fuzzy bits. You can have one or several bits in the “ambiguous region” or out of range and no fuzzy bits detected. This is due to the capability of the WD1772 to recover many errors even on bits that violate the “MFM rules”.

Therefore, by definition, what I call fuzzy bits is a bit that reads with some randomness in successive reads without implication on the underlying technique.

For protection purpose it is not relevant to know the underlying mechanism used to create the Fuzzy bits. However DM uses a carefully crafted “sliding pattern” in ambiguous area that results in fuzzy bits but the assembled byte is always either $68 or $E8 (randomness in the MSB) and this is tested by the protection (see http://dmweb.free.fr/?q=node/210)

No Flux Area is a totally different story that has been discussed at length in other threads. Again I choose that term to indicate what the WD1772 reads and not the underlying mechanism.

[JimDrew]

Funny thing about that patent is that there are no actual weakbits contained in the Dungeon Master protection. The randomness only comes from the WD1772 trying to decode the data outside of the clocking window. At a flux level, writing the data that was originally read re-produces the track exactly.

[DrCoolZic]

Correct. DM only contains fuzzy bits and of course copy at flux level works (as for all fuzzy protections).

[JimDrew]

I still think that term is funny. It's really nothing more than bitcell times between the normal 4us/6us/8us range.

[ijor]

Funny thing about that patent is that there are no actual weakbits contained in the Dungeon Master protection. The randomness only comes from the WD1772 trying to decode the data outside of the clocking window. At a flux level, writing the data that was originally read re-produces the track exactly.

Writing back the same flux transition pattern as read from the drive verbatim, (almost) always produces a working copy of weakbits, fuzzy bits, or whatever you want to call them. It works for any of those protections, not just for DM, at least on the ST.

That is, writing back those random pulses, will produce random weak bits. You may argue that this is not the best way, and that you are not reproducing the original magnetic state of the disk, and you would probably be right. I used to be able to distinguish mentally, just by looking at the flux level pattern, which method was used to produce the random pulses. Was by writing back some pseudo random pulses, by writing back no pulses (yes, I agree it is better), or if the area was actually unformatted and degaussed (as was with most original disks). But the software checking the protection doesn't see the flux level ...

Now, for the DM protection, the one you call fuzzy bits, I would say better not write back exactly the same pattern you read. Perform some equalization and normalization before. Otherwise, you are copying like an analog copier. It would work in most (but not all) cases for the first generation copy. Subsequent generations would degrade the signal and eventually fail.

[AtariZoll]

...
Now, for the DM protection, the one you call fuzzy bits, I would say better not write back exactly the same pattern you read. Perform some equalization and normalization before. Otherwise, you are copying like an analog copier. It would work in most (but not all) cases for the first generation copy. Subsequent generations would degrade the signal and eventually fail.

Well, I'm not some floppy expert, and even not much interested in it, except some for me practical things, but quoted txt. pulled my attention.
Comparing with analog seems here as bad idea. People who will write SCP images on their floppies will certainly use image of original (thanx to Internet) and not image made from copy.
And I think that we can agree that we still can not make 100% accurate copy, just such what will be good enough for pass copy protection.

[JimDrew]

Copies should be identical the original, +/-1 bitcell. The same is true of TRACE and other duplication machines.

[ijor]

Copies should be identical the original, +/-1 bitcell. The same is true of TRACE and other duplication machines.

I don't think duplicators performed drive speed matching (I assume that's what you mean). At least not normally. Multiple original copies of the same master don't have the same track length. You can also infer they didn't really care from the significant difference found often in the very same disk (as a consequence of drive speed warm up).

So in this regard, we can make copies even more accurate than original ones.