List of difficult to copy disks

[DrCoolZic]

i just found the problem
it is now fixed in new aufit will post more information later as it is interresting

[DrCoolZic]

Here are more details.
Gunship and P47 both use what I call intra-sector bit width variation.

The Macrodos/Speedlock is the well known example of this protection: You have the first 128 bytes with a normal width, the next 128 bytes have a bit width slightly above normal, the next 128 bytes have a bit width slightly below normal, and the the last 128 bytes with a normal width. I thought that this was the only pattern possible for intra bit cell variation. But apparently this is not the case.
Note that this also explain why the STX format has been changed to allow storing timing information (originally only Macrodos was supported with internal tables).

Aufit has been tuned to detect Macrodos pattern but other intra bit-width variation. This is now fixed in latest prototype.
here are the Gunship and P47 stx generated by new Aufit prototype (note that now Aufit uses the "compress" mechanism of the STX format - sector not saved if already in track - generating files of smaller size)

They seems to work but please let me know if you find problem.

Gunship-P47.rar

http://info-coach.fr/atari/software/Gam ... sea_compil will be soon updated with more technical details

[kodak80]

Both of these STX work fine for me. AUFIT is a tool I use regularly and I am looking forward to the new version. Keep up the good work.

I still cannot create STX for the other titles in this compilation:

Silent Service - boots to splash screen and then gives HARDWARE_FAILURE message.
F-15 Strike Eagle - does not work when F15.PRG is run
Carrier Command - converted STX works fine

I am using version 0.4c of AUFIT.

[DrCoolZic]

Strange
Here are the SilentService and F15 with new version. Seems to work for me?

SilentService.rar

[JimDrew]

I have seen a few programs for the Amiga with Rob's Copylock that uses 5 or 6 different sets of bitcell lengths on the same track, not just 2 different (higher/lower) bitcell lengths.

Where is the latest Aufit download located?

[DrCoolZic]

Not yet available about 50% of code has changed
So not yet ready for release. Need more tests

[DrCoolZic]

For Jim example of display with new Aufit

layout.png

On the right you can see that sectors of first tracks have a small fuzzy segment followed by a NFA area
Near center a sector with fuzzy bits

Time of lore show shifted track in their glory

layout1.png

Notice the sampling was not good so we get several fuzzy sectors.

Backside of time of lore is very strange with data and NFA layout in a nice way !

layout2.png

[JimDrew]

Nice... I wanted to make my disk display also with a black line between each track to make it easier to see. That also cuts down on the moire.

I did figure out that VB's circle function uses floating point for the radian start/end of the arcs, which is pretty quick as it is handled by the CPU's internal FPU.

So, I will experiment a bit with this. At least with VB's circle function, you just need to set the radius as the offset from the center for each track. So, that fixed point along with 360 degrees converted to a radian gives you how many possible data points there are for the circumference of the circle. Knowing the number of data points then gives you the ability to scale the bitcell times to a data point.

[Jeff_HxC2001]

Here are more details.
Gunship and P47 both use what I call intra-sector bit width variation.

The Macrodos/Speedlock is the well known example of this protection: You have the first 128 bytes with a normal width, the next 128 bytes have a bit width slightly above normal, the next 128 bytes have a bit width slightly below normal, and the the last 128 bytes with a normal width. I thought that this was the only pattern possible for intra bit cell variation. But apparently this is not the case.
Note that this also explain why the STX format has been changed to allow storing timing information (originally only Macrodos was supported with internal tables).

Aufit has been tuned to detect Macrodos pattern but other intra bit-width variation. This is now fixed in latest prototype.
here are the Gunship and P47 stx generated by new Aufit prototype (note that now Aufit uses the "compress" mechanism of the STX format - sector not saved if already in track - generating files of smaller size)

They seems to work but please let me know if you find problem.

Gunship-P47.rar

http://info-coach.fr/atari/software/Gam ... sea_compil will be soon updated with more technical details

Interesting magnetic ghost effect on the Gunship P47
Look at the unformatted side 1 :

Image2.png

BTW about the speed variation: are you talking about the first sector on the track 1 ? I found the same speed variation on the following tracks (the tracks without sector)

[DrCoolZic]

BTW about the speed variation: are you talking about the first sector on the track 1 ? I found the same speed variation on the following tracks (the tracks without sector)

Seems like we are not using same version? On mine track 39-79 are normal Atari track and does not seems to be the case in your image?

P47-Disk.png

Now about speed variation Track1 is an Atari track and the speed variation is used as protection because if timing not written in STX file program do not run.

P47-Track1.png

For other track it is strange. Indeed some but not all of the tracks have speed variation for example Track 2

P47-Track2.png

But some other "data tracks" do not have variation for example track 39 is totally flat and some are marginally flat?

Jeff can you have a look at viewtopic.php?f=95&t=21669#p263573
Hiipy Dave talk about something that can be interesting ???

[Brume]

Well, now I have a big issue with Roadwar 2000 (original disk).

- I firstly copied it with SCP and also verified the disk with Aufit. Track #02 seems to be damaged or has fuzzy bits... Strange, I didn't think this game was really protected, especially on track #02, which isn't usual.
So I tried to clean up the disk (as usual), and tried to copy it again with SCP: same result. Track #02 is faulty. The .STX file generated with Aufit doesn't work either. Aufit says the disk has fuzzy bits on track 02 / sector 06.

- OK, let's try to read the disk with Kryoflux. Once again, it detected a faulty track #02 when reading. The .CTR file doesn't work at all with Steem, but.... The .ST file (done with Kryoflux) works fine!!!
Then I tried to convert the .RAW files with Aufit into .STX format: it doesn't work.
I also used HxCFloppyEmulator in order to convert the .RAW files into something I can read with Steem: the .STX file doesn't work, but... the .MSA runs fine!!! Hey, what happens?

- Actually the original game runs perfectly on the real machine. And Fastcopy doesn't detected any damages on Track #02...

- Also please note I have a second copy of that game (still original): it doesn't have the faulty track #02. I compared both main .prg (between the faulty disk and the healthy one): they have one byte of difference only.

Now I'm really lost. What's wrong with the 'damaged' disk? Why the track #02 can be read with FastCopy, but not with SCP nor KF?

The disk can be downloaded here:
https://mega.co.nz/#F!uFlyAR6a!_r1kGag0Yone9D4cUWLHqw

Thanks for reading... and sorry for interrupting the discussion about Gunship/P47 and other games

[DrCoolZic]

Hello Brume,

Here is what I think. The disk has some problem on sector 6. Some bits (hope you can see them) are located in "ambiguous area". Therefore according to Jim magic they are "fuzzy bits". But in fact they are just some flux that have migrated for some reason. This is a perfect example of why you need to sample several revolutions ... but stop on this subject ...

roadwar-sector-zoom.png

This game does not seems to have protection but is at the limit of what can be read. Now why does it works on a real Atari? As you may know when the TOS read one sector with CRC error it rereads the sector up to 5 times doing a technique that I call shoe shine to clean the head after try Number 3 (shoe shine => move head to track 0 and back to wanted track so that dust will be moved away from head).
So on a real ST the sector will be read several times and as the transitions are marginals it will succeed after some retry.

On the samples you provided:
- SCP always gives bad CRC on all revolutions and no Fuzzy bits are detected. This means that the sector is read incorrectly but with the same content on 5 revolutions. So here there is nothing you can do here.
- KF on the other hand find some fuzzy bits. Play with the revolution buttons of Aufit. You will see that sector 1 & 3 has fuzzy bits AND CRC error while sectors 2, 4, and 5 have only fuzzy bits. As this is almost impossible (real fuzzy bits always imply CRC error) this means that this sector has been read 3 times correctly and 2 times incorrectly. So here you can use a feature of Aufit probably not documented (non even sure it is on .4c?). Just select for example revision 2 (or any revision WITOUT CRC error) and click Write Disk. The STX file will use data from this revolution ... and you get a working STX - Try the attached STX -

roadwar.rar

I hope in future to be able to detect automatically this kind of situation and to be able to automatically select "good data" when available but for now you have to do it manually

[JimDrew]

My guess is that since .st and .msa are just sector based they always contain valid (no errors). A conversion to these formats removes any errors that exist. The data that is "corrected" could be just some graphics data (no code) and it doesn't matter that it is not the original data contents. The track definitely has problems, as reported by everything, including my flux display (see below). You will see the pixels scatter above and below the normal the 3 bitcell zone (4us/6us/8us). You can have this in the track gap (which you can see at the gray vertical line), but you can not have this in the middle of a sector. This track does have weakbits around the middle of the track and copy will produce the same weakbits (yes, even with just a single revolution dump), which I guess would be somewhere around sector 5 or 6, so it makes sense what Aufit reports. You will also notice that this disk suffers from sticking as it rotates (or the drive is really bad). I am guessing it makes quite a bit of noise as it rotates because you can see the flux warble like a sign wave.

Another slight possibility is drive alignment. It could be that the drive you are using for reading is just slightly out of alignment from this disk (or this disk is out of alignment from you drive, depending on how you want to look at it).

Besides cleaning the disk, did you remember to clean the drive head each time before you attempted a dump? A dirty disk can cause the head to become dirty and really unusable by just stepping through the disk once.

[JimDrew]

SCP always gives bad CRC on all revolutions and no Fuzzy bits are detected. This means that the sector is read incorrectly but with the same content on 5 revolutions. So here there is nothing you can do here.

The image he made most certainly has different bitcell times for area in question on each revolution, of course you don't need to look at the other revolutions to know that weakbits exist. You can see that from just the single revolution.

[DrCoolZic]

Ah Ah - you see I guessed correctly that Jim would see fuzzy bits (I am now starting to know his magic)

For me the dive has an above average variation of speed but still within IBM spec. if you look at the output of the DPLL in Aufit (the bit width graph has been magnified) you will see that if fluctuates from 1965 to 2040. The WD1772 according to my tests can cope with up to 10% variation in MFM and 100% variation in FM. That is slow variations not ISV of course

roadwar2000-track2.png

[JimDrew]

I wonder how you are converting bitcell times then. The data in the flux display clearly shows the minimum and maximum bitcell times WAY past 10% - I see ~37% off.

Look at the data between the 5 columns (each column is one revolution). You can clearly see that they are all different in the vertical string of pixels in the middle of each colum... and that is coarse resolution.

[DrCoolZic]

SCP always gives bad CRC on all revolutions and no Fuzzy bits are detected. This means that the sector is read incorrectly but with the same content on 5 revolutions. So here there is nothing you can do here.

The image he made most certainly has different bitcell times for area in question on each revolution, of course you don't need to look at the other revolutions to know that weakbits exist. You can see that from just the single revolution.

This is not fuzzy bits just damage floppy.
The fact that the results are always bad with SCP might just be a lack of chance. Also due to the fact that both devices uses different sample rate might just be the "bad luck" reason
Reading very marginal disk is probably more a question of chance than anything.

By the way this is also why KF as a guided mode where you can specify that if a track/sector is read incorrectly then retries are made like on a real Atari. Of course this also takes much more time to sample.
Brume did you use the guided mode of CTA ?

[DrCoolZic]

I wonder how you are converting bitcell times then. The data in the flux display clearly shows the minimum and maximum bitcell times WAY past 10%!

No 2000 + 10% is 2200 so we well inside the DPLL range.
For example Rob Northen protection uses 5% increase or 2100 µs and the bits are read perfectly

[JimDrew]

You must not be reading my data correctly then! The longest bitcell time is 12.593us (36.47% outside of the 8us norm). The shortest is 3.505us (12.375% outside of the 4us norm). However, the middle zone (typically 6us) has values that go up to 7us and below 5us... or values in the 4us are going past 5us. In any case, those are invalid bitcell times due to being weakbits. Regardless of the cause of the weakbit (physical damage or deliberately caused), they are still weakbits.

The circled areas are well beyond 10%.

[DrCoolZic]

You must not be reading my data correctly then! The longest bitcell time is 12.593us (~37% outside of the 8us norm). The shortest is 3.505us. However, the middle zone (typically 6us) has values that go back 7us and below 5us... or values in the 4us are going past 5us. In any case, those are invalid bitcell times due to being weakbits. Regardless of the cause of the weakbit (physical damage or deliberately wrong), they are still weakbits.

The values of shortest / longest bitcell time has nothing to do with speed variation of the drive. The DPLL as it names indicated performs a PLL function on the flux transitions to adjust freq and position of the "inspection window" ...
Actually amazingly if you look at the results using the SCP files the answer is no. The bits on the 5 revolutions are ALWAYS read the same (exact same values are returned from the read sector). Just unfortunately not the right values. To be honest this is rather unusual but certainly not impossible (this test case is a good example).

I am just playing with two Aufit running simultaneously one with SCP and one with KF. Both read almost the same values in all revolutions
when you go from one rev to the next almost nothing move. So some fluxes are way beyond normal value but stay at same value between revolutions and are therefore decoded the same. With KF sample there is only one fuzzy byte at location 372 in the sector 6.
By looking at timing given in the buffer content window of Aufit it this seems to be located close to 114 ms
The diagram below show the bad transition(s) all other above / below normal transitions are decoded correctly. This is also why it works on real Atari.

roadwar bad-bit.PNG

[JimDrew]

If the flux data is invalid, it is due to a weakbit (or strongbit), and that value will change on consecutive reads - as clearly shown in the image Brume provided.

Another thing - you have a VERY small decoder window. If you made the decoder window wider, perhaps 12%-15% (depending on it looking for a sync or regular data) like the WD1772 really is (not 10% as some have suggested), you might have better success with some data. I have some disks for the Amiga that are flakey (specially on the upper tracks) that needs about 23% of a window to properly decode. I use a 47% window for decoding all format that are supported. I can pull data out of thin air this way.

[Jeff_HxC2001]

Well, now I have a big issue with Roadwar 2000 (original disk).

[...]
- OK, let's try to read the disk with Kryoflux. Once again, it detected a faulty track #02 when reading. The .CTR file doesn't work at all with Steem, but.... The .ST file (done with Kryoflux) works fine!!!
Then I tried to convert the .RAW files with Aufit into .STX format: it doesn't work.
I also used HxCFloppyEmulator in order to convert the .RAW files into something I can read with Steem: the .STX file doesn't work, but... the .MSA runs fine!!! Hey, what happens?

It "works" in ST and MSA format because they don't store the data crc unlike the STX & CTR format... But for sure the ST & MSA data is corrupted at this sector. The CRC is just regenerated by the emulator.

[Jeff_HxC2001]

BTW about the speed variation: are you talking about the first sector on the track 1 ? I found the same speed variation on the following tracks (the tracks without sector)

Seems like we are not using same version? On mine track 39-79 are normal Atari track and does not seems to be the case in your image?

It's the same version. I have just disabled the sector analysis to show the ghost

Here is the full analysis :


Full rez:
http://hxc2001.com/disks_analysis/vrac/ ... 89_scp.png

Now about speed variation Track1 is an Atari track and the speed variation is used as protection because if timing not written in STX file program do not run.
For other track it is strange. Indeed some but not all of the tracks have speed variation for example Track 2

Got the same result here :


Same at track 3,4,5,etc...

But some other "data tracks" do not have variation for example track 39 is totally flat and some are marginally flat?

Yes the speed variation fadeout until the track 25/26...

[JimDrew]

That's very cool, Jeff!

[DrCoolZic]

If the flux data is invalid, it is due to a weakbit (or strongbit), and that value will change on consecutive reads - as clearly shown in the image Brume provided.

Another thing - you have a VERY small decoder window. If you made the decoder window wider, perhaps 12%-15% (depending on it looking for a sync or regular data) like the WD1772 really is (not 10% as some have suggested), you might have better success with some data. I have some disks for the Amiga that are flakey (specially on the upper tracks) that needs about 23% of a window to properly decode. I use a 47% window for decoding all format that are supported. I can pull data out of thin air this way.

It seems that I did not express myself correctly because you totally misinterpreted what I have said.

If I understand what you say you define a “decoder window” where bits in range 4 / 6 / 8µs +/- 47% are decoded as 1, 2, 3?!? Already the % value seems strange as 6000 + 47% = 8820 but whatever value you take this is not the way the WD1772 works. You CANNOT say something like I get value 2 for flux in range 5100 – 6900 again this is definitively not the way it works.

If we were in a perfect word this would be fine, but as we know floppy drives (as the rest of the world) are far from being perfect.

For example in Dungeon Master the data bits are moved slowly from below 5000 to above 5000 fence. With a “decoder window” technique if the drive rotate above or below normal speed most of the bits will decoded incorrectly. This does not means that you won’t be able to reproduce the disk because during this operation you really do not care (unfortunately) how the bits are decoded. But for emulation the bits will be decoded incorrectly and depending on how precisely the protection is checked it may or may not work (luckily it again works in most cases).

So back to what I said and that you have misinterpreted. When I talk about a 10% tolerance this has absolutely nothing to see with the width of the bits. What I said is that the WD1772 has an internal DPLL and the frequency of this DPLL can stay synchronized with the input stream for slow variation of +/- 10% (well above IBM drive specification that indicates that a drive must rotate a 300 RPM +/- 3%).

I am not going to explain again how the WD1772 works as it is clearly explained in my documentation http://info-coach.fr/atari/documents/_m ... ection.pdf page 28. As well as http://info-coach.fr/atari/hardware/FD-Hard.php#FDC_PLL and more specifically http://info-coach.fr/atari/hardware/FD- ... #detail_iw

MFM Example.png

<diagram>

Therefore if you want to decode flux stream correctly (as close to what a WD1772 would do) you must use an algorithm like the one described in the patent 4,780,844 http://info-coach.fr/atari/documents/ge ... 780844.pdf
But if the goal is to decode bits approximately you can use the notion of “decoder windows” as this is probably good enough in most cases. Again it really depends what you are trying to achieve.
Unfortunately in real life it is never possible to look at a flux transition value and say: ha this guy has a value of 2 and it is a fuzzy bits or it belongs to an NFA

In practice inside Aufit, as the DPLL is in software, it is easy to adjust the maximum frequency lock range and the speed to adjust the inspection window. So I experimented different values on damage and normal disk but it turn out that in the Patent the values wisely chosen to get a “fast settling but stable PLL” (believe it or not It seems that these guys knew what they were doing )

If the “rotation speed” of the samples you are trying to decode is already above or beyond 10% you definitively have a problem with your samples and it is usually worthless to use this data.

Hope this clarify a bit the matter?