Brume's dumps

A forum about Atari protected floppy disks analysis, preservation, emulation, tools

Moderators: DrCoolZic, Brume

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Fri May 29, 2015 3:04 pm

I do not want to reply for KF people but good in the case of CTA means good for preservation and preservation means good enough data to guaranty being able to re-create a disk 100% same as original disk and not 100% same as imaged disk.

So not so good data wont produce preservation IPF file (even if would be possible) but data can still be used for emulation by converting for example to stx using Aufit.

Again different goals means different requirements and obviously gives different results.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Fri May 29, 2015 3:20 pm

I have seen recently a TV documentary talking about restoration of paintings at the Louvre Museum.
They do all sort of deep analysis using very sophisticated equipments to find a lot of hidden information. These analysis combined with knowledge of used techniques at the time of the painting allow to restore painting as close as possible to the original.

This is the kind of ideas that I would like to use in Aufit. Some examples are:
- automatic errors correction from multiple revolution (already done)
- automatic flux variation correction to get perfect data: e.g. some sectors have transitions not sampled correctly (i.e. due to motor speed variation) and this results in detection of short/long sectors. This does not hurt for emulation but the result is not conform to original data ...
- better unformatted track/sector detection
- etc.

a lot of work at flux level that would also allow to provided automatically quality information. So that someone could decide to clean FD to try to improve images.

Brume I think you do not need to redo any dump :)
Unless an image is very bad to the point that you cant even emulate you should not have to redo dump. It is always possible to produce better dumps but there is a "compromise" between the time spend and the quality of the dumps ...

JimDrew
Atari Super Hero
Atari Super Hero
Posts: 580
Joined: Mon Nov 04, 2013 5:23 pm

Re: Brume's dumps

Postby JimDrew » Fri May 29, 2015 8:58 pm

The funny thing to me though in this whole "preservation" conversation is that every single disk ever produced, even on a commercial duplication machine, is different. Who has the power to say what is "good" and what is "bad"? You can certainly analyze the flux stream and determine what is typical and what is not, but something with bad flux data could very well have been created that way - either deliberately or by accident. Bounty Bob Strikes Back! for the C64 is a good example. Every single one of the original (v1.0 - unmarked) disks was created in one of many different 1541 disk drives - not a commercial duplicator. v1.1, v1.2, and v1.3 were all produced on various (different) TRACE machines. No person or group of people should have the right to say "sample X" is proper and toss out all other working samples because it is physically impossible for all samples ever made to be compared.
I am the flux ninja

User avatar
dlfrsilver
Atari God
Atari God
Posts: 1412
Joined: Mon Jan 31, 2005 1:41 am
Contact:

Re: Brume's dumps

Postby dlfrsilver » Sat May 30, 2015 12:44 pm

DrCoolZic wrote:Hello sorry for the late reply I am travelling and it would have been tough to reply from the boat to Niagara Falls :)

Several replies explain exactly what's happening. Good or Bad depends on what you are trying to achieve and how you process the data.

If you take a tool like CTA, the goal is to produce an IPF file that contains the exact data that where used to originally create the floppy disk. In order to do that you better work from almost perfect dump data. By design CTA is not very tolerant, for example CTA detects if a disk has been "tempered" (modified on an Atari) and if this is the case it wont let you write the IPF file ...

Current Aufit main goal is to produce images (mainly stx) to be used for emulation. Therefore the only constrain is to verify that we are reading information correctly as this would be done on a real Atari (and even better than on a real Atari). For example on a real Atari the tolerance on MFM transitions is about 10% of the nominal values and there is also automatic read retries performed by the TOS (up to 5 times using so called shoe-shine technique). Without DPLL and retries a large percentage of your floppies wont be read correctly. Aufit has an even more tolerant DPLL and has automatic "read retries" equivalent by reading data from different revolution (one of the good reason o sample at least 5 revolutions -- means 5 retrys).
On top of this main goal Aufit produces different graphs and text outputs that allow to "visualize" the quality of the image, but this interpretation currently has to be done manually by interpreting the outputs (CRC, out of band transitions, histograms, etc...)

To summarize: Current Aufit privileged production of correct stx images rather than perform quality analysis like CTA.
If you want to play a game Aufit should be able to generate correct stx images of relatively bad dumps.
If you want to reproduce a perfect image of the original disk use CTA and produce IPF files but the input quality has to be good.
SCP is in the middle of this two solutions has it just records/writes transitions. So if input transitions are good it will produce a good image but if they are bad it will produce a "not so good" image.

I am thinking of modifying Aufit to try to generate good IPF file from bad input! This should be feasible in many cases has the "layout" for protected and non protected disks are known in most cases... but this is only plans

so good or bad only means something if you define your goal


Thanks for clarify better than i did the whole thing :)
Now SPS France representative since the 19th of June 2014. Proud to be an SPS member !

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Aug 18, 2015 6:13 pm

G.Nius has been updated with a better dump.
More to come soon :wink:

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Tue Aug 18, 2015 7:24 pm

Thanks got it

Amazing
old-gnius.PNG

New
new-gnius.PNG


I can understand that cleaning helps on second sector
But I am puzzled by the fact that the clock is much more stable around sector 5-7
Is this on the same drive ???
You do not have the required permissions to view the files attached to this post.

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Aug 18, 2015 8:02 pm

Forget to say: it comes from a different source. I bought another copy of G.Nius in a lot (unboxed - no manual, but very well preserved).

Jeff_HxC2001
Captain Atari
Captain Atari
Posts: 305
Joined: Fri Sep 21, 2007 7:35 pm
Location: Paris - France
Contact:

Re: Brume's dumps

Postby Jeff_HxC2001 » Wed Aug 19, 2015 5:48 am

kodak80 wrote:So AUFIT is not suitable for checking how good a Kryoflux or SCP dump is then if it is not showing real errors! Great that it can recover from the errors but it would be nice to see the raw track information and any possible errors with my dumps.

I guess HxC Floppy Emulator provides a better image of the disk data and layout (once converted to CTR) as it doesn't correct the errors like AUFIT seems to be.

HxC also shows no errors with the Kryoflux raw dump which makes it very difficult to test how good a dump is. I guess we have to convert to CTRaw to test a Kryoflux dump as this shows the error on track 27?


If the HxC Software show you that a sector is good, this means that the sector is decoded correctly (checked against it's CRCs). Please note that the HxC software choose the best revolution fr each track when you import a stream files. So it is possible that another revolution spot an error at this point.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Wed Aug 19, 2015 6:31 am

Actually it is very difficult to find if a dump is good or not.
As Jeff mention CRC error may indicate a problem (or not).
If a sector is decoded with a good CRC you can be sure that the dump for this sector is good as there is no way a good CRC can be generated by chance.
If a sector has sometimes good and bad CRC the dump is bad but no bad enough you cant recover. In that case Aufit gives an error message but recover.
Another interesting point is the timing consideration: here again it is very difficult to know if the timing for a sector are normal or not. If the variation in the DPLL are above 5% Aufit may detect incorrectly bit width variation. This is usually not a problem in an stx file as this is some extra information not used by the program. I am considering to add a check in Aufit on the variation speed: speed variation is almost instantaneous in a protection and slow on bad dump.
Other problem hard to detect is whether or not a FD has been modified. I am working on this topic currently but it is hard.

I am working on a "mastering view" in Aufit: basically the idea is to check if it is possible from the current dump to match a perfect master description of a FD this is required to generate meaningful IPF file.

...

Jeff_HxC2001
Captain Atari
Captain Atari
Posts: 305
Joined: Fri Sep 21, 2007 7:35 pm
Location: Paris - France
Contact:

Re: Brume's dumps

Postby Jeff_HxC2001 » Wed Aug 19, 2015 7:00 am

DrCoolZic wrote:Other problem hard to detect is whether or not a FD has been modified. I am working on this topic currently but it is hard.


In fact without some specialized tools, you can't be sure : Some master disks was made with a simple computer. So you got write splice and others "Seems modified" symptoms.

User avatar
dlfrsilver
Atari God
Atari God
Posts: 1412
Joined: Mon Jan 31, 2005 1:41 am
Contact:

Re: Brume's dumps

Postby dlfrsilver » Wed Aug 19, 2015 9:54 am

Better than that, some publishers made a master with just the protection applied, and once getting it back, the programmer copied on it the game files with routine for protection check. And then gave the master disk back for duplication.

Other case : the master disk was made on a computer, and was blindly replicated by the trace machine. Hence the seems modified or made on a computer problem.
Now SPS France representative since the 19th of June 2014. Proud to be an SPS member !

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Sep 01, 2015 4:03 pm

Added Super Sprint to the list of dumped games.

User avatar
dlfrsilver
Atari God
Atari God
Posts: 1412
Joined: Mon Jan 31, 2005 1:41 am
Contact:

Re: Brume's dumps

Postby dlfrsilver » Tue Sep 01, 2015 5:05 pm

Your super sprint dump as CTR and KF Raw is good :) But it's seen as modified or not duplicated.
Now SPS France representative since the 19th of June 2014. Proud to be an SPS member !

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Dec 22, 2015 9:52 pm

Added Prophecy I - The Viking Child to the list.
More to come soon (before xmas? ;) )

anormal
Atarian
Atarian
Posts: 1
Joined: Tue Mar 24, 2015 7:24 am

Re: Brume's dumps

Postby anormal » Fri Jan 22, 2016 1:46 pm

@jim, @drcoolzic and the rest: guys, this thread is a goldmine...

I have both a kryoflux and a SCP, and it's difficult to find out exactly the meanings of a lot of things about floppy preservation/emulation.
This thread gives a lot of information in a very clear way. Even when i'm interested in PC/dos preservation, but ideas are the same.

Now i only want to know exactly about DPLL and how it really work from magnetic fluxes->binary, thanks drcoolzic's docs gives a lot of information!

Thanks!!

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Sun Oct 02, 2016 6:09 pm

Added Turrican I & Turrican II.
They are edited and spread by Rainbow Arts (there's no mention of Softgold on the box nor the disks).
Boxes are in Deutsch, French and Italian. Manual are in Deutsch, English, French and Italian.

Well they're not so rare, but the protection is very different compared to the one we can see on DrCoolZic website. Guess it could be interesting, who knows?

User avatar
TheNameOfTheGame
Atari God
Atari God
Posts: 1043
Joined: Mon Jul 23, 2012 8:57 pm
Location: Almost Heaven, West Virginia

Re: Brume's dumps

Postby TheNameOfTheGame » Mon Oct 03, 2016 1:26 am

Thanks very much!

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Tue Oct 04, 2016 8:08 am

Just analyzed Turrican and as Brume mention this version is quite different from my version (or Jim's version).

This one does not seems to have any "serious" protection. Only a strange MFM format 1x512 + 1x128 + 5x1024
Note that CTA from SPS report all track as modified? With my current version of Aufit (not sure if released) that have an experimental write splice detector I find 2 sector splices (front of id and between id/data) for almost all sectors.
Are you sure this is an original?

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Tue Oct 04, 2016 8:14 am

Just analyzed Turrican II with Aufit and CTA and same conclusion: not really protected / same strange format / tracks seems modified.

Note that Aufit detect some of the tracks as long tracks because the clock period is low (~1940) on these FDs and some tracks are therefore marginal

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Oct 04, 2016 8:16 am

Yes, true original and the games were in mint condition. All other games I got from the seller were in perfect condition, I rarely find games in such condition.
One question: when CTA detects modified tracks, does it also mean it can't generate IPF files?

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Tue Oct 04, 2016 9:02 am

Not sure always possible but in that case I was able to generate IPF file :)

You can get it here if you want to try
https://mega.nz/#!p8ZwmCgB!TtTGm5AtjDmG ... mxoQkNaiGM

Pictures
turrican CTA.PNG

turrican aufit.PNG
You do not have the required permissions to view the files attached to this post.

User avatar
Brume
Red eyes
Red eyes
Posts: 4093
Joined: Mon Apr 22, 2002 10:16 am
Location: France
Contact:

Re: Brume's dumps

Postby Brume » Tue Oct 04, 2016 10:04 am

Fantastic, thank you very much for the IPF :thumbs:

ijor
Hardware Guru
Hardware Guru
Posts: 3095
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Brume's dumps

Postby ijor » Tue Oct 04, 2016 11:36 am

DrCoolZic wrote:Turrican ... This one does not seems to have any "serious" protection. Only a strange MFM format 1x512 + 1x128 + 5x1024


And this certainly IS a protection. Don't know what exactly you mean by "serious", but you can't copy that format with standard hardware.

Note that CTA from SPS report all track as modified? With my current version of Aufit (not sure if released) that have an experimental write splice detector I find 2 sector splices (front of id and between id/data) for almost all sectors. Are you sure this is an original?


Multiple write splices doesn't necessarily mean that the disk was modified. Many original disks were recorded with two passes, or what seems to have multiple write splices. And of course that a single write splice doesn't guarantee that the disk was not modified or it's original.

User avatar
DrCoolZic
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2144
Joined: Mon Oct 03, 2005 7:03 pm
Location: France
Contact:

Re: Brume's dumps

Postby DrCoolZic » Tue Oct 04, 2016 12:12 pm

ijor wrote:
DrCoolZic wrote:Turrican ... This one does not seems to have any "serious" protection. Only a strange MFM format 1x512 + 1x128 + 5x1024


And this certainly IS a protection. Don't know what exactly you mean by "serious", but you can't copy that format with standard hardware..

You can't use BIOS calls to copy this track but WD1772 supports 128/256/512/1024 bytes/sect so I think it should be possible to copy these tracks with std Atari HW? In other word I do not see anything that WD1772 cant do?

Ijor wrote:
Note that CTA from SPS report all track as modified? With my current version of Aufit (not sure if released) that have an experimental write splice detector I find 2 sector splices (front of id and between id/data) for almost all sectors. Are you sure this is an original?


Multiple write splices doesn't necessarily mean that the disk was modified. Many original disks were recorded with two passes, or what seems to have multiple write splices. And of course that a single write splice doesn't guarantee that the disk was not modified or it's original.

I do not know the answer to that. Seems like SPS people considers tracks with double sector splices as modified (as reported in CTA).
Aufit just try to detect and report splices without implication. Note that in many cases (like in Turrican) the detection of splices is not that easy. In many cases splices do not result in short/long fluxes but only in wrong sequence of bits. Therefore Aufit algorithm for splice detection uses three criterions ...

ijor
Hardware Guru
Hardware Guru
Posts: 3095
Joined: Sat May 29, 2004 7:52 pm
Contact:

Re: Brume's dumps

Postby ijor » Tue Oct 04, 2016 12:53 pm

DrCoolZic wrote:1x512 + 1x128 + 5x1024
...WD1772 supports 128/256/512/1024 bytes/sect so I think it should be possible to copy these tracks with std Atari HW? In other word I do not see anything that WD1772 cant do?


The FDC can create and write that kind of sectors, but the ST can't write them at the necessary frequency to fit all of them on one track. You need a custom controller, or an FDC clocked faster, or a drive rotating slower.


Social Media

     

Return to “Floppy Disk Preservation”

Who is online

Users browsing this forum: No registered users and 1 guest