Bad Mood : Falcon030 'Doom'

[dml]

Doug,
Here's an interesting story about an offshoot of Doom, 'Marine Doom'. It has a little link at the end of it.

Bizarre story, from top to bottom. I had heard about this before but hadn't seen the details.

[dma]

Ah yes, the first bizarre feeling i had about this WAD has been reading "Sergeant Daniel G. Snyder, USMC" as author instead of usuals demonic themed pseudos.
http://www.doomworld.com/idgames/?file= ... arine1.zip

[Zamuel_a]

When will this version be out so I can test it?

[dml]

When will this version be out so I can test it?

We were working on a problem integrating the last bits of code for MIDI music (most of it hassles with tools really, and having to disable more and more code which interferes with native debuggers) but that all seems to be sorted out as of this morning so it just needs some tidy up work and some config file bindings and then an initial beta version can be released.

I haven't been making any major code changes so it has a chance to settle down. Should be out quite soon!

[Eero Tamminen]

We were working on a problem integrating the last bits of code for MIDI music (most of it hassles with tools really, and having to disable more and more code which interferes with native debuggers) but that all seems to be sorted out as of this morning so it just needs some tidy up work and some config file bindings and then an initial beta version can be released.

Is the MIDI stuff still being integrated?

I'm wondering because I noticed that you had some comments about Hatari MIDI interrupt emulation (on hatari-devel).

[dml]

Is the MIDI stuff still being integrated?

I'm wondering because I noticed that you had some comments about Hatari MIDI interrupt emulation (on hatari-devel).

Yes it is still being integrated. Really the MIDI library etc. is working ok but there have been some real headaches making replay work between two MFP timers without breaking mouse & keyboard, and trying to avoid polling-transmits because it wastes so much CPU time. While figuring that stuff out we ran into some other problems with memory allocation which are getting fixed now.

I was recently on vacation too, so everything stopped at my side

[SteveBagley]

I thought this news article might amuse the people reading this thread Someone's ported Doom to a printer.

Steve

[dml]

I thought this news article might amuse the people reading this thread Someone's ported Doom to a printer.

Steve

Hehe.

He said he had undertaken the project to demonstrate the security problems surrounding devices that would form the "internet of things"

Yeah, right. 4 months of work for that. I think he really did it for the bonus hacker points.

[dml]

The hardware side of the MIDI replay stuff now seems to be working as of last night (famous last words!). There's just one serious bug left that we know of, which continues to cause some problems.

There may be a few more glitches which show up once that's fixed but it must be pretty close now. I can play a couple of levels with keyboard + mouse and MIDI interrupts running, feeding MIDI to a softsynth on my PC without obvious problems and with very low CPU overhead on the Falcon side.

A buffer overrun is still causing a crash sometimes when loading new tracks but this should be fixed soon and I can hopefully return to tidyup for a release.

[dml]

My latest problem seems to be that I can't copy files from one CFlash card to another on my Falcon if the file is bigger than 4MB. So long as I keep the size under 4MB it works.

I'm using HDDriver 8.4x with two CF cards on the same IDE cable. One is byte-swapped for exchanges with PC, the other is native format. I probably didn't run into this before with the bigger WADs since I used ARJ to transfer them in pieces. Only now that my updated PWAD hit 8MB that I noticed it. Have removed half of the content and it seems to be working.

It's interesting that reading the file directly from the 2nd CF on the Falcon seems to show correct data, but copying it to the internal CF and then reading it shows garbage.

This is one of those insane cases that gets in the way of progress and takes forever to figure out by trial and error - anyone had similar experiences?

[calimero]

^ maybe to try ppera harddisk driver...?

[Dal]

I thought this news article might amuse the people reading this thread Someone's ported Doom to a printer.

Steve

Yes, this was very interesting for me also as I nearly ended up working for Canon.

I was a systems consultant for Océ up until they merged with Canon last year. Part of my role in Océ was handling clients who needed support on device security (encrypted print data, hard disk removal/destruction, network access etc).
Firmware injection was always one of my concerns but whenever I raised it, I was always told that it's highly unlikely an employee within a client would do such a thing.

Now the idea has been given media coverage....

Anyway, I'm free of all that now, but I do feel a little relieved I'm not having to get involved with this one. Trying to mobilise R&D was always a pain in the backside within Océ, Canon's organisation is much much deeper.

[dml]

^ maybe to try ppera harddisk driver...?

I definitely need to deal with this before it causes bigger problems. It must be a fault in HDDriver or perhaps a limitation of the partition or format which has been used on the CF. I have worked around it meantime by using ARJ and splitting into 1.44 floppy-sized chunks, using low compression for speed. Having the CRC check performed on extraction is helpful, compared with just trusting that the file is complete...

[dml]

Yesterday I found an evil bug in BadMood, caused by an evil optimization.

I was getting intermittent crashes, causing the game to exit back to the GEM desktop without actually causing a bus error or other exception. Not quite sure how it managed that but I had all of the exception vectors mapped to my own handler, complete with colour cycling and console messages - and it never ever got there. The game interrupts keep running too, so the display and screensplit makes the game look like it is still running. The only telltale that it hits GEM is the disk activity light showing a directory refresh immediately after the lockup (it has a sort of recognizable flicker to it!).

It also refused to happen under Hatari, which made debugging it 'really hard'.

Anyway after much testing and narrowing, I found that it only ever happens when the PMMU is configured to mark some areas of memory non-cacheable to the CPU. In particular it started happening after a 'fix' to this code, which stopped the PMMU from mapping non-cacheable memory correctly for some addresses.

The PMMU nocache trick is useful when you want to 'partially' disable the datacache - i.e. you want reads to be cached, but you don't want writes to invalidate/kick out cache entries which are useful for reading later. Copying textures to a framebuffer is one example. This is especially true if you are using the write-allocate mode which can cache both reads and writes (where the writes are longwords, and long-aligned), since write-allocate mode causes even more destruction to cache entries when misaligned longs or aligned words are written (see 030 user manual for a lot of nasty detail on that stuff).

After reconfiguring the PMMU to map the same memory in different ways, turning the feature on/off, using transparent translation instead, and changing the code which uses the specially mapped memory, it was clearly related to *access* to this memory in some parts of the game - not the code using it, or the PMMU configuration itself. Something to do with the context in which this was being used.

So I divided up the code into the different parts which relied on this feature, and placed each under a separate compile-time switch. I managed to rule out speed/timing issues in the code which is sensitive to that. Also ruled out cases where it was used from interrupts (audio mixing buffers).

Narrowed it down to memset. It seemed that clearing ram marked as non-cacheable produced flaky results. Removing that stopped the crash. After a bit of thought I realized this was probably the only use case where the affected memory would affect program flow control - critical logic. It is often used to clear structures before proper values are initialized. Every other case was just being used for display or audio buffers, or linear filling/copying of data which is for output only. Only memset was used near sensitive structures which could upset logic.

After a bit more thought, it becomes clear that this is dangerous - the same physical memory is being mapped through two different logical addresses (in this case 0x00xxxxxx and 0x0Exxxxxx) but the CPU's data cache has only a single mapping. Performing a memset on 0x0Exxxxxx will bypass the cache, without updating or clearing those entries. Accessing the same physical memory immediately afterwards with the CPU via 0x00xxxxxx will then read garbage, instead of zeroes, because some cache entries may remain tagged with the old values.

eek.

Removing the nocache mask from the memset address fixes the issue. As does clearing the datacache after memset.

Another lesson on the hazards of evil optimizations


NB. It seems to be common to just turn off the datacache on Falcon, because it typically slows code down. This can certainly be true, if the operation is a trivial transfer of memory e.g. copying data, clearing buffers. Those are faster without d-cache because the cache only produces a 'hit' if you read the same location twice in close proximity - something that won't happen during linear operations. It does however happen a lot during scaling, data decoding, data recursion and other more complex access patterns. The performance overhead is made a bit worse by the fact the 68030 caches are longword-based, and the Falcon030 bus is 16bit (so reading a word forces a longword cache entry to be filled - it just happens that often the next word needed is in the same longword, so the overhead in practice is very small despite this). The situation is also made worse by the d-cache algorithm itself inside the chip, which must always perform write-throughs to RAM, and will typically destroy cache entries mapping to the written address, just to keep the cache contents consistent with future reads. By using the trick I describe above on the destination buffer however, this cache-destroying step is prevented, so read hits frequency can be increased even in code which spends a lot of time writing data. Texture mapping can benefit from this.

[dhedberg]

That's what I call a great bug hunt!
Great and informative post Doug! Thanks for sharing!

[mfro]

Another lesson on the hazards of evil optimizations

It's absolutely unrelated to Doom or even the Falcon, so sorry to interfere, but allow me the statement that MMUs are indeed really nasty beasts, honestly.

Recently I had a heavy struggle with one of these specimen on BaS_gcc (the Firebee's firmware).

Coldfires utilize an MMU that has lots of nice features, among others different page sizes and a way to assign different mappings for different PIDs. This comes at a cost, these thing can only hold 32 TLBs for instruction and data pages each. The MMU handles pages faults through assigning new mappings during an access exception, where it just replaces the least recently used mapping with a new one.

BaS_gcc always worked fine with all kinds of programs, but crashed in MiNTs network code for no apparent reason.

One should assume they tell you this in the manuals (but they don't): if you have heavy use of memory pages with code sequences that do not touch the supervisor stack for a long time but touch other memory areas frequently, the MMU considers the page where our poor supervisor stack resides as least recently used and steals it. The code still runs until the next page miss which will trigger an access exception that doesn't find a place where to put its exception stack frame to. Boom. Double fault...

[dml]

One should assume they tell you this in the manuals (but they don't): if you have heavy use of memory pages with code sequences that do not touch the supervisor stack for a long time but touch other memory areas frequently, the MMU considers the page where our poor supervisor stack resides as least recently used and steals it. The code still runs until the next page miss which will trigger an access exception that doesn't find a place where to put its exception stack frame to. Boom. Double fault...

That is evil, indeed.

While debugging that, I was also struggling with randomly occurring spurious interrupts, caused by race conditions between CPU and MFP trying to communicate events while its state was being changed. That's another dark rabbithole. I'm not sure which of the two is more evil. They are both contenders. Getting them both at once was unfortunate.


In my case for PMMU, I tried to sidestep ATC overheads (and special hazards) by filling out a shallow tree, using early terminators. No real page descriptors. So the ATC only has to deal with a few commonly accessed 16MB areas (primary area, HW area and nocache-shadow area).

I did worry at one point about how the CPU/MMU deals with page faults that need to pull in or set up something that is also time-critical (like paging in a mouse handler from disk!) but I don't have anything like that so for now it's an idle worry

[dml]

This is still plodding on, if a bit slowly.

Last night i got pitchbend support into the BadMood MIDI 'softsynth'. This turned out to be a nightmare but I think it's sorted now. The new code needs properly optimized but I'll have to shunt that back a bit.

I picked a difficult testcase - the awesomely wobbly 'dukin' track from Duke Nukem. haha. It has active polyphony within 2 pitch-affected channels, generating lots of pitch-affected voices - which I think is stretching it for this project.


With very loud Doom guitars:

https://dl.dropboxusercontent.com/u/129 ... kin-gt.mp3

With electro synthy sounds instead:

https://dl.dropboxusercontent.com/u/129 ... kin-sy.mp3

Both recorded from Hatari.

There are still a few problems to be sure but it's a challenge to make it work with everything - some tracks still need tweaked or remapped to get them sounding decent and it takes time. This one has a couple of timing dropouts. Some are caused by import-for-edit, and some caused by the player not prefetching the resampling cache on load (those ones 'cure' on the 2nd loop).

Now I just need to re-convert and retest 50 music tracks after the changes :-z

[EvilFranky]

Sounds great Doug!

[dml]

Oh yeah,

And because this is Atari, chip tunes can turn up too.

https://dl.dropboxusercontent.com/u/129 ... di-dma.mp3

I don't know why it would be necessary to play chiptunes by DMA, but hey.

[dml]

In terms of size the soundfont / samplebank is quite big, about 3mb at last count including all the instruments and percussion. However it is shared across about 50 tracks. The music files are about 10k-40k. the chiptune conversions go up to about 200k if they have fine event grain.

[Eero Tamminen]

In terms of size the soundfont / samplebank is quite big, about 3mb at last count including all the instruments and percussion.

mp3 files sounded great! Is there some plan to have also a standalone MIDI-player with your sound synthetizer?

I'd like to test how good it sounds with random MIDI files from other music genres (e.g. classical or jazz).

[dma]

Indeed great renditions, nothing to be ashamed compared to some Sound Blaster cards of the old PC days.

[dml]

Indeed great renditions, nothing to be ashamed compared to some Sound Blaster cards of the old PC days.

Many thanks.

And BTW when I wrote 'by DMA' above I really should have written 'with the DMA & Codec chips'.

(doh! doh! doh!)

[dml]

mp3 files sounded great! Is there some plan to have also a standalone MIDI-player with your sound synthetizer?
I'd like to test how good it sounds with random MIDI files from other music genres (e.g. classical or jazz).

A music disk was suggested and that should work fine.

Playing random MIDI files will probably be quite erratic. But I'll tidy up the convertor so its all standalone and you can find out for yourself.

The two main problems are the soundfont (which is 'optimized' for Doom, with plenty of gaps in it) and some missing controllers. The 'dukin' track actually has a controller which does some sort of cross-mixing between the two lead channels but it's not the volume or panning ones because fruityloops completely ignores it too. So there are probably lots of weird things like that which would need added/fixed for general purpose use. It was really just coded to play specific music for some games and demos - a kind of souped up modplayer - and assumes you have time to tweak things.