Bad Mood : Falcon030 'Doom'

dml · Post by **dml** » Sat Mar 09, 2013 11:48 am

I'll upload a build with MIDI hooked up and enabled, perhaps somebody can test it for me? The .MID file plays ok on my PC and BM seems to load it happily and starts the player. Just need to confirm it's outputting something to the ports.

dml · Post by **dml** » Sat Mar 09, 2013 11:59 am

Here's a test of BM403F'M' with MIDI enabled. The .MID file should be in the same directory as the TTP. An error message will appear on the TOS screen before starting, if it can't be found (but the engine will still go).

BM403FM.zip

I might have a quick look at the WAD midi lumps to see if they are directly compatible.

[EDIT]

BAH - the WAD files hold another format (.MUS) which needs converted to MIDI if using a MIDI replay. They are similar but it's still pretty annoying! WAD tools can do the conversion but it's not much use for directly playing stuff out of existing WADs.

WADs can also hold MIDI files instead, so it would be possible to create a BADMOOD.WAD (PWAD) overlay with new music files in it, to access new or original music. Or loading them directly as files. Not the end of the world but it would have been nice to play them direct from the WAD so existing game files would work as-is.

[EDIT 2]

Seem to be some resources here for extracting the MUS lumps, doing the conversion to MID, and building a PWAD overlay for the music... not actually tried it but somebody may have the time to experiment at least with the .MID export, and have a go at replacing NIKLAS.MID with one of the converted WAD MUS files.

ftp://archives.gamers.org/pub/idgames/utils/sound_edit/

[EDIT 3]

...found source for MUS->MID conversion in C. That may allow conversion at runtime, by placing it between the WAD LUMP and the BM resource cache. That would probably be a win!

dml · Post by **dml** » Sat Mar 09, 2013 2:55 pm

Completed my notes on differences between Doom and BM rendering steps. Most of it is quite easy to correlate. The parts that deal with individual wall columns in Doom is complicated and messy and I'm not sure there's much of interest there except perhaps texturing correctness issues with coordinates/pegging etc. BM deals with the same stuff but it's done earlier and only once per wall surface.

There is an extra shortcut in Doom which isn't present in BM specifically for ignoring 'trigger' walls (I think used to wake up AIs) so I have added this, but it hasn't influenced any metrics gathered - same number of walls get drawn. So that shortcut doesn't seem to be used much, or BM was rejecting them by null texture codes later on.

There is also a shortcut for visplane lookups when floors/ceilings are are invisible due to the player's eye height (e.g. player eye is below the floor level of a raised sector). This shortcut is missing from BM mainly because there is no visplane searching and visplanes aren't referenced this far up the rendering chain - but it might translate to deeper stuff in the BM floor edge generator (DSP work). It's worth a look later if the DSP starts being a bottleneck.

Eero Tamminen · Post by **Eero Tamminen** » Sat Mar 09, 2013 7:21 pm

dml wrote:Here's a test of BM403F'M' with MIDI enabled. The .MID file should be in the same directory as the TTP. An error message will appear on the TOS screen before starting, if it can't be found (but the engine will still go).

I tried that on Linux and MIDI playback of the supplied MIDI file seems to work fine.

* I did as root:

Code: Select all

# modprobe snd-virmidi

* And as user:

Code: Select all

$ qsynth &
$ aconnectgui &
$ hatari --midi-out /dev/snd/midiC1D0 /path/to/BM/

(In my setup virtual MIDI ends up as Card 1 for Alsa, and I'll use the first MIDI device i.e. 0, so the device file is midiC1D0. I have QSynth application already set up for FluidSynth synthetizer, so I don't need tell it what sound font it should use.)

Then I just connect virtual raw MIDI output 0 to FluidSynth input in Alsa GUI before starting Badmood for the first time.

Music starts right after pressing key at startup, so one is entertained while textures are scaled on screen.

The problem with the supplied NIKLAS.MID is that it doesn't setup the midi instruments. When I tried changing that to another MIDI file that does setup them, they were not played. I.e. the BM MIDI player is lacking a bit in features.

When I looked a bit more into MIDI files that didn't work, with:

Code: Select all

for mid in *.mid; do echo $(hexdump -C $mid | head -1) $mid; done

They were ones with multiple tracks in them. I guess each of the tracks in them set the instrument for the track at the beginning, whereas files with single track (potentially with notes going to different midi channels i.e. normally to different instruments), don't have events that set/change the instruments.

Instruments are set by program change messages:
http://en.wikipedia.org/wiki/MIDI#Techn ... ifications
http://www.sonicspot.com/guide/midifiles.html

I wonder is there some tool to mix events from multiple MIDI tracks to a single track? Maybe by playing MIDI file and saving raw MIDI data to another MIDI file with "midid"?

Eero Tamminen · Post by **Eero Tamminen** » Sat Mar 09, 2013 8:25 pm

Eero Tamminen wrote:I wonder is there some tool to mix events from multiple MIDI tracks to a single track? Maybe by playing MIDI file and saving raw MIDI data to another MIDI file with "midid"?

Unfortunately "midid" (DosEmu utility) doesn't work, it says often "bad data" when trying to record sequencer output from Hatari, and the produced MIDI data contains invalid data (negative chunk length(s)).

Does anybody have other ideas / tools that could convert multitrack MIDI file to a one with a single track?

(Best would of course adding support for multiple tracks to BM MIDI player, but I guess there's nobody to dig into that code now.)

Eero Tamminen · Post by **Eero Tamminen** » Sat Mar 09, 2013 8:30 pm

dml wrote:...found source for MUS->MID conversion in C. That may allow conversion at runtime, by placing it between the WAD LUMP and the BM resource cache. That would probably be a win!

If that converts MUS into a single track MIDI file, everything will hopefully work fine as-is. Does it map instruments to general MIDI instruments (specification on what instrument each program number should match to)?

I think using MIDI for music is very Atari thing.

dml · Post by **dml** » Sat Mar 09, 2013 10:22 pm

Eero Tamminen wrote:If that converts MUS into a single track MIDI file, everything will hopefully work fine as-is. Does it map instruments to general MIDI instruments (specification on what instrument each program number should match to)?

I haven't had enough time to study it - the prog is called qmus2mid and I was able to build the single .c/.h under VC8 without changing anything. I did notice that it's not endian-clean so it won't work on the Falcon without hacking around with it and fixing all the fread/fwrite access, and whatever else might be broken.

Eero Tamminen · Post by **Eero Tamminen** » Sat Mar 09, 2013 10:41 pm

Another version of the earlier CPU side graph. This is reduced one with costs propagated up to callers:

Code: Select all

hatari-profile.py -p --compact --no-leafs --no-intermediate -e 8.0 -l 4.0 -f 0  --ignore-to framecounter,ikbd_handler,get_dy,new_vbi,stabilizer_b -g bmsym.txt

As unfortunately there's something not quite right in places where graph loops, these kind of graphs contain now also node's own cost.

Eero Tamminen · Post by **Eero Tamminen** » Sat Mar 09, 2013 10:51 pm

dml wrote:I did notice that it's not endian-clean so it won't work on the Falcon without hacking around with it and fixing all the fread/fwrite access, and whatever else might be broken.

You probably know this already... Standard network netinet/in.h include file provides ntohl/ntohs/htonl/htons (h=host, n=network i.e. big endian order, l=long, s=short) inline conversion routines. I.e. they should work also on other platforms (including Windows I hope), so that converter code remains portable.

dml · Post by **dml** » Sun Mar 10, 2013 9:59 am

Eero Tamminen wrote: You probably know this already... Standard network netinet/in.h include file provides ntohl/ntohs/htonl/htons (h=host, n=network i.e. big endian order, l=long, s=short) inline conversion routines. I.e. they should work also on other platforms (including Windows I hope), so that converter code remains portable.

Yes I think the difficulty in this case will be finding where the endian fixes are actually needed. Fortunately it's a small program, at least half of it commandline handling stuff so it's just a matter of finding the time to look at it properly.

The first experiment should be to extract a MUS from WAD directly, convert it to MID with that tool and test it in the BM player. If that works ok, it's worth trying to put an equivalent pipeline in the BM resource cache. If not, it's probably better to spend the time on a more complete player, or alternatively - supply premade MID files for all known MUS lumps, but play MID lumps directly from PWADs if supplied (it was common to use MID for custom WADs since it supports both?).

Will definitely spend more time on the audio/music area once the visuals are tied up and the game code is supplying some events to use.

Eero Tamminen · Post by **Eero Tamminen** » Sun Mar 10, 2013 2:59 pm

dml wrote:ftp://archives.gamers.org/pub/idgames/utils/sound_edit/

dml wrote:The first experiment should be to extract a MUS from WAD directly, convert it to MID with that tool and test it in the BM player. If that works ok, it's worth trying to put an equivalent pipeline in the BM resource cache. If not, it's probably better to spend the time on a more complete player, or alternatively - supply premade MID files for all known MUS lumps, but play MID lumps directly from PWADs if supplied (it was common to use MID for custom WADs since it supports both?).

I tried the DOS binaries in that directory with DosBox. Extracting the MUS files works with XMUS program, and of the MUS->MIDI converters, MID2MUS1 and QMUS seem to work, whereas MID2MUS2 (midi2mus v0.9?) gave invalid MIDI file. QMUS froze at the end, but the resulting TEMPMID.TMP seemed fine. As QMUS produced smallest MIDI file, memory wise that seems best conversion option.

However, all of these tools produce multi-track MIDI files (which set up instruments correctly), i.e. these MIDI files won't work with current BM MIDI player.

Adding support for that shouldn't be too that hard if the player code would be C. One would need to keep pointers to all tracks in the MIDI file and play the event which time has come. The small complication is that event timings are as deltas to previous events on that same track i.e. one needs to convert them to wall time so that one knows which event from which track one needs to play next. After that, one would know whether the code supports program changes, if not, that should be easier to add.

Another option would be to change QMUS to write a single track MIDI file. Improving BT MIDI-player would IMHO be nicer, but changing QMUS might be easier.

dml · Post by **dml** » Sun Mar 10, 2013 6:55 pm

Eero Tamminen wrote: However, all of these tools produce multi-track MIDI files (which set up instruments correctly), i.e. these MIDI files won't work with current BM MIDI player.

Ok thanks, that's all useful.

It might in fact be easier to trim down (or flatten output from) the other midi parser I have been using (in C) as it can handle multi track data ok, but it's not worth trying to integrate with BM as a C component - would be better to bind it with the Doom code as a separate component/layer just for the music.

I'll also take a look at the existing player but extending it will probably require more time and effort.

Eero Tamminen · Post by **Eero Tamminen** » Sun Mar 10, 2013 9:58 pm

I wrote few lines of code around this:
https://raw.github.com/pichenettes/avri ... idifile.py

to convert multitrack MIDI file to a single track one. However, result of that didn't play with ALSA aplaymidi (which played original multitrack file fine), nor with BM midi player, although Rosegarden sequencer played it fine though. I guess they didn't like some of those extra MIDI messages on the format 0 (single track) MIDI file.

I think having the instrument information is essential, without correct instruments the pieces can sound pretty horrible (and heavy/metal style music used in doom also pretty boring). So it seems that BM MIDI player would need to be improved, doing things in the converter isn't enough.

dml · Post by **dml** » Mon Mar 11, 2013 9:27 am

Eero Tamminen wrote: I think having the instrument information is essential, without correct instruments the pieces can sound pretty horrible (and heavy/metal style music used in doom also pretty boring). So it seems that BM MIDI player would need to be improved, doing things in the converter isn't enough.

Yes the Doom music doesn't quite meet the grade for Atari

but getting it as compatible as practically possible would be good. An upgraded/improved player is probably the best route.

dml · Post by **dml** » Mon Mar 11, 2013 9:58 am

I started the process of moving the expensive parts of wall insertion onto the DSP, and cleaning up some of the DSP stuff to make that more efficient. It will take time and I don't have much of that at the moment so I won't have a new version for at least a week, maybe more.

There's a wall projection & visibility test routine on the DSP which gets used before inserting walls. Interacting with this test, reading the results back, doing all the wall height & offset logic and then inserting wall surfaces as separate tasks is causing most of the CPU overhead after pixel drawing. This is the bit I didn't finish when I left BM307. I think the main reason it was left until last is simply because it involves lots of Doom drawing compatibility logic and it took time and tweaking to get the results looking like the real thing. It was therefore not attractive to optimize (or DSP) too early.

This is now being converted (albeit in three steps) into an input-only pipeline to the DSP which begins with loading the seg vertices, followed by linedef attributes, current/adjacent sector attributes etc. so the DSP can do the rest, inserting its own surfaces. This should have a measurable impact on speed for all but the simplest views, and a large impact on complex views.

The first step will be to defer seg fields through this routine (renamed to R_ViewTestAddLine, nearly analogous with Doom::R_AddLine but with the occlusion test rolled into it instead of before the call) which the DSP can reuse when inserting surfaces (instead of going back and forth between processors). The second step will be to augment this deferred material with other fields used by wall insertion until nearly all of the CPU wall term calculation duties are absorbed by the DSP. The last step will be to move wall insertion logic and events themselves to the DSP, and remove all of the CPU wall handling below the level of Doom::R_AddLine.

At some point during step 3, the remaining CPU code will begin to fit entirely in the CPU cache (currently the iterated part is about 2k!), and itself will speed up significantly. Assuming that all works as planned (and it should) the wall column pixel drawing will be the last bottleneck.

I had some ideas which might allow the wall texturing to use the DSP effectively, but it would involve moving the texture perspective calculation to another point where it would stop being 'free' and would begin to show up as a DSP cost. I use a real divide for this - Doom doesn't. Doom uses several tables instead which consume precious DSP memory (I would need to compress/interleave the tables - not attractive, and one of the tables is resolution dependent too). However I see another way to do it without the divide expense or the tables. If I don't have too much to do already (!) I may get to try that.

I also started renaming bits of BM to map better to the Doom code so it's easier to identify what each bit does. Might be helpful if somebody else wants to mess with it when I'm done.

Eero Tamminen · Post by **Eero Tamminen** » Mon Mar 11, 2013 11:35 am

My MIDI format conversion script had a small bug, it didn't filter out EndOfTrackEvents, and first of the tracks output by QMUS had just of couple of meta events before it ended, and therefore the copied end-event terminated the single track playback before it had even started...

Attached script, which requires above linked midifile.py, filters all MIDI meta events out while joining the track data to a single track.

BM can play the resulting MIDI file just fine, and it does understand instrument settings! So it was just the included NIKLAS.MID that was crap, BM player was OK.

PS. Note that while midifile.py parses QMUS output fine, it doesn't parse all the MIDI files, it will barf on files which data isn't completely correct.

dml · Post by **dml** » Mon Mar 11, 2013 3:40 pm

Eero Tamminen wrote:My MIDI format conversion script had a small bug, it didn't filter out EndOfTrackEvents, and first of the tracks output by QMUS had just of couple of meta events before it ended, and therefore the copied end-event terminated the single track playback before it had even started...

Attached script, which requires above linked midifile.py, filters all MIDI meta events out while joining the track data to a single track.

BM can play the resulting MIDI file just fine, and it does understand instrument settings! So it was just the included NIKLAS.MID that was crap, BM player was OK.

PS. Note that while midifile.py parses QMUS output fine, it doesn't parse all the MIDI files, it will barf on files which data isn't completely correct.

Great stuff! This already improves chances of decent MIDI output quite a bit.

nativ · Post by **nativ** » Mon Mar 11, 2013 4:33 pm

http://www.youtube.com/watch?v=MqqAFhESYEg

Strife running on Amiga 1200 030

Have you seen this FPS project

http://aminet.net/package/game/misc/StrifeAGA-1.0

dml · Post by **dml** » Mon Mar 11, 2013 5:04 pm

nativ wrote:http://www.youtube.com/watch?v=MqqAFhESYEg

Strife running on Amiga 1200 030

Have you seen this FPS project

http://aminet.net/package/game/misc/StrifeAGA-1.0

It's nice to see a new game being produced from the same tech. So many source ports but most of them just different views/extensions on Doom.

Would be good to see something similar happen on Atari (even when I first started working on this project it seemed like an obvious route to spawn at least one new game, and the engine is amenable to feature extensions to suit other games).

Eero Tamminen · Post by **Eero Tamminen** » Mon Mar 11, 2013 7:59 pm

dml wrote:It's nice to see a new game being produced from the same tech. So many source ports but most of them just different views/extensions on Doom.

Besides Strife, there were several other 90's games based on Doom engine:
http://en.wikipedia.org/wiki/Doom_(video_game)#Clones

Strife is kind of interesting because sources to its Doom engine modifications were lost and they were recreated later by Doom source port maintainers:
http://en.wikipedia.org/wiki/Strife_(vi ... evelopment

Amiga Strife seems to be based on Chocolate Doom engine, I wonder is it faster (not using GL etc) than the other ports supporting Strife...

Eero Tamminen · Post by **Eero Tamminen** » Mon Mar 11, 2013 10:55 pm

dml wrote:Great stuff! This already improves chances of decent MIDI output quite a bit.

There was another small bug in the code, it didn't copy PPQN (Pulses Per Quarter Note clock tick value) from the original file to new one. Attached version fixes this + outputs some more info about the MIDI file.

When adding this, I noticed that BM MIDI player doesn't support that either, it plays all MIDI files at same speed. For 240 PPQ, the music sounds *really* slow (at 2.5x slowdown, also really boring).

[EDIT] I added support for converting MIDI files to a different PPQN value & uploaded new version of the script.

What value BM player uses to scale the events?

dml · Post by **dml** » Tue Mar 12, 2013 9:53 am

Eero Tamminen wrote: When adding this, I noticed that BM MIDI player doesn't support that either, it plays all MIDI files at same speed. For 240 PPQ, the music sounds *really* slow (at 2.5x slowdown, also really boring).
[EDIT] I added support for converting MIDI files to a different PPQN value & uploaded new version of the script.
What value BM player uses to scale the events?

I'm not sure - I only looked at the code very briefly. I think it uses a fixed frequency TimerA/D and then there is some constant event timer based on this base frequency. It might be easy to modify it to work correctly - although the base frequency needs to be set high enough not to quantize the playback rate.

(TimerA will be probably needed later for SFX so it would be good to have both systems work together without one breaking the other, so it is probably better that the MIDI player can handle a range of base frequencies via some scaling value without itself needing to modify the base freq!).

Eero Tamminen · Post by **Eero Tamminen** » Tue Mar 12, 2013 12:04 pm

dml wrote:I think it uses a fixed frequency TimerA/D and then there is some constant event timer based on this base frequency. It might be easy to modify it to work correctly - although the base frequency needs to be set high enough not to quantize the playback rate.

(TimerA will be probably needed later for SFX so it would be good to have both systems work together without one breaking the other, so it is probably better that the MIDI player can handle a range of base frequencies via some scaling value without itself needing to modify the base freq!).

In general, AFAIK MIDI file PPQN values should be multiples of 24, for compatibility reasons with older (80's) MIDI devices which clock resolution wasn't so great. Typical values are 96 and 120, but slightly newer orchestral pieces (from end of 90's) often seem to have PPQN of 240, some even higher.

QMUS seems to default Doom music PPQN to 89 for some reason [1]. One can give it another value, but it doesn't scale the music delta times accordingly (like my script does), it just sets the given value to MIDI file header.

[1] and when QMUS shows the music length, it calculates that wrong, using PPQN of 70, instead of 89. AFAIK seconds should be calculated as "Time/(PPQN*2)".

Btw. The 25 music pieces in Doom1 are in length from 5s to 2.5min, with 4 to 13 tracks (corresponding to different instruments?).
The 21 music pieces in Doom2 are in length from 12s to 4min, with 5 to 15 tracks.

Only the number of events at any given moment should affect performance. E.g. pitch bend events could be problematic, as there can be a lot of them in short time, but I don't think Doom music has any extreme cases.

Eero Tamminen · Post by **Eero Tamminen** » Thu Mar 14, 2013 4:17 pm

Btw. Douglas, will you release / open source your Bad mood DSP and CPU side code at some point?

dml · Post by **dml** » Thu Mar 14, 2013 4:40 pm

Eero Tamminen wrote:Btw. Douglas, will you release / open source your Bad mood DSP and CPU side code at some point?

Yes, certainly - was thinking of placing it in a public mercurial repo somewhere but not until the rework is complete. Part of the renaming effort is to make it easier for others to follow/adapt if they want.

Just very busy at the moment to do much with it this week. Will probably see some progress next week.

Atari-Forum

Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'

Re: Bad Mood : Falcon030 'Doom'