Bad Mood : Falcon030 'Doom'

[viking272]

Damage & pickup indicators, transparent weapon, free look, speed. Looks awesome!

I am so going to invest so much time in playing this!!

[dma]

What does feel wrong is playing [in free look] for a time and then switching it off - feels like your neck is in a plaster cast and you can't look up to find a source of danger

Eheh, so true !

wait. is free look possible in PC Doom?

In some alternate engines it is yes, ZDoom for example.

[calimero]

^
Res lines are pick up indicator?

Maybe it would be better to a hurt indicator

And maybe yellow to be pickup...

[Zamuel_a]

Can it run Heretic or Hexen WADs? In those games you could look up and down.

[dml]

Can it run Heretic or Hexen WADs? In those games you could look up and down.

The viewer used to work with those WADs so the engine should be ok. The game code is specifically Doom though. The engine would need to be joined to the Hexen (etc) game code to 'play' it, with similar edits all over the place.

Implementing freelook in GL ports like ZDoom is probably a breeze because it only affects the view transform. The GL layer handles texture mapping, usually via HW but it's done for you in any case.

In a software based Doom engine its a bit more tricky because Doom makes certain assumptions about the horizon being in the middle of the view and the tables used for texture mapping are based on that.

When I first tested freelook the texture horizon remained 'fixed' while the walls and floors moved up and down. It is a little bit fortunate that BMEngine works differently from Doom in this part because it made offsetting the plane easier. The hardest part wasn't so much code, but getting my head around the transform and which terms were involved... the code changes amounted to a few instructions.

[dma]

Indeed true that free-look in a non "real 3D translated" engine is much more of an achievement.

[dml]

Indeed true that free-look in a non "real 3D translated" engine is much more of an achievement.

There are a few more Falcon goodies on the way! Just checked in a few today.

[dml]

So BadMooD recently gained a few extras:

- 10 new intermission screens courtesy of a recent project contributor (as with any contributor, will be appropriately credited when the material first appears in the build!)
- recognizes it's own custom WAD file - i.e. containing the ^BADMOOD marker and uses some Atari-specific material when found
- pulls & displays fullscreen images from said WAD file (416x240 @ VGA, or 384x270 @ RGB). this applies to title screen, intermission screens, credit screen - maybe some other screens later
- can use truecolour PNGs for opaque game graphics, thanks to GT Turbo's decoder. PNG is already supported by modern WAD tools for ZDoom etc.
- freelook, as shown in the last video
- some new shaders and code to make adding more a bit easier
- higher quality colour remapping for textures
- several new optimizations. runs a bit quicker.

New stuff on the way:

- music is in development (direct MIDI replay of original Doom tracks, via the ports - again to be credited to the author when it goes 'live')
- will do a custom TC background/border tile for reduced game window plus a few other cosmetic things
- 'repainted' levels
- going to attempt some new/variated monsters, but will see how that goes

[NGF]

As a complement, would it be possible for soundchip player to play chiptune versions of the doom music if no midi hardware is available? Or would it eat too much cpu?

[dml]

As a complement, would it be possible for soundchip player to play chiptune versions of the doom music if no midi hardware is available? Or would it eat too much cpu?

Yes, it would be *great* to have this.

Last year I was sent a short test by track (by PeP) made with one of the YM trackers, just to try integrating music alongside the SFX system already in the game. The track was great, but didn't work in-situ with the SFX system running.

The exported YM music file has the YM player code embedded in it, and when activated it changes some hardware settings which halts my SFX replay code, causing a conflict. I hacked around with it for a while trying to patch out the hardware changes from the embedded player but couldn't get both of them to work together. It was one or the other. In principle they should run side by side because the SFX uses Codec, while music uses YM.

So I need to contact the tracker author to see if there is a hands-off replay routine which can be called alongside the SFX code without fiddling with hardware settings outside the YM. Once I have that, or once I know how to export the track without those side effects then it should be fine.


I didn't try output from any other trackers because I don't know that anyone is ready and willing to write music for the game using any of those. If that changes anytime (if anyone volunteers to do full tracks with a specific tracker), I'll return to the code and figure out a way to get it working for sure.


I did spend time on a MIDI -> YM convertor at one point but I'm not sure how much more effort I would invest there - it's very difficult to generate anything worthwhile, esp. without writing a YM tracker backend with decent fx etc. - a huge job and I just don't have time for it (...and good results not assured - given that we're dealing with existing tracks. It could still sound really bad!).

The only way good YM tracks will be produced is for artists to create them the hard way, by hand. If we're very lucky one or more will be interested to write some for us

[Eero Tamminen]

As a complement, would it be possible for soundchip player to play chiptune versions of the doom music if no midi hardware is available? Or would it eat too much cpu?

The point of the MIDI playback is that the MIDI music is already included in Doom's WAD files (WAD format uses a subset of the MIDI standard for music). For chiptune to make sense, somebody would need to compose music suitable to some WAD and then insert it into WAD. In addition to BadMood having support for playing it.

As I'm fairly sure chiptunes take more CPU, and I have synthetizer, I myself prefer MIDI & more speed.

[dml]

While YM would take more time than MIDI ***, they should both be low costs compared with everything else that's going on (incl. SFX mixing etc)

The real challenge with with YM is firstly writing the music - and using a replay rout (tracker) that can work beside the SFX/codec method.


It's been mentioned here before, but replay can also be done by recording all YM activity at a sufficiently high rate and blasting the registers with raw data, using rudimentary compression. This can be done as a last resort but it's more or less wasteful in some direction, so it's probably better to get the native replay code working if possible - maybe also fewer glitches in the replay.


*** actually this isn't certain either - depends on what the MIDI rout ends up implementing, and what the YM rout would be doing e.g. buzzers, digi fx etc.

[dml]

For chiptune to make sense, somebody would need to compose music suitable to some WAD and then insert it into WAD. In addition to BadMood having support for playing it.

Yes, exactly - it's the best way to get the original music to play and sound good at the same time. And the Falcon has MIDI ports

[dml]

Found several significant optimizations to the engine. Two I have tried, the others not yet tried but worth exploring.

Here's one of them:

The DSP-based texturing code used for floors and ceilings has the benefit that plotting pixels is really quick compared with any other method. However it's also been shown that sending textures to the DSP for this to work is expensive, and the cost mounts up with active texture count. The situation gets worse as the window is shrunk or low detail modes are used, since the pixel area gets smaller but the texture transfer size remains fixed.

A 64x64 tile contains 4096 pixels, and packed into 5:5:5 format for transfer, thats still 1365 words. The palette lighting table is 32 colours x 32 lightlevels, which is another 1024 words. So that's 2389 words or nearly 5kb per texture.

If there are 12 textures active in one scene, that's nearly 60k of stuff to send to the DSP every frame and this cost shows clearly on profiles. A profile I did at the weekend (albeit, on a complicated map) showed texturing uploads competing with floor pixel drawing, at 16% each of total drawing time with default LOD and window size. It would be nice to squash that without complicating it further.

The engine already collapses drawing states for surfaces with similar properties so they get drawn at once. Surfaces also get generated in front-to-back order because they are a side effect of the BSP based scene traversal algorithm. This means the first floor and ceiling surfaces drawn are the nearest surfaces to the viewer.

Because of perspective effects, the nearest surfaces tend to have the largest pixel area onscreen. In fact, with 12 textures active, 1 or 2 textures make up 95% of all pixels drawn.

This suggests it might be a good idea to just limit the number of DSP texture uploads to (for example) 2 per frame, drawing 95% of whats needed and then switching down to a slower texturing scheme for all of the remaining textures, needing no texture transfers.

This seems to work in practice, and is faster measurably. Benefits of both worlds.


The drawback just now is that only the DSP texturing method can currently do mipmapping/filtering, whereas the other 2 methods (full CPU and CPU/DSP hybrid scheme where DSP does texture address generation only) can only draw from the fixed 64x64 texture size. So some surfaces appear to boil and visibly toggle between drawing methods depending on the viewpoint. This is fixable and doesn't make the optimization any less worthwhile.

[dml]

Have reworked some of the DSP code, optimizing a few important bits and freeing up about 1k of ram (which is a lot to free up, in this program).

With all the recent changes combined, about half of the Doom II maps are playable, with some of the rest being 'tolerable' (a few probably still just too expensive for 16mhz). All of the first 3 episodes of Doom 1 are quite playable. Some of the episode 4 maps are becoming playable.

Going to stop this now and return to finishing the content.

[dhedberg]

Found several significant optimizations to the engine. Two I have tried, the others not yet tried but worth exploring.

Here's one of them:
[...]
This seems to work in practice, and is faster measurably. Benefits of both worlds.

Interesting read and great detective work! So if I understand you correctly, you've already implemented the described optimization? Great!
What about the other ones? Are they of the same magnitude?

Optimizing code is one of the parts of programming that I enjoy the most... at least when it pays off. Hunting down bugs is another of my favorite tasks! May sound odd, but a "good" bug is a great challenge and can be quite a mystery until you find out the cause of it. As a bonus you often learn a lot along the way.

Thanks for keeping us updated!

--Daniel

[LaceySnr]

Hunting down bugs is another of my favorite tasks! May sound odd, but a "good" bug is a great challenge and can be quite a mystery until you find out the cause of it. As a bonus you often learn a lot along the way.

I was thinking you must be some kind of sick person, but actually you make a great argument for hunting bugs being a good think... going to have to start thinking of them more in that light!

[dma]

Making the brain work, is indeed always satisfying.

[dml]

Hi!

Interesting read and great detective work! So if I understand you correctly, you've already implemented the described optimization? Great!
What about the other ones? Are they of the same magnitude?

I have tried to do a number of others - some hard to measure the impact on their own. In theory they should help but sometimes it takes 2-3 optimizations combined to 'unblock' something suddenly when you have 2 processors synchronizing on different problems. So I try to estimate what should work and what isn't worthwhile and fix as much as possible in related chunks. Over time it gets gradually better.

The other big change recently was modifying the floor defragmentation process which occurs every time the ceiling or floor render state changes. Spans of floor surface generated by the vertical spaces between walls collect in x: memory as zones, and occasionally need processed into y: memory and adjacent ones joined together for efficient drawing.

Since this process keeps the DSP busy in bursts, it's bad to have the CPU wait for it to finish so it can store the tracking info on the zone begin/end for drawing later (the CPU renderstate stack keeps track of where the zones are in DSP ram so it can ask for them in renderstate-order to minimize texture switching etc.).

So a long time ago I rearranged it to work asynchronously. e.g.

Code: Select all

 CPU::begin_next_zone        DSP::get_next_zone
                             DSP::defrag_prev_zone
 CPU::wait_result (long spin...............)
 CPU::get_prev_zone          DSP::send_prev_zone

became more like this...

Code: Select all

 CPU::begin_next_zone        DSP::get_next_zone
 CPU::wait_result (short spin..)
 CPU::get_prev_zone          DSP::send_prev_zone
                             DSP::defrag_prev_zone

However this was only good if floor or ceiling changed one at a time. If both changed at once, it still blocked for the 2nd exchange, since it was a single pipe. So I recently changed it again to use separate pipes for floor and ceiling, so each exchange sends C and/or F at once, and the zones are defragmented 1 or 2 at a time, asynchronously. The result is no blocking when both states change at the same time, which is quite common.

There is still some blocking but it has more to do with granularity of work and the DSP being needed for other jobs before other stuff is finished - but its difficult to make a truly clean unidirectional rendering pipeline for this kind of engine.

There are too many ways in which drawing/committing item A to the framebuffer results in future item B becoming invisible - this sort of feedback is efficient for hiding content but inefficient for asynchronous processing. This is probably the best I can do now without starting again from scratch and doing lots of things differently.

Optimizing code is one of the parts of programming that I enjoy the most... at least when it pays off. Hunting down bugs is another of my favorite tasks! May sound odd, but a "good" bug is a great challenge and can be quite a mystery until you find out the cause of it. As a bonus you often learn a lot along the way.

Yes optimizing and finding bugs are both interesting and good practise - I like to hunt bugs by looking at the context/effect, and scanning the source by eye for probable cause. It's a good way to learn to code defensively, carefully and with forward planning. Resorting to a debugger every time can end up taking longer and doesn't really help you write better code (took me years to learn that better code and tidier code often agree but are not the same thing!). Low level debugging teaches you other things instead of course, so its good to do a bit of both in a balanced way. I think relying solidly on debuggers definitely has a hidden cost long term though so it's often a last resort for me or when studying the source is not practical.

I probably find/fix 80%+ of my bugs by thinking about how I wrote something and how it could go awry. I might make changes to source to prove an effect or test a condition, and if that fails or if the effect is too obscure will start tracing code.

Thanks for keeping us updated!

And thanks for your interest

[Eero Tamminen]

I did a quick try of BM code from few days ago with worst frame profiling. This was with "Mountain Factory" WAD, TIMEBASE=4, GCC 2.x.

I'm not sure whether profiling started at suitable place, but at least according to it, texture cache loading is hugely expensive (frame render part is >6s):

Code: Select all

Time spent in profile = 6.82118s.
...
Visits/calls:
  66.39%  66.39%       49330     49330   correct_element
   4.66%                3462             repair_start
   4.66%                3461             repair_next
   4.41%   4.41%        3278      3278   hash_insert
   4.40%                3272             repair_found
   2.76%                2049             set256_2
   1.84%                1364             init_font
...
Executed instructions:
  60.94%  61.02%  61.02%     9773310   9785997   9785997   _D_QuantCubeFillLookupTable
   9.84%   9.85%  71.10%     1577746   1580275  11403034   _D_ImageQuantizerCreateOptimalPa
   9.41%   9.43%   9.43%     1509782   1511580   1511580   D_TextureMipGen_8_16
   5.85%   5.86%   5.86%      937536    939831    939831   correct_element
   3.77%   3.77%   3.77%      604458    605017    605017   D_PatchAnalyseMask
   1.95%   1.95%   7.79%      312006    312725   1249091   create_local_palette_64levels
   1.78%   1.79%   1.79%      286269    286704    286704   _D_ImageQuantizerAddPixelsToHist
   1.38%   1.38%   1.38%      221733    222114    222114   D_PatchTargetGetApproxRGB
   0.90%   0.91%   1.73%      145040    145344    277961   D_CacheFlushPartial
   0.58%                       93266                       R_SCShader_Masked_2x1
   0.57%                       91928                       D_PatchRender_8_8_CM

Used cycles:
  55.19%  55.40%  55.40%    60388228  60620672  60620672   _D_QuantCubeFillLookupTable
  11.54%  11.58%  67.26%    12633060  12674768  73600892   _D_ImageQuantizerCreateOptimalPa
  10.48%  10.52%  10.52%    11468980  11511368  11511368   correct_element
   8.27%   8.30%   8.30%     9049460   9083024   9083024   D_TextureMipGen_8_16
   2.51%   2.52%   2.52%     2747260   2757404   2757404   D_PatchAnalyseMask
   2.05%   2.06%  12.54%     2241388   2253772  13723048   create_local_palette_64levels
   1.46%   1.47%   1.47%     1595940   1603256   1603256   _D_ImageQuantizerAddPixelsToHist
   1.26%   1.27%   2.32%     1382872   1388608   2540632   D_CacheFlushPartial
   1.08%   1.09%   1.09%     1186688   1193176   1193176   D_PatchTargetGetApproxRGB
   1.07%                     1175992                       set256_2
   0.59%                      647448                       R_SCShader_Masked_2x1

Instruction cache misses:
  16.76%  23.70%  23.70%       14388     20346     20346   _D_QuantCubeFillLookupTable
   9.96%                        8553                       init_font
   4.47%   4.65%  17.60%        3842      3990     15109   D_CacheFlushPartial
   2.94%   3.99%   3.99%        2523      3429      3429   D_TextureMipGen_8_16
   2.92%                        2504                       correct_element
   2.81%   2.81%   2.86%        2409      2412      2452   internal_allocate
   2.76%                        2370                       unlink_chunk
   2.71%   2.71%   2.71%        2324      2330      2330   _D_QuantCubeCalcColor
   2.21%   2.22%  12.82%        1900      1905     11005   deallocate_chunk
   2.21%   7.38%  10.60%        1899      6338      9100   internal_deallocate
   2.07%   2.36%  34.33%        1775      2023     29479   _D_ImageQuantizerCreateOptimalPa
...

Can init_font be called during game?

Attached is a callgraph.

[dml]

Hi,

..yes all of that is stuff offline, once-only texture processing. It happens when new WAD material is identified and goes into the BMC/ cache. It isn't repeated for a texture in future and isn't associated with normal loading time.

correct_element is gamma correction for pixel r,g,b, based on gamma commandline arg

(BTW it isn't really calling 'init_font' it's a profiling alasing thing, caused by labels disappearing due to grep. It's actually some other code nearby)

[dml]

Eero, after recent tests I can see that worst-frame profiling will be needed again but for for some specific things.

Optimizing stuff has made the 'shooting' cost more obvious and in some cases it drops multiple frames on big maps when the first few enemies are shot.

I didn't check yet what is causing this - I don't think its loading because it can be repeated on the same map. Maybe some strange nonlinear map overhead for LineAttack operations or sending audio alerts round the map. It's probably the biggest single remaining performance problem anyway.

[Eero Tamminen]

Eero, after recent tests I can see that worst-frame profiling will be needed again but for for some specific things.

Ok, I will look into fixing the script to setup worst frame breakpoints better with the latest BM code. Might not happen this week though, maybe next week.

[calimero]

Very, very cool video!

Classic Game Postmortem - DOOM (2011)

Speakers: Tom Hall and John Romero
history of making Doom, first handed

http://www.gdcvault.com/play/1014627/Cl ... Postmortem


trivia: they mention how colors in distant gets darker; if they used Falcon back than, they would get perfect shadows and backgrounds

[dml]

GDC - was lucky enough to attend that conference a few times. The most interesting bits were not always the best covered.

I think I still have the 1999 or 2000 proceedings 'tome' somewhere in storage...

Very, very cool video!

Classic Game Postmortem - DOOM (2011)

Speakers: Tom Hall and John Romero
history of making Doom, first handed

http://www.gdcvault.com/play/1014627/Cl ... Postmortem

trivia: they mention how colors in distant gets darker; if they used Falcon back than, they would get perfect shadows and backgrounds