Bad Mood : Falcon030 'Doom'

[npomarede]

Finally, managed to assemble a video:

http://www.youtube.com/watch?v=MUWJSUiP ... e=youtu.be

This is using all the latest code, running the full game with a custom made map for testing the liquids. It has all optimizations enabled and is running on a stock 14MB machine. No emulation and no acceleration.

The window is 224 pixels wide, using the default 320-pixel video mode with a 512-pixel virtual linewidth. This is what causes the menus to mess up (no it hasn't crashed).

I turned the status bar off because it's still in C and slows everything else down.

Recorded from VGA monitor using a camcorder.

Really nice results, very smooth ! (btw, what is the music ? It sounds very demo-ish)

[dml]

Really nice results, very smooth !

Thanks!

(btw, what is the music ? It sounds very demo-ish)

a sharp scene-sense you have there.

Look for 'Beta' by Still.

[CiH]

Indeed, where did you get the music from?

Loved the video, it looks like a selection of soups sold at a particularly dystopian IKEA store restaurant!

Press on!

[npomarede]

Look for 'Beta' by Still.

[offtopic] Yes, I remember this demo, really great direction and true artistic composition !
For those interested, have a look here http://www.youtube.com/watch?v=fUawxurEQJU
[/offtopic]

[kristjanga]

I'm sure JagPad support would be easy. However I don't have one If I can't do it, I'm sure somebody else can.

I will ask the forum for a talent to do this.
A JagPad / Powerpad support would be nice and so Atari-ish that when i think about it it is kinda must

[kristjanga]

I'm sure JagPad support would be easy. However I don't have one If I can't do it, I'm sure somebody else can.

I got this from Simon sunnyboy

Adapt as necessary:

Code: Select all

;
; Jagpad Routine
; reading Jagpad A & B
; from some code found on the web
;
; (c) 2007 by Simon Sunnyboy / Paradize
;
; Returns :-
; pad0 = c1numpad,c1midbuts,c1firebuts,c1joypad
; pad1 = c2numpad,c2midbuts,c2firebuts,c2joypad
; contain bitwise representation in the form
; 1 4 7 * 3 6 9 # 2 5 8 0 o p c b a r l d u
; where o=option, p=pause, a/b/c=fire, r/l/d/u=directions. and rest are the phone pad.
;

SUPER
EXPORT Jagpad_ReadPads
EXPORT JagpadA
EXPORT JagpadB

JagpadA:
	move.l pad0,d0
	rts
	
JagpadB:
	move.l pad1,d0
	rts

; jagpad return values after call
pad0:               DC.L 0
pad1:               DC.L 0

; local variables
c1numpad:           DC.L 1
c2numpad:           DC.L 1

Jagpad_ReadPads:
                    movem.l D0-A2,-(SP)
                    clr.l   D0
                    clr.l   D1
                    clr.l   D2
                    clr.l   D3
                    clr.l   D4
                    clr.l   D5
                    clr.l   D6
                    clr.l   D7
                    movea.l #0,A0
                    lea     c1numpad(PC),A1
                    lea     c2numpad(PC),A2
                    move.l  #0,(A1)
                    move.l  #0,(A2)

* read row 1

                    move.w  #$FFEE,D0
                    bsr     read_controller

                    move.w  D1,D0
                    andi.w  #1,D0
                    move.b  D0,D3

                    move.w  D1,D0
                    andi.w  #2,D0
                    lsr.w   #1,D0
                    move.b  D0,D4

                    move.w  D1,D0
                    andi.w  #4,D0
                    lsr.w   #2,D0
                    move.b  D0,D6

                    move.w  D1,D0
                    andi.w  #8,D0
                    lsr.w   #3,D0
                    move.b  D0,D7

                    move.w  D2,D0
                    andi.w  #$0F00,D0
                    lsr.w   #8,D0
                    move.b  D0,D5

                    move.w  D2,D0
                    andi.w  #$F000,D0
                    lsr.w   #8,D0
                    lsr.w   #4,D0
                    movea.l D0,A0

* read row 2

                    move.w  #$FFDD,D0
                    bsr     read_controller

                    move.w  D1,D0
                    andi.w  #2,D0
                    or.b    D0,D4

                    move.w  D1,D0
                    andi.w  #8,D0
                    lsr.w   #2,D0
                    or.b    D0,D7

                    move.w  D2,D0
                    andi.w  #$0F00,D0
                    lsr.w   #8,D0
                    move.w  D0,(A1)

                    move.w  D2,D0
                    andi.w  #$F000,D0
                    lsr.w   #8,D0
                    lsr.w   #4,D0
                    move.w  D0,(A2)

* read row 3

                    move.w  #$FFBB,D0
                    bsr     read_controller

                    move.w  D1,D0
                    andi.w  #2,D0
                    lsl.w   #1,D0
                    or.b    D0,D4

                    move.w  D1,D0
                    andi.w  #8,D0
                    lsr.w   #1,D0
                    or.b    D0,D7

                    move.w  D2,D0
                    andi.w  #$0F00,D0
                    lsr.w   #4,D0
                    or.l    D0,(A1)

                    move.w  D2,D0
                    andi.w  #$F000,D0
                    lsr.w   #8,D0
                    or.l    D0,(A2)

* read row 4

                    move.w  #$FF77,D0
                    bsr     read_controller

                    move.w  D1,D0
                    andi.w  #2,D0
                    or.b    D0,D3

                    move.w  D1,D0
                    andi.w  #8,D0
                    lsr.w   #2,D0
                    or.b    D0,D6

                    move.w  D2,D0
                    andi.w  #$0F00,D0
                    or.l    D0,(A1)

                    move.w  D2,D0
                    andi.w  #$F000,D0
                    lsr.w   #4,D0
                    or.l    D0,(A2)

                    lsl.l   #7,D3
                    lsl.l   #4,D4
                    or.l    D4,D3
                    or.l    D5,D3

                    lsl.l   #7,D6
                    lsl.l   #4,D7
                    or.l    D7,D6
                    move.l  A0,D2
                    or.l    D2,D6

                    move.l  (A1),D2         ; d2 = c1numpad
                    lsl.l   #5,D2
                    move.l  (A2),D5         ; d5 = c2numpad
                    lsl.l   #5,D5

                    or.l    D2,D3
                    or.l    D5,D6

                    move.l  D3,D4
                    andi.l  #$1E00000,D4
                    lsr.l   #4,D4
                    andi.l  #$1FFFF,D3
                    or.l    D4,D3

                    move.l  D6,D4
                    andi.l  #$1E00000,D4
                    lsr.l   #4,D4
                    andi.l  #$1FFFF,D6
                    or.l    D4,D6

                    lea     pad0(PC),A0
                    move.l  D3,(A0)
                    move.l  D6,4(A0)
                    movem.l (SP)+,D0-A2
                    rts

read_controller:
                    move.w  D0,$FF9202
                    move.w  $FF9200,D1
                    move.w  $FF9202,D2

                    andi.w  #$0F,D1
                    andi.w  #$FF00,D2

                    not.w   D1
                    not.w   D2

                    rts
                  
                    END

[dml]

I got this from Simon sunnyboy

Great, I'll try to use it - I left a note in the other thread.

Cheers

[nativ]

Look for 'Beta' by Still.

[offtopic] Yes, I remember this demo, really great direction and true artistic composition !
For those interested, have a look here http://www.youtube.com/watch?v=fUawxurEQJU
[/offtopic]

The artwork in that demo is by the 'Optical Artist' Victor Vaserely.

https://www.google.co.uk/search?q=victo ... &gws_rd=cr

funny thing.. I was trying to compile some images for an ST Vaserely slideshow just before they did that! Got to up my game on that I guess!

[dml]

The artwork in that demo is by the 'Optical Artist' Victor Vaserely.
https://www.google.co.uk/search?q=victo ... &gws_rd=cr
funny thing.. I was trying to compile some images for an ST Vaserely slideshow just before they did that! Got to up my game on that I guess!

[offtopic]
Well you need to hurry up. Demo coders never sleep.
https://www.youtube.com/watch?v=kytM7Y5dbNI
[/offtopic]

[dml]

[tech talk alert]

Tonight I decided to tackle an engine optimization that I have dodged for a long time, because I knew it was going to be pain.

And I was right. But it was worse than that.

There's a large bit of DSP code in there which takes all the empty space between wall edges and makes floors out of them, as separate groups of linked spans in an intermediate buffer. This buffer grows as more floor surfaces are generated with the same state. When the engine encounters an unavoidable change in floor state (texture, light level, height) it 'commits' this intermediate buffer to a permanent buffer which is used for drawing at the end of the scene. This 'commit' is horribly complicated - it takes the spans sharing the same state and uses a series of edge indexing buffers to defragment everything into as few spans as possible so the CPU doesn't have to work so hard. It does this very well but it's really hard to follow the code, and hard to modify it. I wrote it when I was 25 or so, and it shows. :-z

One of the problems with this code, and the permanent buffer used for storing the optimized spans, is the fact that the buffer isn't big enough to hold everything needed to draw stuff later. Well it is big enough, for most maps - but some more complex maps can cause overrun. It also means less space for code and other things which might be added.

So the original code would transmit some stuff (number of spans per line per zone) back to the CPU while defragmenting. This didn't cost the DSP much since the CPU had nothing else to do at the time, except wait for that data. But it does mean the CPU can't continue on with something else. It has to wait there until that process finishes. This amounts to about 3-5% of total time, occasionally more.


Tonight I found a way to encode the same information in the existing buffer without growing it. There are only two bits free in each span (!) but it was enough to do it. I use one bit as a 'newline' bit which tells the CPU to advance to the next line. It then doesn't need to count spans on each line. This added further complications because a stop/exit code becomes necessary, and since some lines can be empty, an absolute line addressing code to 'jump' to the next active line when there was a gap. So this was all very painful to change inside the original DSP code with no address registers free etc..

The spanbuffer is encoded as word pairs like this:

word #1 [newline:1][z:23]
word #2 [unused:1][luma:5][xsize:9][xpos:9]

It now seems to be working as before, without transmitting that superfluous data back to the CPU. It isn't faster, but the CPU function is still waiting for the DSP. I'm going to try to remove that completely to see if some of that 3-5% can be soaked up. Even if the improvement isn't significant it will mean stuff can be reorganised to maximise overlap while that process is busy.

[dml]

Well that worked and it's a bit faster. Improvement varies but it's consistently a few % points anyway and the responsible CPU code has dropped into the background. It hasn't gone away unfortunately, because sometimes both ceiling and floor state change at once and two zones need flushed, so concurrency can only be gained with the 2nd of the two.

There's a bit of cleanup / optimization still to do and I'll move on to something else.

I also rewrote P_PointInSubsector in asm and did some more work on P_CheckSight as and when I had time. The gains in each case are small but it's all adding up.

[EvilFranky]

Is this turning into increased FPS Doug?

And are you keeping the FPS indicator included in the release?

Sent from my Nexus 4 using Tapatalk 4

[dma]

Congrats on the deep code refactoring Doug.
It's indeed like getting deep down in the mine. Not sure what you'll find, have to do, and in which state you'll get out of it, or even if you will...

Also sure that with the possible complexity on level design, such an engine is miles away from any "constant framerate" gaming vision.

(by the way, if some would like to understand the BSP system, i found this explanation pretty good)
(and it seems DOOM rendering engine actually limits itself on some maximums, not to fall into ridiculous "still shots" FPS count i guess (but also crashing if reaching some of those...))

[dml]

Is this turning into increased FPS Doug?

One small bit at a time. There are 30+ functions in the game which each take 1%, so making a 'big optimization' isn't really possible. It has to be handled as lots of small ones and it takes time.

And are you keeping the FPS indicator included in the release?

Not by default, as the FPS indicator is part of a much bigger builtin sampling profiler and it's all implemented using FPU to detect very small changes in speed. It will be left in as an option which has to be deliberately turned on.

Congrats on the deep code refactoring Doug.
It's indeed like getting deep down in the mine. Not sure what you'll find, have to do, and in which state you'll get out of it, or even if you will...

Also sure that with the possible complexity on level design, such an engine is miles away from any "constant framerate" gaming vision.

Yes it's highly variable, although I've had a few more ideas on how that could be managed down a bit, which I will try before doing more with content.

(by the way, if some would like to understand the BSP system, i found this explanation pretty good)

This is quite a good overview of the various IdTech systems - it's just enough detail to see how it works but obviously it gets a lot more complicated in some areas, esp. with sprites (which I'm sure I have missed some details about as it is).

(and it seems DOOM rendering engine actually limits itself on some maximums, not to fall into ridiculous "still shots" FPS count i guess (but also crashing if reaching some of those...))

Yes there were plenty of limits in the original - static buffers and MAX_THIS_AND_THAT but I don't think many of them capped framerate/work directly. They mainly crashed the engine if they were exceeded. I think there was at least one which limits drawing distance (which BadMood does not) so I might take a closer look at that. Hard to imagine it happening often though without looking ugly! Still I should check it before discounting anything that might help with speed.

[dml]

Have applied another optimization tonight which speeds up BM's renderer, a sort of shortcut for scanning of the occlusion bitfield for sections of wall which are invisible. The cost is now much lower for 90% of the occlusion scan with a one-time cost for the first visible column in each visible section of an occluded wall. Again not a massive speedup but e1m2 startpoint rose from 6.13fps to 6.26fps which translates to about 2% gain.

The worst part of rendering now is processing of visible sprite spans (not drawing, but sorting/clipping bits of sprite against walls, doors etc). This (and the sky shader) are probably the last rendering costs I'll look at, since everything else is pretty well optimized now.

[EDIT]

Have confirmed that the sky shading code is really bad. It qualifies as 'debug class' code and is probably part of the reason why some of the Doom2 open-air maps are so slow. It has been giving high readings in the profiler but I also notice running around map11 that it always gets noticably slower as soon as the sky appears. Definitely to be replaced.

Part of the reason for slowness is that it's rendering a visplane (spans or rows) using a type of wall texture (stored as columns). That doesn't help. It's also all CPU-side and has two 64bit multiplies per span drawn (!). I'm going to fix this next.

[Omikronman]

You can save another 2 % by turning off the mouse pointer via interrupt IPL 4.

[dml]

You can save another 2 % by turning off the mouse pointer via interrupt IPL 4.

That's the kind of thing that could actually happen

[Eero Tamminen]

You can save another 2 % by turning off the mouse pointer via interrupt IPL 4.

I don't remember seeing anything about that in Hatari profiles of BadMood. Where that 2% of CPU usage should show up?

[dml]

[coder's rant]

Very evil bug tonight. One of those 'sinking feeling' heisenbugs that force you to turn over every pebble and make a complete mess of the sourcecode in the process.

After optimizing a bunch of things on both CPU and DSP side, I noticed some corruption appearing under the rendering window on occasion. I found it could be repeated only by loading up the game and starting a specific map, shrinking the window a few steps, then rotating left.

The corruption happens immediately the sky appears onscreen, and more appears as the sky area moves across the screen. However if you resize the window, or turn back to face the original direction, it stops. And it won't come back unless you reload the game. Scary.

The corruption part wasn't so bad - it was the strange 'self-repairing' effect that troubled me, requiring a restart to get it to happen again. I couldn't find anything related to sky rendering which looked uninitialized.


Some debugging later, I followed it back through recent changes to the floor/sky spanbuffer, which now has escape codes for newline and absolute line hops. I stabbed a guess that the absolute line code was happening when the corruption happened. And it was. But that's fiddly DSP stuff so debugging it is going to hurt.

Tracing it farther back, I find the reserved 'escape code' ($800000) for absolute line hop is being aliased by a real z-value in the spanbuffer (z=0 + newline). But z=0 isn't legal in there - so more digging and tracing... into some very old perspective calculation stuff which I haven't touched in ages. There I find a divide by zero. But it's not coming from a divide, rather from the divide cache.

As it turns out the divide cache behaves differently when you div by zero, because I didn't implement that case. A normal divide just gives you back overflow and it's always the same. The divide cache however returns a 'hit' on zero, as an uninitialized cache tag, and returns... zero. Which isn't right. But it's worse than that - it only returns zero for a while. Other things using it can cause the cache to miss, and force the answer for zero-div to eventually be corrected, and permanently.

So this is why the problem repaired itself after moving around a bit. The cache was slowly becoming initialized for the zero case, and until that happened, the spanbuffer was getting escape codes and the CPU was being told to draw rubbish across random areas of RAM.


Note to self: when using a divide cache, make sure the zero tag maps to overflow even if you don't actually plan to divide by zero at any time.

[dml]

Apart from that bug, I did some useful things tonight on the project:

- replaced the sky shader with a *much* faster implementation (only one thing left to do - 'fullbright' texture format will save a few %)
- reoriented the sky textures to be row-major instead of column-major format, for faster addressing
- reduced the number of state changes in the sky (lighting, height differences don't affect sky visplanes)
- optimized visplane flushing to ignore floor surfaces above player's eye, ceiling surfaces below player's eye. this was a good one. couldn't figure out a way to do it before now, but had an idea which worked this time.
- a few other optimizations and bugfixes

The game is definitely playable now for most Doom maps and quite a few Doom II as well - providing the AI/gametick is managed with the changes I applied (the ones that completely break demo replay). With a quarter-sized window I was getting something like 8-12 FPS in Doom II's map01 while playing, which is fine.

The status bar is still off, still needs redone. As do the weapon overlays. Probably next.

[dml]

You can save another 2 % by turning off the mouse pointer via interrupt IPL 4.

I don't remember seeing anything about that in Hatari profiles of BadMood. Where that 2% of CPU usage should show up?

I believe it was a bee joke. ...Or he's deep in MonTT and finding embarrassing faults in my code...

[EvilFranky]

Any new gameplay vids Doug?

Sent from my Nexus 4 using Tapatalk 4

[dml]

Any new gameplay vids Doug?

I'll try to make one soon, but too late tonight.

Should have something during next week I think.

[EvilFranky]

Excellent, looking forward to it

Sent from my Nexus 4 using Tapatalk 2

[dml]

Yesterday I fixed a few more bugs and started looking at ways to the cut line-of-sight testing rate as low as possible. Started by collecting some metrics on what Doom does normally.

On e2m2 startpoint I counted something like 300-500 raytests per second and at least 4000 AI ticks per second. So it shouldn't be surprising the game code runs slowly (the raytests alone cross multiple sectors of the map, each testing against multiple walls until the ray is stopped or exhausts)

A quick modification to the line-of-sight tester provides a sort of cache which returns the same result for at least the next second. This is just a hack but was helpful for measurement. This cut down raytests to under 100 per second. Side effects on gameplay were noticed but mainly in the form of exploding barrels not always affecting enemies (these use raytests to apply immediate radius damage and rely on accurate, immediate raytest results - but don't happen too often) and other problems of that kind. The enemies can take up to a second to recognize the player worst case but during normal gameplay this only provides a surprise advantage when turning corners, and only vs weaker enemies. It also seemed a bit more realistic that they don't notice you on the very first frame.

I also checked the effectiveness of the AI management optimizations and with those enabled, the AI tick rate drops from 4000 to about 100 per second. The side effects from this are more serious (enemies get more 'drunk' with increasing distance from player, with them being nearly asleep at the horizon) and will take a bit more work to get this working well for all enemy types, but it's worthwhile because it saves so much time.

Examples of problems with this are fireballs and related projectiles, which need to cross the entire map at constant speed. These were excluded. Pink monsters and skulls charge the player and also need special care. Most of the other enemies work quite well with proximity-managed thinking and it actually stops them all from hearing, seeking and collecting around the player in a short timeframe in open spaces, forcing the player to go find them. The side effects at short distances are less noticable - enemies still chase the player unless you get far enough away and they temporarily lose interest.