Quake 2 on Falcon030

[dml]

In the last iteration I got all the per-polygon-span math down to 4 nops exposure on the CPU side - might get it down as far as 3 but probably not less. The jump tower can be made bigger but the face setup code is still a too bit big to make that change.

So the whole rendering process is now very compact. It's probably not going to improve a lot more for this pixel/span drawing stage on the 16mhz target. Remaining work will be in the per-face setup, using flat fill mode for small, distant faces and making better use of DSP internal memory + code overlays for geometry & clipping work.

Code: Select all

pix	macro
	move.w		(a4),d7
	move.w		(a5,d7.l*2),(a1)+
	endm

...face setup stuff done previously...
*-------------------------------------------------------*
.spnext:
*-------------------------------------------------------*
	moveq		#8-1,d7
	move.w		(a4),d2
	and.w		d2,d7
	lsr.w		#3,d2		
	lea		-2(a0,d1.l*2),a1 ; remember to compensate for -1 termination test
	move.b		.joff(pc,d7.w),d7
	nop ; some padding exposure to let DSP do some heavy lifting work
	nop
	nop
	nop
.jj	jmp		.jj(pc,d7.w) ; execute jump tower
.xlp:	pix
.j7:	pix
.j6:	pix
.j5:	pix
.j4:	pix
.j3:	pix
.j2:	pix
.j1:	pix
.j0:	dbra		d2,.xlp
	
*-------------------------------------------------------*
.send:	moveq		#1,d1			; receive next span
	add.w		(a4),d1				; dst addr (or -1 to terminate)
	bne.s		.spnext

[Guest]

hi doug do you think duke 3d can happen isnt it the sameish engine as quake???
ive got a heep or custom maps and samples{monty python mostly}
fantastic work!!! cant wait to give it a boot

[Zamuel_a]

hi doug do you think duke 3d can happen isnt it the sameish engine as quake???
ive got a heep or custom maps and samples{monty python mostly}
fantastic work!!! cant wait to give it a boot

The Duke3D engine is much simplier so would be even better on the Atari than Quake, but I remember this was up on discussion some time ago, and the conclusion was that the Build engine in Duke3d was messy and not fun to dig into.

[dml]

Just a very quick update:

After posting that last night, I made a few more simple improvements that actually got rid of all of the padding nops and doubled the size of the jumptower. The impact of this on speed of complex scenery is actually quite good - it spends more time drawing pixels and less flyback time on very short spans.

For now it's partly a negative trade because it meant pushing some other code out of DSP fast memory - slowing other areas down - and causing more cache misses on the CPU side (bad!) but these can be bought back later with other changes. The fact that it is faster eveywhere despite this is a good sign.

Code: Select all

*-------------------------------------------------------*
.spnext:
*-------------------------------------------------------*
	move.w		(a4),d2
	moveq		#16-1,d7
	and.w		d2,d7
	lsr.w		#4,d2		
	lea		-2(a0,d1.l*2),a1
	move.b		.joff(pc,d7.w),d7
	moveq		#1,d1	
.jj:	jmp		.jj(pc,d7.w)

*-------------------------------------------------------*

pix	macro
	move.w		(a4),d7
	move.w		(a5,d7.l*2),(a1)+
	endm

.xlp:	pix
.jf:	pix
.je:	pix
.jd:	pix
.jc:	pix
.jb:	pix
.ja:	pix
.j9:	pix
.j8:	pix
.j7:	pix
.j6:	pix
.j5:	pix
.j4:	pix
.j3:	pix
.j2:	pix
.j1:	pix
.j0:	dbra		d2,.xlp
	
*-------------------------------------------------------*
.send:
	add.w		(a4),d1
	bne		.spnext
*-------------------------------------------------------*
.spdone:
*-------------------------------------------------------*

I expect whats left of this code can be shuffled around and optimized a bit more since the nops have gone, buying a few cycles but I don't know how much room is there to make changes before the DSP begins to stall again. Probably not much.

[dml]

Have done a bit more optimization on the code, especially the CPU side of polygon setup. It's been shrunk to a fraction of the original code. The entire rendering stage fits inside the CPU cache. This will break slightly when different surface types are supported - currently there is only one surface type: textured+lit. Unlit and tiled/transparent surfaces also need a rendering unit. The cost of flipping between these should be low because they aren't very common.

Will soon start converting everything to use overlays so the rendering will be broken into roughly 4 stages:

- scene gather (BSP)
- geometry submission (transform, clipping, edge production)
- scan conversion
- drawing (face setup, span setup, texturemapping)

This lets each stage use the 512(-64) words of internal (fast) program memory on the DSP for the intensive parts. In the current build, only the last stage has a portion of code using fast memory. Everything else is stuck in external memory. This reorg should therefore help quite a bit.

* The internal memory isn't exactly 'faster' than external memory - all 32kwords of DSP memory is zero-wait, but internal is more parallel. The gain is a bit like having a wider bus on a more conventional system. Gain can be anything from 1.x times to nearly 3x faster depending on whats being done. Matrix and vector ops benefit especially. In this project it mainly benefits the scan conversion and texturing stages but it does impact all stages to some extent because there is so much going on.

[Cyrano Jones]

Code: Select all

pix	macro
	move.w		(a4),d7
	move.w		(a5,d7.l*2),(a1)+
	endm

.xlp:	pix
.jf:	pix
.je:	pix
.jd:	pix
.jc:	pix
.jb:	pix
.ja:	pix
.j9:	pix
.j8:	pix
.j7:	pix
.j6:	pix
.j5:	pix
.j4:	pix
.j3:	pix
.j2:	pix
.j1:	pix
.j0:	dbra		d2,.xlp
	
*-------------------------------------------------------*
.send:
	add.w		(a4),d1
	bne		.spnext
*-------------------------------------------------------*
.spdone:
*-------------------------------------------------------*

You could load D7 once outside the Macro and save a bunch of cycles.

[mikro]

You could load D7 once outside the Macro and save a bunch of cycles.

I think 'a4' stands for the dsp host port so unfortunately, he could not :)

[dml]

Here's a (shaky) photo taken from the real 16MHz machine, showing a decent increase in speed over the older version of Hatari I normally use for video capture (speed is typically +1fps on a real machine - so it's around 8fps here in Hatari*)

The TV is showing borders around the normal 320x200 screen area, so the rendered window is using approx 70%+ of the available screen. No chunky modes / fat pixels being used.

photo_f030.jpg

* I haven't been able to get this project to work yet with the new Hatari but once that is solved I'll compare again.

[dml]

I think 'a4' stands for the dsp host port so unfortunately, he could not

Yep that's correct - but sure, if anyone notices 030 optimizations then punt them in my direction Part of the reason for posting code - to make sure I don't miss things (forest vs trees and all that!).

There is a faster way to replace what that macro does - i.e. takes less cycles on 030 side but it makes the flyback a bit more complicated and there is a reasonable chance the DSP won't keep up. Not actually tried it so probably worth a go - but at least 70% chance of not working out

[SteveBagley]

Yep that's correct - but sure, if anyone notices 030 optimizations then punt them in my direction Part of the reason for posting code - to make sure I don't miss things (forest vs trees and all that!).

The one thing that I wondered about is whether you can increase the code density in the cache by reducing pix from 6 bytes to 4 like this:

Code: Select all

add.w (a4),a5
move.w (a5), (a1)+

However, it would require enough spare cycles on the DSP to a) double the returned value, b) be calculate the delta from the previous value and c) that that delta is fits between +/- 32767, and I suspect these would push the idea into being unworkable.

Steve

[dml]

The one thing that I wondered about is whether you can increase the code density in the cache by reducing pix from 6 bytes to 4 like this:

Code: Select all

add.w (a4),a5
move.w (a5), (a1)+

However, it would require enough spare cycles on the DSP to a) double the returned value, b) be calculate the delta from the previous value and c) that that delta is fits between +/- 32767, and I suspect these would push the idea into being unworkable.

Steve

Yes this is a relevant optimization (similar to the one I was referring to but not identical) and the range issue is real - but it is possible to get around the range problem for most of the surfaces in the scene because mipmaps ensure textures shrink with distance. Still, the 'textures' are actually temporary blocks in the surface cache and can be quite a bit larger than the source textures, especially if they get tiled out a bit. So a check needs to be done on each surface block to see if the texture size is within the addressable range for that optimization, and use 2 different code paths. But the check can probably be absorbed easily. Flipping between code paths is probably also not too bad since surfaces are rendered in approx distance order - some grouping will happen for free.

Your method is definitely the faster implementation on the CPU side, but as you noted will increase cost on the DSP side to track the delta if not also doubling the address - the reduction on the CPU side will almost certainly trip up the renderer because they are so closely matched at the moment. There are (IIRC) 22-24 cycles available per pixel on the DSP side and they are mostly used up - maybe 20-22, not exactly sure now but its close.

There's a middle-ground version on the CPU side which is faster than the current one (around 20-21 cycles) but still occupying 6 bytes, and doesn't add cost on the DSP side. But it will probably still trip over.

Anyway - both are worth experimenting with. And sometimes a DSP lucky optimization will allow bigger changes on the CPU side, which might open up an opportunity to use the 4-byte version sometime later!



[EDIT]

Actually there may be a chance to use the 4byte delta version if I can ever get the lower resolution DSP-side to work. i.e. perspective correct every N'th pixel (2 or 4) instead of every pixel.

This seems easy but it's actually pretty hard - mainly because its hard to get uniform timing between every pixel. You end up with lots of pixels quite tightly timed and then the last pixel has a big difference. Some hacks can be done to even it out but it adds more time to the start of the span etc. etc. so after a few tries I abandoned it. Still its worth attacking a few times if it can save enough DSP cycles per pixel to switch the CPU side around!

[dml]

I think I got the special surfaces working now (the corrupt lights/panels). This one was a bit of a puzzle because in Q2 they seem to be put into the surface cache, but have no lightmaps. Some of them tile (like water) so that would use up a lot of surface memory and I was trying to avoid that. The are now rendered direct from the texturemap (without tiling) and with some compensation to fix offsets given that they are not being tiled out into the cache.

This results in semi-working water surfaces. They are correctly shaded but tiling is broken - so I'm going to try a quick solution to the tiling problem which avoids writing a new texturemapper with tiling builtin. This will hopefully get all surfaces looking right except transparency and the sky. I'll leave transparency disabled until after the project has been split into overlays.

The sky is also nearly working - it's shaded correctly but badly warped due to trying to perspective correct at large distances. I'll try to sort that separately. The z-clipping problem is already fixed - by making the nearplane movable (better to move z farther out for big objects to avoid overflows on x,y).

[Scarlettkitten]

Sounds like it's coming along nicely Doug

[dml]

Sounds like it's coming along nicely Doug

Sky is now also nearly working. There are still ugly seams but trying to fix that. It also doesn't quite follow the camera around so it looks a bit like you're moving around inside a big painted cardboard box. Once that is fixed I think it will start to look more convincing.

I recently dug up PCM's POVRay skybox generator from our Wayfarer project so custom skies are going to make an appearance here before long

[Scarlettkitten]

That's fantastic

[calimero]

it is really hard to imagine that these screen shots with light maps and per pixel lightning moving on 16MHz machine!

[AdamK]

We're kind of starved. Video pretty please?

[dml]

Hi,

While there are some new additions and speedups, there are still a few things needing fixed.

The sky needs to track the camera, and needs new images generated with custom palettes. Transparency is broken. There is some glitching of textures. Performance isn't as good as it should be while moving around. Memory inefficiency causes crashes on some maps with sky textures loaded. Textures with long filenames get aliased incorrectly via TOS, messing up some maps.

I'll try to fix a few of these at least and then probably try a new vid.

Early screenshots taken during lunch with mostly working sky/water/lava. Will have another go at the weekend.

[Scarlettkitten]

Wow, drool

[Mindthreat]

*walks into interview*

Hi, did you bring a resume?

*slips disk with Quake 2 for Falcon on it*

-*loads disk*-



Congratulations! You're hired!

[dml]

Hah! If only that were the case

[dml]

The sky is working correctly now, acting as a true camera-centerd skybox and without seams.

I had to make some more changes to support model contexts with separate model matrices during the final rendering pass (previously it only affected the geometry pass) so textured surfaces get transformed correctly by any changes to the camera. i.e. camera is always at the centre of the skybox but it is mobile w.r.t. the rest of the scenery.

This work would be needed anyway for smaller models - game objects etc. if they end up carrying textures.

Stil a few things about the sky I'm not happy with - the source textures are 8bit and fitted to the Q2 game palette. This can be changed to work the same way as BadMooD - each image or set of images can use a private palette to take advantage of the truecolour framebuffer. This will look a lot better esp. with custom skybox textures.

The other problem is memory - the sky alone takes 6x textures at 256x256 resolution, so that's 400k in one go. Not a big piece of 14MB, but it all adds up - it's actually a lot in terms of texture budget. 4MB is already lost on surface cache and several more MB on geometry and lightmaps. I'm wasting a lot still, so more can be bought back later. The textures though will need to be paged like BadMooD because Falcon has no virtual memory (unlike a 16MB Windows machine!).

Sky would look a bit better still and render a bit faster if using raw 16bit textures - but that in turn means 800k for sky textures. Probably needs to be specified as an option, or limited to smaller maps only. Will see.

[dml]

A few more steps forward...

- got PCM's POVRay skybox generator producing a decent panorama in 24bit colour
- created ImageMagick script to build skybox maps from 24bit source images and autoname them properly for the engine
- got a custom truecolour skybox rendering on the Falcon - looks nice
- got GtkRadiant set up for map building/editing
- installed qbsp3, qvis3, arghrad and set up a script to compile maps with them
- converted a Q3 test map to Q2 format, got it working in the engine (nearly - arghrad lighting pass fails so there are no lightmaps yet)

[dml]

It's amazing how much inertia you can slam into when you switch from using your own stuff to using other people's stuff :-z

Yikes.

Aside from the sheer number of tools you need to get working to build a map - all of them have weird and almost inexplicable limits which cause the map making process to explode in a variety of different ways. All of them pretty infuriating.

All errors seen so far could be described: "computer says no".

I have just about managed to get my test map lit properly but not without pain, confusion, trial-and-error and googling. And it still doesn't look right.

One of the biggest problems has been the fact that 'Quake 2' map making tools and guides have been trounced/merged/aliased with similar instructions for Quake2World, Quake3, Quake4... and the editor I picked (QERadiant) tries to support them all. So in many cases guides which appear to be talking about Quake2 are actually flipping between all of them and ignoring limits in classic Quake2.

Oh well. That's what I get for resurrecting old technology

Some of the problems I hit straight away:

- map too big to light with default settings
- some objects with no textures on them were still getting used by the lighting pass, and acting like they had textures, causing the lighting tool to explode because of the size of the lightmaps required.
- sky box wasn't marked with sky flag
- not clear how to put sky textures on sky boxes, since sky textures are not .wal format in Q2, but PCX/TGA. can't get QER to load PCX/TGA as textures. so i did it anyway with .WAL to try and get the ambient lighting right. .WAL get remapped to original Q2 palette. looks awful. probably a bad move overall.
- lighting tool keeps running out of radiosity patches, lightmaps, vertices (!?), RAM, other random things...

Despite all of that I'm making a bit of progress. Still need to understand some things about skies and lighting in Q2, but other areas are getting a bit clearer.

The screenshot is from Quake 2 on the PC, running the test map I was trying to light... it gets darker in the tunnel, which is a good sign. There is also a greyish lightsource from below (not visible here) which ties with the lower sky texture. And yellowish colour from above/around, which is right. But the bright sun in the corner of the skybox seems to be absent... hmm.

Image1.png

[shoggoth]

I poll this thread every 100ms now.