Quake 2 on Falcon030

[dml]

The new rasterization method seems to be working, and I can see 'water' surfaces floating (still unsorted) in front of other geometry. It won't look right until sortkeys are assigned but the support for multiple frontsurfaces appears to work.

When I get a bit more time I'll try to render those surfaces with transparency and assign sortkeys properly so they don't float in front of other stuff.

[dml]

Transparent water drawn correctly in software, direct from the BSP, without a ZBuffer. Not only that but it can draw any number of transparency layers correctly. Even the original didn't do that because of the z-test rule OTOH that's unlikely to be found in any of the maps for the same reason.

[EDIT] I had second thoughts about this bit - it could well be supported in the original if the transparencies are BSP ordered as well as ZBuffered and drawn in reverse order - which is probably the case - and can probably be found in the maps after all. Don't know if the software renderer actually draws multiple layers of sorted transparency but I think it can at least - probably it does. Never underestimate the Carmack factor!

This was particularly difficult to get working. It also introduced a nasty bug which locks up the engine at random points and it has evaded 2 debugs so far, so that's going to be fun to find.

grab0067.png

Ok so it just looks like more flat polys, but it will make more visual sense when it has a moving texture on it. It is a transparent plane and not just colour fiddles on the ground polys.

My head exploded during that so I'm going to leave it for a while and recharge batteries.

[troed]

Transparent water drawn correctly in software, direct from the BSP, without a ZBuffer. Not only that but it can draw any number of transparency layers correctly. Even the original didn't do that because of the z-test rule OTOH that's unlikely to be found in any of the maps for the same reason.

At some point I expect Carmack to find this thread.

[calimero]

^
that would be cool!

I remember his interview/blog about coding Wolfenstein 3D for first iPhone... or Jaguar!
I think he enjoy in coding for limiting hardware and this thread is real treasure! (of course, both iPhone and Jaguar are magnitude faster than F030)

Doug, when you add textures, how much expect that it will have impact on fps? 50% slowdown?
If you can, please make option for switching between textured and flat shades polygons.

and what is main limiting factor while texturing world?
slowness of ST-Ram? I suppose that MC68030 got everything prepared and need only to move.w with texels to screen? would, hypothetical, 32bit bus had great impact on performance in this case of texturing e.g. quake?

[dml]

It will be nice if he finds it - hopefully he won't get too upset with my random comments It's nearly always complementary though

Another pic from the first map:

grab0072.png

I found a better place to capture layered transparency, in one of the later singleplayer maps. Seems to confirm the maps do contain such arrangements (and that my version works properly!)

grab0069.png

grab0070.png

grab0071.png

[dml]

Doug, when you add textures, how much expect that it will have impact on fps? 50% slowdown?
If you can, please make option for switching between textured and flat shades polygons.

I don't know yet exactly - it will depend on the effectiveness of the planned optimizations.

Yes I think having flatshading available is useful. The texturing stuff is an experiment in Falcon capabilities but I have never been sure what the end result will be. There are so many ways to speed it up that it will take time to figure out something 'final'.

and what is main limiting factor while texturing world?

There are several limiting factors in this case - CPU speed, STRam read latency, DSP performance... all of these can limit texturing capabilities for this system.

slowness of ST-Ram? I suppose that MC68030 got everything prepared and need only to move.w with texels to screen? would, hypothetical, 32bit bus had great impact on performance in this case of texturing e.g. quake?

Slowness of STRam is always a problem yes.

DSP can serve textures directly - but it's a bit more complicated than that unfortunately - while it is faster to do this in the best case, the textures need to be on the DSP in the first place - and they take a lot of time to upload (across the same host port bus) and only a few kB of space to store them. They are also large, and many. Almost every surface pixel in the game has a unique texture! This is because each pixel in the world is uniquely lit with a lightmap. So even repeated/tiled textures get expanded to unique ones in the surface cache. That's too much uploading of textures for the DSP to be worthwhile (every pixel needs uploaded, just to be be downloaded again). An alternative to surface cache expansion might be combining lightmaps + colour pixels in realtime on the DSP, but that means more DSP compute time and more space for two texture sources etc. etc. the problems mount up.

For this engine, the DSP is also very busy computing perspective correction, so there is time available on the CPU side for more than a single 'move.w'. More efficient use of both sides is achieved by having the CPU address the texture from main memory using two moves, while the DSP does address calculation. This achieves roughly 1:1 on workload. Certain optimizations can still be done on the DSP side to change the work ratio to favour the CPU again (and this will be used - to an extent), but at the cost of increasing the overhead per edge/per span - which then makes complex scenery slower. and so on.....

[Zamuel_a]

I remember back in the day when everyone was playing Q3 deathmatch and they altered the settings so the graphics looked more or less like flat shaded polygons so that it would be easier to see the players. So flat shading is only a feauture

[AnthonyJ]

I remember back in the day when everyone was playing Q3 deathmatch

Not just back in the day - happened with Quake 4 too (although I guess that is "back in the day" too now), and people are still playing Quake Live like that now.

Edit: current QL player configs: http://www.esreality.com/post/2705471/p ... nfig-2015/

[dml]

I remember back in the day when everyone was playing Q3 deathmatch

Not just back in the day - happened with Quake 4 too (although I guess that is "back in the day" too now), and people are still playing Quake Live like that now.

Yeah - and it was common in Q1 and Q2 also. IIRC 'Stuff was done' to prevent that kind of thing if the server config didn't like it

[dml]

BTW a colleague (who is also into computer graphics stuff) gave me a sample of his Quake 3 map loading/display code for DirectX, and I've had a scan through the specs online - there are clearly parts which are very close to Q1/Q2, and other parts not so close (like the bezier stuff).

Anyway I think it might be fun though to try and get Q3 maps loaded and displaying on the Falcon once this one is working well enough.

I'm also going to have a bash at Half-Life - which was, after all, based on IdTech #2 (Quake 2). The file format is almost the same...

[shoggoth]

My head exploded during that so I'm going to leave it for a while and recharge batteries.

http://youtu.be/X0urzOLkoik

Use your belt.

[dml]

My head exploded during that so I'm going to leave it for a while and recharge batteries.

http://youtu.be/X0urzOLkoik

User your belt.

Moral: never venture too far without your belt (or your towel)

[dml]

So the flaky bug mentioned earlier turned out to be something more insidious - and a consequence of not doing enough diligence on Q2 before writing the code for that bit.

I had noticed a very rare cases where polys would flicker/disappear and put it down to a memory-sharing bug which would get found and fixed later. I never found this bug and planned to look for it after the transparency work. Turns out, the transparency work found it for me.

After browsing the Q2 source and looking at other implementations of Q1 and Q2 renderers on PC, I made the hasty conclusion that the maps don't contain any non-convex polys. Because if they did, some of the other circulating code samples couldn't work - right?

Wrong. The maps do contain non-convex polys. Not many, but they are present. Oops.

I opted to clip polys in screenspace - unlike JC, who used the more elegant, simpler 3D frustum clipper - mainly because 2D clipping is more stable for fixedpoint/integer arithmetic and simple 2D classification could be done quickly on DSP. 3D clipping is better if you have some precision and it makes for less code mess. Unfortunately 2D clipping is a nightmare when you have non-convex polys.

So the rare flickery polys turned into lockups when certain transparency-oriented changes turned list edits into list editing loops. This exposed the bug as serious, and exposed the current clipper as inadequate.

Fun!

But at least I see whats wrong, and can fix it. I could just tesselate the non-convex polys as a getout, but prefer to fix the clipper.

[dml]

A quick browse through the HalfLife-2 .BSP specs indicate the content is probably close enough to be rendered with this engine on Falcon, with a separate loader. The important differences are small enough to be reformatted.

I suspect the 'prop' objects used to add detail to the world can be cheated using the same stuff I'll have to do for doors and other BModels anyway.

I don't expect HL2 will render 'fast' but it will be kind of fun if it renders at all. Still a lot to do though before going there.

[TheNameOfTheGame]

A quick browse through the HalfLife-2 .BSP specs indicate the content is probably close enough to be rendered with this engine on Falcon, with a separate loader. The important differences are small enough to be reformatted.

I suspect the 'prop' objects used to add detail to the world can be cheated using the same stuff I'll have to do for doors and other BModels anyway.

I don't expect HL2 will render 'fast' but it will be kind of fun if it renders at all. Still a lot to do though before going there.

Haha you're a madman for sure Good show!

[dml]

I was able to fix the clipper to work with nonconvex polys, so it's all working again using transparency without freezing or glitches.

[dml]

Been busy with other stuff so haven't done very much but did take a few snaps of progress on texturing the world. Mainly working out precision problems and the dreaded 'sparkles' caused by texture addressing errors.

I can see some problems with the texture cache, which I think is responsible for several bugs, one affecting the position/offset of textures on some walls. Another bug is related to lack of memory.

It's still loading full sized textures but drawing from the 2nd largest mip onwards. The largest image is not being used, so I'll be getting rid of that soon to free up ram for surface cacheing.

It's quite difficult to optimize some things for speed vs space when there is already barely enough ram to run the original data as it is.

[mfro]

My head exploded during that so I'm going to leave it for a while and recharge batteries.

http://youtu.be/X0urzOLkoik

User your belt.

Moral: never venture too far without your belt (or your towel)

[AdamK]

dml: after you will be finished with Quake2, maybe you could use gained experience to improve your Quake port? Maybe it could have playable framerate on plain Falcon?

[shoggoth]

High framerates are for wimps.

[dml]

dml: after you will be finished with Quake2, maybe you could use gained experience to improve your Quake port? Maybe it could have playable framerate on plain Falcon?

Should be possible to retrofit the graphics stuff onto Q1 sometime. That says nothing about the game engine though - which itself sucks a ton of CPU and float math.

High framerates are for wimps.

In the very early days of Q2 multiplayer, random people would log into a game server hosted on the LAN in our office. Because we broadcasted the IP oops.

Some of these logins were from the US (our office was in the UK). This led to some visitors essentially moving-by-teleportation because the network latency between them and us was so bad they would just hang in the air for seconds at a time. You could run rings around those players - who could hardly move at times.

But there was a freaky bug in the game physics which sometimes worked to their advantage - they could 'sweep' the crosshair over any player and pull the trigger near the start of the sweep - somehow, the game engine decided whatever lay in that wide arc was a direct hit. The slower your connection, the wider the arc. We called this effect 'sweeprail', and it claimed a good number of frags. Of course, it didn't always work. So they were mostly toast. And sometimes it was just a whiney excuse for being beaten by a player who deserved to be toast but magically escaped.

Ah, Quake nostalgia.

[dml]

Well some good news...

Roughly doubled the performance the texture mapping just now, with better precision, almost no 'sparkles' and without adding new kinds of glitches to the system.

So I'm getting roughly the same performance now at 320x160 that I was getting before at 160x160.

This doesn't mean I'm aiming for 320x - I just decided to stick with 320x for now to help debug visual glitches, texture alignment and so on. Lower res just makes things more difficult.

Some grabs from the latest version showing FPS. Of course its still way too slow but doubling the speed is the right kind of improvement.

grab0081.png

grab0082.png

grab0083.png

grab0085.png

For the technically oriented, the CPU side is now a tower of these:

Code: Select all

	move.w		(a2),d7
	move.b		(a5,d7.w),d0
	move.w		(a6,d0.w*2),(a1)+

If the surface cache can be made efficient enough, I can probably expand the cache to 16bit and end up with this:

Code: Select all

	move.w		(a2),d7
	move.w		(a6,d7.l*2),(a1)+

This change will need a lot of effort in other areas of the engine, and more optimizations to the DSP mapper, but I believe it can be done. This will bring the texturing performance very very close to BadMooD.

The per-face performance is still quite poor and per-span is also not great, so these will still require a lot of effort. It still struggles with dense scenery because of that.

The surface cache itself is tremendously slow, so turning a corner or waving the camera around can cause delays of a second or so as the cache is rendered out - that needs totally rewritten and is again quite effort intensive. All of this will take time.

[dml]

...and a quick capture at 160x80 pixels using the new improved mapper:

https://dl.dropboxusercontent.com/u/129 ... 160x80.avi

Starting to look like realtime yet?

The final version should manage bigger than 160x80 but this is probably getting indicative, for scenery drawing at least.


Not really worth putting this on YT yet, and I'm catching a cold and will need to stop playing with it tonight. So it's linked directly for now as AVI. Sorry!

[Eero Tamminen]

Been busy with other stuff so haven't done very much but did take a few snaps of progress on texturing the world. Mainly working out precision problems and the dreaded 'sparkles' caused by texture addressing errors.

Thanks, the screenshots look awesome!

I can see some problems with the texture cache, which I think is responsible for several bugs, one affecting the position/offset of textures on some walls. Another bug is related to lack of memory.

It's still loading full sized textures but drawing from the 2nd largest mip onwards. The largest image is not being used, so I'll be getting rid of that soon to free up ram for surface cacheing.

It's quite difficult to optimize some things for speed vs space when there is already barely enough ram to run the original data as it is.

Does your code need MMU? If not, you could provide a binary and I'll check how well it works with Hatari Falcon 32-bit addressing / TT-RAM support.

(As mentioned on hatari-devel, while the new Hatari core has now pretty good support both for 030 MMU and TT-RAM, there isn't yet a run-time patch to TOS v4 that would get TT-RAM recognized with MMU.)

[dml]

Does your code need MMU? If not, you could provide a binary and I'll check how well it works with Hatari Falcon 32-bit addressing / TT-RAM support.

(As mentioned on hatari-devel, while the new Hatari core has now pretty good support both for 030 MMU and TT-RAM, there isn't yet a run-time patch to TOS v4 that would get TT-RAM recognized with MMU.)

I don't think the MMU stuff is enabled so it may work. However I'm not sure if its TT-ram safe yet (display ram), I should check that first. A lot of very basic stuff is lifted from BM or other experiments and not nicely coded but will be cleaned up later.