Bad Mood : Falcon030 'Doom'

[EvilFranky]

It's also not the full game - but being able to run around a map at better than 0.2fps would be a good start.

It would be cool to see Doug

[NGF]

Q2 on Falcon, I never thought this day would come

Have you experimented with Wolf3d on Falcon? Surely the machine must be fast enough running Wolf3d in full speed with sfx and music? I mean the ST even have a playable port.

[AdamK]

There is Wolf by ray/tscc: http://s390174849.online.de/ray.tscc.de/wolf3d.htm. On ST you get 15fps, on falcon even more.What more would you want?

[NGF]

The development has stopped on this version. It's impressive and has potential but it's not complete.

[EvilFranky]

What's not complete about it?

[dml]

Have you experimented with Wolf3d on Falcon? Surely the machine must be fast enough running Wolf3d in full speed with sfx and music? I mean the ST even have a playable port.

Yeah I worked on something called STRay on another thread here, which was half-way between Wolf and Doom on an ST and displayed 64 colour 'shades' on ST. It used block raycasting a bit like the original Wolf but sector height was handled more like Doom so floors and ceilings are also textured. This was about as far as I thought things could go on an ST with a sensible framerate. It runs fast on a Falcon but only uses 4-plane drawing suitable for STs.

The Falcon can do a bit more, and I'm interested to see if I can change Q2 to make it work on an integer-based CPU.

[dml]

There is Wolf by ray/tscc: http://s390174849.online.de/ray.tscc.de/wolf3d.htm. On ST you get 15fps, on falcon even more.

Yes Ray's work is cool. I was really impressed when I found out about that.

What more would you want?

More and more

[dml]

So I scrapped the FPU-based software rasterizer and put in a simple edge scanner as a basis for building a new one. Still uses the FPU until I get a chance to rework it properly.

https://www.youtube.com/watch?v=J7KCzRtplNY

Impressive just how _slow_ it is even without surfaces But it does run.

[EvilFranky]

Hahaha you are actually a bit crazy Doug

[dml]

Hahaha you are actually a bit crazy doug [WINKING FACE]

I've made quite good progress in the last hour but its still only 2fps in wireframe mode. The core engine code is setting a ceiling of about 6fps even with drawing stopped. So those areas need a closer look.

Fortunately most of the time is spent in only a few functions so investigating that isn't too bad.


I decided it was worth trying to get a fast wireframe version going. Reasoning is simple: if I can't manage that much, the decision to stop is reached sooner and with less pain

[dml]

I did make some progress last night converting Quake 2 rendering pipeline to use integer/fixedpoint arithmetic instead of pentium-class floating point code. It is far from done but enough to show that the scene still renders properly with integer based matrices and transforms.

I also unravelled the graphics pipeline a bit, to avoid alternating between different kinds of work so similar work can be performed in batches from the instruction cache. So (nearly) all vertices can be transformed, projected in large blocks instead of face-at-a-time. Some of that can/should be DSP'd - particulary projection (I'm less sure the DSP has the dynamic range needed for world vertex transforms on any decent sized map) - but I can't profile the code sensibly until the floating point is removed because the profiler timings are all wrong for counting FPU cycles and misleads optimization. So even if the CPU version isn't optimized it will still give more honest profiling.

The visibility builder has been modified and the data reformatted to perform vis cluster tests with a single assembly instruction (memory bitfield test) and so squeeze the entire scene BSP/face visibility process into the 030 instruction cache. This was very impactive, and now takes less than 2.5%. It will need fully converted to assembler to make it fast enough in the end but was a good start.

A number of other smaller things were done but these were the significant ones. There is no shortage of optimization potential and the engine structure and code quality really makes this easy so achieving a reasonably fast wireframe version @16MHz still looks feasible to me.

Note: Drawing wireframe maps really demonstrated to me how 'chunky' the Q2 visibility system really is. I can find places where large pieces of map are drawn behind solid walls, and clusters 'turn on' quite early even before they can be viewed through windows. I guess this was a preprocessing time constraint and a PVS storage constraint, so I probably can't fix it myself (and don't want to get into tool territory!).

This is perhaps the least favourable thing for a Falcon conversion because it means a lot of faces need to be encountered by the CPU which are not actually going to be drawn. In a final implementation the DSP will need to know about them in order to exclude them from drawing. This is why the early pipeline needs to be reduced as far as possible because it will be a choking point on some maps.

[dml]

Another note before I forget it. I was worried about the setup time needed to make surfaces from textures and lightmaps. It's quite involved and the original version has bilinear filtering per texture pixel and a lot of management going on. The surface cache was given a huge memory pool and I had to reduce that a lot to get the thing to load in 14MB - which would make the process even slower.

However it seems obvious now that the best solution is to just give it a massive surface cache, maybe even larger than the original, and use the MMU and virtualization to page the surfaces in and out to disk. It's probably a lot quicker to read/write contiguous pages than it is to build a lightmap per pixel, and might manage the active memory more efficiently e.g. for individual mipmaps.

Solved! (probably)


I'll move the Q2 stuff to another thread soon, before the BM release updates appear.

[Eero Tamminen]

I can't profile the code sensibly until the floating point is removed because the profiler timings are all wrong for counting FPU cycles and misleads optimization.

Are the instruction counts & function call counts too misleading metric? With them and the Hatari profile post-processing, you should at least be able to get overall callgraph information about the hotspots and what higher level functions cause them.

Hatari FPU instruction timings are known to be off a lot, if you provide some feedback on that on the hatari-devel that would be appreciated.

[dml]

Are the instruction counts & function call counts too misleading metric? With them and the Hatari profile post-processing, you should at least be able to get overall callgraph information about the hotspots and what higher level functions cause them.

I considered this, and was using that information for the first stages. But the true performance is so far from reality that only the callgraph and instruction counts are meaningful. The cycle weights are not.

I'm nearly past the point where I can use the profiling info from floating point code, because I already know the approximate structure of the weighting in the engine (from the callgraphs, instruction counts and various tests on hardware).

However I have almost completely removed the floating point code from the graphics pipeline, so the Hatari profiler is suddenly becoming useful for cycle counting as well.

Converting Quake2 from floating point to integers has been much easier than I expected. I broke exactly one thing in the process, and I know exactly where it is, but haven't solved it yet. Once that is fixed it will be a completely integer-based vertex pipeline and optimizations can begin properly.

Hatari FPU instruction timings are known to be off a lot, if you provide some feedback on that on the hatari-devel that would be appreciated.

I suspect it will not be easy to emulate this well. The chip is partially concurrent with both itself, and with the CPU. So multiple counters need to be maintained to track overlaps and all of that hidden nastiness.

And the relative usage of FPU in the real world for performance code is low (or non-existent) until you get to 68040/68060, where it suddenly falls into 'typical coding' category and important for performance. However the 68060 FPU has completely different timing characteristics and behaves like a parallel integer unit - it may be a while before our desktops/laptops can emulate a 68060 without 99% frameskips

So I doubt that 68881/2 accuracy will ever be supported in Hatari (unfortunately). It's just too complex to get right and too rarely tied to performance code for anyone to care about cycle counts.

[Eero Tamminen]

Right. I also noticed that you had already mailed feedback to hatari-devel (I just got back to Finland and going through all my emails).

Attached are couple of callgraphs for GCC 2.x built T4 version of BM with DoomU WAD, for interactive startup of level 4 i.e. from first P_RunThinkers() call to first A_Chase() call. I.e. no monster activity or shots, just basic rendering of the level startup.

[dml]

Right. I also noticed that you had already mailed feedback to hatari-devel (I just got back to Finland and going through all my emails).

Attached are couple of callgraphs for GCC 2.x built T4 version of BM with DoomU WAD, for interactive startup of level 4 i.e. from first P_RunThinkers() call to first A_Chase() call. I.e. no monster activity or shots, just basic rendering of the level startup.

Thanks for this. While I have been using the profiler a lot for instantaneous inspections i haven't used the auto profiling in a while and it needs done before the release because a lot of stuff has changed!

[dml]

BTW all of the browser based SVG viewers I have tried really suck. Even the original GraphViz plugin for IE which could at least zoom properly. And I'm not keen on Adobe update policies.

Somebody needs to do a proper SVG plugin someday It's not a difficult problem - just allow arbitrary levels of zooming/panning, and don't crash on big graphs, and don't clip/lose bits of graph when the coordinates get bigger than a few pages. Not asking for an awful lot really

</whining>

[dml]

BTW if I don't post something for a few days it will be because my harddisk has failed and I'm trying to salvage my work. The SMART metrics on my main machine have started to head downhill so it's on deathwatch. There's a lot of unique stuff on here which isn't backed up somewhere else (yay!).

I have a new disk but not much time or space to mess with imaging & hw mess right now. Playing with fire really At least Eero and Saulot have clones of BadMooD just in case!

[EvilFranky]

Can you not copy the whole lot to Dropbox or similar?

[dml]

Can you not copy the whole lot to Dropbox or similar?

It would take longer to sift out the public/work/private stuff than it would to just image the drive, so I'll probably do that. A linear pass is also less stressful for the drive. I just need to get my act together and do it.

[EvilFranky]

Cool, well you may already have a plan to do this but here's a tool I found that was exceptionally good at offline disk imaging/cloning which is free and tiny:-

http://hddguru.com/software/HDD-Raw-Copy-Tool/

If the disk is also your system disk then you'll probably need a tool that will allow you to image using VSS. Or use something like Hiren's boot CD to give you a basic Windows environment from DVD so your hard disk is unlocked, then the above tool can be used...

[dml]

Cool, well you may already have a plan to do this but here's a tool I found that was exceptionally good at offline disk imaging/cloning which is free and tiny:-

http://hddguru.com/software/HDD-Raw-Copy-Tool/

If the disk is also your system disk then you'll probably need a tool that will allow you to image using VSS. Or use something like Hiren's boot CD to give you a basic Windows environment from DVD so your hard disk is unlocked, then the above tool can be used...

Thanks for that. Looks useful. I used some other tool with a USB multi-drive bay last time but no idea where that thing went now.

[Eero Tamminen]

Thanks for this. While I have been using the profiler a lot for instantaneous inspections i haven't used the auto profiling in a while and it needs done before the release because a lot of stuff has changed!

Heh, the "profile.sh" script creates them automatically. One just needs to build BM, a WinUAE version of Hatari and setup few variables in profile.conf.

As for viewing GraphViz results, I've found all the SVG and PDF viewers to suck badly at presenting really large graphs, so I view DOT files directly.

While "dotty" coming with GraphViz is fast, so far the best program for viewing DOT files has been XDot:
https://github.com/jrfonseca/xdot.py

I guess it works also on Windows if you install PyGtk:
http://www.pygtk.org/downloads.html

[dml]

HW MIDI replay was integrated today. It's not quite working get in the game but we're working on it.

[CiH]

Doug,

Here's an interesting story about an offshoot of Doom, 'Marine Doom'. It has a little link at the end of it.

https://medium.com/war-is-boring/a-budg ... 29955706ca