Page 6 of 6

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Mon Apr 02, 2018 10:08 pm
by Eero Tamminen
Thanks, it's much better now.

Next suspicious things in the main menu are:
* Timer_InterruptRun...->get_sysvar(), ~15% of instructions / cycles
* main() -> toupper(), 5-10% of instructions / cycles (20% of all calls)
* Driver_Voice_IsPlaying() -> DSP_GetStatus(), ~5% of instructions / cycles, although there's no DSP (emulation = none)

These seems like they were doing something that could potentially be done beforehand, and skipped while in menu. What do you think?

Things that caught my eye in the game play itself:
* GUI_DrawSprite() has most instruction cache misses (23% of all), maybe the active parts could be be optimized to fit cache better?
* GUI_DrawSprite() calls Format80_Decode() which takes 5% of cycles, could the decoding be done beforehand?
* DSP_Play() takes 7% of cycles although I have DSP disabled

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 03, 2018 9:45 am
by nanard
1) I don't think it is worth optimizing the main menu as long as it is 50fps. Have you used F8 key to display FPS ? (you may have to move the mouse cursor in the upper right corner of the screen to see something ;)
2) DSP_xxx functions do not refer to the falcon DSP, but the SOUNDBLASTER DSP. Don't forget it is a MSDOS game. So all DSP_* functions are about Digital Sound. The Soundblaster had a YM3812 (FM synthesis) + a kind of 8051 + DAC for playing digital sound which is called Digital Sound Processor (DSP)
3) Audio in OpenDUNE Atari/TOS is using STE/TT DMA Sound. Because it only supports fixed frequencies, sound has to be converted. It is also converted from 8bit unsigned samples to 8bit signed samples. DSP_Play() does take CPU time for that.
I guess it is possible' to use Falcon DSP for the resampling, but I've absolutly no skills in that matter, I need help :(
4) I will have a look at GUI_DrawSprite(), it may be worth optimizing :)

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 03, 2018 7:01 pm
by penguin

Just thought I let you know that I reviewed your OpenDune port for ST-Computer magazine (sorry, German only) ... =currentde

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 03, 2018 7:45 pm
by nanard
@penguin : viele Danke !

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 03, 2018 9:19 pm
by Eero Tamminen
One of the things DML did with Bad Mood was converting the Doom data to be more optimal for how BM works. Same could also help Dune. Anything that requires run-time conversion, could be have an offline tool that converts the data to the optimal format, which would then be used at run-time.

(In BM, that tool was actually built into to BM, on first run it first converted the data and saved the converted data as separate files to file system. On successive runs, it skipped the conversion step.)

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 03, 2018 10:26 pm
by nanard
In Opendune there isn't much data to convert :) Everything can be done at loadtime.
As the game was initially under MSDOS in realmode, it had to fit in the 640K (550K or so in fact...) so some loading and unpacking are done at the last minute. This can be improved :) But I still want the game to be playable on 4MB Falcons

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 10, 2018 4:03 pm
by nanard
new optimizations (minimap redrawing) : ...

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Tue Apr 10, 2018 11:17 pm
by Eero Tamminen
Thanks, it's getting more playable, or I'm getting more used to things getting slow when there are a lot of items moving on the map (with emulated Falcon). :-)

I guess I could try next profiling emulated TT a bit instead of Falcon.

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Wed Apr 11, 2018 10:12 am
by nanard
Is there a "how to" do profiling with hatari somewhere ? I see the script, but I don't know how to generate the profile with data from the start :)

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Wed Apr 11, 2018 9:25 pm
by Eero Tamminen
It's all explained in the manual. Below is quick intro, look into help output and Hatari manual when you need more info: ... #Profiling

Just enter the debugger and do:

Code: Select all

> profile on
> c

And everything will happen automatically when your program includes debug symbols. Hatari will then profile all the code between the times you enter to debugger (by keyboard shortcut or breakpoint).

To save profile for your code that gets run between previous debugger invocation and current one:

Code: Select all

> profile save profile-assembly.txt

To improve profile post-processing results, you should always get debug symbol addresses for your code (instead of relying the symbols in the saved profile data being enough):

Code: Select all

$¬†gst2ascii -a -l  opendune.tos > opendune.sym

(you could do that as last step in your opendune build.)

And then to post-process the profile file:

Code: Select all

$ -r opendune.sym -s -t -p -g profile-assembly.txt

This will show the top functions based on number of calls, executed instructions, used cycles, instruction cache misses and data cache hits. "-p" option adds information for the (inclusive) costs of all the further functions called by a given function. "-g" option generates callgraphs for them.

To view the generated callgraphs, you need a program that can display Graphviz *.dot graph files. It can be either Graphviz itself, but on Linux I prefer program called "xdot":

Code: Select all


XDot is available on Debian derived from the "xdot" package, or in source form from:

If you want to see where the time goes inside ROM (TOS), I recommend using EmuTOS instead of Atari TOS, and adding following hatari_profile option for the symbols file included with EmuTOS:

Code: Select all

-a etos512k.sym

There are several options that can be used to make the generated callgraphs clearer. Main one is "--ignore-to" option. Give that all the functions which collect lots of calls in the callgraphs, but which are either interrupt handlers (i.e. "calls" aren't real), or some generic functions of which you aren't interested about. For example I use this to disregard opendune input interrupt handlers:

Code: Select all

--ignore-to ikbd,ikbd_mousex,ikbd_mousey

You might want to ignore generic functions like "_BitArray_Set,_BitArray_Test,_memcpy".

Or if there's some particular function about which usage you're interested, you can ask for a callgraph of only stuff that goes through that particular function with the "--only" post-processing option.

When you profile a large amount of code, the main problem is usually that callgraphs have too much information. You may need to experiment with different hatari_profile options for removing details you aren't interested about.

PS. Profiler will assign costs for instructions following a symbol address to whatever symbol preceeds them. If symbol file misses important function symbols, callgraph function can be quite misleading. If in doubt, check things from the saved profile assembly file and the symbols list. E.g. MiNTlib builds used for a long time bad linker options, which dropped MiNTLib symbols e.g. for heavy time functions.

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Thu Apr 12, 2018 3:27 pm
by nanard
Thanks. I see that I need to optimize

I optimized GFX_DrawSprite() and also improved the way FPS are displayed on atari : ...

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Thu Apr 12, 2018 11:19 pm
by Eero Tamminen
Btw. it's possible to automate profiling with Hatari so that it will automatically find for you a slowest frame in your game while you play, and provide profile for that. That was used in profiling BadMood, where it was especially useful because Doom supports gameplay record & replay. For OpenDune, you would need to play the game yourself while debugger profiles the frames.

That requires setting up some chained breakpoints in suitable place in code which check instruction counter and if it's larger than for previous profile, save the profile over previously saved profile data, otherwise they just no-op to clear the instruction counter.

Note: Debugger has also cycle counter variable, but that follows lowest 32-bit bits for Hatari's global cycle counter. However, debugger CPU & DSP Instruction counter variables count only instruction executed between debugger invocations, when debugger's per-instruction tracking (like profiling) is enabled. And these counters are automatically zeroed each time emulation is continued from debugger.

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Fri Apr 13, 2018 2:07 pm
by nanard ... (Edit: The previous ZIP was compiled with a bug, many sprites were not decoded)
in this new version, the sprites are "Format80" decoded on loading. it should take maybe 500KB more memory but saves on CPU :)
don't hesitate to use F8 key to enable FPS display.

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Sun Apr 15, 2018 11:01 pm
by Eero Tamminen
Tested the new version by playing through one level. It plays pretty nicely, the main problem is that keyboard input for scrolling is buffered. When things get slow, it's really annoying to wait game scroll past the area I wanted before it starts scrolling it back. -> Could how much game buffers key events for movement be limited to a smaller number? (Game actions should probably excepted from this limitation)

Btw. It's a good idea to verify that optimizations really speed up things in general (instead of e.g. moving the load elsewhere) by accurate measurements, because humans are easily swayed by expectations. If they expect things to be faster, they often report them to be faster, although they in reality aren't (there are studies about this).

You can use Hatari profiler also for timing things. Just enable profiling and put breakpoints to start and end of what you want to time.

Main issue in timing OpenDune is that there's no record & replay, and what many of the functions do different things depending on what stage the game is (for example handle different amount of sprites). With manual game play it's hard to guarantee that the what's running is exactly same as the earlier.

Do you have some solution for that?

Re: Dune II / OpenDUNE for Falcon (and TT)

Posted: Mon Apr 16, 2018 10:18 am
by nanard
there is record and play !
use debug_log_game in opendune.ini
see ... ME.txt#L95

Also with the FPS display I can see that the game is actually faster :)