Dune II / OpenDUNE for Falcon (and TT)

All about games on the Falcon, TT & clones

Moderators: Mug UK, [ProToS], lp, moondog/.tSCc., Moderator Team

User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

Thanks, it's much better now.

Next suspicious things in the main menu are:
* Timer_InterruptRun...->get_sysvar(), ~15% of instructions / cycles
* main() -> toupper(), 5-10% of instructions / cycles (20% of all calls)
* Driver_Voice_IsPlaying() -> DSP_GetStatus(), ~5% of instructions / cycles, although there's no DSP (emulation = none)

These seems like they were doing something that could potentially be done beforehand, and skipped while in menu. What do you think?

Things that caught my eye in the game play itself:
* GUI_DrawSprite() has most instruction cache misses (23% of all), maybe the active parts could be be optimized to fit cache better?
* GUI_DrawSprite() calls Format80_Decode() which takes 5% of cycles, could the decoding be done beforehand?
* DSP_Play() takes 7% of cycles although I have DSP disabled
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

1) I don't think it is worth optimizing the main menu as long as it is 50fps. Have you used F8 key to display FPS ? (you may have to move the mouse cursor in the upper right corner of the screen to see something ;)
2) DSP_xxx functions do not refer to the falcon DSP, but the SOUNDBLASTER DSP. Don't forget it is a MSDOS game. So all DSP_* functions are about Digital Sound. The Soundblaster had a YM3812 (FM synthesis) + a kind of 8051 + DAC for playing digital sound which is called Digital Sound Processor (DSP)
3) Audio in OpenDUNE Atari/TOS is using STE/TT DMA Sound. Because it only supports fixed frequencies, sound has to be converted. It is also converted from 8bit unsigned samples to 8bit signed samples. DSP_Play() does take CPU time for that.
I guess it is possible' to use Falcon DSP for the resampling, but I've absolutly no skills in that matter, I need help :(
4) I will have a look at GUI_DrawSprite(), it may be worth optimizing :)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
penguin
Captain Atari
Captain Atari
Posts: 191
Joined: Tue Dec 24, 2013 10:43 am

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by penguin »

@nanard

Just thought I let you know that I reviewed your OpenDune port for ST-Computer magazine (sorry, German only)
http://st-computer.atariuptodate.de/ind ... =currentde
AtariUpToDate - Atari ST/TT/Falcon software database and version tracker: http://www.atariuptodate.de
st-computer magazine - https://st-computer.atariuptodate.de/
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

@penguin : viele Danke !
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

One of the things DML did with Bad Mood was converting the Doom data to be more optimal for how BM works. Same could also help Dune. Anything that requires run-time conversion, could be have an offline tool that converts the data to the optimal format, which would then be used at run-time.

(In BM, that tool was actually built into to BM, on first run it first converted the data and saved the converted data as separate files to file system. On successive runs, it skipped the conversion step.)
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

In Opendune there isn't much data to convert :) Everything can be done at loadtime.
As the game was initially under MSDOS in realmode, it had to fit in the 640K (550K or so in fact...) so some loading and unpacking are done at the last minute. This can be improved :) But I still want the game to be playable on 4MB Falcons
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

new optimizations (minimap redrawing) :
http://nanard.free.fr/opendune/opendune ... im-TOS.zip
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

Thanks, it's getting more playable, or I'm getting more used to things getting slow when there are a lot of items moving on the map (with emulated Falcon). :-)

I guess I could try next profiling emulated TT a bit instead of Falcon.
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

Is there a "how to" do profiling with hatari somewhere ? I see the hatari_profile.py script, but I don't know how to generate the profile with data from the start :)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

It's all explained in the manual. Below is quick intro, look into hatari_profile.py help output and Hatari manual when you need more info:
https://hg.tuxfamily.org/mercurialroot/ ... #Profiling

Just enter the debugger and do:

Code: Select all

> profile on
> c
And everything will happen automatically when your program includes debug symbols. Hatari will then profile all the code between the times you enter to debugger (by keyboard shortcut or breakpoint).

To save profile for your code that gets run between previous debugger invocation and current one:

Code: Select all

> profile save profile-assembly.txt
To improve profile post-processing results, you should always get debug symbol addresses for your code (instead of relying the symbols in the saved profile data being enough):

Code: Select all

$ gst2ascii -a -l  opendune.tos > opendune.sym
(you could do that as last step in your opendune build.)

And then to post-process the profile file:

Code: Select all

$ hatari_profile.py -r opendune.sym -s -t -p -g profile-assembly.txt
This will show the top functions based on number of calls, executed instructions, used cycles, instruction cache misses and data cache hits. "-p" option adds information for the (inclusive) costs of all the further functions called by a given function. "-g" option generates callgraphs for them.

To view the generated callgraphs, you need a program that can display Graphviz *.dot graph files. It can be either Graphviz itself, but on Linux I prefer program called "xdot":

Code: Select all

xdot profile-assembly-2.dot
XDot is available on Debian derived from the "xdot" package, or in source form from:
https://github.com/jrfonseca/xdot.py

If you want to see where the time goes inside ROM (TOS), I recommend using EmuTOS instead of Atari TOS, and adding following hatari_profile option for the symbols file included with EmuTOS:

Code: Select all

-a etos512k.sym
There are several options that can be used to make the generated callgraphs clearer. Main one is "--ignore-to" option. Give that all the functions which collect lots of calls in the callgraphs, but which are either interrupt handlers (i.e. "calls" aren't real), or some generic functions of which you aren't interested about. For example I use this to disregard opendune input interrupt handlers:

Code: Select all

--ignore-to ikbd,ikbd_mousex,ikbd_mousey
You might want to ignore generic functions like "_BitArray_Set,_BitArray_Test,_memcpy".

Or if there's some particular function about which usage you're interested, you can ask for a callgraph of only stuff that goes through that particular function with the "--only" post-processing option.

When you profile a large amount of code, the main problem is usually that callgraphs have too much information. You may need to experiment with different hatari_profile options for removing details you aren't interested about.

PS. Profiler will assign costs for instructions following a symbol address to whatever symbol preceeds them. If symbol file misses important function symbols, callgraph function can be quite misleading. If in doubt, check things from the saved profile assembly file and the symbols list. E.g. MiNTlib builds used for a long time bad linker options, which dropped MiNTLib symbols e.g. for heavy time functions.
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

Thanks. I see that I need to optimize
GUI_DrawSprite()
c2p1x1_8_falcon
GFX_DrawSprite()
Format80_Decode()
GUI_Widget_Viewport_Draw*

I optimized GFX_DrawSprite() and also improved the way FPS are displayed on atari :
http://nanard.free.fr/opendune/opendune ... _2-TOS.zip
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

Btw. it's possible to automate profiling with Hatari so that it will automatically find for you a slowest frame in your game while you play, and provide profile for that. That was used in profiling BadMood, where it was especially useful because Doom supports gameplay record & replay. For OpenDune, you would need to play the game yourself while debugger profiles the frames.

That requires setting up some chained breakpoints in suitable place in code which check instruction counter and if it's larger than for previous profile, save the profile over previously saved profile data, otherwise they just no-op to clear the instruction counter.

Note: Debugger has also cycle counter variable, but that follows lowest 32-bit bits for Hatari's global cycle counter. However, debugger CPU & DSP Instruction counter variables count only instruction executed between debugger invocations, when debugger's per-instruction tracking (like profiling) is enabled. And these counters are automatically zeroed each time emulation is continued from debugger.
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

http://nanard.free.fr/opendune/opendune ... im-TOS.zip (Edit: The previous ZIP was compiled with a bug, many sprites were not decoded)
in this new version, the sprites are "Format80" decoded on loading. it should take maybe 500KB more memory but saves on CPU :)
don't hesitate to use F8 key to enable FPS display.
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

Tested the new version by playing through one level. It plays pretty nicely, the main problem is that keyboard input for scrolling is buffered. When things get slow, it's really annoying to wait game scroll past the area I wanted before it starts scrolling it back. -> Could how much game buffers key events for movement be limited to a smaller number? (Game actions should probably excepted from this limitation)

Btw. It's a good idea to verify that optimizations really speed up things in general (instead of e.g. moving the load elsewhere) by accurate measurements, because humans are easily swayed by expectations. If they expect things to be faster, they often report them to be faster, although they in reality aren't (there are studies about this).

You can use Hatari profiler also for timing things. Just enable profiling and put breakpoints to start and end of what you want to time.

Main issue in timing OpenDune is that there's no record & replay, and what many of the functions do different things depending on what stage the game is (for example handle different amount of sprites). With manual game play it's hard to guarantee that the what's running is exactly same as the earlier.

Do you have some solution for that?
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

there is record and play !
use debug_log_game in opendune.ini
see https://github.com/OpenDUNE/OpenDUNE/bl ... ME.txt#L95

Also with the FPS display I can see that the game is actually faster :)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
agranlund
Atari nerd
Atari nerd
Posts: 49
Joined: Sun Aug 04, 2019 1:49 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by agranlund »

Hey nanard, this is great stuff! Are you still developing this for the Atari?

My ST is quite fast and I wanted to give it a go so I grabbed the latest sources from Github.
Here's a rather quick proof-of-concept at supporting 16 color ST Low resolution, and DMA sound detect to prevent it from crashing on DMA-less machines.

520ST with TF534 accelerator (68030 @ 50Mhz, 4MB 32bit TT-RAM):

https://www.youtube.com/watch?v=TCZuY365tIE

https://www.youtube.com/watch?v=8VJrFX1PlTQ

What are the chances of getting this into the official branch? Are you guys taking pull requests?
The changes are unintrusive and lives in the Atari video/sound backend.

Cheers,
--Anders
User avatar
nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 141
Joined: Mon Apr 04, 2016 2:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nanard »

Wow ! nice work !
I'm quite busy right now, but I will of course consider any good pull request !
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44 /|\ Stacy 4
nemodhs
Atari User
Atari User
Posts: 43
Joined: Sat Aug 31, 2013 2:29 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by nemodhs »

Looks really nice. Any chance some of your Scumm VM optimizations find their way in there as well? :D
agranlund
Atari nerd
Atari nerd
Posts: 49
Joined: Sun Aug 04, 2019 1:49 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by agranlund »

nanard wrote: Sun Jun 28, 2020 7:57 pm Wow ! nice work !
I'm quite busy right now, but I will of course consider any good pull request !
Same here! I'll try to read up on Git and figure out how to submit the pull request :)
I'm a bit of a noob with Git, it's all about Perforce at work :lol:
agranlund
Atari nerd
Atari nerd
Posts: 49
Joined: Sun Aug 04, 2019 1:49 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by agranlund »

nemodhs wrote: Sun Jun 28, 2020 8:57 pm Looks really nice. Any chance some of your Scumm VM optimizations find their way in there as well? :D
I doubt it to be honest. Still have unfinished business with the ScummVM port and I'm reluctant to taking on something else in parallel.
Thinking about lifting some of the non-DMA sound stuff from ScummST to get sound effects on my Atari though. And maybe the YM midi emulation..

I recon OpenDune could possibly be made to run on 8Mhz machines given a lot of work.. Dune2 ran on the Amiga 500 after all :)
But these types of changes quickly become very intrusive as it would have to operate in an Atari centric way right at the core instead of in a final platform translation pass. The slowest supported Atari pretty much becomes the "lead platform" (for that branch).
If I had the blessing from the OpenDune team, and find myself with nothing to do down the line, it could be fun giving it a go!
User avatar
Eero Tamminen
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2182
Joined: Sun Jul 31, 2011 1:11 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by Eero Tamminen »

agranlund wrote: Mon Jun 29, 2020 4:17 pm I'll try to read up on Git and figure out how to submit the pull request :)
I'm a bit of a noob with Git, it's all about Perforce at work :lol:
I've had very little use of Perforce, but AFAIK most Perforce users are migrating to Git, due to speed, distributed usage etc.

As to Git hub pull request, I think process goes something like this (I've done this only with Gitlab, but I think Github works the same way):
* fork the upstream repo from Github UI
* clone your new repository locally (Github UI shows the "git clone" command for this)
* create a new branch for the changes locally: "git checkout -b <new branch name>"
* add your changes to the new branch checkout
* commit those changes, preferably each distinct logical change as separate commit [1]
* push the new commits to your Github repo: git push
* submit that branch from your repo Github UI as a pull request to upstream repo

[1] it's easier to commit changes in small steps, and later combine them with "git rebase -i origin/master", than split changes later.
agranlund
Atari nerd
Atari nerd
Posts: 49
Joined: Sun Aug 04, 2019 1:49 pm

Re: Dune II / OpenDUNE for Falcon (and TT)

Post by agranlund »

Eero Tamminen wrote: Wed Jul 01, 2020 9:31 am
agranlund wrote: Mon Jun 29, 2020 4:17 pm I'll try to read up on Git and figure out how to submit the pull request :)
I'm a bit of a noob with Git, it's all about Perforce at work :lol:
I've had very little use of Perforce, but AFAIK most Perforce users are migrating to Git, due to speed, distributed usage etc.

As to Git hub pull request, I think process goes something like this (I've done this only with Gitlab, but I think Github works the same way):
* fork the upstream repo from Github UI
* clone your new repository locally (Github UI shows the "git clone" command for this)
* create a new branch for the changes locally: "git checkout -b <new branch name>"
* add your changes to the new branch checkout
* commit those changes, preferably each distinct logical change as separate commit [1]
* push the new commits to your Github repo: git push
* submit that branch from your repo Github UI as a pull request to upstream repo

[1] it's easier to commit changes in small steps, and later combine them with "git rebase -i origin/master", than split changes later.
Thanks Eero!
Post Reply

Return to “Games”