DOOM on atari st

[dml]

Carmack has just done this interview: http://www.wired.com/gamelife/2013/12/j ... mack-doom/ - in which he said:

There were some intermediate steps that I did for a technology engine that I did for Origin at the time that added lighting and floors and ceilings—it was still tile-based, but the graphics side of things that people look at, the core things where it had lighting, where you could have flashing darkened areas, was no longer tile-based

(that intermediate game was ShadowCaster - don't think I saw that one at the time, dunno how I missed it!). Is this the missing link that you were guessing existed?

I don't know much about ShadowCaster but it is interesting that there were actually some intermediate steps!

[AnthonyJ]

I don't know much about ShadowCaster but it is interesting that there were actually some intermediate steps!

Me neither. Some interesting technical details can be guessed from this analysis of the data though: http://www.doomworld.com/vb/everything- ... r-modding/

[Desty]

Shadowcaster was a great game - didn't know he had anything to do with it. 'Twas very smooth on my friend's weak 486.

[GokMasE]

I feel that the last portions of this thread is a virtual cliffhanger..
Seeing how the BadMood alpha turned out in the end, it is getting quite intriguing to imagine what the ST code will end up looking like!

Regards,

/J

[dml]

Hi!

I feel that the last portions of this thread is a virtual cliffhanger..
Seeing how the BadMood alpha turned out in the end, it is getting quite intriguing to imagine what the ST code will end up looking like!

I had to wind this down temporarily, to get BadMood released just after new year. That took a bit longer than expected. Once the bugs in that are fixed I'll probably start spinning this one back up again in parallel, and surface sometime later.

[jvas]

Hi!

I feel that the last portions of this thread is a virtual cliffhanger..
Seeing how the BadMood alpha turned out in the end, it is getting quite intriguing to imagine what the ST code will end up looking like!

I had to wind this down temporarily, to get BadMood released just after new year. That took a bit longer than expected. Once the bugs in that are fixed I'll probably start spinning this one back up again in parallel, and surface sometime later.

Are you on steroids ???

[dml]

Are you on steroids ???

Ha - I should have said that it won't be *Doom* on the ST, but the project is still being developed into something.

[yerzmyey]

Any 3D game for Atari ST is MOST appreciated.

[Ragstaff]

Haven't visited the forums for a while but I stumbled across this thread and have just finished reading.
Amazing stuff!! I can't wait to get home and try running some of those STRay demo's. I'm gobsmacked looking at the screenshots!
I've always been intrugued by first-person engines on the ST, from Midimaze, Legends of Valour, through to Hellgate, Substation, Imminent Destruction, Defjams screen in the Checkpoint demo (which you could control with the keys if you hit... esc, I think?), Rays Wolf3d, and now this.
Of course I'm also eager to see if you have been able to get back onto this project since putting your attentions back onto Badmood!

FYI, I recall you made one comment in this thread wondering how much CPU Ray uses clearing the floor and ceiling, even if it was none. Ray's todo list for v0.9 (never released) included "* a delta clear to draw floors/ceilings more quickly".
Doesn't apply to you as your floors and ceilings are textured obviously, but thought I'd throw that in to answer your question a little!

I'd be fascinated to see if you, having done this detailed study into first-person rendering techniques and features, could find any other optimisations to Ray's already-amazing Wolf3D engine. Or vice versa. You two should collaborate some time

EDIT - he also did a "doom floor" in the Outline 2007 invite: http://www.pouet.net/prod.php?which=30634

Finally, also not of much consequence to you but you said you weren't aware of the Legends of Valour engine at all. There's a video here: https://www.youtube.com/watch?v=JNKjtODjZCQ
Played on Amiga 500, not quite as fast as the ST version. No textured floors or ceilings, but has outdoor areas, windows, variable-height walls, a fair variety in textures, and unique characters walking around whom you can fight or talk to.
The screen tears as you move around so obviously it's drawing the directly into the frame buffers without any VBL syncing/timing (sorry, I'm not a programmer, but I hope you get what I mean).
As per the video you can change the size rendered window. On my Mega STE with cache on, IIRC, I used to play it at the largest setting and was OK.

[dml]

Hi,

Amazing stuff!! I can't wait to get home and try running some of those STRay demo's. I'm gobsmacked looking at the screenshots!

thanks.

I've always been intrugued by first-person engines on the ST, from Midimaze, Legends of Valour, through to Hellgate, Substation, Imminent Destruction, Defjams screen in the Checkpoint demo (which you could control with the keys if you hit... esc, I think?), Rays Wolf3d, and now this.
Of course I'm also eager to see if you have been able to get back onto this project since putting your attentions back onto Badmood!

TBH still busy on BM finishing off the audio & music, which has quietly grown into a project of its own - complete with bugs and extra work to go.

There is work remaining on the ST project which needs to come off the shelf but I need to put BM to bed first.

FYI, I recall you made one comment in this thread wondering how much CPU Ray uses clearing the floor and ceiling, even if it was none. Ray's todo list for v0.9 (never released) included "* a delta clear to draw floors/ceilings more quickly". Doesn't apply to you as your floors and ceilings are textured obviously, but thought I'd throw that in to answer your question a little!

Yes it probably would make sense for him to try that although the clear time is probably minor compared with the rest, so it would be a small improvement.

I'd be fascinated to see if you, having done this detailed study into first-person rendering techniques and features, could find any other optimisations to Ray's already-amazing Wolf3D engine. Or vice versa. You two should collaborate some time

It's hard to find meaningful optimizations for that I did find a few but they are small.

Wolf has several advantages which help simplify - textures don't need to tile vertically, and all walls are the same height. These two facts mean you can generate a unique bit of code for every possible z - (more realistically: every possible steprate). If the eye height can't move (I forget, but I think it is fixed in Wolf?) then you have another advantage - the starting y for each column is also fixed according to z. So you get clipping for free, by simply omitting the right bits of each generated column at each given z. Job done!

I'm not sure how Ray does y-clipping, but I bet it looks something like the above. It is popular to reserve some margin/dead area above and below the framebuffer and let stuff spill over - but that has its limits in 3D because stuff up close gets bigger faster (using a margin is more technically called 'guardband clipping') and it wastes fillrate in the worst case.

So Doom (and STRay) break the 'Wolf rules', and code-generating columns in that way won't work. There are too many things going on at once - vertical start/end positions, steprate, tiling offsets - and restricting the combinations just puts more limitations on the drawing and before you know it you are back to Wolf3D. These requirements also break the clipping trick I describe above - so I had to find a different way, while still avoiding waste pixels.

So most of the effort I spent on the walls wasn't on improving the speed of scaling, but on all that other stuff - preserving those capabilities without paying the price of making it slower.


I found a tidy solution to tile textures while using fast code-generated columns, to recover sub-pixel texturing precision for the starting coordinates of each column, and arbitrary height columns - using what I think are some 'new' techniques. I say new because they depend a lot on mipmapping arithmetic and I don't think anyone else bothers with mipmaps because its just more complexity and effort for normally not much extra use I found some ways to make it very useful in the math to save memory and cycles, which is nice.

In the end, I managed to make the scaling a little bit faster than Wolf (cycles per texel), but only by using a wider variety of codegen patterns. Something Ray had already done to an extent, from what I can see - I just took it a bit further to save more cycles, at the cost of making the code generator much more complicated (and so, more debugging and testing). There are no particularly new ideas introduced in that optimization, just more work. The actual saving from this is probably small - a few %.

So any 'new stuff' which lurks in there will be related to:

- better precision from generated code
- better reuse of generated code
- variable height columns with generated code
- tiling with generated code

...and not on new ways to fill pixels (which is just more generated code). I think that's a fair summary of that thing.

EDIT - he also did a "doom floor" in the Outline 2007 invite: http://www.pouet.net/prod.php?which=30634
Finally, also not of much consequence to you but you said you weren't aware of the Legends of Valour engine at all. There's a video here: https://www.youtube.com/watch?v=JNKjtODjZCQ

Thanks - I missed a lot of stuff from that time and still catching up with it - esp. the ones without YT links and which don't like my default STE TOS I moved machines recently and still to copy over a lot of Atari related things like ROMs!

L.O.V - I remember the game quite well from the mag reviews etc. but know very little about the tech. I don't remember if I saw the game running. Will look when I get a chance.

[GokMasE]

A YouTube video demonstrating this concept surfaced through the Quake2 thread so I though it might be appropriate to mention it here too!

Here is an URL to a video of Dougs' STRay: Hybrid Wolf3D / Doom engine on Atari STE: https://www.youtube.com/watch?v=QCvx2O5M69E

While the 2-field rendering isn't coming across very good through YouTube, maybe a binary would be good if anyone would care to give it a run on real hardware? (or a proper emulator, of course)


This "proof of concept" is impressive enough to suggest that the STe would have been capable of running something even more delicate than the gouraud shaded world of Substation, which in its turn was groundbreaking stuff back in 1995 or there about. I would love to see someone put this engine to use in a game

Regards,

/Joakim

[calimero]

yes, binaries would be cool

doug, I have few question:
do you use anything specific to STe or it will work on simple ST?
how much memory it require?
what kind of maps (file format) it use? and what editor can produce it?
what is resolution is in youtube video?
and how did you achieve 64 colors? did you use palette swaping? (although I am pretty sure that it is impossible to benefit from palette swaping in doom-like game)

[dml]

yes, binaries would be cool

I thought I had posted the last binary towards the end of this thread, but looking back perhaps I forgot. There's an earlier one but maybe not the latest one used in the vid. Reading the posts quickly it looks like I was upgrading it in some way... probably trying to make it a lot smaller.

I'll post a link to that binary here soon - but as stated in previous posts it only runs for now on a 4mb STE (or STeem or Hatari) due to a lot of precalc for the floor and other tables which I never spent any time reducing. It all gets loaded/generated and hangs around.

doug, I have few question:
do you use anything specific to STe or it will work on simple ST?

It may run on an ST as it is, but will probably look horrible due to the palette being STE-specific. However an ST-specific palette can be generated which will still work.

I didn't try it on an ST but it doesn't use other features, at least nothing important.

how much memory it require?

Currently, a lot - over 3mb IIRC. Mostly inefficiency related to the floor - which became a bit pointless anyway because the texture I ended up using really doesn't need that level of detail o precalc storage at all. It just ended up looking more appropriate for the demo.

what kind of maps (file format) it use? and what editor can produce it?

It's very basic - just typed in by hand as a table. Width * Height cells, with each byte 'cell' referencing a sector table. The sector table defines floor heights and texture for that cell (any of which can be modified in realtime).

So it doesn't really need an editor but if it did, would be easy enough. You could even edit the map while walking around by point-click since a raycast can tell you whats under the screen coordinates at any point.

what is resolution is in youtube video?

The capture was 320x200x16 cols, but the active window is something like 128x or 160x because of the expensive C2P. I don't remember for sure but certainly not much bigger than 1/2 on both axes.

If using just 16 colours, C2P is cheaper so a slightly bigger window is afforded.

and how did you achieve 64 colors? did you use palette swaping? (although I am pretty sure that it is impossible to benefit from palette swaping in doom-like game)

No palette swapping. Bitmap swapping using interlaced C2P - it depends on a specially computed palette which yields correlated pairing of palette colours to make target colours. You need to write two framebuffer fields per frame, and swap them at 50 or 60hz, so that's 4 framebuffers for a simple double-buffered display. The graphics refresh rate can be much lower than the interlace rate though.

Computing the required palette is a project I did back in 2012-ish.

The writeup for that is on my site with image & game examples here: http://www.leonik.net/dml/sec_crypto.py ...and there's an AF thread about it as well somewhere.

Anima recently did a version of that idea too using a web based service instead of a standalone tool, I think aimed more towards images than moving graphics but in the end it produces the same sort of computed palette for interlacing without palette swapping. He has a separate thread with examples on AF.

[dml]

A bit of detail on how to get from original truecolour texture images down to 64-colour ST/STE images in the framebuffer...

1) Generate mipmaps for the source textures, filtered down step by step until 2x2 pixels (or whatever limit suits, or whatever axis degenerates first).

At this point there's an opportunity to bake depth-cue lighting into the mipmaps, since they are selected at rendertime by exact distance. This is what I did.

Also best to gamma-correct the mipmaps before and after filtering, to improve detail (an energy preserving filter).

Third trick is to modulate depth-cue by absolute luminosity, so light emitters don't darken with distance compared with basic surfaces.

All of these are used in the demo, although subtle.

2) Concatenate all the processed textures into one super-texture for colour reduction. In fact my tool just uses a histogram, with a pixel count for each unique colour present. It takes less space and is faster to compute stuff from.

3) Generate specially coded 16-colour palette with implicit pairing to generate (up to) 64 total colours when interlaced as pairs. The resulting 64-colour palette is used to colour-reduce the original textures to 6-bit texturemaps (stored as 8-bit chunky, with 2 dead bits).

Note: more than 64 colours are possible from 16 colour base palette - in practice up to about 120-130 usable ones. But 6bit is a limit in this demo to keep C2P table sizes somewhat sensible.

4) Generate pair of index tables which translates each of the 64 texture colour indices into a C2P field (odd and even)

5) Draw stuff using 6-bit source textures (once only per frame) into a chunky framebuffer.

6) Perform two C2P passes, into a dual-field backbuffer alternating at 50hz. Each pass uses a different indexing table. (in fact 2 separate passes not needed - I perform one pass and write to 2 buffers at the same time)

7) Voila - 64 colour rendering using 4 bit planes at the price of a little bit of flicker, some extra framebuffer ram, and slightly more expensive C2P.

[dml]

I think this was the last binary: https://dl.dropboxusercontent.com/u/129 ... STRAY6.zip

[AtariCrypt]

Downloading that for sure!!

Steve

[dml]

I posted a caveat a few times about running it under emus - its best viewed on a real STE, run from a harddisk or CFlash or equivalent.

It's certainly a lot more convenient to run in STeem or Hatari but I found there are problems getting a decent 50hz refresh on those when the physical VGA display is actually running at 70-120Hz, so results vary an awful lot (flicker and framedrop) depending on configuration.

[AtariCrypt]

yeah i totally get that.. there's been some nice interlace pic slideshows that i wanted to upload to my yt channel but Hatari just didn't perform. It's tough when the Mac's refresh rate is different for things like this.. Real ST is best. But that's just stating the obvious : )
Cheers

Steve

[calimero]

No palette swapping. Bitmap swapping using interlaced C2P - it depends on a specially computed palette which yields correlated pairing of palette colours to make target colours. You need to write two framebuffer fields per frame, and swap them at 50 or 60hz, so that's 4 framebuffers for a simple double-buffered display. The graphics refresh rate can be much lower than the interlace rate though.

yes, I follow that thread. (there is also example of Metal Slug make with this technique, right?)
I just want to ask for details how did you apply this technique on textures but I read it in next post

It's very basic - just typed in by hand as a table. Width * Height cells, with each byte 'cell' referencing a sector table. The sector table defines floor heights and texture for that cell (any of which can be modified in realtime).

Is it in ascii format or... ?
Does maps need some compiling before use? (but textures would require preprocesing anyway... :/)

Anima recently did a version of that idea too using a web based service instead of a standalone tool, I think aimed more towards images than moving graphics but in the end it produces the same sort of computed palette for interlacing without palette swapping. He has a separate thread with examples on AF.

Animas web tools is quite impressive, I spent quite a lot time playing with it.

[FedePede04]

hi Doug
The music in the video, what is that, it is really awesome,it could be great if you could uploaded as a mp3

i have been thinking for a long time, how about if they made a video filter on emulators, something like a temporal / motions blur filters, where you interpolate 2 pictures and div the output with 2.
it could maybe smooth out the raster fx?

idea if they show the Atari (RGB) frame like this,
0 add with 1 output div 2,
1 add with 2 output div 2,
2 add with 3 output div 2,
etc.
you would always show 2 picture, and if it is a simple palette raster change, it would always have two of the opposite frame in the picture, and in the theory should make the picture steady,
i know there will be cases, where it gets more complex, where you have to make some edge detection, to get it working probably.

but i think that it could be done,what do you think and do you have some idea on the subject?

Edit16-03:
change some of the txt,i wrote it late, so some of it did not make any sense, so i hope it does now

[calimero]

I just try it on Falcon with Backward III

colors are somehow wrong: instead of brown walls and floor I have red one

(I try also without Backward and it works but color are still wrong - it only run crazy fast on 16MHz )

[dml]

@FedePede04

Yes if this sort of frame blending was implemented in the emulators it should deal with the problem so long as source images are not skipped but always combined.

@calimero

Not sure what's going on there - it wasn't written for Falcon but doesn't do anything very interesting to make it incompatible. Sounds like the Falcon/Videl palette hardware is still activated so it may just not be in an ST-compatible video mode.

[calimero]

Not sure what's going on there - it wasn't written for Falcon but doesn't do anything very interesting to make it incompatible. Sounds like the Falcon/Videl palette hardware is still activated so it may just not be in an ST-compatible video mode.

that was my thought so I try it with Backward (Atari ST "emulator" for Falcon) but result are same... weird colors.

[Eero Tamminen]

Did you start it from ST compatible "ST low" mode?

(I get weird screen output if I start it from incompatible mode in Hatari, so demo doesn't seems to set screen mode by itself.)

[calimero]

Did you start it from ST compatible "ST low" mode?

(I get weird screen output if I start it from incompatible mode in Hatari, so demo doesn't seems to set screen mode by itself.)

First I try ST compatible "ST low" mode = weird colors

than I try to set Backward to be as much compatible with ST as possible but result was same = weird colors