Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

All about ST/STE demos

Moderators: Mug UK, lotek_style, Moderator Team

Zarchos
Captain Atari
Captain Atari
Posts: 263
Joined: Wed May 09, 2012 7:38 pm
Location: FRANCE

Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Zarchos » Mon Dec 11, 2017 6:34 am

https://youtu.be/wuOmoV5edRk

Awesome animation !!
Falcon, Atari 1040 STE, 520ST, 800xl, xegs, Amigas, Archimedes, RISC PCs, Iyonix, Omega, BBC B, Atom, Electron, ZX 81, Spectrum 48/128/+2/+3, Russian clones, Sam Coupe, V6Z80P, QLs inc. Q68, and more !
2200 m2 museum on its way https://youtu.be/xjB6_Ez-3BA
Shorter video here : https://youtu.be/UEZisfkcN1Y

Currently porting SOTB to the Archie : https://www.youtube.com/user/Archimedes75009/featured

User avatar
alexh
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2679
Joined: Wed Oct 20, 2004 1:52 pm
Location: UK - Oxford
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby alexh » Mon Dec 11, 2017 9:01 am

But it was listed in the demo competition!? Should have been either WILD or Animation category.

User avatar
Orion_
Captain Atari
Captain Atari
Posts: 389
Joined: Sat Jan 10, 2004 12:20 pm
Location: France
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Orion_ » Mon Dec 11, 2017 9:41 am

first of all, this is not a demo, this is a video codec, and a poor one considering the huge file of more than 50Mbytes
for example, the same animation was ported on Megadrive which also have a 68k @8mhz and it fits into 8Mbytes
https://www.youtube.com/watch?v=2vPe452cegU

the real challenge here would have been to make this video fit into one or 2 floppy, but here you need a full hard drive just for it ...

the original animation is here https://www.youtube.com/watch?v=G3C-VevI36s
and was ported to countless computer/console platforms.. http://www.pouet.net/search.php?what=ba ... &type=prod
My retro games shop including Atari ST/Falcon/Firebee games ! -- Free Atari games/demos/tools -- Free Falcon demos/tools
Atari Mega STe 4MB + SD2SCSI 1GB + NOVA ET4000 + Pico PSU + Gotek HxC // Atari STe 2MB

User avatar
troed
Atari God
Atari God
Posts: 1398
Joined: Mon Apr 30, 2012 6:20 pm
Location: Sweden

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby troed » Mon Dec 11, 2017 10:07 am

Of course it's a demo. We would've laughed at the space requirements in 1989 but we would've appreciated the codec nonetheless.

(Omega GROTESQUE on steroids)

/Troed of SYNC

Zarchos
Captain Atari
Captain Atari
Posts: 263
Joined: Wed May 09, 2012 7:38 pm
Location: FRANCE

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Zarchos » Mon Dec 11, 2017 12:41 pm

Orion_ wrote:first of all, this is not a demo, this is a video codec, and a poor one considering the huge file of more than 50Mbytes
for example, the same animation was ported on Megadrive which also have a 68k @8mhz and it fits into 8Mbytes
https://www.youtube.com/watch?v=2vPe452cegU

the real challenge here would have been to make this video fit into one or 2 floppy, but here you need a full hard drive just for it ...

the original animation is here https://www.youtube.com/watch?v=G3C-VevI36s
and was ported to countless computer/console platforms.. http://www.pouet.net/search.php?what=ba ... &type=prod



There are some huge differences between the STE version and the MD version :
I see the STE version has 16 shades from black to white ; when the MD version has only black and white.
Believe me, it makes a big difference : if you want to encode each picture, you need many more infos. (as 90% of your convertd pictures use the full spectrum of 16 shades, if you set up the parameters of your picture converter correctly).
To me, it is just not possible to have Bad Apple in 16 (used) colours without a huge file to load from, anyway.
I have been working on that for the Archie (on my YT channel, various videos, including batch conversion using ChangeFSI ; and also something these STE coders haven't done : removing the myriads of 'lonely pixels' in the final pictures, giving ugly artefacts).
Falcon, Atari 1040 STE, 520ST, 800xl, xegs, Amigas, Archimedes, RISC PCs, Iyonix, Omega, BBC B, Atom, Electron, ZX 81, Spectrum 48/128/+2/+3, Russian clones, Sam Coupe, V6Z80P, QLs inc. Q68, and more !
2200 m2 museum on its way https://youtu.be/xjB6_Ez-3BA
Shorter video here : https://youtu.be/UEZisfkcN1Y

Currently porting SOTB to the Archie : https://www.youtube.com/user/Archimedes75009/featured

User avatar
Orion_
Captain Atari
Captain Atari
Posts: 389
Joined: Sat Jan 10, 2004 12:20 pm
Location: France
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Orion_ » Mon Dec 11, 2017 1:44 pm

Zarchos wrote:There are some huge differences between the STE version and the MD version :

Yep, 45Mbytes of difference :)

Zarchos wrote:when the MD version has only black and white.

No, the MD version has 2 bitplane, 4 colors

Zarchos wrote:To me, it is just not possible to have Bad Apple in 16 (used) colours without a huge file to load from, anyway.

I'm a 68k coder too so I know what I'm talking about, check the Thunderdome demo from Checkpoint, you have a 4 minutes audio file with some video animation, and all that in only 1 floppy disk, because the coder have achieved an awesome audio & video compression technic
https://www.youtube.com/watch?v=gHwUchzEG8k&t=2m15s
In this Bad Apple demo, the sound is not even compressed, it's plain raw (just open the data file in a music player and hear for yourself)
My retro games shop including Atari ST/Falcon/Firebee games ! -- Free Atari games/demos/tools -- Free Falcon demos/tools
Atari Mega STe 4MB + SD2SCSI 1GB + NOVA ET4000 + Pico PSU + Gotek HxC // Atari STe 2MB

Zarchos
Captain Atari
Captain Atari
Posts: 263
Joined: Wed May 09, 2012 7:38 pm
Location: FRANCE

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Zarchos » Mon Dec 11, 2017 2:33 pm

ok 4 colours for the MD version, I didn't notice that watching the video.
Still that encodes a quarter of the infos of what I think the STE would have to encode, if it displays, as I believe, 16 colours (without further checking each picture to count the colours displayed, I'd say there are up to 16 displayed). ***

I also read this about the MD version :
Two modes: 15 FPS without any artifacts or 30 FPS with vertical interlacing

The Thunderdome demo is much easier to pack : complexity of pictures, compared to BA demo, has nothing in common, or so little.
Still an amazing work, of course. (Don't get me wrong : of course I admire the wits and the talents of the authors).

*** Of course a clever algo will encode infos for number of displayed colours -1 (this colour being the one a clever algo will fill in the display with).
In the case of the MD, that means effort is now on 3 colours, when for the STE that would be 15.
Now we are in a ratio of 5 times more infos for the STE compared to the MD. (Yes : I love my Ataris :cheers: )

Algo can be further enhanced by splitting screen in stripes (variable in number and size, and vertical or horizontal too, even a mix is possible if not stretching from one edge to another) to more closely match the preponderant colour to fill in the parts of the screen (or better : surrounding the other colours to show).
Many possibilities to do sthing clever, and much to check to know whether there is a gain or not (with speed vs size of the infos to take into account).
Falcon, Atari 1040 STE, 520ST, 800xl, xegs, Amigas, Archimedes, RISC PCs, Iyonix, Omega, BBC B, Atom, Electron, ZX 81, Spectrum 48/128/+2/+3, Russian clones, Sam Coupe, V6Z80P, QLs inc. Q68, and more !
2200 m2 museum on its way https://youtu.be/xjB6_Ez-3BA
Shorter video here : https://youtu.be/UEZisfkcN1Y

Currently porting SOTB to the Archie : https://www.youtube.com/user/Archimedes75009/featured

arty
Atari User
Atari User
Posts: 31
Joined: Mon Dec 11, 2017 10:49 am

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby arty » Mon Dec 11, 2017 3:16 pm

It seems Bad Apple opening animation with OST is new kind of multiplatform religion. It has to be known everywhere and adopted to specific roots of any retro platform :)
It was interesting to see it first time as challenge for 8 bit platforms as C64 or Sega Master System, because of obvious restrictions of memory size and data storage tech on them. IMHO, from other hand 16bit or 32bit platforms are far capable and mighty for this .
Anyway, thanks for author(s) of STE-version and after all it's good and quite emotional animation to be known everywhere :)
…………ıllıllı just a friend ıllıllı…………

fenarinarsa
Atari User
Atari User
Posts: 43
Joined: Sat Mar 15, 2014 11:23 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby fenarinarsa » Mon Dec 11, 2017 10:28 pm

Oh my god another thread XD I didn't expect to have that many talks about this conversion.

So yes it may be considered as a simple video codec, but as I'm originally a demo coder and I used my old skills to do that, you can also consider it to be a demo. It's up to anyone, I don't mind at all. As for the compo, well, I didn't ask anything, I just sent it to the SV orgas and they did whatever they wanted with it. I wouldn't even mind for it to be shown outside any compo or not shown at all.

I see people compare it to other video players or other demos (thunderdome, blabla/ppera's player). Fair enough, but the first one doesn't have that much frames (I think) and bad audio quality. The latter is amazing too, but is a general purpose player that couldn't play bad apple at full framerate with this audio quality. Note that again I don't mind about being compared, it's just that I didn't even watched those players to do bad apple, I started from scratch without knowing what was really possible or not.

I thought about this conversion during ~2 years but as I didn't do any m68k code for 25 years it took a long time for me to do anything. Then a conversion was released by StackDesign. I was really glad about it, so I tried it on my STE and... it *doesn't work*! It works only on emulators like Hatari because HDD access timings are not emulated and are immediates. And even so, it uses an RLE compression which makes the ST struggle, so it's often out of sync.

I wondered if the ST was really that slow and that nothing better could be done in 30fps and full resolution. I really thought it was impossible. I did a quick first test in GfA basic by loading a pi1 sequence form my UltraSatan and it was slow as hell (2~3fps). Then I merged all pi1 in one file, and it was faster (~5fps).

Of course I opened Pandora's box, I wanted to try different solutions and I took my best vasm (which I used to patch R-Type DX a few months ago) and got into work.

In C# I did a simple deltapacking from that pi1 sequence that generates a really basic code (move.w/addq), which is a well-known technique that I never used myself back in the 90s. As expected this reduced each frame quite a lot, then I executed it and got... 10fps.
I did various optimization to the code generation but never got past 12fps if I remember correctly. I have some videos of the progress, I will do a small making of video later.

The bottleneck was actually the GEMDOS FREAD call which is, as you know, a synchronous call. So the main loop that went like FREAD/render/FREAD/render was slowed down because of the FREAD.

That was the time I got into ACSI documentation and Ppera's website. I didn't want to go low-level, I don't have the skills and I know it can be a real hell to make things work, and I wanted to stick with FREAD. I learnt that the max transfer rate is around 1,5Mbps. So why couldn't it load faster?

What I found out by doing many tests is that reading a file in big chunks (like 100kB) is way faster than doing small reads. I actually thought like a modern asshole developper that doesn't mind about doing 2 bytes reads from the HDD, but on ST it is of course not the way to go. I don't know how exactly Ppera's driver works, but I guess it may be the same with any other drivers.

So I tried to parallelized HDD reads. When I succeeded, boom: 60fps. Sixty, not 30! It was actually faster than the video needed.

This is the thing I'm proud of, being able to use GEMDOS and render at the same time, without using the VBL (that I need to keep to count elapsed frames). The main loop does the HDD reads, while the renders goes into the... HBL! It's the only interrupt that doesn't stall the VBL frame counter.
There may be other ways of doing this, but it works this way :)

To be exact the HBL is triggered with a Timer B. And if you run the demo on a real STE, you can press + to see the load/play pointers and left shift to see the CPU and blitter timings. Then you see that when a 100kB chunk is loading, the blitter/CPU rendering is a lot slower. That is because the disk DMA steals cycles from the rendering when emptying the FIFO, making the loading actually asynchronous.

I was really amazed, because I was ready to lower the graphic quality to 3, maybe 2 bitplanes. I didn't have to do that. And when I added sound, it all ran smoothly, I could push the quality all the way up to 50kHz stereo just because I wanted the best soundtrack possible.

That was my goal all along, the best quality ever for the animation and sound with GEMDOS loading. This is not the codec that made it possible, but the parallelized loading.

After that, I did optimize the "codec" by using the blitter. I could have released it without using the blitter (it still can be disabled in my generator) but I *never* succeeded in using that bloody blitter in the 90s and I wanted my payback. At first I had to use Steem's debugger to know what I did wrong because the blitter went crazy many times. And strangely when I used the 'tas' instruction as shown in Atari's docs, it crashed on my Mega STE. I don't know why but now I avoid tas even if theorically the ST has no issue with it (like the Amiga does). My opinion is just that my code is crap :)

So I used some blitter tricks, the main being that the delta-packing is done in vertical stripes of 1, 2 or 4 bitplanes depending on the frame (the generator finds the best setup). Those stripes are copied into the file and copied back to the frame with a simple blitter copy. There is also a later optimization that detects empty stripes (black) and full stripes (white) and no data is saved in the file since the blitter can fill 1's and 0's with no source.
Using the blitter reduce the filesize by ~40%. There is still some generated code when the stripes are too small.

And, finally, I used a modified gray code for the palette to minimize bit change for shades and black/white state change (black=0000, white=0001 instead of 1111). This is particularly efficient with the blitter.

Of course there should be more ways of getting the file smaller. I had some other ideas for the blitter I didn't have the time to implement. I checked the talks here about packers, and none could fit my requirements since they're tailored for floppy loading rates. But maybe... after all the rendering leaves 50-80% of CPU time. If you find a way to unpack 200kB/s of graphic data, be my guest.
You can try to compress the audio, but I highly doubt the STE would be able to lossless depack 100kB of sound per second. Prove me wrong :)

Anyway, it was more an exercise for me than anything else. I had fun doing it. I did reduce the data, but first of all I wanted the best quality I could get of my old STE. I didn't mind the file size: I have many 1GB cards that are mostly empty. An Atari STE with 1GB of mass storage! Why would I sacrifice quality since I had the space to do anything I wanted. :wink:
Last edited by fenarinarsa on Fri Dec 15, 2017 10:04 pm, edited 5 times in total.

distantminds
Atari maniac
Atari maniac
Posts: 98
Joined: Thu Sep 29, 2005 5:03 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby distantminds » Mon Dec 11, 2017 10:45 pm

thanks fenarinarsa, i enjoyed reading that. the demo/animation was nicely executed also - nice one :)

Zarchos
Captain Atari
Captain Atari
Posts: 263
Joined: Wed May 09, 2012 7:38 pm
Location: FRANCE

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Zarchos » Mon Dec 11, 2017 11:04 pm

distantminds wrote:thanks fenarinarsa, i enjoyed reading that. the demo/animation was nicely executed also - nice one :)


Seconded :cheers:
Falcon, Atari 1040 STE, 520ST, 800xl, xegs, Amigas, Archimedes, RISC PCs, Iyonix, Omega, BBC B, Atom, Electron, ZX 81, Spectrum 48/128/+2/+3, Russian clones, Sam Coupe, V6Z80P, QLs inc. Q68, and more !
2200 m2 museum on its way https://youtu.be/xjB6_Ez-3BA
Shorter video here : https://youtu.be/UEZisfkcN1Y

Currently porting SOTB to the Archie : https://www.youtube.com/user/Archimedes75009/featured

User avatar
Anima
Atari Super Hero
Atari Super Hero
Posts: 664
Joined: Fri Mar 06, 2009 9:43 am
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Anima » Tue Dec 12, 2017 5:35 am

Thanks fenarinarsa for the explanation.

fenarinarsa wrote:After that, I did optimize the "codec" by using the blitter. I could have released it without using the blitter (it still can be disabled in my generator) but I *never* succeeded in using that bloody blitter in the 90s and I wanted my payback.

In fact, this keeps the spirit of the community alive. Thanks again for your work.

User avatar
ggn
Atari God
Atari God
Posts: 1202
Joined: Sat Dec 28, 2002 4:49 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby ggn » Tue Dec 12, 2017 7:21 am

As someone who has released 2 demos that stream in data from the hard drive during runtime using GEMDOS (i.e. system is still running in the background) (http://www.pouet.net/prod.php?which=64940 http://www.pouet.net/prod.php?which=64940) I can only agree that this is not as trivial as it sounds!

Also fully agree about demo sizes - packing down demos into a small space was really essential back when floppy disk was the only sensible way of distribution. I'm pretty sure that with the design goals fenarinarsa set (0 compression artifacts, 50khz sound) this wouldn't have been possible to do from floppy, otherwise the quality would suffer. And it would have taken an awful lot of time to research all methods and come down to this conclusion.

In closing, what do people prefer more? A new 50mb demo (because yes I do consider this a demo) or no demo at all?
is 73 Falcon patched atari games enough ? ^^

wietze
Captain Atari
Captain Atari
Posts: 215
Joined: Fri Mar 01, 2013 10:52 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby wietze » Tue Dec 12, 2017 8:04 am

ggn wrote:In closing, what do people prefer more? A new 50mb demo (because yes I do consider this a demo) or no demo at all?


More please.

User avatar
spiny
Disk Imager Supreme
Disk Imager Supreme
Posts: 2461
Joined: Mon Aug 11, 2003 11:53 pm
Location: just outside bristol
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby spiny » Tue Dec 12, 2017 9:31 am

fenarinarsa wrote:
Anyway, it was more an exercise for me than anything else. I had fun doing it. I did reduce the data, but first of all I wanted the best quality I could get of my old STE. I didn't mind the file size: I have many 1GB cards that are mostly empty. An Atari STE with 1GB of mass storage! Why would I sacrifice quality since I had the space to do anything I wanted. :wink:


thanks for the post explaining some of the tech details. It may be a 'stream' but I still see it as a proper demo, and like you say, space is cheap these days. Obviously, nobody would watch it if it had to ship on 50+ floppy disks :D and even 20 years ago 'big' hard drives were rare, but its NOT 20 years ago, we should use the hardware available today.

User avatar
spiny
Disk Imager Supreme
Disk Imager Supreme
Posts: 2461
Joined: Mon Aug 11, 2003 11:53 pm
Location: just outside bristol
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby spiny » Tue Dec 12, 2017 9:33 am

ggn wrote:
In closing, what do people prefer more? A new 50mb demo (because yes I do consider this a demo) or no demo at all?



more :)

User avatar
AdamK
Captain Atari
Captain Atari
Posts: 255
Joined: Wed Aug 21, 2013 8:44 am

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby AdamK » Tue Dec 12, 2017 10:02 am

Thanks fenarinarsa, from the moment I got my hands on your prod, I wanted to ask 'how you did it' :)

I like your prod a lot. I have one question: it seems that yo're not using whole screen, how much does that save?

As for the future: more please :)
Atari: FireBee, Falcon030 + CT60e + SuperVidel + SvEthlana, TT, 520ST + 4MB ST RAM + 8MB TT RAM + CosmosEx + SC1435, 1040STFM + UltraSatan + SM124, 1040STE 4MB ST RAM + 8MB TT RAM + CosmosEx + NetUSBee + SM144 + SC1224, 65XE + U1MB + VBXE + SIDE2, Jaguar, Lynx II, 2 x Portfolio (HPC-006)

Adam Klobukowski [adamklobukowski@gmail.com]

junosix
Captain Atari
Captain Atari
Posts: 235
Joined: Sun Jul 08, 2007 3:22 pm
Location: Plymouth

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby junosix » Tue Dec 12, 2017 10:14 am

Nice work!

BlankVector
Captain Atari
Captain Atari
Posts: 446
Joined: Wed Oct 24, 2007 7:52 pm
Location: France
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby BlankVector » Tue Dec 12, 2017 10:48 am

Well, even if it does not fit on a single floppy, that's really good stuff :D
I don't remember having seen anything similar on STe before.

When Falcon was released, a major feature was audio recording "direct to disk".
You actually managed to read a video "direct from disk" on STe, quite new stuff :mrgreen:
Subscribe to my new channel Vretrocomputing on YouTube and Facebook.

fenarinarsa
Atari User
Atari User
Posts: 43
Joined: Sat Mar 15, 2014 11:23 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby fenarinarsa » Tue Dec 12, 2017 10:55 am

AdamK wrote:I like your prod a lot. I have one question: it seems that yo're not using whole screen, how much does that save?


Yes I do, it's 320x200 so it's the whole regular screen.
The borders can't be used unless you use fullscreen techniques, but they're not compatible with disk loading (HDD loading uses DMA, which borks the synchronization needed to do fullscreen).

tat
Retro freak
Retro freak
Posts: 14
Joined: Wed Nov 12, 2014 10:07 am
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby tat » Tue Dec 12, 2017 12:08 pm

I enjoyed it.

I wonder if reducing the size will be the new "16x16 sprite record".

tat

User avatar
CiH
Atari God
Atari God
Posts: 1119
Joined: Wed Feb 11, 2004 4:34 pm
Location: Middle Earth (Npton) UK
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby CiH » Tue Dec 12, 2017 12:36 pm

Add my upvote for more please.
"Where teh feck is teh Hash key on this Mac?!"

User avatar
viking272
Captain Atari
Captain Atari
Posts: 312
Joined: Mon Oct 13, 2008 12:50 pm
Location: west of London, UK

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby viking272 » Tue Dec 12, 2017 1:28 pm

Nice one, enjoyed that demo, more please!

marss
Atari freak
Atari freak
Posts: 66
Joined: Thu Nov 07, 2002 12:29 pm

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby marss » Tue Dec 12, 2017 6:53 pm

more, please

User avatar
Zorro 2
Administrator
Administrator
Posts: 2195
Joined: Tue May 21, 2002 12:44 pm
Location: Saint Cloud (France)
Contact:

Re: Bad Apple by Fenarinarsa (Atari STe demo) !!!!!

Postby Zorro 2 » Wed Dec 13, 2017 9:35 am

Much people appreciated this compo at the Silly Venture and this demo had taken the advantage via a best display by the video projector with less colours than other productions. For me it's not really a demo because if I compare with all releases at this party like the Paradox "Wonderful World", technically you are not in the same place and because I saw the real and final result of this demo at this party - Y'a pas photo les gars !
For me, you should have been on WILD compo but I don't care, there were much and great ST productions at this party :)
Member of NoExtra Team


Social Media

     

Return to “Demos - General”

Who is online

Users browsing this forum: No registered users and 2 guests