MP3 playing on STE - Decoding through the CosmosEx

News, questions and bugs reports about CosmosEx by Jookie. Now we have a Raspberry Pi in our machines!

Moderators: Jookie, Moderator Team

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 12:13 am

I finally made it :D
The MP3/WMA/OGG/... are decoded and resampled on the CosmosEx thanks to ffmpeg
samples are streamed to the STE through ACSI, and played thanks to DMA sound :)

to get it working :

1) use this branch of the ce_main_app : https://github.com/miniupnp/ce-atari/tree/mediastreaming_r

2) get ffmpeg working on the CosmosEx. you can compile it by yourself or get mine :
http://nanard.free.fr/ce_pi_usr_local_20160416.tar.bz2
you won't have enough space free on the / file system, so create a new partition on the SDCard of the PI just for /usr/local. I made myself a 500MB one

Code: Select all

# df -h
Filesystem                   Size  Used Avail Use% Mounted on
/dev/root                    491M  344M  120M  75% /
devtmpfs                     215M     0  215M   0% /dev
tmpfs                        219M  192K  219M   1% /run
tmpfs                        219M   41M  178M  19% /var/volatile
/dev/mmcblk0p5               488M  214M  239M  48% /usr/local

then cd / ; tar xvjf /tmp/ce_pi_usr_local_20160416.tar.bz2
Check that ffmpeg is running right.
you may need to add /usr/local/lib to /etc/ld.so.conf , and then run ldconfig

3) put the "media player" cemediap.ttp in your configdrive (/ce/app/configdrive)
https://github.com/miniupnp/AtariST/tree/master/CosmosExMediaPlayer

Just double click on any .MP3 / .WMA / .OGG (on a translated / shared drive)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

User avatar
Jookie
Hardware Guru
Hardware Guru
Posts: 1245
Joined: Wed Feb 04, 2004 6:54 pm
Location: Kosice, Slovakia
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Jookie » Tue Apr 26, 2016 1:02 am

Wow, that's crazy and cool at the same time ;)

This will probably become the part of the main app (if nanard allows ;) ), so there will be no need for the compilation, partition changes, and so on, just a regular update (but possibly on the new Raspbian instead of Yocto - we'll see)

The things that now need to be done for the broader use:
  • playback support for ST - without DMA sound - this will be used on many machines
  • GEM UI for the version of app, which isn't just a TTP (selection of file through file dialog, play next and previous)
  • the mentioned linux image, which has all this set up, so regular users won't have to do the installation manually

Great job, keep on going :cheers:

Jookie

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 1:37 am

It's probably possible to compile lighter ffmpeg and libs. (without symbols, etc.) so it could fit into the standard yocto image.

ffmpeg is eating 60%to 70% CPU on the Raspberry PI to decode and resample a MP3, so we probably need an optimized version (using ARM NEON ?).
Currently, Video is out of reach.
For the record : I first tried to use "sox", but it was way too slow !

Digitized sound on plain ST is possible but costs much CPU. Help is appreciated :)
I need to have end of DMA transfer signaled by IRQ.
At the moment, it is polled. That's not a problem as STE DMA sound eats 0 cpu time :)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 1:52 am

anyway I need to write a proper doc of the protocol for streaming from the CosmosEx so anyone can write it's own atari application.
First I need to clean up a few things :)
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

User avatar
Jookie
Hardware Guru
Hardware Guru
Posts: 1245
Joined: Wed Feb 04, 2004 6:54 pm
Location: Kosice, Slovakia
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Jookie » Tue Apr 26, 2016 7:11 am

nanard wrote:Digitized sound on plain ST is possible but costs much CPU. Help is appreciated :)
I need to have end of DMA transfer signaled by IRQ.
At the moment, it is polled. That's not a problem as STE DMA sound eats 0 cpu time :)


As on ST it will be most probably the only app that would be running under TOS, I don't see problem with using too much CPU.

The ST sample playback routine could be taken from some demo or some open source app. When I look at the github of ggn, I see that some playback is in the DHS demo Drone (or the same in DHS ST demosystem), and I guess many other will have something similar there, so probably it would be only take-modify-use thing ;)

tin
Atari freak
Atari freak
Posts: 70
Joined: Mon Jul 23, 2012 7:59 am
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby tin » Tue Apr 26, 2016 8:13 am

Yes! This is indeed great stuff. Finally we can play AdamKs SNDHRECORD on the real hardware :)

@nanard: Let me know if you need any help with ST playback.

User avatar
AdamK
Captain Atari
Captain Atari
Posts: 234
Joined: Wed Aug 21, 2013 8:44 am

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby AdamK » Tue Apr 26, 2016 9:24 am

Great :) Step 2: video player, Stem 3: profit ;)

It takes a lot of CPU time on Raspi, something is wrong there IMO.

Btw. could you make some docs about programming CE? I looked at the source, but I had not enough time to dig in, some docs and/or lib for data transfer would be helpful.
Atari: FireBee, Falcon030 + CT60e + SuperVidel + SvEthlana, TT, 520ST + 4MB ST RAM + 8MB TT RAM + CosmosEx + SC1435, 1040STFM + UltraSatan + SM124, 1040STE 4MB ST RAM + 8MB TT RAM + CosmosEx + NetUSBee + SM144 + SC1224, 65XE + U1MB + VBXE + SIDE2, Jaguar, Lynx II, 2 x Portfolio (HPC-006)

Adam Klobukowski [adamklobukowski@gmail.com]

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 9:59 am

I'm currently streaming a mp3 (192kbps 44.1kHz stereo) from the internet for 15 minutes or so and ffmpeg takes 16% and cosmosex main app 13% :

Code: Select all

  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
11771  7760 root     R    25468   6%  16% ffmpeg -v 0 -i http://vdl.stream-lat.org:8000/voixdulat_mp3 -ar 50066 -f au
 7760   911 root     R     533m 122%  13% /ce/app/cosmosex

That's stable. I think it was taking 60% because of buffering.

@tin : If anyone can point me to a good ST digitized sound playback routine, if possible for signed 8bits samples, that would be great.
But as most routines are using a table, signed/unsigned 8bits should just be a problem of changing the table :)

@jookie : if the aim is to play video, any CPU cycle should be saved, on the PI as well as on the Atari !
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 10:01 am

I've checked, the BCM2708 cpu doesn't support NEON instructions :(
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

User avatar
leonard
Moderator
Moderator
Posts: 640
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby leonard » Tue Apr 26, 2016 11:06 am

that's really cool project! I guess there is many other cool ideas to do with the cosmoex cpu!

I'll have a look at your mediaplayer source code, I guess I can add STF replay compatible!
Leonard/OXYGENE.

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Tue Apr 26, 2016 11:49 am

@leonard : the streamed data is based on the .AU file format.
I made a small player : https://github.com/miniupnp/AtariST/tre ... r/steauply
I'm still working on a better version (interrupt based, without polling)

tools to make .AU files :
$ sox input.mp3 -r 50066 -b 8 -e signed-integer ouput.au
(add -c 1 to convert to mono)
$ ffmpeg -i 'input.mp3' -ar 25033 -f au -c pcm_s8 -ac 1 output.au
(-ac 1 is for mono)

I guess you can do something like 15 or 20kHz mono with the YM, right ?

----------
about PI CPU usage : ffmpeg take around 15% for MP3 decoding, but 65% for WMA decoding !
For video, we would need to stay low (bellow 70% I guess) because there are more processing to do before sending to the ST : dithering / etc + chunky to planar conversion.
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

User avatar
Jookie
Hardware Guru
Hardware Guru
Posts: 1245
Joined: Wed Feb 04, 2004 6:54 pm
Location: Kosice, Slovakia
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Jookie » Tue Apr 26, 2016 12:33 pm

nanard wrote:For video, we would need to stay low (bellow 70% I guess) because there are more processing to do before sending to the ST : dithering / etc + chunky to planar conversion.


This should be possible with the proper video player, which uses GPU of Raspberry Pi for video decoding, and which will hopefully output the video through framebuffer, from where the data could be captured...

User avatar
qq1975b
Atari God
Atari God
Posts: 1064
Joined: Tue May 15, 2012 9:15 am
Location: Barcelona

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby qq1975b » Tue Apr 26, 2016 2:47 pm

wow! this is amazing. That's a good example of taking advantage of all the available resources ;)
Trying to learn...

tin
Atari freak
Atari freak
Posts: 70
Joined: Mon Jul 23, 2012 7:59 am
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby tin » Tue Apr 26, 2016 5:38 pm

nanard wrote:I guess you can do something like 15 or 20kHz mono with the YM, right ?

Thats absolutely no problem, don't underestimate the ST ;) 15kHz is like one sample every scanline on 50Hz. Even with interrupt overhead that's no problem at all. Depending on what you do (e.g. don't use tables and prepare ym data on Pi directly) you can tune the needed CPU time pretty good. In theory, you can play up to 222kHz an ST (disregarding HD DMA overhead of course), but that's probably not needed here :)

The only thing that might be a bit tricky is to synchronize the DMA transfers in a way, that the sound doesn't distort. I might be misremembering, but an FDC DMA sector took somewhat around 3 scanlines and I would suspect the same for HD DMA? If this is true, higher sample rates could get slight (almost inaudible) distortions due to DMA stopping the 68k and hogging the bus during transfer. Unfortunately there is no HBL or VBL where this transfer can be hidden in this case ;)

EDIT: misremebered scanline/Hz ratio :)

User avatar
Cyprian
Atari God
Atari God
Posts: 1404
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Cyprian » Tue Apr 26, 2016 9:00 pm

nanard wrote:I finally made it

really great idea!

tin wrote:FDC DMA sector took somewhat around 3 scanlines and I would suspect the same for HD DMA

The DMA fills (read from the ST ram) its 16byte FIFO buffer with full CPU bus speed - 2MB/s. Therefore 50Khz stereo 8Bit should steals more or less 3% memory bandwidth. It is about 8 scanlines per frame.
Jaugar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
SDrive / PAK68/3 / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Aranym / Steem / Saint
http://260ste.appspot.com/

tin
Atari freak
Atari freak
Posts: 70
Joined: Mon Jul 23, 2012 7:59 am
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby tin » Tue Apr 26, 2016 11:13 pm

Cyprian wrote:The DMA fills (read from the ST ram) its 16byte FIFO buffer with full CPU bus speed - 2MB/s. Therefore 50Khz stereo 8Bit should steals more or less 3% memory bandwidth. It is about 8 scanlines per frame.

Ah, yes, thanks for refreshing my memory :)

The CE can currently send about 1MB/s to the ST. So I suspect one sector to take roundabout 512*8 cycles until it is read from ACSI. In this time we probably have 32 16 byte FIFO flushes with 64 cycles each followed by a bus release. After each bus release we might have 64 cycles free, given the CE sends its sector at a constant rate - and that rate is the limiting factor, not some setup/housekeeping in the CE itself. The atomic part of a (normal) sample YM register update does fit very well into those 64 cycles. So it's very possible that the sound won't be distorted as the YM update is only slightly shifted a few cycles (64 at worst).

So even 50kHz stereo will work with ACSI on ST (YM/parallel port dongle combo). We have about 150 cycles per sample (bus bandwith eaten by ACSI DMA is substracted here) after all. That is plenty.

Only if the CE would send it's sector with the full possible DMA bandwith there's a problem - then we'll have distortions. But I doubt that the bus isn't released during a sector transfer from CE. Will test that later.

User avatar
Cyprian
Atari God
Atari God
Posts: 1404
Joined: Fri Oct 04, 2002 11:23 am
Location: Warsaw, Poland

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Cyprian » Wed Apr 27, 2016 12:09 pm

very sorry but my calculation wasn't fully correct.
The MMU allows the CPU/DMA to access ST-Ram (via CPU bus) 2 000 000 times per second. It is 4MB/s because of 16bit data path. And with that speed DMA should fills its FIFO. 50Khz stereo 8Bit should steals more or less 3% (50 066 / 2 000 000) CPU memory bandwidth - about 8 scanlines per frame.


The other part is how fast DMA can flush the data from FIFIO to the CosmosEx. You mentioned 1MB/s - from the CPU point of view it doesn't matter whether it is 1MB/s or 2MB/s. That transfer is done between DMA FIFO and CosmosEx and it has no impact onto the CPU.
Jaugar / TT030 / Mega STe / 800 XL / 1040 STe / Falcon030 / 65 XE / 520 STm / SM124 / SC1435
SDrive / PAK68/3 / CosmosEx / SatanDisk / UltraSatan / USB Floppy Drive Emulator / Eiffel / SIO2PC / Crazy Dots / PAM Net
Hatari / Aranym / Steem / Saint
http://260ste.appspot.com/

User avatar
alexh
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2581
Joined: Wed Oct 20, 2004 1:52 pm
Location: UK - Oxford
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby alexh » Wed Apr 27, 2016 4:02 pm

AdamK wrote:It takes a lot of CPU time on Raspi, something is wrong there IMO.

AFAIK Raspberry Pi has hardware MP3 decoder and so everything is done on the CPU

User avatar
leonard
Moderator
Moderator
Posts: 640
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby leonard » Wed Apr 27, 2016 9:02 pm

prepare ym data on Pi directly


tin is right, a very fast STF player is possible, even with more than 8bits sample quality. It requieres to prepare some data on rasp. On the ST side we can use almost 12bits quality YM table. And if you remove any TOS or ROM access in the main player core, we even can use registers without save/restore. Then the interrupt routine should looks like that:

Code: Select all

timerA:   move.w   (a6)+,a5   ; prepared data, points in low memory on a 12bits YM table
      move.l   (a5)+,d7
      move.l   (a5),(a4)   ; a4=$ffff8800.w
      movep.l   d7,0(a4)
      rte


of course the code would be a bit slower because you have to handle the buffer looping.

Code: Select all

sLoop:   lea      buffer(pc),a6
      bra.s      sBack

timerA:   move.w   (a6)+,a5   ; prepared data, points in low memory on a 12bits YM table
      cmpa.l   #buffer+bufferlen*2,a6
      beq.s   sLoop
sBack:   move.l   (a5)+,d7
      move.l   (a5),(a4)   ; a4=$ffff8800.w
      movep.l   d7,0(a4)
      rte


if it's not a problem to copy more data each frame, rasp can duplicate the end of the buffer to avoid the loop code. ( but cmpa and beq.s are not a big deal)
Leonard/OXYGENE.

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Wed Apr 27, 2016 11:01 pm

leonard wrote:

Code: Select all

sLoop:   lea      buffer(pc),a6
      bra.s      sBack

timerA:   move.w   (a6)+,a5   ; prepared data, points in low memory on a 12bits YM table
      cmpa.l   #buffer+bufferlen*2,a6
      beq.s   sLoop
sBack:   move.l   (a5)+,d7
      move.l   (a5),(a4)   ; a4=$ffff8800.w
      movep.l   d7,0(a4)
      rte


if it's not a problem to copy more data each frame, rasp can duplicate the end of the buffer to avoid the loop code. ( but cmpa and beq.s are not a big deal)

you do it that way, because a branch is slower when taken, right ? :)

could you explain me how YM digitized sound work ?
I understand that is is done by setting the volume on the 3 YM channels which is registers $8 $9 and $A of YM
but the volume itself is only 4 bits, right ? how do you get 8 or almost 12 bit quality ?
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

tin
Atari freak
Atari freak
Posts: 70
Joined: Mon Jul 23, 2012 7:59 am
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby tin » Thu Apr 28, 2016 1:32 am

leonard wrote:of course the code would be a bit slower because you have to handle the buffer looping.

Well, in a "we're ditching TOS"-szenario, it's probably faster to just not bother and let the hardware deal with that - just by requesting a special sector from CE at buffer end which includes a sample entry that generates an address or bus error.

For lower replay rates while TOS/Gem is active, I would propose that the PI generates sth like that (per sample):

Code: Select all

move.l #0,8800.w
move.l #0,8800.w
move.l #0,8800.w
add.w  #8*3+6+2,$timervector+2.w
rte

Where 0 is replaced with the respective samples' ym registers/values of course. Depending on the sample rate and the resulting DMA bus time the data transfer needs and steals from CPU (2 cycles per byte as Cyprian pointed out, so 68 in total per sample), this should be pretty fast (and also doesn't need buffer checks).
The PI could even try to optimize register volume usage in a way, that consequent register changes (and thus move.ls) are removed (e.g. only two or one register change might be needed on some samples, because the needed register value is already set).

Unfortunately at 50kHz this method produces a bit too much data for the CE to transfer :-)

User avatar
leonard
Moderator
Moderator
Posts: 640
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby leonard » Thu Apr 28, 2016 9:20 am

you do it that way, because a branch is slower when taken, right ?


exactly, 4 cycles win each sample

YM can "simulate" a 8 bits DAC using three 4 bits voices because YM volumes are logarithmic. So you can use small log values to "fine adjust" the global volume.
Leonard/OXYGENE.

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Thu Apr 28, 2016 7:28 pm

leonard wrote:YM can "simulate" a 8 bits DAC using three 4 bits voices because YM volumes are logarithmic. So you can use small log values to "fine adjust" the global volume.

OK, if volume is set to value v (0 to 15) in YM register $8, $9 $A, real level of voice is exp(v)
Maximum total level is 3exp(15), minimum 3exp(0)=3 (well I guess you can even turn off the voice :)
which exp() base is used ? 2? (which means +1 => +3dB) less so +1 => +1dB ?
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

nanard
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 109
Joined: Mon Apr 04, 2016 2:11 pm

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby nanard » Thu Apr 28, 2016 9:18 pm

according to this document : http://www.ym2149.com/ym2149.pdf
voltage is doubled (+3dB) when volume level is +2 so the exp() base is sqrt(2)
Each level is +1.5dB
4MB STE + CosmosEx /|\ MegaST4 + MegaFile 44

User avatar
Kirkman
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 121
Joined: Fri Sep 03, 2010 2:29 am

Re: MP3 playing on STE - Decoding through the CosmosEx

Postby Kirkman » Fri Apr 29, 2016 7:48 pm

All I can say is wow.


Social Media

     

Return to “CosmosEx”

Who is online

Users browsing this forum: No registered users and 2 guests