LZMA - best packing method ?

GFA, ASM, STOS, ...

Moderators: simonsunnyboy, Mug UK, Zorro 2, Moderator Team

Post Reply
AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

LZMA - best packing method ?

Post by AtariZoll »

It's used in some latest archivers, so in UPX too - they prefer now it. Possible to make TOS SFX with it, but only if it is given in command line. With reason - depacking is extremely slow. I disassembled depacker - it is very long (2.3 KB code), and I don't think that there is place to make is significantly faster. Yes, there is source code available, but it's not exactly same with what I see in SFX file made with UPX 3.93, plus is not good for Devpac.
I packed 1 screen data with LZMA - using freeware xz packer, ZIP, WinRAR and JEK 2.01.
LZMA: 4686 bytes
ZIP - 5595
WinRAR with best packing - 5250
JEK - 6844
Needs about 5 secs to depack 1 screen data (32KB), what is actually not worse than some depackers used in Atari SW, but with less efficient packing.
Packing on Atari is out of my consideration - I tried UPX.TTP, and it crashed as always. Will not bother to solve, because it would be too slow anyway.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.
mlynn1974
Captain Atari
Captain Atari
Posts: 362
Joined: Mon Mar 03, 2008 10:33 pm
Contact:

Re: LZMA - best packing method ?

Post by mlynn1974 »

I haven't seen the LZMA depacker for the Atari ST. It's interesting that there is one, but I would expect it to be slow.

There are a number of considerations for depackers on systems with limited memory:
1. Memory required by the depacker itself including stack size,
2. Can the decompressed file be written to an absolute or relative address,
3. Dictionary sizes and maximum string match lengths,
4. Decompresssion time.
If the compressor is written in C, depending the optimiser, it might not be as efficient as one written in native assembly.

How does PackFire compare with your test picture?
http://www.pouet.net/prod.php?which=54840

From memory there were lots of neat features in Pack-Ice 2.4 making it better than the older packers at the time.
Still got, still working: Atari 4Mb STe, 520STFM, 2.5Mb STF.
Hardware: Cumana CSA 354, Ultimate Ripper, Blitz Turbo, Synchro Express II (US and UK Versions).
AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Post by AtariZoll »

I tried briefly PackFire. In large model it made little shorter files than xz packer. Looking into output file, and depacker src. I'm pretty sure that PackFire uses LZMA method too :D It is good to have src for Atari depacker. I will deal with it little - obviously it will be shorter than 2KB . Will see depacking speed - will be not fast - there are multiplications for instance, as there are in depacker I used.

1. It needs 16KB stack space, where dictionary is placed too
2. You serve dest address in one address register of CPU before calling depacker, so it is absolute. Don't see for what relative addressing would be good.
Especially as it must deal with larger files than 64KB.
3. I don't know details - see 1. It worked well with over 50 files I packed for 23 games on 1 floppy project. Longest was over 300 KB,
4. Instead time the speed: about 4-6 KB/sec .

I will test PackFire depacking speeds.
I don't think that we even should mention ICE in same thread with LZMA :mrgreen: It is slow, and ratio is nothing impressive.

Did depacking test. Large model depacking code len. is 700 bytes, and is PC-relative - and that's is what is useful. Speed seems some 10-20% better than UPX depacker speed. So, definitely better implementation of LZMA for Atari, 68000 :D Thanx for pointing on it.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.
leonard
Moderator
Moderator
Posts: 665
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: LZMA - best packing method ?

Post by leonard »

For depacking speed of various depacker on various atari machine, you can refer to numbers here: http://www.atari-forum.com/viewtopic.ph ... 91#p316491 (it includes depacker routine size, and packfire tiny and large model .pft and .pfl)

Packfire has really good compression ratio, thanks to arithmetic coding algorithm (one of the key of LZMA). The drawback is the need of arithmetic operation, ie MUL in our case. Depacking is really slow compared to huffman entropy decoder. If you accept to lose just a little packing ratio but still want to use modern arithmetic class decoder, you could use shrinkler! the depacker is two times faster than Packfire. Really impressive. ( .shk in my benchmark )
Leonard/OXYGENE.
mikro
Hardware Guru
Hardware Guru
Posts: 2207
Joined: Sat Sep 10, 2005 11:11 am
Location: Kosice, Slovakia
Contact:

Re: LZMA - best packing method ?

Post by mikro »

I don't know what your problem with UPX is, I use it for packing all my releases and obviously it works.
AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Post by AtariZoll »

mikro wrote:I don't know what your problem with UPX is, I use it for packing all my releases and obviously it works.
I don't have that to add - Atari TTP works not . And I don't care actually.
How much time takes to pack some 300 KB executable on 8 MHz Atari ? Do you pack only executables, or some binary files too ?
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.
mikro
Hardware Guru
Hardware Guru
Posts: 2207
Joined: Sat Sep 10, 2005 11:11 am
Location: Kosice, Slovakia
Contact:

Re: LZMA - best packing method ?

Post by mikro »

AtariZoll wrote:I don't have that to add - Atari TTP works not . And I don't care actually.
How much time takes to pack some 300 KB executable on 8 MHz Atari ? Do you pack only executables, or some binary files too ?
Of course I pack it on PC. :) Executables only, depacking time is not really that bad, hardly noticeable. But bear in mind my software usually targets Falcons/CT60/FireBee, i.e. high end machines.
User avatar
troed
Atari God
Atari God
Posts: 1460
Joined: Mon Apr 30, 2012 6:20 pm
Location: Sweden

Re: LZMA - best packing method ?

Post by troed »

I use UPX for everything, executables and binaries. Requires some manual stripping and modifications by my build system.
AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Post by AtariZoll »

mikro wrote: Of course I pack it on PC. :) Executables only, depacking time is not really that bad, hardly noticeable. But bear in mind my software usually targets Falcons/CT60/FireBee, i.e. high end machines.
Depacking time is excellent with default NRV packing method - well default only in Atari mode. Otherwise LZMA is default now in PC mode - told me one of UPX guys.
It is easy when you deal with executables only. Packing binary files with NRV is troublesome- because used NRV packer is closed source.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.
rockyone
Atari Super Hero
Atari Super Hero
Posts: 511
Joined: Thu Jan 20, 2011 8:47 pm
Location: France
Contact:

Re: LZMA - best packing method ?

Post by rockyone »

I have failed to run PackFire.exe with Windows 7 32 and 64-bit
AtariZoll
Fuji Shaped Bastard
Fuji Shaped Bastard
Posts: 2978
Joined: Mon Feb 20, 2012 4:42 pm
Contact:

Re: LZMA - best packing method ?

Post by AtariZoll »

rockyone wrote:I have failed to run PackFire.exe with Windows 7 32 and 64-bit
That's command line program - run it in console.
Famous Schrodinger's cat hypothetical experiment says that cat is dead or alive until we open box and see condition of poor animal, which deserved better logic. Cat is always in some certain state - regardless from is observer able or not to see what the state is.
leonard
Moderator
Moderator
Posts: 665
Joined: Thu May 23, 2002 10:48 pm
Contact:

Re: LZMA - best packing method ?

Post by leonard »

UPX depacks faster than ARJBETA -m7 but has lower compression ratio. ( for instance, We Were @ demo won't fit on two floppies if I use UPX as a packer (I did the test))
If you really want best compression ratio and acceptable depacking speed, use ARJBeta.
Leonard/OXYGENE.
Post Reply

Return to “Coding”