Arcade cores

[brunosilva]

Storing the CRC of individual <part> elements (the ones that reference a rom file in a zip) and looking up zip entries by CRC instead of file name seems like it would be the easiest way to detect individual rom mismatches and report them as an error.

The tradeoff there is that if there's ever a new/updated dump of some component the mister loader will fail. I'd argue that's perhaps a good thing, as it means some assumptions the core was developed under have now changed. As it stands now it is vulnerable to a mame 'naming reorganization' breaking things (although those do not happen as frequently as some people seem to fear).

Thinking out loud: wouldn't a lookup via CRC also mean you don't need to have multiple rom part=0 entries to accommodate merged/nonmerged/split sets? Instead you'd have one entry and just turn the zip file name attribute into an ordered-by-priority list? So something like zip="nonmerged.zip|parent.zip' ?

Hi

maybe its a good idea... in a flash I (and probably you also) can generate the mra with crc and not path of the file inside the zips (maybe it could be there only to help/information".
for end user maybe worst... find a file by crc without any more reference.
but the "mra engine" must be rewrite...
not my call

[brunosilva]

@elvisa - you already got this here: https://mega.nz/#F!DI92zKBb!ucHvIuROpM0R6j6kw6yy-A

what did i do:
- pickup xml for the 3 sets for 0.217
- striped only the roms needed and created 3 smaller xml
- used romvault to get files and layouts for each xml
- used my routine to check the mra vs files in each set and it looked that is all there

(there is more mra than its in this set - i will add asap to this set) -we are setting up things so its "easy" to add more hacks/versions to alternatives folder

[zakk4223]

Storing the CRC of individual <part> elements (the ones that reference a rom file in a zip) and looking up zip entries by CRC instead of file name seems like it would be the easiest way to detect individual rom mismatches and report them as an error.

The tradeoff there is that if there's ever a new/updated dump of some component the mister loader will fail. I'd argue that's perhaps a good thing, as it means some assumptions the core was developed under have now changed. As it stands now it is vulnerable to a mame 'naming reorganization' breaking things (although those do not happen as frequently as some people seem to fear).

Thinking out loud: wouldn't a lookup via CRC also mean you don't need to have multiple rom part=0 entries to accommodate merged/nonmerged/split sets? Instead you'd have one entry and just turn the zip file name attribute into an ordered-by-priority list? So something like zip="nonmerged.zip|parent.zip' ?

Hi

maybe its a good idea... in a flash I (and probably you also) can generate the mra with crc and not path of the file inside the zips (maybe it could be there only to help/information".
for end user maybe worst... find a file by crc without any more reference.
but the "mra engine" must be rewrite...
not my call

I just implemented this in my main mister fork: https://github.com/zakk4223/Main_MiSTer

If a 'crc' attribute exists on a part tag, it will try to lookup the rom part via the CRC, and falls back to using the file name if that fails (or there is no crc attribute).

Additionally, 'zip' attribute values can now be of the form 'zip1.zip|zip2.zip' and the zips will be searched in the order they appear in the string.

With this you can use a single "rom" element for a clone and set the zip attribute to something like 'centiped3.zip|centiped.zip' and it will find the rom parts in a merged, non merged or split set.

I need to regenerate a bunch of MRA files with CRC attributes to do a larger test, but my initial tests have gone ok so far.

If it is agreed this approach is a good idea I'm happy to generate the PR.

[Sorgelig]

If it is agreed this approach is a good idea I'm happy to generate the PR.

i've looked to your code and it looks good. Idea is also good, so i agree with this.

I see mame has crc32, so can grab it directly from there.

[elvisa]

Storing the CRC of individual <part> elements (the ones that reference a rom file in a zip) and looking up zip entries by CRC instead of file name seems like it would be the easiest way to detect individual rom mismatches and report them as an error.

The tradeoff there is that if there's ever a new/updated dump of some component the mister loader will fail. I'd argue that's perhaps a good thing, as it means some assumptions the core was developed under have now changed. As it stands now it is vulnerable to a mame 'naming reorganization' breaking things (although those do not happen as frequently as some people seem to fear).

Thinking out loud: wouldn't a lookup via CRC also mean you don't need to have multiple rom part=0 entries to accommodate merged/nonmerged/split sets? Instead you'd have one entry and just turn the zip file name attribute into an ordered-by-priority list? So something like zip="nonmerged.zip|parent.zip' ?

This is essentially what MAME does, via the "DAT format" information it stores which also enables other checksums (MAME stores currently CRC32, MD5 and SHA1SUM entries for all ROM parts). Grab the MAME binary and run "mame -listxml" to see it in action. If you don't want to run MAME every time, "mame -listxml > arcaderoms.xml" produces a file you can import into most ROM managers (rename it to .dat if .xml doesn't work).

It looks up several known zip names, and then reads in the ROM parts by checksum. A failure is then thrown to the screen on the specific part that failed, not the ZIP file as a whole.

In reverse, these checksum lists can be used to verify an entire ZIP file and/or rebuild it from a pool of downloads. Tools exist already to produce comprehensive "have" or "miss" lists to let a user know exactly what they can and can't play without having to go through trial and error. That's how tools like ROMVault, RomCentre, ClrMAMEPro all do this now, and could easily be used to manage MiSTer ROMs too.

The DAT files I produced on my GitHub link above were via a lengthy process of collecting various ROMs, building them via the "build_rom.sh" scripts in the MiSTer repos, verifying the final MD5 of the built ROM, and then on successful exit copying the ROM parts back to another named directory to then run "DIR2DAT" over them to create the DAT files. I'm happy to keep contributing/updating these, however all of that information is already built into MAME and could be used by MiSTer right now.

If it is agreed this approach is a good idea I'm happy to generate the PR.

i've looked to your code and it looks good. Idea is also good, so i agree with this.

I see mame has crc32, so can grab it directly from there.

Great news! I only wish I could help at a code level.

[brunosilva]

If it is agreed this approach is a good idea I'm happy to generate the PR.

i've looked to your code and it looks good. Idea is also good, so i agree with this.

I see mame has crc32, so can grab it directly from there.

nice

i will leave for now the name and i will add crc32... at least for a while?

[brunosilva]

@elvisa

check my post... no need to do all that

[zakk4223]

If it is agreed this approach is a good idea I'm happy to generate the PR.

i've looked to your code and it looks good. Idea is also good, so i agree with this.

I see mame has crc32, so can grab it directly from there.

nice

i will leave for now the name and i will add crc32... at least for a while?

The name is still used for error messages, so best to leave it in.

[brunosilva]

ok i wil leave the filename without folder

[brunosilva]

hi
no issues in your github so I place here some "requests"

i think you are only contemplating having one rom index="0" in xml, right?

1) i build for now a new rom=0 before the 3 sets to test. its not possible to if crc match, dont process the other rom 0's?

md5[29dbfb61bc9e45f2cebe63743de85a57]
md5-calc[29dbfb61bc9e45f2cebe63743de85a57]
file_finish: 28672 bytes sent to FPGA <= here its a success

but it still tries to process the other roms.
md5[29dbfb61bc9e45f2cebe63743de85a57]
md5-calc[d41d8cd98f00b204e9800998ecf8427e]
mismatch
file_finish: no data, discarded
md5[29dbfb61bc9e45f2cebe63743de85a57]
md5-calc[d41d8cd98f00b204e9800998ecf8427e]
mismatch
file_finish: no data, discarded
md5[29dbfb61bc9e45f2cebe63743de85a57]
md5-calc[d41d8cd98f00b204e9800998ecf8427e]
mismatch
file_finish: no data, discarded

2) if the zip file doesnt exist, dont try to find every file inside the zip?
file: /media/fat/_Arcade/mame/amidar.zip/am2d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed

mame_root /media/fat/_Arcade
file: /media/fat/_Arcade/mame/amidar.zip/am2d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2d does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2e, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2e, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2e does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2f, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2f, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2f does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2h, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2h, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2h does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2j, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2j, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2j does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2l, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2l, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2l does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2m, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2m, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2m does not exist
file: /media/fat/_Arcade/mame/amidar.zip/am2p, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/am2p, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
am2p does not exist
file: /media/fat/_Arcade/mame/amidar.zip/2716.a5, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/2716.a5, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
2716.a5 does not exist
file: /media/fat/_Arcade/mame/amidar.zip/2716.a6, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/2716.a6, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
2716.a6 does not exist
file: /media/fat/_Arcade/mame/amidar.zip/amidarus.5c, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/amidarus.5c, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
amidarus.5c does not exist
file: /media/fat/_Arcade/mame/amidar.zip/amidarus.5d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidar.zip, error:file open failed
file: /media/fat/_Arcade/mame/amidars.zip/amidarus.5d, start=0, len=0
FileOpenZip(mz_zip_reader_init_file) Zip:/media/fat/_Arcade/mame/amidars.zip, error:file open failed
amidarus.5d does not exist
md5[29dbfb61bc9e45f2cebe63743de85a57]
md5-calc[d41d8cd98f00b204e9800998ecf8427e]

3) i think the error that you display is the last error that you get in the process. if zips dont exist, isn't possible to give a error saying zip not found? i know that now could be more hard because now we have more than one zip and depending on the set used, the zip could not exist and it isn't an error.

4) because mister will try to use crc, and some files could have the same name but different crc, its possible to display crc in console logs?

5) if no crc present in mra, mister will default to filenames? or it will be crc only?
(this is already answered by text in PR)

Thanks

[brunosilva]

double post... delete

[TempleSidekick]

I appreciate all the work that's being done to improve the cores, but the switch to these mra files was very abrupt for me since I had just been running the updater script for over a year and never checking these forums. I had gotten the arcade cores working before but I couldn't find this new setup documented, so I'm just putting this here for anyone else wondering why their arcade cores are suddenly looking for .zip files.

Your _Arcade directory now has a bunch of .mra files. They list the .zip files they depend on. Put those .zip files in _Arcade/mame and you should be back in business.

Overall it seems much nicer than having to generate the roms but definitely confusing at first. Thanks everyone for putting so much work into making these!

[elvisa]

@elvisa

check my post... no need to do all that

I'm very excited to see this in action. It'll be a huge step forwards for MiSTer ROM management.

[Locutus73]

I appreciate all the work that's being done to improve the cores, but the switch to these mra files was very abrupt for me since I had just been running the updater script for over a year and never checking these forums. I had gotten the arcade cores working before but I couldn't find this new setup documented, so I'm just putting this here for anyone else wondering why their arcade cores are suddenly looking for .zip files.

Your _Arcade directory now has a bunch of .mra files. They list the .zip files they depend on. Put those .zip files in _Arcade/mame and you should be back in business.

Overall it seems much nicer than having to generate the roms but definitely confusing at first. Thanks everyone for putting so much work into making these!

If you add MAME_ARCADE_ROMS="false" and MAME_ALT_ROMS="false" in your ini the previous setup (old rom files along rbf in _Arcade) will be restored. This setup should still work.
Since yesterday the updater considers MAME_ARCADE_ROMS="" and MAME_ARCADE_ROMS="" as do nothing and that’s the default value, instead of defaulting to "false" (which tries to revert the current structure to the old style).
http://www.atari-forum.com/viewtopic.ph ... 50#p390654

I guess you’d like to use the new setup: if you ended with MRA files into _Arcade, you willingly manually set MAME_ARCADE_ROMS="true". If this is the case, you must put the MAME and HBMAME zipped ROMs into _Arcade/mame and _Arcade/hbmame (if, how and where to find them is up to you). Otherwise as described above, the updater can revert to the previous file/directory structure for you.

Regards.
Locutus73

[TempleSidekick]

@Locutus73 Thanks for the clarification!

To be fair, I didn't willingly set MAME_ARCADE_ROMS="true". After trying to figure out what was going on myself and then ending up on this forum, I came across a post by brunosilva where he had updated the updater script so I haphazardly grabbed everything and ran it. Apparently that did include the .ini file with that parameter.

What _would_ be really convenient, though, is if the build_mame_roms script took the new method into account and just pulled down the necessary zips. Maybe that's coming, maybe it's somewhere that I can contribute.

Side question while I'm here, is there any good way of saving the high scores? I understand why save states aren't implemented, but is there some cool website or tool that looks cool for manual entries vs a pad of paper near the TV? Part of the arcade experience is high-score chasing, so I'm just wondering how y'all are getting your fix.

[raibol1]

Hallo,

does anyone know where to download spyhunter.rbf??
Or is there an manual, how i can compile this core for mister??

Thanks in advance.

Greets

Rai

[NML32]

Hallo,

does anyone know where to download spyhunter.rbf??
Or is there an manual, how i can compile this core for mister??

Thanks in advance.

Greets

Rai

https://github.com/alanswx/Arcade-SpyHunter_MiSTer

[raibol1]

@NML32:

Thank you very much!

Greets

Rai

[alanswx]

Hallo,

does anyone know where to download spyhunter.rbf??
Or is there an manual, how i can compile this core for mister??

Thanks in advance.

Greets

Rai

https://github.com/alanswx/Arcade-SpyHunter_MiSTer

It is close. I have some other arcade updates I need to finish before I work on getting spy hunter and the other cores online.

[wakodump]

I am trying out the new mra format. I put the correct files on /_Arcade/mame and hbmame.

All the cores on /_Arcade work

All the cores on /_Arcade/_alternative will not load.

I have the correct rom files, but when I try to start for example any 1942 core or hack they don't load. I just get back to the core select menu.
I am currently up to date with the update script. Anyone has had this happen to them? Do you have a clue of what could I be doing wrong.

[bigrico]

So this is my structure...

/_Arcade/*.mra
/_Arcade/cores/*.rbf
/_Arcade/mame/*.zip
/_Arcade/hbmame/*.zip

I have the ini pointing to these folders, none of the arcade cores are launching. Any help?

[djmartins]

So this is my structure...

/_Arcade/*.mra
/_Arcade/cores/*.rbf
/_Arcade/mame/*.zip
/_Arcade/hbmame/*.zip

I have the ini pointing to these folders, none of the arcade cores are launching. Any help?

That should work, hook up the console and see what it is trying to load and not finding....
I suspect a ROM CRC mismatch but do that and you can read the errors messages and know for sure what's going on.

[crocky]

Does anyone know why the direction pad gets flipped on the gamepad on Galaga if the rotation is turned off? Same behaviour on Galaxian.

[esmith13]

Hi all,

Can anyone tell me where I can go to get arcade cores that are not part of the main updater script?
I know some are locked for patrons only for a while, but by my count there should be at least half a dozen arcade cores I could get right now that just aren't part of the main update script.
Since I switched over to using MRAs It's even more obvious how many rbf files I'm missing.

Any help would be appreciated!

Thank you in advance!

[jca]

Hi all,

Can anyone tell me where I can go to get arcade cores that are not part of the main updater script?
I know some are locked for patrons only for a while, but by my count there should be at least half a dozen arcade cores I could get right now that just aren't part of the main update script.
Since I switched over to using MRAs It's even more obvious how many rbf files I'm missing.

Any help would be appreciated!

Thank you in advance!

A while back someone posted a spreadsheet listing all cores (I think it was NML32).
Here is the link.