MiSTer updater script

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, Greenious, spiny, Sorgelig, Moderator Team

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

MiSTer updater script

Postby Locutus73 » Tue Dec 11, 2018 11:00 am

Hi, I made a small Bash script in order to update all MiSTer cores, including menu.rbf and the main MiSTer Linux executable. Simply put the attached mister_updater.sh in your SD and launch it through SSH. You can edit it (Win users, please use Notepad++) in order to change some self-explanatory variables (paths, deleting old cores, etc.). I take no responsibility for any data loss or anything, if your DE10-Nano catches fire it’s up to you: use the script at your own risk.

Regards.

Locutus73
You do not have the required permissions to view the files attached to this post.

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5089
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Postby Sorgelig » Tue Dec 11, 2018 11:19 am

Good addition!
Sometimes i forget to update the cores on SD myself :)

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Tue Dec 11, 2018 6:28 pm

Sorgelig wrote:Good addition!
Sometimes i forget to update the cores on SD myself :)

Obviously, needless to say, if you like it, feel free to use it wherever you want, i.e. the SD installer or the Wiki.

Now, back to something we discussed months ago: I think a menu in the main MiSTer for launching scripts would be cool. I understand that displaying the output would be tricky (not totally mandatory tho) and that the full Linux cores can be used to launch scripts and do complex stuff... but a menu would be still cool... and again I know it’s open source so anyone interested can code the function :lol:

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Tue Dec 11, 2018 9:50 pm

Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables.

Regards.

Locutus73
You do not have the required permissions to view the files attached to this post.

esmith13
Atariator
Atariator
Posts: 28
Joined: Tue Oct 30, 2018 8:06 pm
Contact:

Re: MiSTer updater script

Postby esmith13 » Tue Dec 11, 2018 11:57 pm

Excellent work and thank you!

I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!

User avatar
Newsdee
Atari God
Atari God
Posts: 1463
Joined: Fri Sep 19, 2014 8:40 am

Re: MiSTer updater script

Postby Newsdee » Wed Dec 12, 2018 2:16 am

esmith13 wrote:I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!


If we go for this, I can suggest four folders: _Computer_8bit and _Computer_16bit. Just because there are quite a bunch of both.

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Wed Dec 12, 2018 10:26 am

Newsdee wrote:
esmith13 wrote:I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!


If we go for this, I can suggest four folders: _Computer_8bit and _Computer_16bit. Just because there are quite a bunch of both.


It would be nice, but I’d like to use the Wiki as the source of any structure/data parsed in order to split cores. So, as @esmith13 suggested, I could split cores just like they are in the wiki itself, so Cores, Console Cores, Arcade Cores and Service cores, but I have no way to know if a core is 16 or 32 bit. Any hint is appreciated.

Regards.

Locutus73

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5089
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Postby Sorgelig » Wed Dec 12, 2018 10:49 am

I don't see the need to split the cores to 8 and 16 bit. It's absolutely doesn't matter.
Current division for Consoles, Arcades and Computer cores has meaning as they are differently operated by user.
There is no difference between 8 and 16 bits. For example TurboGrafx16 is advertised as 16 bit but really CPU is 8bit. At the same time this core compete and sometimes win over Genesis which is 16bit.
Amiga originally can be treated as 16bit computer but actually CPU is 32bit.
So, 8/16 bits are really doesn't matter.

User avatar
Newsdee
Atari God
Atari God
Posts: 1463
Joined: Fri Sep 19, 2014 8:40 am

Re: MiSTer updater script

Postby Newsdee » Wed Dec 12, 2018 2:53 pm

It's not so much about accuracy but splitting the list between older computers and newer ones, and keep the choices on screen shorter.

I don't split consoles since its all similar in operation, but a Mac runs differenly to an Apple II (if anything just boot time). So I go to different menues depending on the kind of experience I'm in the mood for.

Anyway just a matter of taste. :P

MattE888
Atari freak
Atari freak
Posts: 51
Joined: Fri Nov 02, 2018 5:34 pm

Re: MiSTer updater script

Postby MattE888 » Wed Dec 12, 2018 11:06 pm

Locutus73 wrote:Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables.

Regards.

Locutus73


Now we just need an update script that updates the update script! Just kidding, thank you for doing this!

uigiflip
Atari freak
Atari freak
Posts: 58
Joined: Sun May 13, 2018 6:41 pm

Re: MiSTer updater script

Postby uigiflip » Wed Dec 12, 2018 11:59 pm

is there way to put 8bit console rbfs into my 8bit consoles folder and 8bit micro rbfss in my 8bit micros folder etc?
Replay 2, Mister, FPGA Replay + 68060 Daughterboard

esmith13
Atariator
Atariator
Posts: 28
Joined: Tue Oct 30, 2018 8:06 pm
Contact:

Re: MiSTer updater script

Postby esmith13 » Thu Dec 13, 2018 1:27 am

uigiflip wrote:is there way to put 8bit console rbfs into my 8bit consoles folder and 8bit micro rbfss in my 8bit micros folder etc?


That's what we're discussing. Right now you can only use 2 folders - one for arcades and the other for everything else. You can already customize the paths for those two.

warham
Atari maniac
Atari maniac
Posts: 90
Joined: Fri Oct 12, 2018 7:17 am

Re: MiSTer updater script

Postby warham » Thu Dec 13, 2018 7:02 am

Im with Sorgelig :p
if the functionality is added to be able to added extra folder locations I hope there is a way to disable it also. People like myself do not distinguish between 8,16,32, etc. I just know if its a console, arcade, pc, utility and prefer not to search through endless menus.

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Thu Dec 13, 2018 8:55 am

warham wrote:Im with Sorgelig :p
if the functionality is added to be able to added extra folder locations I hope there is a way to disable it also. People like myself do not distinguish between 8,16,32, etc. I just know if its a console, arcade, pc, utility and prefer not to search through endless menus.

Whatever extra directory, it can always be disabled: i.e. the current script has two variables for defining two target directories:
CORES_PATH="$BASE_PATH"
ARCADE_CORES_PATH="$BASE_PATH/_Arcade"
You can set both variables to the same path actually disabling any distinction.
Anyway, if/when I’ll implement extra directories (I’m interested in implementing other functionalities first), the script will distinguish between cores parsing the MiSTer wiki, so Cores, Console Cores, Arcade Cores and Service cores as in the wiki.

Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Thu Dec 13, 2018 9:05 am

@Sorgelig or whoever can help me.
Please excuse my ignorance. I have a GitHub account I used just for posting “issues” on other people repositories. Now, I know how to add a repository to my profile, so I could share the script through GitHub, but how could I add the repository to https://github.com/MiSTer-devel ? I mean, I see there are repositories there, i.e. Filters_MiSTer, managed by other people than Sorgelig, am I missing something?

Thank you very much in advance.
Best regards.

Locutus73

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5089
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Postby Sorgelig » Thu Dec 13, 2018 10:34 am

I need to add you into MiSTer group. What is your user name on github?
Upd: i've invited Locutus73 on github. Accept it and you will be able to create your repository in MiSTer-devel.

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Thu Dec 13, 2018 10:44 am

My GitHub user name is Locutus73
[EDIT]
I accepted the invite and I'll create the repository, thank you very much.
[EDIT2]
I created the repository
https://github.com/MiSTer-devel/Updater_script_MiSTer
from now onwards all the updater updates will be there

Thank you in advance.
Regards.

Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Fri Dec 14, 2018 7:56 pm

Hi, I just updated the script to Version 1.2
Added support for distinct directories for computer cores, console cores, arcade cores and service cores (default values are "_Computer", "_Console", "_Arcade", "_Utility"
Added an option for removing "Arcade-" prefix from arcade core names (default value is true)

https://github.com/MiSTer-devel/Updater_script_MiSTer

Best regards
Locutus73

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Sat Dec 15, 2018 10:16 am

One command trick for updating MiSTer with my script from your computer, without actually installing it on the SD and always using the latest updater version:

using Linux or OSX

Code: Select all

ssh root@MiSTer "curl -ksLf https://github.com/MiSTer-devel/Updater_script_MiSTer/blob/master/mister_updater.sh?raw=true | bash -"

using Windows after installing PuTTY https://www.chiark.greenend.org.uk/~sgt ... atest.html

Code: Select all

plink -ssh -pw 1 root@MiSTer "curl -ksLf https://github.com/MiSTer-devel/Updater_script_MiSTer/blob/master/mister_updater.sh?raw=true | bash -"


Regards.
Locutus73

warham
Atari maniac
Atari maniac
Posts: 90
Joined: Fri Oct 12, 2018 7:17 am

Re: MiSTer updater script

Postby warham » Sun Dec 16, 2018 2:11 pm

It's a beautiful thing :) Thank you Sir.

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Sun Dec 16, 2018 4:03 pm

Hi, I just updated the script to version 1.3

Version 1.3 - 2018.12.16 - Added Kernel, Linux filesystem and bootloader updating functionality; added autoreboot option.
Version 1.2 - 2018.12.14 - Added support for distinct directories for computer cores, console cores, arcade cores and service cores; added an option for removing "Arcade-" prefix from arcade core names
Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables
Version 1.0 - 2018.12.11 - First commit

Beware: updating Linux may be dangerous, if you don't want to take any risk, please comment the SD_INSTALLER_URL assignment this way
#SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer"

https://github.com/MiSTer-devel/Updater_script_MiSTer

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Sun Dec 16, 2018 5:09 pm

Released version 1.3.1 of the updater script

Version 1.3.1 - 2018.12.16 - Disabled Linux updating as default behaviour.
uncomment #SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Installer-Win64_MiSTer" to enable it

Sometimes linux.img remains the old one, I have to investigate.

https://github.com/MiSTer-devel/Updater_script_MiSTer

[EDIT]
When I extract the release_20181109.rar I overwrite all files in the linux directory including linux.img which is the mounted linux filesystem. This shouldn't be a problem, because the kernel loaded in memory should maintain the handle to the old file, while unrar should uncompress the new one (unix systems let you move/delete open files), but it seems that sometimes the old linux.img was retained. Any idea?

Regards.
Locutus73

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5089
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Postby Sorgelig » Sun Dec 16, 2018 7:14 pm

if you log into console, root filesystem (which is linux.img) is remounted to read-write. And then if you replace it, you risk to corrupt something...

Locutus73
Captain Atari
Captain Atari
Posts: 486
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Postby Locutus73 » Sun Dec 16, 2018 8:06 pm

Sorgelig wrote:if you log into console, root filesystem (which is linux.img) is remounted to read-write. And then if you replace it, you risk to corrupt something...

I agree, but in theory, Linux should distinguish between file handles and directory entries: I mean, in theory the system mounts linux.img in read-write mode, but I should be able to delete it, actually removing the entry from the directory, while the handle remains still live, so I should be able to extract the new linux.img, while the deleted (from the directory) one is still pointed by the handler.
What I have observed so far:
1) If I extract the new linux.img over the older one, sometimes I have the new one other times I have the old one
2) If I delete the old linux.img before extracting the new one, I get the new one after the reboot, but the dd command in the updateboot script fails (corrupting the bootloader)
Now I’m trying to first extract and update the bootloader, then deleting the old linux.img and extracting the new one just before the reboot.

Regards.
Locutus73

Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 5089
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Postby Sorgelig » Sun Dec 16, 2018 8:44 pm

linux.img is special.
It's residing in exFAT volume which is not native to Linux FS, and may not follow handler used by Linux.
Also, since it's root FS which is actually loop - it may have its own quirks.


Return to “MiSTer”

Who is online

Users browsing this forum: No registered users and 7 guests