MiSTer updater script

https://github.com/MiSTer-devel/Main_MiSTer/wiki

Moderators: Mug UK, Zorro 2, spiny, Greenious, Sorgelig, Moderator Team

Locked
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

MiSTer updater script

Post by Locutus73 »

Hi, I made a small Bash script in order to update all MiSTer cores, including menu.rbf and the main MiSTer Linux executable. Simply put the attached mister_updater.sh in your SD and launch it through SSH. You can edit it (Win users, please use Notepad++) in order to change some self-explanatory variables (paths, deleting old cores, etc.). I take no responsibility for any data loss or anything, if your DE10-Nano catches fire it’s up to you: use the script at your own risk.

Regards.

Locutus73
You do not have the required permissions to view the files attached to this post.
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Post by Sorgelig »

Good addition!
Sometimes i forget to update the cores on SD myself :)
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Sorgelig wrote:Good addition!
Sometimes i forget to update the cores on SD myself :)
Obviously, needless to say, if you like it, feel free to use it wherever you want, i.e. the SD installer or the Wiki.

Now, back to something we discussed months ago: I think a menu in the main MiSTer for launching scripts would be cool. I understand that displaying the output would be tricky (not totally mandatory tho) and that the full Linux cores can be used to launch scripts and do complex stuff... but a menu would be still cool... and again I know it’s open source so anyone interested can code the function :lol:

Regards.

Locutus73
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables.

Regards.

Locutus73
You do not have the required permissions to view the files attached to this post.
esmith13
Atari User
Atari User
Posts: 32
Joined: Tue Oct 30, 2018 8:06 pm
Contact:

Re: MiSTer updater script

Post by esmith13 »

Excellent work and thank you!

I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!
User avatar
Newsdee
Atari God
Atari God
Posts: 1561
Joined: Fri Sep 19, 2014 8:40 am

Re: MiSTer updater script

Post by Newsdee »

esmith13 wrote: I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!
If we go for this, I can suggest four folders: _Computer_8bit and _Computer_16bit. Just because there are quite a bunch of both.
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Newsdee wrote:
esmith13 wrote: I realize you parse the GitHub link text for arcade cores vs other cores, but is there any way to split core updates 3 ways to the folders '_Arcade', '_Computer', and '_Console'? That would make this the ULTIMATE updater script!!!
If we go for this, I can suggest four folders: _Computer_8bit and _Computer_16bit. Just because there are quite a bunch of both.
It would be nice, but I’d like to use the Wiki as the source of any structure/data parsed in order to split cores. So, as @esmith13 suggested, I could split cores just like they are in the wiki itself, so Cores, Console Cores, Arcade Cores and Service cores, but I have no way to know if a core is 16 or 32 bit. Any hint is appreciated.

Regards.

Locutus73
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Post by Sorgelig »

I don't see the need to split the cores to 8 and 16 bit. It's absolutely doesn't matter.
Current division for Consoles, Arcades and Computer cores has meaning as they are differently operated by user.
There is no difference between 8 and 16 bits. For example TurboGrafx16 is advertised as 16 bit but really CPU is 8bit. At the same time this core compete and sometimes win over Genesis which is 16bit.
Amiga originally can be treated as 16bit computer but actually CPU is 32bit.
So, 8/16 bits are really doesn't matter.
User avatar
Newsdee
Atari God
Atari God
Posts: 1561
Joined: Fri Sep 19, 2014 8:40 am

Re: MiSTer updater script

Post by Newsdee »

It's not so much about accuracy but splitting the list between older computers and newer ones, and keep the choices on screen shorter.

I don't split consoles since its all similar in operation, but a Mac runs differenly to an Apple II (if anything just boot time). So I go to different menues depending on the kind of experience I'm in the mood for.

Anyway just a matter of taste. :P
MattE888
Atari freak
Atari freak
Posts: 51
Joined: Fri Nov 02, 2018 5:34 pm

Re: MiSTer updater script

Post by MattE888 »

Locutus73 wrote:Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables.

Regards.

Locutus73
Now we just need an update script that updates the update script! Just kidding, thank you for doing this!
uigiflip
Atari freak
Atari freak
Posts: 63
Joined: Sun May 13, 2018 6:41 pm

Re: MiSTer updater script

Post by uigiflip »

is there way to put 8bit console rbfs into my 8bit consoles folder and 8bit micro rbfss in my 8bit micros folder etc?
Replay 2, Mister, FPGA Replay + 68060 Daughterboard
esmith13
Atari User
Atari User
Posts: 32
Joined: Tue Oct 30, 2018 8:06 pm
Contact:

Re: MiSTer updater script

Post by esmith13 »

uigiflip wrote:is there way to put 8bit console rbfs into my 8bit consoles folder and 8bit micro rbfss in my 8bit micros folder etc?
That's what we're discussing. Right now you can only use 2 folders - one for arcades and the other for everything else. You can already customize the paths for those two.
warham
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 135
Joined: Fri Oct 12, 2018 7:17 am

Re: MiSTer updater script

Post by warham »

Im with Sorgelig :p
if the functionality is added to be able to added extra folder locations I hope there is a way to disable it also. People like myself do not distinguish between 8,16,32, etc. I just know if its a console, arcade, pc, utility and prefer not to search through endless menus.
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

warham wrote:Im with Sorgelig :p
if the functionality is added to be able to added extra folder locations I hope there is a way to disable it also. People like myself do not distinguish between 8,16,32, etc. I just know if its a console, arcade, pc, utility and prefer not to search through endless menus.
Whatever extra directory, it can always be disabled: i.e. the current script has two variables for defining two target directories:
CORES_PATH="$BASE_PATH"
ARCADE_CORES_PATH="$BASE_PATH/_Arcade"
You can set both variables to the same path actually disabling any distinction.
Anyway, if/when I’ll implement extra directories (I’m interested in implementing other functionalities first), the script will distinguish between cores parsing the MiSTer wiki, so Cores, Console Cores, Arcade Cores and Service cores as in the wiki.

Regards.

Locutus73
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

@Sorgelig or whoever can help me.
Please excuse my ignorance. I have a GitHub account I used just for posting “issues” on other people repositories. Now, I know how to add a repository to my profile, so I could share the script through GitHub, but how could I add the repository to https://github.com/MiSTer-devel ? I mean, I see there are repositories there, i.e. Filters_MiSTer, managed by other people than Sorgelig, am I missing something?

Thank you very much in advance.
Best regards.

Locutus73
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Post by Sorgelig »

I need to add you into MiSTer group. What is your user name on github?
Upd: i've invited Locutus73 on github. Accept it and you will be able to create your repository in MiSTer-devel.
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

My GitHub user name is Locutus73
[EDIT]
I accepted the invite and I'll create the repository, thank you very much.
[EDIT2]
I created the repository
https://github.com/MiSTer-devel/Updater_script_MiSTer
from now onwards all the updater updates will be there

Thank you in advance.
Regards.

Locutus73
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Hi, I just updated the script to Version 1.2
Added support for distinct directories for computer cores, console cores, arcade cores and service cores (default values are "_Computer", "_Console", "_Arcade", "_Utility"
Added an option for removing "Arcade-" prefix from arcade core names (default value is true)

https://github.com/MiSTer-devel/Updater_script_MiSTer

Best regards
Locutus73
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

One command trick for updating MiSTer with my script from your computer, without actually installing it on the SD and always using the latest updater version:

using Linux or OSX

Code: Select all

ssh root@MiSTer "curl -ksLf https://github.com/MiSTer-devel/Updater_script_MiSTer/blob/master/mister_updater.sh?raw=true | bash -"
using Windows after installing PuTTY https://www.chiark.greenend.org.uk/~sgt ... atest.html

Code: Select all

plink -ssh -pw 1 root@MiSTer "curl -ksLf https://github.com/MiSTer-devel/Updater_script_MiSTer/blob/master/mister_updater.sh?raw=true | bash -"
Regards.
Locutus73
warham
Obsessive compulsive Atari behavior
Obsessive compulsive Atari behavior
Posts: 135
Joined: Fri Oct 12, 2018 7:17 am

Re: MiSTer updater script

Post by warham »

It's a beautiful thing :) Thank you Sir.
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Hi, I just updated the script to version 1.3

Version 1.3 - 2018.12.16 - Added Kernel, Linux filesystem and bootloader updating functionality; added autoreboot option.
Version 1.2 - 2018.12.14 - Added support for distinct directories for computer cores, console cores, arcade cores and service cores; added an option for removing "Arcade-" prefix from arcade core names
Version 1.1 - 2018.12.11 - Added support for additional repositories (i.e. Scaler filters and Game Boy palettes), renamed some variables
Version 1.0 - 2018.12.11 - First commit

Beware: updating Linux may be dangerous, if you don't want to take any risk, please comment the SD_INSTALLER_URL assignment this way
#SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Inst ... n64_MiSTer"

https://github.com/MiSTer-devel/Updater_script_MiSTer
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Released version 1.3.1 of the updater script

Version 1.3.1 - 2018.12.16 - Disabled Linux updating as default behaviour.
uncomment #SD_INSTALLER_URL="https://github.com/MiSTer-devel/SD-Inst ... n64_MiSTer" to enable it

Sometimes linux.img remains the old one, I have to investigate.

https://github.com/MiSTer-devel/Updater_script_MiSTer

[EDIT]
When I extract the release_20181109.rar I overwrite all files in the linux directory including linux.img which is the mounted linux filesystem. This shouldn't be a problem, because the kernel loaded in memory should maintain the handle to the old file, while unrar should uncompress the new one (unix systems let you move/delete open files), but it seems that sometimes the old linux.img was retained. Any idea?

Regards.
Locutus73
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Post by Sorgelig »

if you log into console, root filesystem (which is linux.img) is remounted to read-write. And then if you replace it, you risk to corrupt something...
Locutus73
Atari Super Hero
Atari Super Hero
Posts: 579
Joined: Wed Feb 07, 2018 6:13 pm

Re: MiSTer updater script

Post by Locutus73 »

Sorgelig wrote:if you log into console, root filesystem (which is linux.img) is remounted to read-write. And then if you replace it, you risk to corrupt something...
I agree, but in theory, Linux should distinguish between file handles and directory entries: I mean, in theory the system mounts linux.img in read-write mode, but I should be able to delete it, actually removing the entry from the directory, while the handle remains still live, so I should be able to extract the new linux.img, while the deleted (from the directory) one is still pointed by the handler.
What I have observed so far:
1) If I extract the new linux.img over the older one, sometimes I have the new one other times I have the old one
2) If I delete the old linux.img before extracting the new one, I get the new one after the reboot, but the dd command in the updateboot script fails (corrupting the bootloader)
Now I’m trying to first extract and update the bootloader, then deleting the old linux.img and extracting the new one just before the reboot.

Regards.
Locutus73
Sorgelig
Ultimate Atarian
Ultimate Atarian
Posts: 6348
Joined: Mon Dec 14, 2015 10:51 am
Location: Russia/Taiwan

Re: MiSTer updater script

Post by Sorgelig »

linux.img is special.
It's residing in exFAT volume which is not native to Linux FS, and may not follow handler used by Linux.
Also, since it's root FS which is actually loop - it may have its own quirks.
Locked

Return to “MiSTer”