Building Linux kernel

[Sorgelig]

I went by other way which should be faster and less resource consuming than initrd/initramfs

[kolla]

I went by other way which should be faster and less resource consuming than initrd/initramfs

OK - still may be useful.
With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image

Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

Code: Select all

# cd /media/fat/Amiga
# losetup -f mister.hdf
# cat /proc/partitions
major minor  #blocks  name

   1        0       8192 ram0
   1        1       8192 ram1
   7        0     655360 loop0
   7        1     104580 loop0p1
   7        2     218106 loop0p2
   7        3     200970 loop0p3
   7        4     131418 loop0p4
 179        0    7716864 mmcblk0
 179        1    7202816 mmcblk0p1
 179        2     512000 mmcblk0p2
 179        3       1023 mmcblk0p3
#
# mount -t affs /dev/loop0p4 /mnt/amiga/dh3
# df -T /mnt/amiga/dh3
Filesystem     Type 1K-blocks   Used Available Use% Mounted on
/dev/loop0p4   affs    131417 116626     14792  89% /mnt/amiga/dh3

Yay

Next mission is adding some usb wifi support and add wpa_supplicant to userland.

[Sorgelig]

Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

do you mean you couldn't mount partitions on loop devices before?

With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image

it's easy in theory, but not so easy in practice, especially if you use u-boot which has slightly different view on "right" kernel image.

Next mission is adding some usb wifi support and add wpa_supplicant to userland.

don't forget that Linux part is supposed to be a read-only the whole time. Unless you use a special key combination or command line command to reboot the MiSTer, every reboot and poweroff is unsafe. You can easily corrupt your linux partition if it will be required to be mounted as read-write.

[kolla]

Anyways, found my way around uboot, added "loop.max_part=8" to boot command line, and now I can mount partitions from MBR and RDB image files:

do you mean you couldn't mount partitions on loop devices before?

Correct - could only mount "filesystem" images.

With a filesystem small enough, entire linux filesystem could be the initrd, built into the kernel image

it's easy in theory, but not so easy in practice, especially if you use u-boot which has slightly different view on "right" kernel image.

Yes, I can imagine - one can always cheat using a "dummy kernel" that just kexec to the actual kernel, but it quickly gets a little complicated.

Next mission is adding some usb wifi support and add wpa_supplicant to userland.

don't forget that Linux part is supposed to be a read-only the whole time. Unless you use a special key combination or command line command to reboot the MiSTer, every reboot and poweroff is unsafe. You can easily corrupt your linux partition if it will be required to be mounted as read-write.

Yes, I am aware. My idea is just add wpa_supplicant, and configure it to use a wpa_supplicant.conf located on the fat partition.

[kolla]

OK, I could need a little hint on how one build userland, rootfs and the stuff in addon.tar - is this based on the de10-nano distro that came with the box?

[Sorgelig]

Correct - could only mount "filesystem" images.

it seems there is no kernel config option for this. I will add this option to command line then.

OK, I could need a little hint on how one build userland, rootfs and the stuff in addon.tar - is this based on the de10-nano distro that came with the box?

it's based on buildroot with my manual/scripting modifications.

[kolla]

it's based on buildroot with my manual/scripting modifications.

Would be nice to see how you do it...

Anyways, I cheated - I copied over binaries from a raspberry pi, wpa_supplicant with friends, a few libs and config, and presto - the mister is now on wifi!

Would be nice to do it "properly" though.

I put wpa_supplicant.conf in /media/fat/config, though I suspect that dir is intended solely for soft cores, but it just seems "messy" to have it on /media/fat :p

[Sorgelig]

give me files you've used from RPi. I will see what can i do. May be i will be able to compile same files from buildroot.
In upcoming linux version, there will be new layout where linux folder on fat partition will be used to hold linux image and kernel. Probably same folder can be used for linux related configs.

[kolla]

give me files you've used from RPi. I will see what can i do. May be i will be able to compile same files from buildroot.
In upcoming linux version, there will be new layout where linux folder on fat partition will be used to hold linux image and kernel. Probably same folder can be used for linux related configs.

That would be cool - you have private message with link etc

[Sorgelig]

New Linux release has WiFi support.
Put your SSID/Pass to _wpa_supplicant.conf and rename it to wpa_supplicant.conf

[kolla]

Awesome, looking forward to try it out tonight!

[kolla]

Hehe, by luck the exact kernel modules that I need for my USB wifi sticks were not enabled

For sake of (my) convenience, maybe you could add for next release...

Code: Select all

-# CONFIG_RT2800USB_RT53XX is not set
-# CONFIG_RT2800USB_RT55XX is not set
+CONFIG_RT2800USB_RT53XX=y
+CONFIG_RT2800USB_RT55XX=y

So, I built my own kernel, but then realized that I need a zImage_dtb and not just a zImage - I have looked at various online resources, but not really found a how-to for going from having a zImage to having a zImage_dtb, seems to vary a bit from device to device?

Thanks again

[Sorgelig]

zImage_dtb is a simple concatenation of zImage and socfpga.dtb

I've updated the kernel. Get it from here: https://github.com/MiSTer-devel/Linux_I ... tor_MiSTer

[kolla]

Brilliant! I needed a piece of firmware too (rt2870.bin - forgot to mention it), just put it in the firmware.tar.gz file, ran the create_img.sh, copied over the resulting image, and presto - the mister is online with wifi symbol in menu and all

So, with my own kernel I can just do a

Code: Select all

cat zImage socfpga.dtb > zImage_dtb

or vice versa?

Thanks a lot!

[Sorgelig]

I needed a piece of firmware too (rt2870.bin - forgot to mention it), just put it in the firmware.tar.gz file, ran the create_img.sh, copied over the resulting image, and presto - the mister is online with wifi symbol in menu and all

Kernel source doesn't include firmwares for all devices due to license. Thus some wifi dongles will require additional steps to make them work. My ASUS dongle with realtek chip works without any additions.

So, with my own kernel I can just do a
cat zImage socfpga.dtb > zImage_dtb

right.

[Doozer]

I update the sources regularly. You can see the change log in kernel repository.

Hi Sorgelig,

I need a clarification to the process to get a kernel able to launch MiSTer. I am stuck at bad init sequence because no mtd is discovered when the kernel is launched.

I can compile the kernel (with duplicate config from your 4.15 running kernel) without issue. After the compilation, I am adding the dtd at the end and I am able to boot on the new kernel. Nevertheless the system refuses to get the MiSTer binary from the root filesystem. The rootfs is normally passed to the kernel by uboot but it seems that the new kernel does not see it. You help would be greatly appreciated.

Code: Select all

reading /linux/zImage_dtb
5627599 bytes read in 556 ms (9.7 MiB/s)
## Flattened Device Tree blob at 01556670
   Booting using the fdt blob at 0x1556670
   reserving fdt memory region: addr=0 size=1000
   Loading Device Tree to 03ff5000, end 03fff85e ... OK

Starting kernel ...

[    0.000000] L2C: platform provided aux values permit register corruption.
[    0.689483] rtc-m41t80 1-0068: read error
[    0.693484] rtc-m41t80 1-0068: hctosys: unable to read the hardware clock
[    0.845211] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
[    0.858287] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.5.0+ #0
[    0.864180] Hardware name: Altera SOCFPGA
[    0.868208] [<c0018af8>] (unwind_backtrace) from [<c0014494>] (show_stack+0x20/0x24)
[    0.875929] [<c0014494>] (show_stack) from [<c02a1b5c>] (dump_stack+0x90/0xa4)
[    0.883133] [<c02a1b5c>] (dump_stack) from [<c00e3f58>] (panic+0xe8/0x23c)
[    0.889999] [<c00e3f58>] (panic) from [<c065d974>] (__irq_alloc_descs+0x0/0x1e4)
[    0.897367] CPU1: stopping
[    0.900069] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.5.0+ #0
[    0.905963] Hardware name: Altera SOCFPGA
[    0.909971] [<c0018af8>] (unwind_backtrace) from [<c0014494>] (show_stack+0x20/0x24)
[    0.917687] [<c0014494>] (show_stack) from [<c02a1b5c>] (dump_stack+0x90/0xa4)
[    0.924884] [<c02a1b5c>] (dump_stack) from [<c0017124>] (handle_IPI+0x2d8/0x2f4)
[    0.932252] [<c0017124>] (handle_IPI) from [<c0009528>] (gic_handle_irq+0x94/0x98)
[    0.939792] [<c0009528>] (gic_handle_irq) from [<c0015014>] (__irq_svc+0x54/0x70)
[    0.947241] Exception stack(0xdf50bf48 to 0xdf50bf90)
[    0.952273] bf40:                   00000001 00000000 00000000 c00217c0 ffffe000 c097c4a0
[    0.960418] bf60: 00000000 c09df034 c097c454 00000000 c097c4a8 df50bfa4 df50bfa8 df50bf98
[    0.968558] bf80: c00107e4 c00107e8 60000013 ffffffff
[    0.973594] [<c0015014>] (__irq_svc) from [<c00107e8>] (arch_cpu_idle+0x48/0x4c)
[    0.980972] [<c00107e8>] (arch_cpu_idle) from [<c00651e4>] (default_idle_call+0x30/0x3c)
[    0.989034] [<c00651e4>] (default_idle_call) from [<c0065440>] (cpu_startup_entry+0x250/0x358)
[    0.997613] [<c0065440>] (cpu_startup_entry) from [<c0016bd0>] (secondary_start_kernel+0x16c/0x174)
[    1.006622] [<c0016bd0>] (secondary_start_kernel) from [<000095cc>] (0x95cc)
[    1.013645] ---[ end Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.

Please move the discussion to a separate thread if necessary.

[Sorgelig]

1) make sure you've update uboot,kernel and linux image with latest version using SD installer.
2) you may enable more debug messages by removing "v" variable in uboot environment
3) you need to build the kernel from my repository as it has important patches to be able to boot from loop device.

with duplicate config from your 4.15 running kernel

probably you've mistyped it but correct kernel version if 4.5.x

From your message it's unclear where is the problem.

[Doozer]

Yes, I am speaking about kernel 4.5 from your repository. The issue was related to the .config file not being the latest one. I was using de10-nano_defconfig from this forum. Now I use an extract from your kernel dated Feb 15 (in attachment).

Here are the steps I use for the compilation and dtb concatenation.

Code: Select all

make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- oldconfig
make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- -k zImage dtbs
cat arch/arm/boot/zImage arch/arm/boot/dts/socfpga_cyclone5_de10_sockit.dtb > zImage_dtb

The kernel is now booting and launching MiSTer properly. Thanks.

[Sorgelig]

You don't need extract anything. Current config is always in repository: https://github.com/MiSTer-devel/Linux-K ... _defconfig

[Locutus73]

This evening I compiled the Kernel for the first time (it was the first time I crosscompiled anything too), so I want to share my recipe.
I started from a brand new minimal Debian 9.4.0 amd64, just the basic system, no gui, nothing, just the terminal (and the ssh server for convenience), so here's the steps I followed.

Code: Select all

cd ~

sudo apt-get install build-essential
sudo apt-get install bc
sudo apt-get install liblz4-tool
sudo apt-get install device-tree-compiler
sudo apt-get install git
sudo apt-get install libncurses5-dev
sudo apt-get install libncursesw5-dev

wget https://releases.linaro.org/components/toolchain/binaries/7.2-2017.11/arm-linux-gnueabihf/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz
tar -xvf gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf.tar.xz

git clone -b socfpga-4.5 https://github.com/MiSTer-devel/Linux-Kernel_4.5.0_MiSTer

export ARCH=arm
export CROSS_COMPILE=$PWD/gcc-linaro-7.2.1-2017.11-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-

cd Linux-Kernel_4.5.0_MiSTer

make clean
make mrproper
make MiSTer_defconfig
make zImage modules dtbs

cd ~
cat Linux-Kernel_4.5.0_MiSTer/arch/arm/boot/zImage Linux-Kernel_4.5.0_MiSTer/arch/arm/boot/dts/socfpga_cyclone5_de10_sockit.dtb > zImage_dtb

In order to compile the last xpad driver, just before the make steps, I performed

Code: Select all

cd ~
cd Linux-Kernel_4.5.0_MiSTer/drivers/input/joystick
mv xpad.c xpad.c.old
wget https://raw.githubusercontent.com/paroj/xpad/master/xpad.c
cd ~

But I think Sorgelig is going to merge the last driver on github very soon.

[EDIT]
added
sudo apt-get install libncurses5-dev
sudo apt-get install libncursesw5-dev
in order to make make menuconfig work

Locutus73

[kolla]

Looks like the socfpga_cyclone5_de10_sockit.dtb is now socfpga_cyclone5_de10_nano.dtb? At least that is what I use.

Anyways, since I am doing small changes to the kernel config and need to recompile it every now and then (and because I tend to use macs as build systems these days), I just as well made a docker container for building the MiSTer kernel. It is quite rudimentary and "hardcoded", but it works, for now... https://github.com/kowoba/Linux-Kernel_MiSTer_build

[kolla]

https://github.com/kowoba/Linux-Kernel_MiSTer_build

I don't think anyone else has used my docker yet, but nevertheless, I have fixed on it a bit, and updated the README accordingly.

Next project is to make a similar docker for building custom user-land images (I have needs and wishes that are outside the scope of the default build)

[alfishe]

https://github.com/kowoba/Linux-Kernel_MiSTer_build

Thank you for you efforts. Basically it works. Probably few things to polish:
1) Exclude kernel compilation into Docker image build phase
2) Put several shell scripts to have compilation in different modes:
- With default MiSTer kernel config (no interactive prompts)
- Load MiSTer config but allow to customize before compilation
3) Publish to dockerhub

[kolla]

https://github.com/kowoba/Linux-Kernel_MiSTer_build

Thank you for you efforts. Basically it works. Probably few things to polish:
1) Exclude kernel compilation into Docker image build phase

And build the entire kernel everything from scratch every time, for minor kernel changes? Kinda what I want to avoid

2) Put several shell scripts to have compilation in different modes:
- With default MiSTer kernel config (no interactive prompts)
- Load MiSTer config but allow to customize before compilation

I have no need for for this - if I want default config, it is already there, I just exit the menu, and since it is already prebuilt, it will be done quickly.
Custom config, if present, is already loaded before presenting the menu config.

As I mentioned in the readme, the purpose of this docker, for me, is not to build default kernels, but to easily build custom kernels with the features I want

3) Publish to dockerhub

[/quote][/quote]

Thanks for feedback - feel free to fork, adjust and carry on with this

[kolla]

I updated the Dockerfile a few days ago to reflect updates with MiSTer, so now it build 4.19 kernel using gcc 6.2.1, just like the "official" kernel. I have a few specific reasons for building kernel myself (IPv6 mostly), and this Dockerfile does all I want. As usual, feel free to fork and improve

Oh, I switched from github.com to gitlab.com... so:

https://gitlab.com/kowoba/Linux-Kernel_MiSTer_build