OpenWRT dist-upgrade, or how to update your OpenWRT automatically

I've been thinking in writing this post for a long time and with OpenWRT 21.02 hopefully coming I thought it was the perfect time so that you could benefit from it.

Those of us who run Debian love the way you can go from one version to the new one (Bullseye is coming this weekend, btw) without needing to reinstall the machine each time you update, a simple apt dist-upgrade will take care of everything.

I love Debian, but for the really small things I enjoy OpenWRT a lot and I've always missed the Debian dist-upgrade way of things on OpenWRT.

At work we have a lot of OpenWRT routers, so we had a quite automated way of managing things, so that when we had to update, we did the sysupgrade and then logged on the machine and executed something that installed back all the extra things we needed. It was then that I though... hey, what if we use the sysupgrade.conf thing to protect a service that would be run on the first boot of the device with the new version of OpenWRT and then this service would install all the packages we need?

And that's how the reinstall script (calling it a service would be too much) was born. This script will take care of reinstalling all the things that you tell him to install after a sysupgrade, so, all you need to do is to identify what extra packages you have installed and write the names on your /etc/reinstall.conf file, one package per line, also, if you want a service disabled, you would write its name prepending it with a "-", and that's it.

The only limitation that I've found that would annoy me is that reinstall needs network connection to work, so... if you need some of the extra packages to stablish the network connection (like a 4G connected router wich needs its drivers, a full wpa client to connect with WPA-EAP, ...), reinstall will fail, I suppose I could give an option to predownload the packages but that would need the target version of OpenWRT, so maybe that would mean that maybe reinstall should download the sysupgrade image itself along with the packages and launch the sysupgrade, ... well, who knows, maybe I end up writing a beast, but right now... it needs to be able to have network connection with OpenWRT out of the box to be able to reinstall

This is a /etc/reinstall.conf of one of my machines which runs as an AP, so I don't want dhcp or other stuff, but I want some extra packages to be able to access external storage and things like that:

-dnsmasq -uhttpd -odhcpd nmap diffutils usbutils kmod-usb-storage

BTW: this AP of mine is an ASL26555 with 16MB of RAM which has just been updated from the 19.07 series to 21.02.0-rc4 using reinstall without any problem.

So... you need the reinstall.conf file and we need the script itself, which is at the end of the post, you must save it as /etc/init.d/reinstall and then do a "chmod 755 /etc/init.d/reinstall", but as I told before... we must setup sysupgrade so that reinstall survives after it, so you must add to /etc/sysupgrade.conf at least the three reinstall lines that I have here on this example so that it ends looking something like this:

## This file contains files and directories that should ## be preserved during an upgrade. /etc/firewall.user /etc/crontabs/root /root /etc/init.d/reinstall /etc/rc.d/S99reinstall /etc/reinstall.conf

So... now you have it all setup... when you are going to do a sysupgrade... you must first do a "/etc/init.d/reinstall enable" in order to enable the service, so that it runs when sysupgrade reboots the device, that's when reinstall installs the wanted packages.

The service will log its actions on /root/reinstall.log by default, then disable itself so that it is not run anymore, and then reboot the machine so that it ends up right how you wanted it to be. After that you can log on the machine to see how everything went, and hopefully your log will look something like this:

_______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M ----------------------------------------------------- OpenWrt 21.02.0-rc4, r16256-2d5ee43dc6 ----------------------------------------------------- Downloading 'http://www.google.com' Connecting to 142.250.185.4:80 Writing to '/dev/null' Download completed (14036 bytes) Thu Aug 12 17:29:39 CEST 2021 Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_core Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/Packages.sig Signature check passed. Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_base Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/Packages.sig Signature check passed. Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/luci/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_luci Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/luci/Packages.sig Signature check passed. Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_packages Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/Packages.sig Signature check passed. Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/routing/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_routing Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/routing/Packages.sig Signature check passed. Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/telephony/Packages.gz Updated list of available packages in /var/opkg-lists/openwrt_telephony Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/telephony/Packages.sig Signature check passed. Disabling -dnsmasq Disabling -uhttpd Disabling -odhcpd Installing nmap Installing nmap (7.80-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/nmap_7.80-3_mipsel_24kc.ipk Installing libpcap1 (1.9.1-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/libpcap1_1.9.1-3_mipsel_24kc.ipk Installing libstdcpp6 (8.4.0-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/libstdcpp6_8.4.0-3_mipsel_24kc.ipk Installing zlib (1.2.11-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/zlib_1.2.11-3_mipsel_24kc.ipk Installing libpcre (8.44-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/libpcre_8.44-3_mipsel_24kc.ipk Configuring libpcre. Configuring libpcap1. Configuring libstdcpp6. Configuring zlib. Configuring nmap. Installing diffutils Installing diffutils (3.7-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/diffutils_3.7-3_mipsel_24kc.ipk Configuring diffutils. Installing usbutils Installing usbutils (013-2) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/usbutils_013-2_mipsel_24kc.ipk Installing librt (1.1.24-3) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/librt_1.1.24-3_mipsel_24kc.ipk Installing libusb-1.0-0 (1.0.24-4) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/base/libusb-1.0-0_1.0.24-4_mipsel_24kc.ipk Installing libevdev (1.10.1-1) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/libevdev_1.10.1-1_mipsel_24kc.ipk Installing libudev-zero (0.4.5-2) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/libudev-zero_0.4.5-2_mipsel_24kc.ipk Installing usbids (0.347-1) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/packages/mipsel_24kc/packages/usbids_0.347-1_mipsel_24kc.ipk Configuring libevdev. Configuring librt. Configuring libusb-1.0-0. Configuring libudev-zero. Configuring usbids. Configuring usbutils. Installing kmod-usb-storage Installing kmod-usb-storage (5.4.137-1) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/kmod-usb-storage_5.4.137-1_mipsel_24kc.ipk Installing kmod-scsi-core (5.4.137-1) to root... Downloading https://downloads.openwrt.org/releases/21.02.0-rc4/targets/ramips/rt305x/packages/kmod-scsi-core_5.4.137-1_mipsel_24kc.ipk Configuring kmod-scsi-core. Configuring kmod-usb-storage. Everything went Ok, reinstall has finished without errors.

I guess, that's all I have to say, hope you enjoy it, keep in mind that embedded devices are always tricky and that if your machine is not stable... you should probably not do automatic things like this on it, as always I take no resposability on anything, use it at your own risk, as for me, I trust OpenWRT so much that I have just reinstalled my ASL26555 from outside using it and everything went Ok ;-)

So... here is reinstall

Edit: Originally I had pasted the code in the page, but that was an error due to how bad blogger works, so... now I have published it on github and added the link to it

Flashing a Samsung stock ROM using heimdall from the command line.

I've read many times how to flash a Samsung official rom from Windows using Samsung's official tools and some other times I've read complex ways to do it using the Heimdall's grafical interface, ... but I never felt any of this ways was for me.

Fortunately I always flash custom ROMs instead, so I never had the need to flash a stock one, till recently, when I wanted to test andOTP on old Android versions, that's when I wanted to install the ancient stock versions of a couple of Samsung phones, and luckily I came to a quick commandline script that did it all for me.

WARNING: this procedure will wipe out all your data on the device in a way that you won't be able to recover it, I'm not resposable for any data loss or any damage to the device that any of the things I describe here may cause to the devices.

First of all you must unpack the stock rom (typically a zip file that inside has a whatever.tar.md5 file which is really a tar file, not a md5 one, so, you untar the tar.md5 file and you get the images of the phone's partitions (recovery.img, modem.bin, boot.img, ...) you can now remove that .tar.md5 file.

So... if you have clear that you are going to delete all your data on the phone and want to continue, I assume you have made a good backup of your data and you have verified that the backup is ok, or that you don't mind loosing it all. In any way...

You must start by wiping your data partition from your recovery or from the system itself by doing a factory reset and then going directly do the bootloader. The easiest way is by selecting reset to bootloader on the recovery after wiping data, or rebooting pressing the bootloader key convination for your device, but making sure that you didn't boot into the system after doing the wiping.

If you are sure you have formated data and booted directly to bootloader, you may need to confirm on bootloader that you want to "Continue" to flash your rom, that way you'll get to the "Downloading..." droid.

Now that we are on the bootloader on download mode we do:

heimdall print-pit --no-reboot > pit

and after we have downloaded the pit file:

heimdall flash --resume $(for i in *.*;do grep -B 1 $i pit|tr '\n' ' ';echo;done|sed "s/.*ame: \([^ ]*\) .*ame: \(.*\)/--\1 \2/"|tr '\n' ' ')

And you are done. The script will flash all the partitions that are included on the stock rom and after that it will reboot for the system to do its job after flashing, so... it will be a first time boot that will take a while, but that's it.

Just some side notes, using --no-reboot and then --resume has never really worked for me, maybe it was a problem with heimdall's version or my devices or whatever, in those cases the second heimdall command will fail, you must reboot to booloader again (without going to system, otherwise you'll have to format data again) and execute the flash command again without the --resume.

Windows and Linux software Raid dual boot BIOS machine

One could think that nowadays having a machine with software raid doing dual boot should be easy, but... my experience showed that it is not that easy.

Having a Windows machine do software raid is easy (I still don't understand why it doesn't really work like it should, but that is because I'm used to Linux software raid), and having software raid on Linux is also really easy. But doing so on a BIOS booted machine, on mbr disks (as Windows doesn't allow GPT on BIOS) is quite a pain.

The problem is how Windows does all this, with it's dynamic disks. What happens with this is that you get from a partitioning like this:

/dev/sda1 * 2048 206847 204800 100M 7 HPFS/NTFS/exFAT /dev/sda2 206848 312580095 312373248 149G 7 HPFS/NTFS/exFAT /dev/sda3 312580096 313165823 585728 286M 83 Linux /dev/sda4 313165824 957698047 644532224 307,3G fd Linux raid autodetect

To something like this:

/dev/sda1 63 2047 1985 992,5K 42 SFS /dev/sda2 * 2048 206847 204800 100M 42 SFS /dev/sda3 206848 312580095 312373248 149G 42 SFS /dev/sda4 312580096 976769006 664188911 316,7G 42 SFS

These are the physical partitions as seen by fdisk, logical partitions are still like before, of course, so there is no problem in accesing them under Linux or windows, but what happens here is that Windows is using the first sectors for its dynamic disks stuff, so... you cannot use those to write grub info there :-(

So... the solution I found here was to install Debian's mbr and make it boot grub, but then... where do I store grub's info?, well, to do this I'm using a btrfs /boot which is on partition 3, as btrfs has room for embedding grub's info, and I setup the software raid with ext4 on partition 4, like you can see on my first partition dump. Of course, you can have just btrfs with its own software raid, then you don't need the fourth partition or anything.

There are however some caveats on doing all this, what I found was that I had to install grub manually using grub-install --no-floppy on /dev/sda3 and /dev/sdb3, as Debian's grub refused to give me the option to install there, also... several warnings came as a result, but things work ok anyway.

One more warning, I did all this on Buster, but it looks like for Grub 2.04 which is included on Bullseye, things have gotten a bit bigger, so at least on my partitions there was no room for it, so I had to leave the old Buster's grub around for now, if anybody has any ideas on how to solve this... they are welcome.