Docs · 1.0

System: Upgrade and Restart

This chapter covers System → Upgrade: how to upgrade keelOS, how the two boot slots work, how to switch and roll back, how to schedule the switch for the night, and what happens to running VMs during an upgrade. It needs the host:write permission.

1. Two slots

The boot partition of keelOS has two slots, A and B. Each slot holds a complete copy of keelOS:

• One slot is running (marked Running on the page);
• When you upgrade, the new version is written into the slot that is not running. Writing does not restart anything and VMs keep running;
• Then you switch to that slot. If the new version does not come up, the host goes back to the previous slot by itself;
• Rolling back simply means switching to the other slot, where the old version is still stored.

So a failed upgrade costs little: the old version stays in the other slot until your next upgrade overwrites it.

2. The upgrade page

Open System → Upgrade. At the top is the version Running now (the keelOS version number and Slot A · running), next to the four steps of an upgrade: Check, Download, Write, Restart. The highlighted one is the current step, and below it is what this step does. Further down are two Boot slots cards showing each slot's version, when it was written, and which slot is the next boot.

3. Upgrading

Step 1: Check

Choose where the new version comes from:

• From a URL: by default keelOS's release address (the latest version on GitHub) is filled in; click Check for updates. You can change it to another ISO address; once changed, a button beside the box restores the default address.
• From this computer: choose a keelOS .iso on your computer and click Upload. Use this on a host without internet access.

Checking only looks at what is at that address and does not download. If you are already on the latest version, the page says "keelOS ... is already the latest version". Otherwise a dialog opens showing:

• Whether this version, compared with the one running, is an update (upgrade), older (a rollback) or the same version;
• The size, and the release notes (if there are none, what changed relative to the running version);
• A disclaimer. You must tick I have read this and agree before you can click Download.

Without agreeing, nothing is downloaded or written.

Step 2: Download

The page shows a progress bar, how much is downloaded, the speed and the time left; Cancel download can be clicked at any time. An interrupted download resumes; if one source sends no data for 120 seconds, keel switches to keelos.dev and carries on.

Before downloading you can tick Write to slot X and switch right after the download (remembered in this browser): then after the download it no longer stops to ask, but writes and switches automatically. Use it if you do not want to watch over it, but read §5 first.

Step 3: Write

After the download (or upload) finishes the page says "keelOS ... is ready to be written to slot X"; click Write to slot X. You can click Discard to give up. What is written is the slot that is not running, so nothing restarts and VMs keep running. If that slot already holds the same version, the page warns that writing again will overwrite it.

Step 4: Restart (switch)

When writing is done the page says "Slot X holds keelOS ..., which is newer than the running version", and the button is Upgrade to .... Clicking it opens the switch dialog, see §4. To see whether a newer version exists, click Check for a newer version.

4. Switching: now or scheduled

Click Upgrade to ... (or the matching button for a rollback or trial) and Switch to slot X opens. The dialog first lists the VMs running now and what will happen to each when you switch (see §5), then says in a sentence what this switch will do. Then choose one of two:

• Switch now: click the button and the host starts switching. The page is covered by a "the host is restarting" notice showing how long it has waited; when the host is back the page refreshes itself and tells you the result: switched to slot X, or the switch did not come up, the host went back to slot Y.
• Set time: under Switch at choose a time (on the host's clock) and click Set time; or click Tonight 03:00 to fill it in with one click and then click Set time. At that time it switches automatically. Meanwhile the page says "planned: switch to slot X" and you can click Cancel.

Note: The switch time must be in the future. When a nested-virtualization VM cannot be checkpointed for now, Switch now cannot be clicked, and the dialog explains what to do, see Nested virtualization.

A switch is in essence "try once": until the new version is up and keel has confirmed it works, the next boot from the firmware still goes back to the old slot. If the new version does not come up, hangs or crashes, the host goes back to the previous slot by itself and the page says the switch to slot X did not come up: the host went back to slot Y. The reason is recorded on the host; run keel upgrade --status to see it.

If the host has not come back after 5 minutes, the notice suggests looking at the host's screen or the serial console (IPMI SOL). If the host has a keelRM management port, the notice also shows root's state and the last lines of the console live.

5. What happens to VMs when you switch

keel decides the way to switch from the new version; you do not choose:

In the switch dialog each VM has its own row saying whether it will be suspended, shut down or something else; the ones that must be shut down are highlighted. In the second case the dialog states plainly that the host will restart completely.

6. Rolling back

If the new version has a problem, the other slot still holds the old version: click Roll back to ... on that slot's card; the dialog is the same as above, now or scheduled. New settings that the old version does not know are ignored but kept in the file, and are still there when you upgrade again.

A slot with no old version shows empty. If you write again, the old version is overwritten and you cannot roll back, so do not hurry to upgrade again before the new version has proved stable.

7. Major versions: try first, then confirm

Some versions change formats on disk (for example storage pool features, the save format of suspended VMs, how the settings file is written). Such a version works in two stages:

1. Trial: after the switch the host runs the new version, but what is on disk stays in the old format. A card at the top of the page says Trying keelOS ..., you can go back to ..., with the buttons Try ... and Roll back .... You can go back to the old version at any time, and suspended VMs carry on under the old version.
2. Confirm the upgrade: when the trial has gone well (the page reminds you after 7 days of running), click Confirm the upgrade…. The dialog lists what this step does: enable the new features on the storage pools, save VMs suspended from now on in the new format, rewrite the settings file in the new way, turn on the new functions, and make the version in the old slot unable to boot. Tick I understand that I cannot go back to ... afterwards and click Confirm the upgrade. You do not need to stop VMs, and it finishes in a few seconds.

After confirming you cannot go back. Some versions cannot write a format the old version understands; before switching the page says plainly "after this upgrade you cannot go back", and when going back, suspended VMs are shut down first.

8. Restarting the host

The upgrade page has no separate restart button unless something is waiting for a restart: a new hypervisor, hypervisor settings or the keelRM switch take effect only at the next host restart. Then Restart the host appears, and this time it is a full restart (running VMs are shut down first and started again afterwards). Everyday restart and shutdown are under System → Maintenance, see System: Maintenance, Time and Host Console.

9. hyp and native mode

hyp is keelOS's hypervisor layer. If the last boot on hyp did not finish (hang, crash, power loss), the host's next boot from the firmware automatically starts without hyp (native mode); VMs still run, but they are shut down when the host restarts. The bottom of the upgrade page shows hyp's state: hyp: on, hyp: try, hyp: off. To try again, click Try hyp once; after you confirm, hyp is tried once at the next restart through the firmware, and if it does not come up, the one after that starts without hyp.

10. Upgrading keelNAS, keelAuth and the manager

keelNAS, keelAuth and the keel manager are VMs on the host, each upgraded on its own: the system disk is swapped, the data disk is left alone, and the old version is kept for 7 days for rolling back. Do it on their own pages, see Accounts, Tokens and Permissions and NAS.

11. Limits in 1.0

• The upgrade source is a release address (which you can change) or an uploaded file; the page does not check for new versions by itself from time to time, so click Check for updates.