<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="4.3.3">Jekyll</generator><link href="https://oxcrag.net/feed.xml" rel="self" type="application/atom+xml" /><link href="https://oxcrag.net/" rel="alternate" type="text/html" /><updated>2026-01-24T14:00:08+01:00</updated><id>https://oxcrag.net/feed.xml</id><title type="html">oxcrag.net</title><subtitle>Tech and other nerdery.</subtitle><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><entry><title type="html">Keychron K8 Max Review</title><link href="https://oxcrag.net/blog/2026/01/18/Keychron-K8-Max-Review.html" rel="alternate" type="text/html" title="Keychron K8 Max Review" /><published>2026-01-18T00:00:00+01:00</published><updated>2026-01-18T00:00:00+01:00</updated><id>https://oxcrag.net/blog/2026/01/18/Keychron-K8-Max-Review</id><content type="html" xml:base="https://oxcrag.net/blog/2026/01/18/Keychron-K8-Max-Review.html"><![CDATA[<p>I recently spent some money on a mechanical keyboard, and thought I’d share some thoughts on it. I’m not a professional reviewer, nobody’s paying me to write, and nobody’s sending me free stuff. I also don’t usually spend extraordinary amounts of money on gear, so this is a subjective review of a single product, and the comparisons are to other products that are available to me.</p>

<p>The keyboard I purchased is a <a href="https://www.keychron.com/products/keychron-k8-max-qmk-wireless-custom-mechanical-keyboard-iso-layout-collection?variant=42215480819801">Keychron K8 Max</a> with “Super Brown” - tactile - switches. The purchase was actually catalyzed by another purchase: my Commodore 64 Ultimate. It made me remember the feeling of high-quality, long-stroked keys, and caused me to look up what’s available on the PC side. Luckily, just when I was researching keyboards, I found this specific Keychron keyboard for about one third off MSRP, and decided to try it out.</p>

<p>I’ve had a cheap gaming keyboard with blue - clicky - switches for years, but it wasn’t much to write home about: Loud, cheap-feeling, but completely decent for its intended purpose, which was mashing keys in games without encountering rollover issues.  Unsurprisingly, this keyboard feels radically different, from its physical heft, via the much nicer-feeling key surfaces and keyboard curvature, to the rewarding “thock” of the keys if fully depressed with some force.</p>

<p>The Commodore 64 Ultimate I mentioned earlier comes with linear switches (“Red” or similar). The tactile switches of the Keychron keyboard are relatively similar: strike the keys fast, and you probably won’t notice a lot of a difference, but if you work them slowly, you’ll feel a slight change in pressure a couple of millimeters down - that’s the tactile bit. Typing fast, you get a slightly muffled sound if you allow the keys to bottom out. That can be avoided with some practice by not depressing the buttons all the way, or at least already having slowed the finger movement down by the way the key hits its bottom position. The other, slightly lighter sound, comes from the keys returning to their upmost position. None of the noises is jarring: to me as a user, it becomes an almost meditative background sound when typing like this. I’d say this keyboard should be completely acceptable to your neighbors in an open office environment too.</p>

<h2 id="using-the-keyboard-with-a-mac">Using the keyboard with a Mac</h2>
<p>Compared to a canonical Magic Keyboard, the layout of the Control, Option and Command keys is a bit off: Since on the Magic keyboard they are inset by the width of an additional key (the Fn/Globe key to the left of the left Control key), I needed to spend some more time with the keyboard to see whether I’d get used to it or not. Could I get used to doing mostly with my pinky finger what I used to do with my thumb? It turns out about one working day was enough to get comfortable, if not fluent with the new key arrangement. Getting there included making some changes to the keyboard’s default configuration.</p>

<p>Using the <a href="https://launcher.keychron.com">Keychron Launcher web app</a>, I made the Caps Lock key take over as a mode toggle, to give access to the keyboard’s second layer. I then mapped Del to the second layer’s backspace key - mimicking the functionality of Fn+backspace on the Magic Keyboard. I also mapped the arrow keys in Vim style (left, down, up, right) to HJKL on the second layer. Combined with the Shift, Option and Command keys, this means I don’t have to move my fingers over to the separate arrow keys and ins/del/home/end/PgUp/PgDn cluster in normal use. They’re still available, but not necessary.</p>

<p>The second thing that takes some getting used to with this keyboard, is using it alongside a Magic Trackpad. The latter is, of course, designed to be flush with the Magic Keyboard, while the Keychron K8 Max sits several centimeters higher, especially with its rear feet down.  It’s more of a stretch, and I’m sure someone who was more bothered by the difference could 3D print a stand for the trackpad, to raise it to the level of the top of the keyboard, but honestly it’s not that bad, if you ask me.</p>

<p>In short, this keyboard turned out to be a joy to work on with the Mac.</p>

<h2 id="using-the-keyboard-with-a-linux-pc">Using the keyboard with a Linux PC</h2>
<p>When switched to “Windows/Android” mode, the Keychron K8 Max behaves just like any standard PC keyboard. Alt and Windows key caps are included in the box, to replace the default Mac keys between the Control keys on either side, and the space bar. I would prefer a choice of OS agnostic meta keys, but Windows is undeniably the default choice on the PC side, so I guess I can’t complain too much about this. In PC mode too, the keyboard is programmable using the Keychron Launcher web app; utilizing the keyboard´s banks 2 and 3 rather than banks 0 and 1 that are used in Mac mode.</p>

<p>One thing that isn’t immediately obvious, is that the security model of Linux by default prevents software from making changes to the keyboard’s programming. The workaround is to either make the requested changes from the Mac, or to configure a <code class="language-plaintext highlighter-rouge">udev</code> rule for the keyboard:</p>

<p>First I created a file, <code class="language-plaintext highlighter-rouge">/etc/udev/rules.d/99-keychron.rules</code>; and then I filled it with the following information, partly based on the output of the <code class="language-plaintext highlighter-rouge">lsusb</code> command, and partly based on advice from a Kagi search:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>KERNEL=="hidraw", SUBSYSTEM=="hidraw", ATTRS{idVendor}=='3434', ATTRS{idProduct}=="0a81", MODE=="0666", GROUP=="users", TAG+="uaccess", TAG+="udev-acl"
</code></pre></div></div>

<p>Then I restarted the <code class="language-plaintext highlighter-rouge">udev</code> service and forced it to re-read its rules:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>udevadm control <span class="nt">--reload-rules</span> <span class="o">&amp;&amp;</span> <span class="nb">sudo </span>udevadm trigger
</code></pre></div></div>

<p>After this, I could use the Keychron Launcher via a Chromium-based browser to update the keyboard configuration as I wanted: At this point I only added the same shortcuts to the keyboard’s upper memory banks that I already had for the Mac, as I think I’ll enjoy them.</p>

<h2 id="other-notes">Other notes</h2>
<p>The Max model of the Keychron K8 has three connection types available: Wired via an included USB-C to USB A cable, wireless via a 2.4 GHz dongle - similar to what Logitech has done for many years - and wireless via Bluetooth, where the keyboard allows pairing with up to three different devices.</p>

<p>The 1000Hz polling rate - nice for gaming - is only available via wired or 2.4GHz wireless connections. Using Bluetooth, polling rate is lowered to one tenth of that: Still plenty for regular productivity use. The way I’ve set it up today, is I use the 2.4GHz dongle in my Linux gaming computer, and then I use two of the available Bluetooth connections for my private and work Macs respectively. I never use both Macs docked to my screen simultaneously, so there’s really no extra hassle switching the keyboard between them; and when I want to play a game, I flip the two mode switches on the left-hand side of the keyboard, and I’m ready to go.</p>

<h2 id="summary">Summary</h2>
<p>The Keychron K8 Max was a very nice upgrade for my typing experience. It rewards sticking to good typing form, and I really enjoy the feedback it provides when I type on it. For a PC user, it can be used out-of-the-box with no changes required at all. A Mac user <em>may</em> use it as-is, but if you are a power user, accustomed to making the most of the keyboard on your laptop, or of the smaller model Magic Keyboard, you may want to slightly adjust some of its default behaviors.</p>

<p>This could turn out to be an expensive purchase, though: While this keyboard is perfect for my Linux gaming computer, I’m already toying with the idea of purchasing a more compact model for use with my Macs: I will keep experimenting with my writing form and with customizing the key maps to see what I can get away with: I don’t think I’ll want to live with the compromises of going all the way down to a 60-key keyboard like the <a href="https://hhkb.io">Happy Hacking Keyboard</a>, but you never know…</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="Keychron" /><category term="keyboards" /><category term="macOS" /><category term="Linux" /><category term="reviews" /><summary type="html"><![CDATA[I recently spent some money on a mechanical keyboard, and thought I’d share some thoughts on it. I’m not a professional reviewer, nobody’s paying me to write, and nobody’s sending me free stuff. I also don’t usually spend extraordinary amounts of money on gear, so this is a subjective review of a single product, and the comparisons are to other products that are available to me.]]></summary></entry><entry><title type="html">Server Monitoring with Grafana</title><link href="https://oxcrag.net/blog/2026/01/04/server-monitoring-with-grafana.html" rel="alternate" type="text/html" title="Server Monitoring with Grafana" /><published>2026-01-04T00:00:00+01:00</published><updated>2026-01-04T00:00:00+01:00</updated><id>https://oxcrag.net/blog/2026/01/04/server-monitoring-with-grafana</id><content type="html" xml:base="https://oxcrag.net/blog/2026/01/04/server-monitoring-with-grafana.html"><![CDATA[<p>I’ve set up a self-hosted <a href="https://grafana.com/docs/grafana/latest/">Grafana</a> instance complete with metrics storage using <a href="https://grafana.com/docs/mimir/latest/">Mimir</a>, central log storage with <a href="https://grafana.com/docs/loki/latest/">Loki</a>, and remote sending of data from my servers using the <a href="https://grafana.com/docs/alloy/latest/">Alloy</a> agent.</p>

<p>I couldn’t find any good resource describing the complete process, so I’ve documented my setup, and am <a href="https://oxcrag.net/projects/self-hosted-grafana-server/">sharing the documentation via my Projects page</a>.</p>

<p>Happy monitoring!</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="Grafana" /><category term="Loki" /><category term="Mimir" /><category term="Alloy" /><category term="homelab" /><category term="selfhosting" /><category term="Linux" /><category term="SiteReliabilityEngineering" /><category term="SRE" /><category term="HAProxy" /><summary type="html"><![CDATA[I’ve set up a self-hosted Grafana instance complete with metrics storage using Mimir, central log storage with Loki, and remote sending of data from my servers using the Alloy agent.]]></summary></entry><entry><title type="html">Creating VMs in separate ZFS filesystems</title><link href="https://oxcrag.net/blog/2025/11/16/Creating-VMs-in-separate-ZFS-filesystems.html" rel="alternate" type="text/html" title="Creating VMs in separate ZFS filesystems" /><published>2025-11-16T00:00:00+01:00</published><updated>2025-11-16T00:00:00+01:00</updated><id>https://oxcrag.net/blog/2025/11/16/Creating-VMs-in-separate-ZFS-filesystems</id><content type="html" xml:base="https://oxcrag.net/blog/2025/11/16/Creating-VMs-in-separate-ZFS-filesystems.html"><![CDATA[<p>I’m running my VMs on a plain Ubuntu server with KVM/QEMU, with VMs stored on ZFS.</p>

<p>Up until now, I’ve created my VMs as QCOW2 files in a single ZFS filesystem; e.g. all VMs have lived in <code class="language-plaintext highlighter-rouge">/ssdpool/vmimages</code>. The <code class="language-plaintext highlighter-rouge">vmimages</code> directory wasn’t a child ZFS file system to <code class="language-plaintext highlighter-rouge">ssdpool</code>, but a plain directory created by VirtManager once upon a time.</p>

<p>There are a couple good reasons to deepen the structure, though:</p>

<p>First of all, it simplifies rollbacks from snapshots as each VM lives in its own ZFS filesystem.</p>

<p>Second, VM storage can be assigned its own ZFS properties that fit with the type of data stored in it - though honestly that’s less of an issue for me, as I generally just use QCOW2 images, as I mentioned.</p>

<p>Third, it makes it possible to purge unused disk images from my backups before the end of my backup retention, serving as an improvement to my storage utilization in cases where I’m sure I won’t need the data again.</p>

<p>So how do I go about changing this behavior, then?</p>

<h2 id="creating-zfs-file-systems">Creating ZFS file systems</h2>

<p>The first step is to create a new ZFS file system under my <code class="language-plaintext highlighter-rouge">ssdpool</code>. I set up the existing structure when I had less experience with ZFS, and the <code class="language-plaintext highlighter-rouge">vmimages</code> substructure is actually just a directory in <code class="language-plaintext highlighter-rouge">ssdpool</code>, rather than being its own file system, as is the way I would set it up today.</p>

<p>I will create a new pool, <code class="language-plaintext highlighter-rouge">ssdpool/vdisks</code>. Then I will create separate ZFS file systems in this structure for each VM and migrate them into their new home.</p>

<p>As described by <a href="https://mercenarysysadmin.com">Jim Salter</a> in a <a href="https://klarasystems.com/articles/tuning-recordsize-in-openzfs/">Klara Systems article on ZFS recordsize tuning</a>, the QCOW2 images I use for my virtual machines should work best with a recordsize parameter that coincides with the native block size of the QCOW2 image; 64KB.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo zfs create ssdpool/vdisks -o recordsize=64K
</code></pre></div></div>

<p>We can verify that it went well:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo zfs get recordsize ssdpool/vdisks
NAME            PROPERTY    VALUE    SOURCE
ssdpool/vdisks  recordsize  64K      local
$
</code></pre></div></div>

<p>At this point it is prudent to set the correct permissions on the new file system for KVM:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo chown -R libvirt-qemu:kvm /ssdpool/vdisks
</code></pre></div></div>

<p>Child file systems will, by default, inherit ZFS properties from their parents, so to create a filesystem for one of my virtual machines shouldn’t require me to explicitly define the record size. Let’s test that hypothesis:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo zfs create ssdpool/vdisks/testsrv1
sudo zfs get recordsize ssdpool/vdisks/testsrv1
NAME                        PROPERTY    VALUE    SOURCE
ssdpool/vdisks/testsrv1     recordsize  64K      inherited from ssdpool/vdisks
$ 
</code></pre></div></div>

<p>Perfect. this simplifies our future VM creation.</p>

<h2 id="migrating-kvm-virtual-machines">Migrating KVM virtual machines</h2>

<p>Changing the storage path for a VM seems to require the following steps:</p>

<ul>
  <li>Shut down the VM</li>
  <li>Move the VM files to their new location</li>
  <li>Edit the VM to find its files in their new location</li>
  <li>Start the VM back up</li>
</ul>

<p>Let’s do this.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo virsh shutdown testsrv1
</code></pre></div></div>

<p>Then we wait until the machine has shut down:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo virsh list --all
 Id   Name                      State
------------------------------------------
 -    testsrv1                  shut off
</code></pre></div></div>

<p>We move the VM virtual disk(s) to its new home, and ensure the permissions are correct:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo mv testsrv1.qcow2 /ssdpool/vdisks/testsrv1/
sudo chown libvirt-qemu:kvm /ssdpool/vdisks/testsrv1/testsrv1.qcow2 
</code></pre></div></div>

<p>We edit the virtual machine to update its disk location:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>virsh edit testsrv1
</code></pre></div></div>

<p>In the file, we locate the disk block(s) and update the path found in the <code class="language-plaintext highlighter-rouge">source file</code> XML block, and then save and exit the editor.</p>

<p>Finally we start the VM again, and hopefully we haven’t messed anything up:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>virsh start testsrv1
</code></pre></div></div>

<p>Of course migrating the VM is only the most visible step of this change. We’re not done until we have confirmed our backups. Let’s revisit that in the <a href="https://oxcrag.net/blog/2025/11/16/zfs-backup-strategy-with-sanoid-and-syncoid.html">next post</a>.</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="Linux" /><category term="KVM" /><category term="ZFS" /><summary type="html"><![CDATA[I’m running my VMs on a plain Ubuntu server with KVM/QEMU, with VMs stored on ZFS.]]></summary></entry><entry><title type="html">ZFS Backup Strategy with Sanoid and Syncoid</title><link href="https://oxcrag.net/blog/2025/11/16/zfs-backup-strategy-with-sanoid-and-syncoid.html" rel="alternate" type="text/html" title="ZFS Backup Strategy with Sanoid and Syncoid" /><published>2025-11-16T00:00:00+01:00</published><updated>2025-11-16T00:00:00+01:00</updated><id>https://oxcrag.net/blog/2025/11/16/zfs-backup-strategy-with-sanoid-and-syncoid</id><content type="html" xml:base="https://oxcrag.net/blog/2025/11/16/zfs-backup-strategy-with-sanoid-and-syncoid.html"><![CDATA[<p>In my <a href="https://oxcrag.net/blog/2025/11/16/Creating-VMs-in-separate-ZFS-filesystems.html">previous post</a>, I discussed how I’ve migrated VMs to new storage. This gave me cause to also take a look at my backup configuration, to ensure I can still come back from catastrophic events.</p>

<p>I use <a href="https://github.com/jimsalterjrs/sanoid"><code class="language-plaintext highlighter-rouge">Sanoid</code> and <code class="language-plaintext highlighter-rouge">Syncoid</code></a> for this purpose. Let’s see how that may look.</p>

<h2 id="primary-backup">Primary backup</h2>

<p>My backup strategy is multi-pronged. I take frequent snapshots of the file system where my VMs live. This is convenient for oupsies, but of course doesn’t protect me should a catastrophic storage failure occur. Snapshots and the eventual cleanup of them are managed by Sanoid.</p>

<p>The relevant parts of my <code class="language-plaintext highlighter-rouge">/etc/sanoid/sanoid.conf</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[ssdpool]
        use_template = ssdpool
        recursive = yes
        process_children_only = no

[template_ssdpool]
        frequently = 0
        hourly = 36
        daily = 30
        monthly = 0
        yearly = 0
        autosnap = yes
        autoprune = yes
</code></pre></div></div>

<p>We can see that the <code class="language-plaintext highlighter-rouge">ssdpool</code> file system gets recursive snapshots every hour plus daily. Thirty-six of these hourly snapshots are kept, and then I keep 30 of the daily snapshots on the primary storage.</p>

<p>In every step, I can verify that this actually works by listing ZFS snapshots:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo zfs list -t snapshot
</code></pre></div></div>

<h2 id="secondary-backup">Secondary backup</h2>

<p>To protect against pool failures, I have a secondary pool in my hypervisor host, set up for backup storage. I call Syncoid from cron for this purpose.</p>

<p>This is handled by this part of my <code class="language-plaintext highlighter-rouge">/etc/cron.d/syncoid</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>02 * * * * root syncoid -r --no-sync-snap --compress=lzo --quiet ssdpool backuppool/ssdpoolbackup
</code></pre></div></div>

<p>On the second minute of every hour of every day, I recursively send ZFS snapshots from <code class="language-plaintext highlighter-rouge">ssdpool</code> to <code class="language-plaintext highlighter-rouge">backuppool/ssdpoolbackup</code>.</p>

<p>Of course, I also have to ensure the backup volume doesn’t fill up with snapshots. Sanoid to the rescue again:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>        
[backuppool/ssdpoolbackup]
        use_template = backuppoolssd
        recursive = yes
        process_children_only = no

[template_backuppoolssd]
        frequently = 0
        hourly = 36
        daily = 30
        monthly = 0
        yearly = 0
        autosnap = no
        autoprune = yes

</code></pre></div></div>
<p>For <code class="language-plaintext highlighter-rouge">backuppool/ssdpoolbackup</code> I don’t create new snapshots, but I keep the same pruning configuration that I have for the primary snapshots.</p>

<h2 id="tertiary-backup">Tertiary backup</h2>
<p>Finally I have a completely separate physical server where I store an additional copy of my backups. This one is configured to connect to the primary server and fetch snapshots, again using syncoid. This way of doing things makes it slightly harder to mess things up on the backup server from the primary server.</p>

<p>The cron configuration line for syncoid on this machine looks like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>20 * * * * root syncoid --sshkey=/root/.ssh/syncoid -r --no-sync-snap --compress=lzo --quiet syncoid@prodsrv1:backuppool/ssdpoolbackup backuppool/prodsrv1/ssdpool
</code></pre></div></div>

<p>It’s very similar to the one running on the main server, but fetches the snapshots from the backup pool on the main server over an ssh tunnel, and then stores them in a local backup pool. As you see, I’ve given it a delay of a few minutes compared to the crontab on the main server, so in most circumstances we should manage to have the backup copy done in a timely manner for minimal data loss in case of an issue.</p>

<p>Of course here too, I make sure to clean up snapshots to avoid filling the disks over time:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>[backuppool/prodsrv1/ssdpool]
        use_template = backupssd
        recursive = yes
        process_children_only = no
        
[template_backupssd]
        frequently = 0
        hourly = 36
        daily = 90
        monthly = 0
        yearly = 0
        autosnap = no
        autoprune = yes
</code></pre></div></div>

<p>The only big difference to the primary and secondary backup copies, is that I store daily snapshots for longer on the backup server.</p>

<h2 id="testing-backups">Testing backups</h2>

<p>To verify that the backup works, we can perform a small experiment: 
I have an otherwise empty VM, where I’ve simply created a text file in my home directory:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ echo Test &gt; is_this_file_gone.txt
$ cat is_this_file_gone.txt
Test
</code></pre></div></div>

<p>After waiting for a snapshot to be made, I remove the test file:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ rm is_this_file_gone.txt
$ cat is_this_file_gone.txt
cat: is_this_file_gone.txt: No such file or directory
</code></pre></div></div>

<p>Now let’s restore the VM from my tertiary copy. If that works, we know that the intermediate backup routines work too.</p>

<p>From my backup server, I will send the snapshot back to the production machine, then see if I can start the VM from the snapshot.</p>

<p>First we find the relevant snapshots:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ zfs list -t snapshot
NAME                                                                                 USED  AVAIL  REFER  MOUNTPOINT
backuppool/prodsrv1/ssdpool/vdisks/testsrv1@autosnap_2025-11-16_11:00:25_daily         0B      -    96K  -
backuppool/prodsrv1/ssdpool/vdisks/testsrv1@autosnap_2025-11-16_11:00:25_hourly        0B      -    96K  -
backuppool/prodsrv1/ssdpool/vdisks/testsrv1@autosnap_2025-11-16_12:00:25_hourly        0B      -  2.84G  -
</code></pre></div></div>

<p>Now let’s take a specific snapshot and send it back:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo syncoid --sshkey=/root/.ssh/syncoid \
    -r \
    --no-sync-snap \
    --compress=lzo \
    --include-snaps=autosnap_2025-11-16_12:00:25_hourly \
    backuppool/prodsrv1/ssdpool/vdisks/testsrv1 \
    syncoid@prodsrv1:ssdpool/restoretest
</code></pre></div></div>

<p>To test the contents of the snapshot non-destructively, on the main server, we’ll stop the VM, move its expected disk image out of the way, and copy in the disk image we pulled from our backup.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd /ssdpool/vdisks/testsrv1
sudo virsh shutdown testsrv1
# After waiting for the VM to stop:
sudo mv testsrv1.qcow2 testsrv1.qcow2.actual
sudo cp --sparse=always /ssdpool/restoretest/testsrv1.qcow2 ./
sudo virsh start testsrv1
</code></pre></div></div>

<p>Once the machine starts up, we can test our backup:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$cat is_this_file_gone.txt
Test
</code></pre></div></div>

<p>As we’re happy everything works, we clean up the VM directory and restart the machine</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo virsh shutdown testsrv1
sudo rm testsrv1.qcow2
sudo mv testsrv1.qcow2.actual testsrv1.qcow2
sudo virsh start testsrv1
</code></pre></div></div>

<p>I always like to perform a dry-run before running destructive ZFS changes:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo zfs destroy -nvpr ssdpool/restoretest
destroy	ssdpool/restoretest@autosnap_2025-11-16_12:00:25_hourly
destroy	ssdpool/restoretest
</code></pre></div></div>

<p>This looks like exactly what we want to do. Remove the <code class="language-plaintext highlighter-rouge">n</code> argument to execute the command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo zfs destroy -vpr ssdpool/restoretest
</code></pre></div></div>

<p>And that’s it, really. I’ve confirmed that snapshots work and that I can restore them.</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="ZFS" /><category term="Sanoid" /><category term="Syncoid" /><summary type="html"><![CDATA[In my previous post, I discussed how I’ve migrated VMs to new storage. This gave me cause to also take a look at my backup configuration, to ensure I can still come back from catastrophic events.]]></summary></entry><entry><title type="html">Making bookmarks work in Safari 26</title><link href="https://oxcrag.net/blog/2025/10/28/making_bookmarks_work_in_safari_26.html" rel="alternate" type="text/html" title="Making bookmarks work in Safari 26" /><published>2025-10-28T00:00:00+01:00</published><updated>2025-10-28T00:00:00+01:00</updated><id>https://oxcrag.net/blog/2025/10/28/making_bookmarks_work_in_safari_26</id><content type="html" xml:base="https://oxcrag.net/blog/2025/10/28/making_bookmarks_work_in_safari_26.html"><![CDATA[<p>The Safari 26 update, changed how the Bookmarks menu works by default. I’m sure it’s based on the use case of most Safari users, and that most people simply a) don’t have a lot of bookmarks, or b) store them unstructured in their bookmarks menu. For both of those use cases, the new default view should work fine, either to find bookmarks visually by name, or by using the search field.</p>

<p>Personally, however, I’m a heavy user of bookmarks, at least for work purposes, and I’ve long been arranging them by category and subcategory. After the Safari 26 update, my logically arranged tree was a complete mess.</p>

<p>In the default view there are three usability issues:</p>
<ul>
  <li>Actual bookmarks are a completely different size and format from folders.</li>
  <li>Folders and bookmarks are separated.</li>
  <li>The folder tree is invisible - you only see the folders and bookmarks that exist inside the specific folder where you currently are.</li>
</ul>

<p>I have good news and bad news: 
The bad: I think this is unfixable in macOS 15, so if you’re stuck on that version, you’re probably SOL.
The good: There’s a workaround in macOS 26.</p>

<p>There are two steps required to make the Bookmarks menu become readable again:</p>
<ul>
  <li>Via the ellipsis menu, select <code class="language-plaintext highlighter-rouge">Compact</code> view to make bookmarks not stand out as a sore thumb.</li>
  <li>Via the ellipsis menu, <em>un</em>-select <code class="language-plaintext highlighter-rouge">Show folders on top</code>, to restore the tree view.</li>
</ul>

<p>This doesn’t preclude the ability to search bookmarks, which is good, but it allows you to visually identify categories again even if you have more than a handful of bookmarks and more than one level of folders.</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="macOS," /><category term="Safari" /><summary type="html"><![CDATA[The Safari 26 update, changed how the Bookmarks menu works by default. I’m sure it’s based on the use case of most Safari users, and that most people simply a) don’t have a lot of bookmarks, or b) store them unstructured in their bookmarks menu. For both of those use cases, the new default view should work fine, either to find bookmarks visually by name, or by using the search field.]]></summary></entry><entry><title type="html">Thoughts on openSUSE Tumbleweed</title><link href="https://oxcrag.net/blog/2025/08/02/Thoughts-on-openSUSE-Tumbleweed.html" rel="alternate" type="text/html" title="Thoughts on openSUSE Tumbleweed" /><published>2025-08-02T00:00:00+02:00</published><updated>2025-08-02T00:00:00+02:00</updated><id>https://oxcrag.net/blog/2025/08/02/Thoughts-on-openSUSE-Tumbleweed</id><content type="html" xml:base="https://oxcrag.net/blog/2025/08/02/Thoughts-on-openSUSE-Tumbleweed.html"><![CDATA[<p>I’ve been running openSUSE Tumbleweed for a little while now, and feel that I can speak a bit about it.</p>

<p>Please keep in mind that I’ve been ignoring my own advice to run a system the way its creators intended, and I’ve been using it with Gnome rather than with Plasma, so what I say here may not necessarily be valid if you choose the default desktop environment.</p>

<p>Compared to Fedora, which is very Gnome-centric, openSUSE clearly wants you to use the various tools provided under the YaST umbrella to manage the machine and its installed software. The tools clearly have been thought through, and being GUI tools, they provide a lot of discoverability to a new user. However, there’s always the feeling that they’re rooted in the UX mindset of late nineties/early noughties Microsoft. Think of a setting and you’ll probably find it, but first you need to find the right YaST sub-program.</p>

<p>Compared to Fedora’s Gnome Software interface, the YaST software installer is clearly superior at providing an overview of what’s installed and what will be installed to fulfill the dependencies if you add another software package to your computer. The issue is that again, the experience is.. practical? Definitely. Nice? Not really. Add to this that the SUSE software manager has no integration at all with Gnome Software, and you get weirdness like Gnome notifying you that there are software updates available, but when you click the notification, Software opens and tells you there are no updates to be had.</p>

<p>Weirdly, if you search for update, one of the early hits is the YaST2 online_update tool. That doesn’t show any available patches either. If you by chance open YaST2 Software Management, you can install new software, but there’s nowhere there to update installed packages. So down to the terminal we go, and then we need to know that zypper update and zypper dup (“dist upgrade”) are distinct functions, and while you’re warned against using the dist upgrade command on “regular” SUSE, on Tumbleweed, conversely, you may <a href="https://www.reddit.com/r/openSUSE/comments/zx8s1k/why_do_we_still_have_zypper_up_for_tumbleweed/">apparently</a> see negative consequences if you happen to try a regular update rather than a dist upgrade for keeping your system updated.</p>

<p>All in all, I see how for someone who came from Windows to Linux, SUSE - and especially SUSE with the Plasma desktop environment - could be of help in extending their comfort zone to include a discoverable and serious-feeling Linux distribution. But after a couple of weeks of usage, I have a hard time seeing where it would improve my own life. And as I’ve alluded to in both this post and in a <a href="https://oxcrag.net/blog/2025/07/21/openSUSE-first-thoughts.html">previous one</a>, this system isn’t exactly user friendly enough that I would recommend it to a novice Linux user either. So what am I missing? Who is it for?</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="Linux" /><category term="openSUSE" /><category term="Tumbleweed" /><category term="review" /><summary type="html"><![CDATA[I’ve been running openSUSE Tumbleweed for a little while now, and feel that I can speak a bit about it.]]></summary></entry><entry><title type="html">GitOps website deployment with Forgejo and Jekyll on FreeBSD</title><link href="https://oxcrag.net/blog/2025/07/26/GitOps-website-deployment-with-Forgejo-and-Jekyll-on-FreeBSD.html" rel="alternate" type="text/html" title="GitOps website deployment with Forgejo and Jekyll on FreeBSD" /><published>2025-07-26T00:00:00+02:00</published><updated>2025-07-26T00:00:00+02:00</updated><id>https://oxcrag.net/blog/2025/07/26/GitOps-website-deployment-with-Forgejo-and-Jekyll-on-FreeBSD</id><content type="html" xml:base="https://oxcrag.net/blog/2025/07/26/GitOps-website-deployment-with-Forgejo-and-Jekyll-on-FreeBSD.html"><![CDATA[<h2 id="background">Background</h2>
<p>A while back I had an out-of-band discussion with a visitor to this website, where he pointed out that the procedure I had documented in my <a href="/blog/2023/09/09/GitOps-website-deployment-with-Gitea.html">previous article</a> on triggering a Jekyll build via a Git hook on FreeBSD didn’t really work anymore. I couldn’t replicate the issue with my setup, so although the report was valid, I didn’t follow up on my own server at that time.</p>

<p>Unfortunately, the underlying issue seems to do with a) how FreeBSD handles Ruby, and b) how specific Jekyll dependencies in Ruby seem to be built only for Linux nowadays. As could be expected, a recent update to Ruby on FreeBSD broke the Jekyll builds on my Git forge.</p>

<h2 id="possible-solutions">Possible solutions</h2>
<p>From where I was, I saw two immediate ways forward: I could set up a new Forgejo server on some common Linux operating system, or I could set up a Linux based Forgejo runner instead of using Git hooks. And then <a href="https://mastodon.social/@mpts">@mpts</a> pinged me a <a href="https://mastodon.social/@mpts/114902254876625283">comment</a>:</p>

<blockquote>
  <p>When it comes to the Linux-only SASS library, a solution might be to run it with #Linuxulator on #FreeBSD.</p>
</blockquote>

<p>I had <a href="https://wiki.freebsd.org/Linuxulator">read</a> about the Linuxulator but never implemented it before, and this sounded like a fun way to spend a Saturday, so I decided to attempt this approach.</p>

<h2 id="implementation-steps">Implementation steps</h2>
<p>First off, I simply followed the <a href="https://docs.freebsd.org/en/books/handbook/linuxemu/">instructions for installing the Linuxulator</a> in the FreeBSD handbook. 
I elected to use Ubuntu, and of course chose Jammy (22.04 LTS) rather than Focal (20.04 LTS), as the latter is EOL. I would have preferred to use 24.04 simply for not having to touch the solution again in a while, but it isn’t yet supported by the Linuxulator.</p>

<p>I fixed the FreeBSD server’s <code class="language-plaintext highlighter-rouge">/etc/fstab</code> as per instructions, and of course made the necessary changes to the <code class="language-plaintext highlighter-rouge">apt</code> config in the Linuxulator environment.</p>

<p>The next step was to <code class="language-plaintext highlighter-rouge">chroot</code> into the Linuxulator environment and install the necessary packages:</p>
<ul>
  <li>build-essential</li>
  <li>git</li>
  <li>ruby</li>
  <li>ruby-dev</li>
</ul>

<p>I also copied the lines for the <code class="language-plaintext highlighter-rouge">git</code> user and group from the FreeBSD machine’s <code class="language-plaintext highlighter-rouge">/etc/passwd</code> and <code class="language-plaintext highlighter-rouge">/etc/group</code> into the corresponding files in the Linuxulator environment.</p>

<p>The next step was to create a home directory for the git user and populate it so that my git hook script (see further down) would work.</p>

<p>At this point, though, the Linuxulator could only be used as root, and that’s unworkable, for the very obvious reason that the Git hook script will run as the <code class="language-plaintext highlighter-rouge">git</code> user - and of course because it will be executing the Ruby bundler, which isn’t happy running as root.</p>

<p>To allow non-root users to chroot, I added the following line to <code class="language-plaintext highlighter-rouge">/etc/sysctl.conf.local</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>security.bsd.unprivileged_chroot=1
</code></pre></div></div>

<p>My current <code class="language-plaintext highlighter-rouge">post-receive</code> git hook for the website repo looks like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c">#!/usr/bin/env sh</span>

<span class="nb">export </span><span class="nv">WEBSERVER</span><span class="o">=</span>websrv2.oxcrag.net
<span class="nb">export </span><span class="nv">PRE_CHROOT_HOME</span><span class="o">=</span>/compat/ubuntu/usr/local/git
<span class="k">if</span> <span class="o">[</span> <span class="o">!</span> <span class="nt">-d</span> <span class="nv">$PRE_CHROOT_HOME</span>/tmp <span class="o">]</span><span class="p">;</span> <span class="k">then
    </span><span class="nb">mkdir</span> <span class="nv">$PRE_CHROOT_HOME</span>/tmp
<span class="k">fi
</span><span class="nb">export </span><span class="nv">PRE_CHROOT_TMP_GIT_CLONE</span><span class="o">=</span><span class="nv">$PRE_CHROOT_HOME</span>/tmp/oxcrag.net
<span class="nb">export </span><span class="nv">POST_CHROOT_PUBLIC_WWW</span><span class="o">=</span><span class="nv">$PRE_CHROOT_TMP_GIT_CLONE</span>/_site
<span class="c"># Verify target dir is really empty</span>
<span class="nb">rm</span> <span class="nt">-Rf</span> <span class="nv">$PRE_CHROOT_TMP_GIT_CLONE</span>
git clone <span class="nv">$GIT_DIR</span> <span class="nv">$PRE_CHROOT_TMP_GIT_CLONE</span>

<span class="nb">chroot</span> <span class="nt">-n</span> /compat/ubuntu bin/bash <span class="nt">-c</span> <span class="s1">'

export HOME=/usr/local/git

# Install Ruby Gems to ~/gems
export GEM_HOME=$HOME/gems
export PATH=$GEM_HOME/bin:$PATH

TMP_GIT_CLONE=$HOME/tmp/oxcrag.net
GEMFILE=$TMP_GIT_CLONE/Gemfile
PUBLIC_WWW=$TMP_GIT_CLONE/_site
gem install bundler --no-document
BUNDLE_GEMFILE=$GEMFILE bundle install
BUNDLE_GEMFILE=$GEMFILE bundle exec jekyll build -s $TMP_GIT_CLONE -d $PUBLIC_WWW

exit
'</span>
<span class="k">if </span>scp <span class="nt">-r</span> <span class="nv">$POST_CHROOT_PUBLIC_WWW</span> deployer@<span class="nv">$WEBSERVER</span>: <span class="p">;</span> <span class="k">then 
    </span><span class="nb">echo</span> <span class="s2">"Website deployment finished"</span>
<span class="k">else
    </span><span class="nb">echo</span> <span class="s2">"Website deployment failed"</span>
    <span class="nb">exit
</span><span class="k">fi

</span><span class="nb">rm</span> <span class="nt">-Rf</span> <span class="nv">$PRE_CHROOT_TMP_GIT_CLONE</span>
<span class="nb">exit</span>
</code></pre></div></div>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="FreeBSD" /><category term="Forgejo" /><category term="Nginx" /><category term="webserver" /><category term="Jekyll" /><category term="Linuxulator" /><summary type="html"><![CDATA[Background A while back I had an out-of-band discussion with a visitor to this website, where he pointed out that the procedure I had documented in my previous article on triggering a Jekyll build via a Git hook on FreeBSD didn’t really work anymore. I couldn’t replicate the issue with my setup, so although the report was valid, I didn’t follow up on my own server at that time.]]></summary></entry><entry><title type="html">openSUSE - First Thoughts</title><link href="https://oxcrag.net/blog/2025/07/21/openSUSE-first-thoughts.html" rel="alternate" type="text/html" title="openSUSE - First Thoughts" /><published>2025-07-21T00:00:00+02:00</published><updated>2025-07-21T00:00:00+02:00</updated><id>https://oxcrag.net/blog/2025/07/21/openSUSE-first-thoughts</id><content type="html" xml:base="https://oxcrag.net/blog/2025/07/21/openSUSE-first-thoughts.html"><![CDATA[<p>I started typing this as a Mastodon post but realized I had too much to say, so I wrote this entry instead.</p>

<h2 id="background">Background</h2>
<p>While I’m more or less fluent across the Debian derivatives, RHEL derivatives, and FreeBSD, I have never <em>really</em> used a SUSE Linux, despite the company being based here in Europe where I live.</p>

<p>Since Fedora is my normal choice, I figured the rolling release of openSUSE, Tumbleweed, would make the best comparison and what I would use if I was to switch.</p>

<p>I’ve been involved in some discussions regarding the novice friendliness and usability of Linux based operating systems related to the upcoming end-of-life of Windows 10, and I thought I’d try to approach this operating system from that perspective.</p>

<h2 id="installation">Installation</h2>
<p>openSUSE is surprisingly hard to find. A web search for “suse” on <a href="https://duckduckgo.com">DuckDuckGo</a> shows the Wikipedia article for the company, and the web site for the commercial SUSE Linux Enterprise offerings. These are of course utterly pointless for a hobbyist wanting to set up a regular laptop. You need to know that what you’re looking for is called <a href="https://www.opensuse.org/">openSUSE</a>.</p>

<p>Once past that hurdle, downloading the distribution and writing it to a USB stick is comparable to any other Linux distribution: Not something you’d expect just anyone to manage or even want to do, but not so hard that it would scare off a technically minded 13-year-old. I consider the first choice on any page to be the default one that most people should take, and on the openSUSE web page, Tumbleweed is that choice.</p>

<p>Once booted, the installer is graphical, so shouldn’t scare anybody away. Most default choices seem to be sane, but I would like to see two main changes to the installer:</p>
<ul>
  <li>The first screen, where you choose your network settings is not intuitive. You have to highlight your preferred network connection and then click on a separate key to edit its settings. Also, setting your machine’s hostname is a completely separate flow. Nobody who understands a bit about networking will have any issues completing the task, but there definitely exists a happy-path here that the installer could guide you through rather than forcing you to discover the various options on your own.</li>
  <li>The default disk partitioning flow has good defaults, but I miss a choice for full-disk encryption without having to dig down into the advanced settings. Also there was no easy way to manually make changes like resizing existing partitions or similar and then get a new disk layout recommendation based on the new configuration. This is less of a problem, though, as I would expect most who install a system like this would perform a clean install of a single system.</li>
</ul>

<h2 id="out-of-the-box-experience">Out-of-the-box experience</h2>
<p>I know that SUSE has traditionally been a Plasma (KDE) Linux distribution in the same way Red Hat has defaulted to Gnome. But since I do prefer Gnome, and the installer provides Gnome as its second option, that’s what I opted to use. This means I can’t speak for the Plasma user experience for someone who starts openSUSE Tumbleweed, but in Gnome, a novice is left with very little help. There is a welcome screen, but it completely fails to guide the user to YaST - arguably SUSE’s big selling point - even for someone who knows to search for it.</p>

<p>Searching for Software brings up the Gnome <code class="language-plaintext highlighter-rouge">Software</code> app, which in this distribution only seems to be an interface for installing Flatpak apps. The next search hits are two instances of <code class="language-plaintext highlighter-rouge">YaST Software ...</code> which is abbreviated so the user is left to guess what each program does, and two instances of <code class="language-plaintext highlighter-rouge">Myrlyn</code> with no further explanation. Again, for a novice, this is not discoverable, and not a great initial user experience.</p>

<h2 id="final-thoughts">Final thoughts</h2>
<p>I already like Tumbleweed as a Gnome-based operating system. The team behind the operating system seem to have done a very good job in many of the areas that you’re likely to care about over time. I like that they’ve seemingly implemented a variation of the FreeBSD style boot environment concept in Snapper, made possible thanks to BtrFS snapshot capability. I would frankly prefer to see more distributions than Ubuntu take ZFS seriously, but this at least goes some of the way - and further than you’d get out-of-the-box with Fedora or Ubuntu.</p>

<p>Again, I don’t know how the Plasma first login experience looks in Tumbleweed, but what I saw from Gnome is very likely to scare new users off - completely unnecessarily. There’s not a whole lot needed, but explain the benefits of YaST, provide a big honking icon pinned to the dock, and announce its presence and general usefulness via the Welcome screen.</p>

<p>I’m not sure whether it’s just my narrow social circle, but SUSE just doesn’t seem to have the mind share of Ubuntu or Fedora, and having tried it, I don’t really see why not. Again, there’s some work to be done in the new user experience, but I really think openSUSE <em>should</em> be more popular than it seems to be. The next time you’re bored with your operating system and want to shake things up a bit, try it out!</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="Linux" /><category term="openSUSE" /><category term="reviews" /><summary type="html"><![CDATA[I started typing this as a Mastodon post but realized I had too much to say, so I wrote this entry instead.]]></summary></entry><entry><title type="html">Fixing GUI stuttering in Xorg on FreeBSD on Lenovo x260</title><link href="https://oxcrag.net/blog/2025/06/18/fixing-gui-stuttering-in-xorg-on-freebsd-on-x260.html" rel="alternate" type="text/html" title="Fixing GUI stuttering in Xorg on FreeBSD on Lenovo x260" /><published>2025-06-18T00:00:00+02:00</published><updated>2025-06-18T00:00:00+02:00</updated><id>https://oxcrag.net/blog/2025/06/18/fixing-gui-stuttering-in-xorg-on-freebsd-on-x260</id><content type="html" xml:base="https://oxcrag.net/blog/2025/06/18/fixing-gui-stuttering-in-xorg-on-freebsd-on-x260.html"><![CDATA[<p>I’ve been seeing similar GUI freezes in FreeBSD, on my Lenovo x260, <a href="https://oxcrag.net/blog/2024/03/29/Lenovo-X260-as-a-Unix-workhorse.html">as I did in Fedora</a>: When using Xorg, my Lenovo x260 would occasionally become unresponsive for several seconds and then proceed as if nothing had happened.</p>

<p>Turns out the fix is very similar to the one for Fedora. Just add the following line to your <code class="language-plaintext highlighter-rouge">/boot/loader.conf</code>:</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>hw.i915kms.enable_psr=0
</code></pre></div></div>

<p>This turns off a feature called Panel Self-Refresh which seems buggy on this computer.</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="FreeBSD" /><category term="troubleshooting" /><category term="x260" /><summary type="html"><![CDATA[I’ve been seeing similar GUI freezes in FreeBSD, on my Lenovo x260, as I did in Fedora: When using Xorg, my Lenovo x260 would occasionally become unresponsive for several seconds and then proceed as if nothing had happened.]]></summary></entry><entry><title type="html">Reboot issues in Q35/UEFI KVM guests</title><link href="https://oxcrag.net/blog/2025/06/07/reboot-issues-in-kvm-guests.html" rel="alternate" type="text/html" title="Reboot issues in Q35/UEFI KVM guests" /><published>2025-06-07T00:00:00+02:00</published><updated>2025-06-07T00:00:00+02:00</updated><id>https://oxcrag.net/blog/2025/06/07/reboot-issues-in-kvm-guests</id><content type="html" xml:base="https://oxcrag.net/blog/2025/06/07/reboot-issues-in-kvm-guests.html"><![CDATA[<p>For a while, I’ve been having issues when soft rebooting some of my older KVM guest machines: They would hang on reboot, and when connecting to the VM console via the Virtual Machine Manager, it was obvious that they had hung before the POST screen. It only happened on Q35 UEFI machines, and only, as mentioned, on older ones, so after some digging I started suspecting I could have an issue with the virtual machine version.</p>

<p>I used Virtual Machine Manager to edit the XML of one of the problematic machines, and located the relevant section:</p>
<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;type</span> <span class="na">arch=</span><span class="s">"x86_64"</span> <span class="na">machine=</span><span class="s">"pc-q35-4.2"</span><span class="nt">&gt;</span>hvm<span class="nt">&lt;/type&gt;</span>
</code></pre></div></div>

<p>Next question was what version to choose. I tried the <code class="language-plaintext highlighter-rouge">kvm</code> command on the hypervisor and quickly found the correct parameters to use, and then also the answer to my question:</p>

<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$ </span>kvm <span class="nt">-machine</span> <span class="nb">help
</span>Supported machines are:
<span class="o">(</span>...<span class="o">)</span>
q35                  Standard PC <span class="o">(</span>Q35 + ICH9, 2009<span class="o">)</span> <span class="o">(</span><span class="nb">alias </span>of pc-q35-6.2<span class="o">)</span>
<span class="o">(</span>...<span class="o">)</span>
</code></pre></div></div>

<p>I replaced the relevant part of the VM definition XML, changing the <code class="language-plaintext highlighter-rouge">machine</code> part:</p>

<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nt">&lt;type</span> <span class="na">arch=</span><span class="s">"x86_64"</span> <span class="na">machine=</span><span class="s">"q35"</span><span class="nt">&gt;</span>hvm<span class="nt">&lt;/type&gt;</span>
</code></pre></div></div>

<p>Upon applying the change, the line auto changed into the expected <code class="language-plaintext highlighter-rouge">pc-q35-6.2</code>.</p>

<p>The VM had to be fully stopped and restarted for the change to apply, but after that, it could soft reboot from the command line with no issues, so I performed the same change to the other VMs that had the same problem.</p>

<p>The real test will be to see how they behave after the next unattended upgrade, but so far so good.</p>]]></content><author><name>Mikael Hansson</name><email>blog@oxcrag.net</email></author><category term="blog" /><category term="KVM" /><category term="troubleshooting" /><summary type="html"><![CDATA[For a while, I’ve been having issues when soft rebooting some of my older KVM guest machines: They would hang on reboot, and when connecting to the VM console via the Virtual Machine Manager, it was obvious that they had hung before the POST screen. It only happened on Q35 UEFI machines, and only, as mentioned, on older ones, so after some digging I started suspecting I could have an issue with the virtual machine version.]]></summary></entry></feed>