Friday, August 17, 2018

Red Hat Container Development Kit 3.5 - Installation Issue with Libvirt (KVM) Group Membership

If you follow the installation instructions for Red Hat's Container Development Kit 3.5, on RHEL 7.5, most everything works well, until you get to the point of starting the virtualized minishift host:

$ minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://mirror.openshift.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.9.31' is valid ... OK
-- Checking if requested OpenShift version 'v3.9.31' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ... 
   Driver is available at /usr/local/bin/docker-machine-driver-kvm ... 
   Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting local OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
   Memory:    4 GB
   vCPUs :    2
   Disk size: 20 GB
-- Starting Minishift VM ................... OK
-- Registering machine using subscription-manager
   Registration in progress ........ OK [16s]
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ... 
   Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ... 
   Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
   Importing 'registry.access.redhat.com/openshift3/ose:v3.9.31'  CACHE MISS
   Importing 'registry.access.redhat.com/openshift3/ose-docker-registry:v3.9.31'  CACHE MISS
   Importing 'registry.access.redhat.com/openshift3/ose-haproxy-router:v3.9.31'  CACHE MISS
-- OpenShift cluster will be configured with ...
   Version: v3.9.31
-- Copying oc binary from the OpenShift container image to VM ................ OK
-- Starting OpenShift cluster ..................
Using nsenter mounter for OpenShift volumes
Using public hostname IP 192.168.42.109 as the host IP
Using 192.168.42.109 as the server IP
Starting OpenShift using registry.access.redhat.com/openshift3/ose:v3.9.31 ...
OpenShift server started.
The server is accessible via web console at:
    https://192.168.42.109:8443
You are logged in as:
    User:     developer
    Password: <any value>
To login as administrator:
    oc login -u system:admin
-- Applying addon 'anyuid':.
 Add-on 'anyuid' changed the default security context constraints to allow pods to run as any user.
 Per default OpenShift runs containers using an arbitrarily assigned user ID.
 Refer to https://docs.openshift.org/latest/architecture/additional_concepts/authorization.html#security-context-constraints and
 https://docs.openshift.org/latest/creating_images/guidelines.html#openshift-origin-specific-guidelines for more information.
-- Applying addon 'xpaas':.........................................................................................................................
XPaaS OpenShift imagestream and templates installed
See https://github.com/openshift/openshift-ansible/tree/release-3.9/roles/openshift_examples/files/examples/v3.9
-- Applying addon 'admin-user':..
Error determining Docker settings for image import: dial tcp: missing address


You get a failure when trying to connect to Docker on the VM.  The problem is that, if you strictly follow the instructions, your user is not a member of the 'libvirt' group.  This is easy to rectify, though.  All you need to do is:

1) Remove the existing minishift VM:

$ minishift delete
You are deleting the Minishift VM: 'minishift'. Do you want to continue [y/N]?: y
Deleting the Minishift VM...
Minishift VM deleted.
$ rm -rf ~/.minishift


2) Add yourself to the requisite group:

$ sudo usermod -a -G libvirt `whoami`
$ newgrp libvirt
$ groups
libvirt wheel ajacocks


3) And relaunch minishift:

$ minishift setup-cdk
Setting up CDK 3 on host using '/home/ajacocks/.minishift' as Minishift's home directory
Copying minishift-rhel7.iso to '/home/ajacocks/.minishift/cache/iso/minishift-rhel7.iso'
Copying oc to '/home/ajacocks/.minishift/cache/oc/v3.9.31/linux/oc'
Creating configuration file '/home/ajacocks/.minishift/config/config.json'
Creating marker file '/home/ajacocks/.minishift/cdk'
Default add-ons anyuid, admin-user, xpaas, registry-route, che, eap-cd installed
Default add-ons anyuid, admin-user, xpaas enabled
CDK 3 setup complete.
$ minishift start
-- Starting profile 'minishift'
-- Check if deprecated options are used ... OK
-- Checking if https://mirror.openshift.com is reachable ... OK
-- Checking if requested OpenShift version 'v3.9.31' is valid ... OK
-- Checking if requested OpenShift version 'v3.9.31' is supported ... OK
-- Checking if requested hypervisor 'kvm' is supported on this platform ... OK
-- Checking if KVM driver is installed ...
Driver is available at /usr/local/bin/docker-machine-driver-kvm ...
Checking driver binary is executable ... OK
-- Checking if Libvirt is installed ... OK
-- Checking if Libvirt default network is present ... OK
-- Checking if Libvirt default network is active ... OK
-- Checking the ISO URL ... OK
-- Checking if provided oc flags are supported ... OK
-- Starting local OpenShift cluster using 'kvm' hypervisor ...
-- Minishift VM will be configured with ...
Memory: 4 GB
vCPUs : 2
Disk size: 20 GB
-- Starting Minishift VM ................. OK
-- Registering machine using subscription-manager
Registration in progress ........ OK [16s]
-- Checking for IP address ... OK
-- Checking for nameservers ... OK
-- Checking if external host is reachable from the Minishift VM ...
Pinging 8.8.8.8 ... OK
-- Checking HTTP connectivity from the VM ...
Retrieving http://minishift.io/index.html ... OK
-- Checking if persistent storage volume is mounted ... OK
-- Checking available disk space ... 1% used OK
Importing 'registry.access.redhat.com/openshift3/ose:v3.9.31' CACHE MISS
Importing 'registry.access.redhat.com/openshift3/ose-docker-registry:v3.9.31' CACHE MISS
Importing 'registry.access.redhat.com/openshift3/ose-haproxy-router:v3.9.31' CACHE MISS
-- OpenShift cluster will be configured with ...
Version: v3.9.31
-- Copying oc binary from the OpenShift container image to VM ............... OK
-- Starting OpenShift cluster ....................
Using nsenter mounter for OpenShift volumes
Using public hostname IP 192.168.42.36 as the host IP
Using 192.168.42.36 as the server IP
Starting OpenShift using registry.access.redhat.com/openshift3/ose:v3.9.31 ...
OpenShift server started.
The server is accessible via web console at:
https://192.168.42.36:8443
You are logged in as:
User: developer
Password: <any value>
To login as administrator:
oc login -u system:admin
-- Applying addon 'xpaas':.........................................................................................................................
XPaaS OpenShift imagestream and templates installed
See https://github.com/openshift/openshift-ansible/tree/release-3.9/roles/openshift_examples/files/examples/v3.9
-- Applying addon 'admin-user':..
-- Applying addon 'anyuid':.
Add-on 'anyuid' changed the default security context constraints to allow pods to run as any user.
Per default OpenShift runs containers using an arbitrarily assigned user ID.
Refer to https://docs.openshift.org/latest/architecture/additional_concepts/authorization.html#security-context-constraints and
https://docs.openshift.org/latest/creating_images/guidelines.html#openshift-origin-specific-guidelines for more information.
-- Exporting of OpenShift images is occuring in background process with pid 3944.


Hopefully I can get the docs updated, so this isn't necessary, in the near future.

Wednesday, August 1, 2018

Suspend Issues with the Lenovo ThinkPad X1 Carbon 6th Generation

UPDATE 2018/09/12:
And the final solution seems to be at hand!  Lenovo has released 0.1.30, and that apparently re-adds S3 sleep to the ACPI tables:

https://brauner.github.io/2018/09/08/thinkpad-6en-s3.html

UPDATE 2018/09/07:
So, it appears that there is a new firmware version available from Lenovo, 0.1.28, the patch no longer cleanly applies, and Adrian has re-rolled it:

https://lisas.de/~adrian/?p=1328

Thanks, Adrian, that's a much easier way to reliably patch the ACPI tables!

I'll update this post, later, with an updated step-by-step guide.

ORIGINAL POST:
Those of you who have the new (and very nicely built!) Lenovo ThinkPad X1 Carbon 6th generation, and who run Linux, may have noticed that suspend doesn't work properly.  I'm running Fedora 28, but this problem has been reported on Arch and Ubuntu, as well.  The issue seems to be that the X1 is using a new suspend technology called "Windows Modern Standby," or S0i3, and has removed classic S3 sleep.  S0i3 technology was added to the mainline Linux kernel with v4.13, but there is apparently a BIOS bug preventing this from working properly.

To work around this problem, we need to patch the ACPI DSDT tables, created by the BIOS, to re-add S3 sleep.  There's a fantastic guide here, which was written by Erik Sonnleitner of the University of Applied Sciences of Upper Austria.

To summarize his excellent directions:
  • In the BIOS, set "Thunderbolt BIOS Assist Mode" to Enabled, and disable secure boot.
  • Install "iasl" and "cpio"
  • Once the system has booted whichever Linux distribution that you use, dump the DSDT table
sudo cat /sys/firmware/acpi/tables/DSDT > dsdt.aml
  • Reverse compile the DSDT table
iasl -d dsdt.aml
  • Download the patch, and patch your decompiled code
    • I had to hand-patch hunk #7, by hand, but this wasn't a big deal
patch --verbose < X1C6_S3_DSDT.patch
  • Compile your updated DSDT code
iasl -ve -tc dsdt.dsl
  • Create a new ACPI override package
mkdir -p kernel/firmware/acpi
cp dsdt.aml kernel/firmware/acpi
find kernel | cpio -H newc --create > acpi_override
sudo cp acpi_override /boot

  • Now, you need to update your grub.cfg
sudo sed -i 's/quiet"/quiet mem_sleep_default=deep"/' /etc/default/grub
echo GRUB_EARLY_INITRD_LINUX_CUSTOM=acpi_override | sudo tee -a /etc/default/grub
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
  • The second change won't work immediately, on Fedora, as GRUB_EARLY_INITRD_LINUX_CUSTOM isn't yet supported
  • So, we need to update the grub boot config, and then manually update the initrd entries to add the new ACPI overlay, before booting the initial ramdisk
sudo sed -i 's/initrd16 \/initramfs/initrd16 \/acpi_override \/initramfs/' /boot/grub2/grub.cfg
  • Upon reboot, suspend should work properly.