Upgrading Ubuntu 12.04 to 14.04 breaks encrypted LVM

My laptop runs Ubuntu and is fully encrypted (since version 10.04). Upgrade from 10.04 to 12.04 was smooth in the sense that my system booted fine, asking for the passphrase to unlock the LVM. However, when I upgraded from 12.04 to 14.04, things broke and my laptop no longer booted properly as the LVM never got encrypted. I had to do the following to get my laptop working again (after many rounds of trial and error):

  • Boot a live usb Ubuntu session, de-crypted the LVM, and chroot’ed to run as the original OS
  • Finish the upgrade session via apt-get update && apt-get upgrade
  • It appears Ubuntu 14.04 installed some new package (did not write name down) that manages LVM or disks somehow (based on googling the error message). I removed this package.
  • Saw lvm issues, so installed the package lvm2
  • I made sure both dm-crypt and lvm2 were installed, and were accessible in initramfs, as cryptsetup was removed from initramfs since version 13.10. Had to do something with the following CRYPTSETUP issue.
  • Based on this post, I modified various files, but things still did not boot properly. I believe what finally fixed it was explicitly pointing to the LVM by /dev/sda5 in the GRUB_CMDLINE_LINUX line in /etc/default/grub.

The following is summary of these files for me. /etc/crypttab:

# <target name> <source device>         <key file>      <options>
# sdb5_crypt UUID=731a44c4-4655-4f2b-ae1a-2e3e6a14f2ef none luks
sdb5_crypt UUID=731a44c4-4655-4f2b-ae1a-2e3e6a14f2ef none luks,retry=1,lvm=vg01


## vinh created http://www.joh.fi/posts/2014/03/18/install-ubuntu-1310-on-top-of-encrypted-lvm/
# CRYPTROOT=target=sdb5_crypt,source=/dev/disk/by-uuid/f1ba5a54-ac7e-419d-8762-43da3274aba4

Then run update-initramfs -k all -c in order to update the initramfs images.

Have this line in /etc/default/grub:


Run update-grub.

Again, I think the key is the source definition in the previous line. I kept trying to refer to it by uuid but that did not work.

Remote unlocking LUKS encrypted LVM using Dropbear SSH in Ubuntu

I recently performed a full disk encryption on my server using dm-crypt + LUKS. I did not address remote unlocking of the disk then because I did not know how. Remote unlocking is highly desirable I might not be physically near the server when a restart is necessary.

To remotely unlock the disk, one needs an ssh server running during startup (boot). Then, ssh into the server and unlock the disk with the passphrase. I originally was going to follow this post to perform remote unlocking via early-ssh. However, I couldn’t figure out how to do so. It appears early-ssh is no longer needed as the solution can be easily implemented with Dropbear SSH Server and Busybox in Ubuntu; see the documention at /usr/share/doc/cryptsetup/README.remote.gz.

It took me quite some time to figure out how to set things up. I first had issues with logging into the Dropbear server (normal user accounts won’t work); this post helped me figure out how to log in. Then I had a difficult time with how to unlock the disk once I’m in the server. The solution is elegantly described here and here.

Set up Dropbear SSH Server

sudo apt-get install dropbear busybox ## do not install early-ssh

There is an error in the dropbear hook script in initramfs-tools. To fix it, do

find /lib -name libnss_files.so.2
## me:

At around line 30 in /usr/share/initramfs-toosl/hooks/dropbear, replace =cp lib/libnss_ “${DESTDIR}/lib/”= with =cp lib/x86_64-linux-gnu/libnss_ “${DESTDIR}/lib/”= (if early-ssh is installed, it will give further errors related to this).

Now, run:

update-initramfs -u

Enable the root account in Ubuntu as only the root user can login to Dropbear SSH Server during boot (entire disk is encrypted):

sudo passwd root
## enter root password
## to disable root account:
## sudo passwd -dl root

Now, in your laptop (not server), copy over the private key in order to login to Dropbear SSH Server:

scp user@remote.server:/etc/initramfs-tools/root/.ssh/id_rsa ~/.ssh/remote_dropbear_id_rsa

NOTE: It appears you HAVE to to use the generated private key in order to login. Login with password will not work. I also tried copying my laptop’s public key into the server’s /etc/initramfs-tools/root/.ssh/authorized_keys so that I can use my laptop’s key to login but that did not work. I might have to translate my laptop’s private key to dropbear’s formatin order for it to work. Since I have to use another file regardless, I’ll just use Dropbear’s private key.

Disable root login for OpenSSH as it is unsafe to login as root (we only allow root to login when Dropbear SSH server is running during startup and restrict root all other times):

## change in /etc/ssh/sshd_config
PermitRootLogin no

If I restart the server now, Dropbear SSH Server will run after some time when the system is waiting for the passphrase to unlock the disk. To SSH into the Dropbear server, do:

ssh -o "UserKnownHostsFile=~/.ssh/known_hosts.initramfs" -i "~/.ssh/remote_dropbear_id_rsa" root@my.server

Remote Unlocking

It appears the original method to unlock the disk does not work with Ubuntu 11.04:

ssh -o "UserKnownHostsFile=~/.ssh/known_hosts.initramfs" -i "~/.ssh/remote_dropbear_id_rsa" root@my.server "echo -ne "encryptionpassphrase" > /lib/cryptsetup/passfifo"

The error is due to Plymouth. Uninstalling or tinkering with Plymouth could cause other errors (like allowing remote unlocking to work but one loses the ability to unlock in at the server’s physical console). To get remote unlocking to work, follow the manual method described here:

## log into dropbear
## locate the process id (first column) for the /scripts/local-top/cryptroot script
kill -9 pid ## PID from previous
## look for a wait-for-root script and note the timeout on the command line; mine: 30
## wait 30 seconds
## enter passphrase
## locate process ID for /bin/sh -i
kill -9 PID

A more concise command is:

pid=`ps | grep "/scripts/local-top/cryptroot" | cut -d " " -f 3`; kill -9 $pid; sleep 35; /scripts/local-top/cryptroot; pid=`ps | grep "/bin/sh" | cut -d " " -f 3`; kill -9 $pid; exit

The disk should unlock and you can now ssh normally into the server (root not allowed!). YAY!

I’m sure one can automate this last portion using a script. Also, I would like to add a startup script that emails me when the server is waiting for a passphrase. This will be useful if the system restarts due to a power outtage without me knowing.

Full disk encryption on Ubuntu with dm-crypt + luks


In this post, I will outline my experience doing a full disk encryption on an Ubuntu computer. Note that this option is available through the installer only on the server edition or the alternate CD of ubuntu (not desktop).

Why would one want to encrypt their disk? A few scenarios:

  1. Suppose someone steals your laptop. Do you want them to have access to your files? With full disk encryption, they won’t even be able to boot up the laptop.
  2. Suppose you send in your disk for repair or exchange. Do you want your personal files to be freely accessible by others?
  3. Suppose the goverment wants to infringe on your right to privacy. Do you want them to easily access your files? Any access to my files will have to be consented by me.

To achieve full disk encryption, what we will do is set up an encrypted LVM. Before getting started, read this to know more about the benefits of an LVM. Then read this which explains the difference between a RAID setup and LVM. They are different things, and can be configured together. Then read this post which benchmarks the performance of the system with an encrypted and unencrypted disk. The difference in performance is negligible for the benefit of having secured data. Also look at this post which shows how one can gain access to an encrypted LVM drive; the bulk of the information came from here.

My setup: I have two 1.5TB disks set up using hardware RAID via the mobo’s BIOS. I then followed the instructions outlined here for setting up the encrypted LVM; the only difference is that I have a RAID1 configuration and did not a separate volume for “/home”. The setup is identical. Prior to trying this out, my concerns were addressed in the comments of that page and here. I ran into some issues as described in the comments of that page. Basically, I got a blank or unresponsive screen after the BIOS pages. I was not asked for my passphrase. Rebooting the computer yields the grub boot menu. Selecting recovery-mode, I was asked for my passphrase before the recovery menu appeared. I then selected boot normally and the server started. This was quite annoying because I did not want to do that many steps just to get a system booted each time. After a few hours of trying to find out what’s wrong and re-installing (thinking the culprit was the RAID setup), I found out that the passphrase is asked for in TTY7 (Control-alt-F7). I didn’t see it because I think TTY1 is Ubuntu’s Server default, hence I saw a blank or unresponsive screen. Now I know the installation process went well and it wasn’t because of RAID. However, I will have to go to TTY7, type in passphrase, and go back to TTY1 to log in. I guess this issue isn’t too problematic since it is a remote server, and don’t plan to be in front of it at each reboot. I plan to follow this post to set up early-ssh and dropbear to be able decrypt the drive via ssh. I haven’t figured out how to use it yet though because my username and password isn’t accepted by dropbear. I’ll update this post once I figure out how to login and submit the script to decrypt the drive.

In the future, I plan to add two more hard drives configured as RAID1. I guess I can just encrypt the drive like usual via dm-crypt and automount it by modifying crypttab/fstab.

UPDATE 9/9/2011 Changing passphrase by adding the new one and removing the old one

Changing a passphrase in dm-crypt was discussed here. Since I was on RAID1 and encrypted my entire LVM, I couldn’t operate on devices like /dev/sda5, etc. Actually, sda# and sdb# weren’t even in /dev/ even though they were listed in sudo fdisk -l. I tried cryptsetup luksDump on /dev/sda, /dev/sdb, and all in /dev/mapper/. The only one that was a valid LUKS device was pdc_dejidcjhg5. Thus, I did

sudo cryptsetup luksAddKey /dev/mapper/pdc_dejidcjhg5 ## added my new long passphrase
sudo cryptsetup luksRemoveKey /dev/mapper/pdc_dejidcjhg5 ## entered in passphrase I wanted removed
sudo cryptsetup luksDump /dev/mapper/pdc_dejidcjhg5 ## should show slot 0 is disabled, slot 1 is enabled

Encrypt hard drives and usb drives with dm-crypt and TrueCrypt


I recently explored encrypting usb drives and external hard drives (well, any hard drive really). I always wanted to lock all of my hard drives just to feel more secure about my files, but never got around to learning how to. In this post, I’ll describe my experience for usb drives and hard drives that aren’t used as the main disk for a computer to boot. I’ll describe full disk encryption for Ubuntu (where the OS resides) in another post.

I explored two methods. The first is using DM-Crypt with LUKS, which is the free and truly open-source solution. The second is using TrueCrypt, which is free (as in beer) and is open-source/proprietary (you can see source code, but I don’t know about modifying and redistributing). A more extensive list can be found here.

This post describes how one can go about encrypting the drive with dm-crypt. You can format the drive as FAT32, ext2, ext3, or anything really (this describes how to format a drive as FAT32 on the command line). On Linux, you just have to use cryptsetup to unencrypt the drive first. Then mount it for use. Ubuntu Desktop does this automatically (asks you for your passphrase) when the drive is plugged in. On a Windows machine, you have to install FreeOTFE in order to decrypt the drive. If the drive is FAT32, you will have access to it automatically after mounting the encrypted drive. If the drive is ext2 or ext3, you will have to install Ext2Fsd first in order to mount the drive. As of now, there isn’t a way to decrypt the drive on Mac OS X.

TrueCrypt is available on all three major platforms: Windows, Mac, and Linux. Thus, it is better than dm-crypt for usb drives in the sense that you can also use them on a Mac. You can use it to encrypt an entire disk or create an encrypted container file (pseudo partition?) to place files you want secured into. The latter approach is good for smaller files. For archives (big size), the FAT32 combination won’t work due to the 4.7GB file size limitation. Getting started with TrueCryt is quite easy as you just have to follow the GUI. To access the drive on any machine, you have to install TrueCrypt. If the drive is ext2 or ext3, again, you have to install Ext2Fsd on Windows. To mount an ext2 or ext3 drive on a Mac, you have to install ext2fuse, of which MacFuse is a pre-requisite.

What will I use? For a hard drive that I know I will only use with Linux, I’ll go the dm-crypt approach. For drives that I might bring around and plug into other platforms, I will either go with the dm-crypt + ext3 or TrueCrypt + ext3 approach. Why ext3? FAT32 is recognized on all three platforms, PS3, and most other devices that you can plug in the USB drive. However, if the disk is encrypted, the places I can plug the disk into is VERY limited (i.e. a computer). There’s no reason to prefer FAT32. Sure, I would have to install additional software on a Mac or Windows machine in order to read ext2/3. However, to unencrypt on those platforms, I would have needed to install TrueCrypt or FreeOTFE anyways, so this addtional install is trivial. Hmm, since I don’t really plug my drives into a Mac anymore, I think I’ll just stick with dm-crypt. The trick to all this is to have a spare unencrypted usb drive that you can use on a daily basis =].

Change passphrase by adding a new one and removing the old one

I thought it was impossible, but it’s quite easy. Just use cryptsetup luksAddKey /dev/MYDEVICE to add a new passphrase, and cryptsetup luksRemoveKey /dev/MYDEVICE to remove the old passphrase.