1 Overview

luksipc is a tool to convert (unencrypted) block devices to (encrypted) LUKS devices in-place (therefore it's name LUKS in-place conversion). This means the conversion is performed without the need of copying all data somewhere, recreating the whole disk (i.e. create a LUKS device, create a new filesystem on the mapped LUKS device, copy all data back). Instead, the process is reduced to:

1. Unmounting the filesystem
2. Resizing the filesystem to shrink about 10 megabytes (2048 kB is the current LUKS header size -- but do not trust this value, it has changed in the past!)
3. Performing luksipc
4. Adding custom keys to the LUKS keyring

2 Documentation

This page is not the most recent documentation, because I've moved the most recent docs to GitHub, where the development snapshot is also hosted. You'll find the most recent documentation there.

3 Getting started

3.1 Theory of operation

3.1.1 Rough explanation

Image that you have a non-encrypted hard disk. The file system has been created right on the device (/dev/sda6 in this case), which means it fits really snugly:

Now first in order to use luksipc you need to resize the file system itself. This is done, for example, with "resize_reiserfs" for a ReiserFS filesystem. This only decreases the amount of space the file system occupies within the partition and is usually very fast. Resizing has to be at least the size of the LUKS header. In the following case I use 10 MiB to be on the safe side:

After this, luksipc comes into play. It performs an in-place encryption of the data and prepends the partition with a LUKS header:

3.1.2 In-depth explanation

Now what's the big deal about all this, anyways? The jumping point is that the LUKS header must be prepended to the partition. This means it has to be created where part of the file system resides. All data is then offset by this constant amount in a LUKS scenario (this is about 2048 kB). Imagine you have a disk which consists of five "chunks". One chunk is a block of space which is exactly the space the LUKS header takes (and also the amount that the encrypted disk is smaller than the unencrypted disk):

Then, you will have to resize your filesystem at least one chunk (which means the last chunk will not be occupied anymore). The disk space is reduced to 4 chunks:

Then luksipc is started: What it now does is it first loads chunk 0 into memory, since this is the chunk which will be overwritten by the LUKS header in the next step:

Then, the disk is LUKS-formatted, which creates a LUKS header. The data of chunk 0 is not lost, it's still kept in memory. Creating the LUKS header creates a second, "virtual" LUKS disk (i.e. /dev/mapper/luksipc), which is exactly one chunk shorter than the original disk. Every chunk on the virtual disk maps to one chunk on the physical disk, only offset by one chunk (i.e. virtual chunk 0 maps to physical chunk 1, virtual 1 maps to physical 2 and so on). The data that can be read from the virtual device at that point is garbage, because when you read from the LUKS device at that point, dm-crypt will decrypt the unencrypted data of the physical partition.

Then, a second chunk, chunk 1, is read into memory:

This is the time when the first chunk can be written to the encrypted disk. chunk 0 is written to the virtual chunk position 0 (which maps to physical chunk 1):

Then, chunk 2 is read in again. Since we have written chunk 0 to the virtual disk, we can reuse the memory space of chunk 0, since that doesn't need to be kept in memory anymore.

Again a chunk can now be written, this time chunk 1:

And a chunk can be read again, chunk 3 (overwriting the position of chunk 1 in memory):

Now that we've read all chunks, the remaining two chunks in memory can be written to the LUKS disk. First chunk 2:

Afterwards, chunk 3 is written:

Now we're all finished and luksipc is done converting the partition to LUKS!

3.2 Aborting the process

Obviously, when you abort the process in the middle of the conversion (meaning your disk is only half-converted), this is a bad situation. While actually the disk is almost completely readable (one half on the unencrypted part, the other on the LUKS device), you will have a very hard time, since you don't know where exactly that cut is. Furthermore, one chunk needs to be kept in memory at all times. This chunk is lost. Therefore, luksipc offers a solution. Should you, for whatever reason, need to abort the process, luksipc will create a "resume.bin" file for you. In this file it will write the current block device position and one chunk of data. Later on, by just calling luksipc with the "--resume" option, the process can be resumed from that point on.

3.3 Disclaimer

Before you attempt to do anything with the device, be warned: You may lose some of your data, even all of it (for example if you put the LUKS device key in /dev/shm and reboot your system before you add another key to the keyring). Power failures are also bad (since you will not have a resume file in that case). Furthermore, luksipc may have bugs that wreak havoc on your data. Also keep in mind that luksipc relies completely on a perfect disk. If your disk has read-problems, it likely will abort somewhere within the middle of the process (most likely also without creating a resume file). If your disk is faulty, get a new one instead of trying to crypt it. And, most importantly: Always have a backup. Now, let's be honest here: You probably don't have a backup. If you had the disk space, you wouldn't have the need to convert data in-place. Or maybe you have one and it's really old. Or the data is not really that important. In any case, please please please do not assume that everything will run smoothly. It may not. You have been warned. I will not be held responsible for any of your actions.

That said let me point out that I trusted my software (after thourough testing) enough to let it convert a 1 TB partition without having a backup. This worked nicely. However, your milage may vary.

3.4 Running it

In the package, there is a regression/ subdirectory which performs quite a lot of tests. If you're unsure about how reliable it is or how it works, play around with those first to get a feeling of how it works. If you have any special options that you're passing to luksFormat and you feel that you might run into trouble - for god's sake, please first try it on some loopback block device first to be on the safe side.

Apart from that, running it is really straightforward (here I'm testing on a 800 MB loopback device where the underlying file is on /dev/shm in case you're wondering -- a hard disk will be much slower). Do not ever forget to resize your filesystem. I'm demonstrating how this is done for ReiserFS here:

joequad [~/luksipc-0.04]: resize_reiserfs -s -10M /dev/loop0
resize_reiserfs 3.6.21 (2009 www.namesys.com)

You are running BETA version of reiserfs shrinker.
This version is only for testing or VERY CAREFUL use.
Backup of you data is recommended.

Do you want to continue? [y/N]:y
Processing the tree: 0%....20%....40%....60%....80%....100%                           left 0, 0 /sec

nodes processed (moved):
int        0 (0),
leaves     1 (0),
unfm       0 (0),
total      1 (0).

check for used blocks in truncated region

ReiserFS report:
blocksize             4096
block count           204288 (204800)
free blocks           196070 (196582)
bitmap block count    7 (7)

Syncing..done


resize_reiserfs: Resizing finished successfully.

Afterwards, just to confirm that the device has really been resized, we can double-check:

joequad [~/luksipc-0.04]: debugreiserfs /dev/loop0
debugreiserfs 3.6.21 (2009 www.namesys.com)


Filesystem state: consistent

Reiserfs super block in block 16 on 0x700 of format 3.6 with standard journal
Count of blocks on the device: 202240
Number of bitmaps: 7
Blocksize: 4096
[...]

This means, the filesystem occupies 202240 blocks of 4096 bytes each. This is 828375040 bytes. The whole disk is 800 MB or 800 * 1024 * 1024 bytes = 838860800. The difference of these two is 10485760, which means we gained exactly the ten megabytes that we needed. Then, you can run luksipc:

joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0
WARNING! All data on /dev/loop0 is to be LUKSified! Ensure that:
   1. You have resized the contained filesystem appropriately
   2. You have ensured secure storage of the keyfile
   3. Power conditions are satisfied (i.e. your Laptop is not running off battery)
   4. You have a backup of all data on that device

    /dev/loop0: 800 MB = 0.8 GB
    Keyfile: /root/initial_keyfile.bin
    LUKS format parameters: None given

Are all these conditions satisfied, then answer uppercase yes: YES
[I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes)
[I]: Performing dm-crypt status lookup
[I]: Performing luksFormat
[I]: Performing luksOpen
[I]: Size of cryptodisk is 837808128 bytes (798 MB + 1044480 bytes)
[I]: 1052672 bytes occupied by LUKS header (1028 kB + 0 bytes)
[I]: Starting copying of data...
[I]:  0:00:  13.1%       105 MB / 798 MB   116.3 MB/s       693 MB left
[I]:  0:00:  25.9%       207 MB / 798 MB   102.4 MB/s       591 MB left
[I]:  0:00:  38.7%       309 MB / 798 MB   100.3 MB/s       489 MB left
[I]:  0:00:  51.4%       411 MB / 798 MB    97.3 MB/s       387 MB left
[I]:  0:00:  64.2%       513 MB / 798 MB    93.7 MB/s       285 MB left
[I]:  0:00:  77.0%       615 MB / 798 MB    91.3 MB/s       183 MB left
[I]:  0:00:  89.7%       717 MB / 798 MB    88.9 MB/s        81 MB left
[I]: Disk copy completed successfully.
[I]: Performing luksClose

luksipc will have created a key file /root/initial_keyfile.bin that you can use to gain access to the newly created LUKS device:

joequad [~/luksipc-0.04]: cryptsetup luksOpen --key-file /root/initial_keyfile.bin /dev/loop0 mydisk

One thing that you should definitely do is add the key that you want to use for your device, maybe afterwards removing the initial keyfile:

joequad [~/luksipc-0.04]: cryptsetup luksAddKey --key-file /root/initial_keyfile.bin /dev/loop0
Enter new passphrase for key slot: 
Verify passphrase:
joequad [~/luksipc-0.04]: cryptsetup luksKillSlot /dev/loop0 0
Enter any remaining LUKS passphrase: 

If you really want to get the last out of your hard disk, you can then resize it again to occupy the whole LUKS device:

joequad [~/luksipc-0.04]: resize_reiserfs /dev/mapper/mydisk
resize_reiserfs 3.6.21 (2009 www.namesys.com)

ReiserFS report:
blocksize             4096
block count           204528 (204288)
free blocks           196310 (196070)
bitmap block count    7 (7)

Syncing..done


resize_reiserfs: Resizing finished successfully.

3.5 Testing the software

If you do not trust luksipc, you can first try it out using a loop device. For this, within the regression package there's a tool that can create pseudo-random test data incredibly fast. It's called prng and takes only one parameter: The amount of bytes that it should generate. So for generating a test device, let's use 800 Megs:

joequad [~/luksipc-0.04]: ./prng $((800*1024*1024)) >/dev/shm/disk

800 MB is 819200 kB. Remember that the LUKS header is 1028 kB, so only the first 818172 kB will be available on the final encrypted device. Let's first make a MD5 hash over the original data (only first 818172 kB obviously):

joequad [~/luksipc-0.04]: dd if=/dev/shm/disk bs=1k count=818172 | md5sum
818172+0 records in
818172+0 records out
837808128 bytes (838 MB) copied, 2.29805 s, 365 MB/s
1c603e62ea6da631bc0dc3a6ed2f61f0  -

Now let's luksipc the device, first create a loop device that maps to the file:

joequad [~/luksipc-0.04]: losetup /dev/loop0 /dev/shm/disk

Then luksipc it -- and abort right in the middle!

joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0
[...]
[I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes)
[I]: Performing dm-crypt status lookup
[I]: Performing luksFormat
[I]: Performing luksOpen
[I]: Size of cryptodisk is 837808128 bytes (798 MB + 1044480 bytes)
[I]: 1052672 bytes occupied by LUKS header (1028 kB + 0 bytes)
[I]: Starting copying of data...
[I]:  0:00:  13.1%       105 MB / 798 MB   140.1 MB/s       693 MB left
[I]:  0:00:  25.9%       207 MB / 798 MB   113.7 MB/s       591 MB left
^C[C]: Shutdown requested by user interrupt, please be patient...
[I]: Gracefully shutting down.

Then, resume from the resume file (resume.bin):

joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0 --resume resume.bin
WARNING! Resume LUKSification of /dev/loop0 requested.
   1. The resume file really belongs to the correct disk
   2. Power conditions are satisfied (i.e. your Laptop is not running off battery)

    /dev/loop0: 800 MB = 0.8 GB
    Resume file: resume.bin

Are all these conditions satisfied, then answer uppercase yes: YES
[I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes)
[I]: Starting copying of data...
[I]:  0:00:  50.3%       402 MB / 798 MB   120.7 MB/s       396 MB left
[I]:  0:00:  63.1%       504 MB / 798 MB    95.8 MB/s       294 MB left
[I]:  0:00:  75.8%       606 MB / 798 MB    88.3 MB/s       192 MB left
[I]:  0:00:  88.6%       708 MB / 798 MB    85.6 MB/s        90 MB left
[I]: Disk copy completed successfully.
[I]: Performing luksClose

Now let's open the device:

joequad [~/luksipc-0.04]: cryptsetup luksOpen --key-file /root/initial_keyfile.bin /dev/loop0 newdisk

And check if the MD5 sum matches:

joequad [~/luksipc-0.04]: cat /dev/mapper/newdisk | md5sum
1c603e62ea6da631bc0dc3a6ed2f61f0  -

Indeed, it does! Now let's use dd for this again to measure the crypt overhead just for fun:

joequad [~/luksipc-0.04]: dd if=/dev/mapper/newdisk bs=1k | md5sum
818172+0 records in
818172+0 records out
837808128 bytes (838 MB) copied, 9.18947 s, 91.2 MB/s
1c603e62ea6da631bc0dc3a6ed2f61f0  -

You can see that the performance has decreased a lot (91 MB/s compared to 365 MB/s -- still it is a lot more than most hard disks will be able to utilize).

4 Download

4.1 Development Snapshot

The luksipc HEAD is now on GitHub. I do not use GitHub to track internal development (this I do on my private repository to hide the countless really, really embarassing development bugs), but I will keep it in sync with my private repo once I do a release. So if you have patches, you can submit pull requests via GitHub and I'll try to incorporate them there (whenever I get around to it, which sometimes is embarassingly long to be honest).

4.2 Released Versions

Filename	Last Changed	Description
luksipc-0.04.tar.gz	2015-05-28	Greatly improved handling of disk I/O errors (graceful shutdown in more situation instead of simply bailing out) Separated resume file specification and actual request for resuming an aborted luksipc process (--resume vs. --resume-file) Unified exit code handling Possibilities to do fault injection in order to efficiently develop and test code to increase robustness Included whole test framework in release
luksipc-0.03.tar.gz	2015-05-25	Allow reLUKSification of devices (i.e. converting LUKS to LUKS) Checking of mount status of file systems Resume files now have additional sanity checks Fast CRC64-based PRNG generator for filling volumes with check data Major code cleanups and refactoring Major regression testing facilities (auto-aborting and resuming on large volumes and on loop devices) Help page of luksipc now looks more professional Partition backup file (128 MiB) is always generated at the start of a LUKSification process Option to deactivate safety checks via command line parameter (--no-seatbelt)
luksipc-0.02.tar.gz	2015-05-18	Fixed interpretation of return code of "cryptsetup status" which had changed with more recent cryptsetup versions to reflect the correct error if no such LUKS name was known. Thanks to Eric Murray and Christian Pulvermacher for reporting this issue. Forced chunk size to be 10 MiB instead of the default of 3 MiB. Thanks to John Morrissey for the bug report (under some weird circumstances, the LUKS header apparently can become a lot larger). Fixed a couple of warnings and used stricter compiler flags. Switched to -stc=c11 to be able to use static assertions. Improved error handling for wrong command line parameters (log level integer parsing). Improved error handling at cleanup (unsynced luksClose may fail at the first try because the device is still busy, sync() filesystems and try up to three times now) Display estimated remaining time until finish. Assert resume file can be written to disk by writing it at very start once and then seeking to its start. Added README file with detailed instructions. More helpful help page
luksipc-0.01.tar.gz	2011-10-15	Ability to convert devices to LUKS format without having to copy the contained data over. Licensed under the GNU General Public License v3