ShredOS : HDD degaussing with style

Background

Have you ever had the need to properly erase HDD or SSD before disposing them ? Or selling them ? You do not want to have someone else playing around with your data right ? What we usually do is using a bootable Linux Live CD and use dd style command to write random data on the disk right ? But what if you have 10 disks in your machine ? What if you need a report to proof the world that wiping is a success ? I came across this challenge recently.

Degaussing ?

Degaussing is a process that entirely and securely erase the content of a hard drive or flash drive using a very strong magnet. This is a requirement in most enterprises when disposing electronic hardware. Either for trashing the hardware (in that case a physical shredding mechanism is a must) or if you are donating or selling your old hardware to a 3rd party.  In my case, we needed to send the hardware back once the leasing period is over. Since you have no idea of what the lesser will do with the hardware, you'd better take care. You do not want to destroy the drive since it won't fit into the return condition established by the leasing contract. However, you need to mandate an external company to perform the degaussing. Hopefully, there are some softwares in the market that help with this process.

One option is to perform a proper erase using the famous dd command :

user@debian:/$ sudo dd if=/dev/zero of=/dev/sde status=progress

3840728785408 bytes (3.8 TB, 3.5 TiB) copied, 50417 s, 76.2 MB/s

dd: writing to '/dev/sde': No space left on device

7501476529+0 record in
7501476529+0 record out

3840755982336 bytes (3.8 TB, 3.5 TiB) copied, 50417 s, 76.1 MB/s  

In a nutshell, the dd command is taking an input file (if=) and transfer it bit-by-bit to the output file (of=). Since in the Linux/Unix world everything is file, it also applies to disks.

The command above is actually copying 0x00 "zeroes" to the block device /dev/sde. At the end of the process, the entire /dev/sde disk will be filled with zeroes. As simple as that.

Ok but, I always been told data deleted with "rm" cannot be recovered ? Yes, practically lambda users are not able to recover data when flushed with "rm". But, "rm" is only removing the file reference in the disk's allocation table (FAT), the data remains in place. If you are good enough to recover the data address, you can still recover the data itself. So, a regular delete is definitely not powerful enough, we need to overwrite the disk with other data to make sure they are gone. Ideally, do it more than once. A 3 passes wipe is usually recommended.

This is probably a good start but it raises few concerns : 

- What if you have many disks to erase ? One at a time ? It will take a very long time

- Are you sure the wiping is efficient ? What about cached data ? What about SSDs ?

- Can we use multiple pass to make sure the erasure is efficient ?

- How can I create a report per disk serial numbers with all the matching physical details ?

- How can I proceed with a remote machine ?

The challenges

I recently had a mission : prepare 3 physical servers with 7 disks each (8 TB and 3 TB) for shipping back to a leasing company when the project was over.

• Challenge #1 : I'm physically 6 time zones away from the server and nobody can be in the datacenter easily until pickup and packing for shipping.
• Challenge #2 : I need to be sure all disks are wiped and no data will be usable
• Challenge #3 : We are ISO certified and need proper report for each disk wipe activities
• Challenge #4 : I need it to be completed asap

I can certainly have used the dd command but it will have taken days if not weeks to flush a single server, requires to do one disk at a time. I have virtual console access using IPMI and HTML5 terminal. But what about the report ? What about forgotten disks ? How to make sure I'm not missing anything ?

Introducing ShredOS

I googled a bit a after few mins I've found that somebody else had the same idea : create a bootable image where I can select the disks I want to wipe and quickly initiate the process with clear view on the progress.

ShredOS is a dedicated Linux Live CD (very light about 260 MB) who can be written on a USB stick or can be mounted remotely to easily wipe out/shred your data. This will definitely help overcoming those challenges !

The project's home page is located on GitHub here.

ShredOS author's nickname is PartialVolume and I managed to have a chat with him. Here's what he said about ShredOS : 

PartialVolume : My involvement with ShredOS began around 2018, stemming from a work-related need to securely wipe a single disk. I explored various software options, prioritizing those that were open-source and provided detailed reports. This search led me to nwipe, a derivative of dwipe, the foundational program in DBAN. Nwipe had undergone changes under Andrew Beverley after Blanco acquired the DBAN domain and modified it, including an advertisement for Blanco's products, but ceased updates around 2015. Beverley later handed nwipe's reins to Martijn Van Brummelen, a Debian maintainer, under whom I joined as a collaborator with full commit access.

While nwipe impressed me, it was plagued by stability issues like segmentation faults. From 2018 to 2020, I dedicated significant effort to rectifying these bugs, aiming to enhance its stability across various Linux distributions. During this period, I also contributed to the original ShredOS, developed by Nadenislamarre. However, as often happens in open-source projects, the original maintainer's time became constrained, prompting me to fork ShredOS to gain greater control and implement desired modifications. Thus, ShredOS.x86_64 emerged.

The initial version of ShredOS was compact, around 12MB, due to limited hardware driver support. In contrast, the latest iteration of ShredOS.x86_64 has expanded to approximately 228MB, reflecting a concerted effort to accommodate a broad range of hardware, including the most recent technologies.

ShredOS is developed using Buildroot, which isn't a traditional Linux distribution but rather a makefile that facilitates creating Linux systems, commonly for embedded systems. Unlike standard distributions designed for versatile hardware, Buildroot typically focuses on singular hardware applications, such as in coffee makers, microwave ovens, or routers. However, ShredOS employs Buildroot uniquely by incorporating a broader range of drivers, necessary to support various systems, exceeding the usual scope for an embedded system.

Furthermore, Buildroot lacks a dynamic repository in its final build, meaning it doesn't support typical package management commands like 'apt' for installing additional software post-deployment. Users desiring extra software must either raise an issue or build a custom version tailored to their needs. A significant benefit of this approach is the ability to compress the entire system to just over 200MB, allowing it to load into memory similarly to a live CD/DVD, offering a compact and efficient solution.

Let's see how ShredOS can help me with my challenges !

Challenge #1

I needed to find a system that can be used with my Out-Of-Band management platform. I have a couple of Supermicro servers and I need to be able to boot the system from IPMI and use the virtual console with HTML5

Supermicro BMC is offering this capability

When the image is mounted, you can chose in the server's boot menu what device to boot from. ShredOS will load integraly into the server's memory (boot take some time, around 3 minutesin my case).

Challenge #1 Achieved !

Challenge #2

ShredOS is very visual and immediately shows all devices that could be flushed in a clear and easily understandable UI.

This screen is the main ShredOS screen where you can select disk you want to place actions on. This way, you will barely forget to flush a disk !  Simply use the arrows and spacebar to make selection.

Challenge #2 Achieved !

Challenge #3

ShredOS is offering the capability to generate a PDF reports when wiping process is completed. This is very handy to have a report showing the drive details (serial number, model, brand, size, ....) and the result of the wiping. So, we have strong evidence in case of external audit. ISO 27K is very sensitive about data leakage and hardware disposal.

Here is a sample report for one of our destroyed HDDs :

Furthermore, ShredOS offers various wiping method options : 

You can also perform a verify action when the wiping is completed. This will ensure no blocked bit in the disk were present during the wiping and confirming your wipe is properly completed.

Challenge #3 Achieved !

Challenge #4

The speed is indeed key when wiping. Most of the time, your servers are having more than one disk and if you use the standard dd it can lead to 2 issues : you can forget disks on the way and other issue is the lack of parallel actions : you need to wait for the first one to be completed, then start the next one, etc ... it can take weeks to complete. ShredOS offers the ability to select all the disks, the wiping method and then you simply start the process and you can forget it for few days. When you come back, it's completed !

Here is ShredOS in action : 

When the wiping is completed, you just have to push <Enter> to start generating the PDFs. This part was initially a bit confusing, I was hoping to have PDF generated immediately. Apparently, this is not the case and you need to "terminate" the process.

PDF are generated in the / folder. Do not search for any subfolders, they are in root filesystem.

Challenge #4 Achieved !

Transferring the reports

After a few days,the process is at last completed, this is time to retrieve the logs/reports. But wait. How ? Only way in my case is to set an IP address on the ShredOS environment since I'm not local to the machine. Once you have found a suitable IP, you can type the following command in another virtual terminal (ALT + F2) : 

$ ip addr add 192.168.x.x/24 dev eth2

$ ifconfig eth2 up

$ ip link show eth2  

In this example, we assume the target machine is on the same subnet. If not, you need to configure a default gateway with a route add command.

Next, we are moving to the folder that contains our reports and we connect to our ftp server :

$ cd /

$ lftp ftp://192.168.x.x

$ lftp 192.168.x.x:> login anonymous

Password:

lftp anonymous@192.168.x.x:> !ls

-rx-r--r-- 1 root root    12928 Dec 13 14:44 nwipe_log_20231213-130352.txt

lftp anonymous@192.168.x.x:> put nwipe_log_20231213-130352.txt

12928 bytes transferred in 2 seconds (6.2KiB/s)

  

You can transfer all reports at once with the mput command

$ lftp anonymous@192.168.x.x:> mput *pdf

2251846 bytes transferred in 2 seconds (1.21MiB/s)

Total 7 files transferred

I have created a light FTP server on a toolbox VM running Windows 10. I'm using this software : Xlight FTP.

Of course, any FTP server out there will do the job.

If you need more details about lftp, you can check this reference : https://linuxconfig.org/lftp-tutorial-on-linux-with-examples.

Conclusion

SharedOS is well maintained and the community is very active, the lead developer/maintainer is Nick (aka PartialVolume), he is based in UK and is very friendly ! Kudos to him for assisting me using his product and commenting this post.

We have decided to use ShredOS in my enterprise in our common wiping procedures, it is handy and efficient.

I hope this help ;-)