Friday, October 16, 2009

Recovering a deleted file from an Ext4 file system

Well, it seems that no matter how many times it accidentally happens and you tell yourself that you won't make that mistake again it seems that the mistake finds a way of repeating itself.

A while back I was going through some old copies of files from my home directory and getting rid of stuff that was taking up too much disk space. I find it easier to do things from the command-line so there I was. Two terminal windows open... comparing files... seeing what I had extra copies of... removing the old out-of-date copy... I had probably been doing this for a couple of hours and was just about done. I ran across a directory named Larry that I didn't need any more so I simply did a rm -rf Larry and pressed return. About 3 seconds later my stomach jumped up into my throat, sweat started pouring off my forehead, my chest got tight... this was all because I realized that I was in the wrong terminal window and it just so happened that the present work directory that I was in for that terminal window was /home and my user directory happened to be named Larry. I hit CTRL+C and assessed the damage.

Well, as it turned out I couldn't tell that anything important was missing but I knew that there were things that I wouldn't realize were gone until I needed them. So, I found a nice little utility called ext3grep that allowed me to track down the deleted files and restore them. It saved my life! It turned out there was a lot of files deleted and many of them were very important. At that moment I told myself that when using rm -rf I will be more careful.

As a secondary precaution I started doing hourly diff backups of my important directories in case something happened like this again. Well, that was so 2008! Since then I upgraded my OS and haven't gotten around to setting up the hourly diff backups. I am still careful with the rm -rf command but wouldn't you know it!

Today I was creating a new project in Eclipse. I don't like putting all of my projects in the root workspace folder so instead I organize them by type. In this case, my new project was an example EJB but instead of the project being created in workspace/mycoolprojects/ejb/myprojectname it was created in workspace/mycoolprojects/ejb. Not a big deal but now this project's contents was intermixed with all my other project folders. I started by refactoring and moving the project to the correct location but Eclipse complained that the project would overlap another project. So, instead of dealing with it I simply deleted the myprojectname project. When asked if I wanted to delete the contents on disk I simply said yes and ignored the warning saying it couldn't be undone. All seemed fine for about 15 seconds when that voice I hear on occasions called me an idiot and told me how, from Eclipse's perspective, the project was workspace/mycoolprojects/ejb and therefore it wouldn't be surprising if Eclipse just deleted the ejb directory and everything in it. A quick check revealed that, as usual, the voice was right.

Well, I will say that in this case there was only one project in the ejb folder that I was upset about and that was the one that I had been working on just 6 hours earlier. It had not been committed to SCM or backed-up (again, haven't yet setup hourly diff backups). So I then went on the search for how to restore the missing stuff. At the time I didn't remember the utility I used last time this happened so I did a lot of searching. ... probably spent two hours searching. All I could find is a lot of pessimistic answers saying that it was virtually impossible to restore accidentally deleted files on an ext3 or ext4 file system.

Well, I knew this just wasn't true because I knew I had done it before and it actually seemed relatively easy last time. Then finally I found it, ext3grep! It all came back to me! So, I installed the package from the repository and fired it up. Unfortunately it simply returned some strange error about the superblock type not matching some hex value. I then did a bit more searching (now that I knew what I was looking for) and found that ext3grep just won't work with an ext4 file system.

I went back to searching hoping that someone had ported or was working on port of ext3grep to work with ext4 file systems. I found some good hits and a couple of projects that looked promising. And then I stumbled on what appeared to be a solution. extundelete (http://extundelete.sourceforge.net/options.html)! It even had the option that I was looking for, --resotre-directory!

Well, unfortunately the --restore-directory option simply told me how many files that I had lost and basically that I wasn't going to get them back. I was devastated and was now approaching the "spent too much time" milestone. But I decided to give extundelete one more try but this time I used its --restore-file option. Sure enough, it worked! I got a file back! Luckily, I had a directory list of all the contents of my ejb directory which meant that all I had to do was pass the list in to extundelete and to make a long story end, I got all my files back!

Thank you extundelete author(s) and community!!