Sunday, November 10, 2013

Fixing Raspberry Pi filesystem corruption

While Raspberry Pi is a computer in its core, it is treated like an embedded device in most cases -- we can't expect a proper shutdown of Pi in most cases. This results in corruption of the / (root) file system (ext4) due to unclean unmount. 

In case you have been reinstalling Pi on to the sdcard every time this happens, you don't really have to reinstall. In most cases, mount the card via a card reader and run 

sudo fsck.ext4 /dev/xxxx  [xxxx the device file of the root partition]

(I assume linux by default). This should fix the / file system and the card will let your Pi boot now. If this doesn't help, try fixing the /boot partition:

sudo fsck.vfat  /dev/xxxx  [xxxx - the device file of the boot partition]

That said, you don't really have to do this. The clean and guaranteed protection against this problem could be provided by mounting the filesystem as read-only. Thus the filesystem is completely intact and will have no chance of corruption. In most cases, this should be fine (unless you have a reason to persist something on to the root filesystem). If you still need to write something to disk, write it to a different partition so it doesn't prevent Pi from booting if that goes corrupted.

How to mark the / and /boot partitions read-only:

You could do this from Pi itself, or on a different machine. Just edit /etc/fstab and add 'ro' in the flags as shown below:

/dev/mmcblk0p1  /boot       vfat    defaults,ro                0       2
/dev/mmcblk0p2  /           ext4    defaults,ro,noatime        0       1

Now reboot the Pi and it should boot with both these mount points in read-only mode. You could do a mount to verify, it should look something similar to this:

/dev/root on / type ext4 (ro,noatime,data=ordered)
devtmpfs on /dev type devtmpfs (rw,relatime,size=216108k,nr_inodes=54027,mode=755)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=44876k,mode=755)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=89740k)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
/dev/mmcblk0p1 on /boot type vfat (ro,relatime,fmask=0022, dmask=0022, codepage=cp437, iocharset=ascii,shortname=mixed, errors=remount-ro)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,relatime,size=89740k)

Note: Some services (like apache2) might fail to start, because they can't write to disk. You might have to check individual services to figure out what they are trying to write and resolve it appropriately. For eg., apache2 writes log files /var/log/apache2. It is best to disable logging. If there is not too much logging, you could map the log folder to /tmp (tmpfs -- as shown in mount).

as root:
mkdir /tmp/apache2 
chmod a+wx /tmp/apache2 
cd /var/log
rm -rf apache2
ln -s /tmp/apache2 apache2

There is no worry of file system corruption any more. You could switch off Pi any time. I have been using this setup (in my PowerStrip Project) for many months now, without issues.