The Butterscotch Waterfall
Seeing “ZFS: i/o error - all block copies unavailable” at boot on your FreeBSD box?

I recently rebooted a FreeBSD box that uses a ZFS root filesystem configuration and was greeted by the error message “ZFS: i/o error - all block copies unavailable”. I was unable to boot the system or reach the loader prompt. After digging through the various forum and mailing list posts in an attempt to find a solution, I was starting to get the feeling my hard disk itself might have been the culprit. This was slightly worrying since the system in question is a non-redundant, single disk configuration.

Turns out my filesystem was almost full, at 95% capacity. So, if you’ve come across this boot error, despite making absolutely no changes to your ZFS or FreeBSD system, and you’re stuck with an installation you can’t boot, hopefully you’ve ended up here.

Here’s what to do:

  • Boot a live rescue system from either the regular installation discs or the memstick image. The 9.x series is your best bet since it has complete ZFS support without needing to faff about loading the necessary kernel modules at boot. You’ll still be able to import/mount an 8.x system’s pool.
  • Import and mount your ZFS pool with: zpool import -o mountpoint=/mnt -f poolname
  • Your filesystem(s) will now be available at /mnt
  • Free up some space by deleting things. I had around 6GB free when I hit the ZFS error and was able to successfully boot my system again once my disk had around 10-12GB free.
  • Reboot

In a situation like this, the first thing most people do is think back to anything that might have changed in their system. In my case, no changes had been made. However, I did copy several large video files on to my NFS-shared home filesystem before rebooting. Certainly not something I usually would have thought about.

So, if you’ve been hit by “ZFS: i/o error - all block copies unavailable” and you’re sure you haven’t done anything to cause it, have a think about what your filesystem usage might be. Hopefully this simple fix will do the job.

Long distance study with RMIT and Open Universities Australia

Over the coming weeks and months I will attempt to document my thoughts and progress in doing a ‘Bachelor of Technology (Computing Studies)’ through RMIT university via OUA.

Here’s the the basic story; I’m 25 years old, I live on the Mid North Coast of NSW, my education is high school level, and I work a boring retail job pretty much full time. Gaining some IT credentials, for me, is about finally having something to show for all the knowledge and skills I’ve taught myself over the years. It’s also mainly an attempt to avoid working ‘dead-end’ jobs forever.

I’m a Unix addict. Linux and BSD in particular. I have been since I was around 13, so by now I’ve got a good grasp of general system administration and coding. 

For somebody like me, the first few units of the Bachelor of Technology degree seem fairly mundane. A kind of necessary evil. Or a chance to brush up, depending on how you look at it. The juicy bits I’m interested in are units like C programming, C++, Java and web development frameworks I’m unfamiliar with, such as ASP.NET. Areas where I’m at an intermediate level, but need to fill some gaps to gain confidence.

I’m doing this blog in the hope that somebody in a similar situation might gain some insight in to what this sort of degree can be like for somebody that, unlike a lot of potential IT students, has a strong understanding of some more advanced subjects and theories.

I’m currently studying two courses per ‘Study Period’, so I’ll try to give you an idea of what it’s like for somebody with half a clue and a heap of after-work spare time, to juggle work and semi-fulltime study through OUA.

Check back over the next couple of weeks for some thoughts on the first couple of assignments in ‘Introduction to Information Technology’ and ‘Introduction to Programming’.