Monthly Archives: April 2015

NetBSD on a Raspberry Pi

Update 2015-12-03:According to a reader comment (below), NetBSD for RPi has matured significantly since I wrote this post. That sounds great to me! But I have not tested yet.

As a long time Linux user I have always had some kind of curiosity about the BSDs, especially NetBSD and its minimalistic approach to system design. For a while I have been thinking that perhaps NetBSD is the perfect operating system for turning a Raspberry Pi into a server.

I have read anti-BSD rants like this “BSD, the truth“, and I have also appreciated pkgsrc for Mac OS X. I felt I needed got get my own opinion. It is easy to have a romantic idea about “Old Real UNIX”, but my limited experience with IRIX and Solaris is not that positive. And BSD is another beast.

For the Raspberry Pi (Version 1, Model B) it is supposed to be possible to run both (stable) NetBSD 6.1.5 and (beta) NetBSD 7.0. It seemed, after all, that the beta 7.0 was the way to go.

At first it was fine

I followed the official instructions and installed NetBSD 7.0. I (first) used the (800MB) rpi.img. I set up my user:

# useradd zo0ok
...
# mkdir /home
# mkdir /home/zo0ok
# chown zo0ok:users /home/zo0ok
# usermod -G wheel zo0ok

Then it was time to configure pkgsrc and start installing packages.

The Disk Problem
I did a quick check to see how much available space I have, before installing stuff. To my surprise:

# df -h
Filesystem         Size       Used      Avail %Cap Mounted on
/dev/ld0a          650M       623M      -5.4M 100% /
/dev/ld0e           56M        14M        42M  24% /boot
kernfs             1.0K       1.0K         0B 100% /kern
ptyfs              1.0K       1.0K         0B 100% /dev/pts
procfs             8.0K       8.0K         0B 100% /proc
tmpfs              112M       8.0K       112M   0% /var/shm

It seemed like the filesystem had not been (automatically) expanded as it should be according to the instructions above. So i followed the manual instructions to resize my root partition, with no success whatsover.

So I ran disklabel to see if NetBSD recognized my 8GB SD-card…

# /sbin/disklabel ld0
# /dev/rld0c:
type: SCSI
disk: STORAGE DEVICE
label: fictitious
flags: removable
bytes/sector: 512
sectors/track: 32
tracks/cylinder: 64
sectors/cylinder: 2048
cylinders: 862
total sectors: 1766560
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # microseconds
track-to-track seek: 0  # microseconds
drivedata: 0 

8 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:   1381536    385024     4.2BSD      0     0     0  # (Cyl.    188 -    862*)
 b:    262144    122880       swap                     # (Cyl.     60 -    187)
 c:   1766560         0     unused      0     0        # (Cyl.      0 -    862*)
 d:   1766560         0     unused      0     0        # (Cyl.      0 -    862*)
 e:    114688      8192      MSDOS                     # (Cyl.      4 -     59)

Clearly, NetBSD thought this SD-card was 900MB rather than 8GB, and this is why it failed to automatically resize it.

The sysinst install
I was anyway not very comfortable with getting a preinstalled/preconfigured 800MB system with swap and everything, so I formatted the 8GB SD card with my digital camera (just to be sure the partition table did not contain anything weird), downloaded (6MB) rpi_inst.img and wrote it to the SD card.

NetBSD installation started properly, and I was looking forward to install over SSH. According to the instructions I was supposed to start the DHCP somehow. But DHCP seemed on (the RPi got an IP) but SSH was off, so I installed using keyboard.

Quite immediately I was informed that NetBSD failed to recognise the “disk geometry” properly. I tried the SD card in Linux which reluctantly reported that it had 166 heads and 30 sectors per track (it sounds like nonsense). So I gave this information to the NetBSD sysinst program and now the SD card seemed to be 7.5GB.

Then followed a long and confused period of time when I tried to be smart enough to come up with any working partition scheme that NetBSD could accept. The right procedure was:

  1. Choose entire disk
  2. Confirm to delete the (required) 56MB dos partition
  3. Partition, pretending to be unaware of the need of a dos partition
  4. Magically, in the end, it added the dos partition

I am clearly stupid. There are no words for how confused I am about the a:, c: and e: partitions (that seems to reuse the DOS naming, but for other purposes), the empty space, the disk labels, the BSD partitions inside a (non existing) primary partition.

Anyway, just after I gave up and then gave it a final try I convinced sysinst to install. Then came a phase of choosing download paths, which clearly was non-trivial since I installed a Beta, and I am fine with that.

Installation went on. In the end came a nice menu where I could configure stuff. I liked it! (I wish I knew how to start it later). It managed to get my network settings from DHCP (except the GW), but it failed to configure and test the network itself (despite it had downloaded everything over the network just a few minutes ago). I configured a few other things, I restarted, network was working and I was happy… for a while.

I configured pkgsrc, and it seems ALL other systems where pkgsrc exist have been blessed with the pkgin tool, except NetBSD where you are supposed to do all the job yourself. Well, I added the PKG_PATH to the .shrc (of my user, not root) and enjoyed pkg_add.

(not) Compiling NodeJS
I want to install node.js on my NetBSD Raspberry Pi. It is not in pkgsrc (which is it for Mac OS X, but whatever) so I had to build it myself. I am used to building node.js and I was looking forward to fix all the broken dependencies. If I had ever gotten there.

I downloaded the source and started unpacking it… it is about 10000 files and 100MB of data. My SD card (a SanDisk Ultra, class 10) is not super fast, dd-ing the image to it earlier wrote at a speed of 3MB/s. The unpacking speed of node.js; roughly 1 file per second. I realised I need a (fast) USB-drive or a faster SD card, so I (literally) went out to town, bought a fast USB drive (did not find the SD card I wanted) and a few other things. When I came back more than 8000 files had been extracted and less than 2000 remained. I started reading about how to partition and format a USB drive for NetBSD, and at some point I inserted it in the Raspberry Pi. A little later I noticed my ssh sessions were dead, and the RPi had restarted. It turns out what reality was worse than the truth in “BSD, the truth”:

[…] the kernels of the BSDs are also very fault intolerant.

The best example of this is the issue with removing USBs. The problem appears when USBs are removed without unmounting them first. The result is a kernel panic. The astounding aspect of this is that this problem has been exhibited by all the major BSD variants Free, Open, Net and DragonflyBSD ever since USB support was implemented in them 5 to 6 years ago and has never ever been fixed. FreeBSD mailing lists even ban people who dare mention about it. In Linux, such things never and happen and bugs as serious as this gets fixed before a release is made.

Fact is, NetBSD 7.0 Beta for RPi, crashes, immediately, when I insert a USB drive.

This actually did not make me give up. I really restarted the system with the USB drive inserted, with the intention of treating my USB drive as a fixed disk and not inserting/removing it unless I shut the RPi down first. This was when I did give up: deleting the 16GB dos partition and creating a NetBSD filesystem was just too difficult for me. Admittedly, my patience was running out.

More on memory card performance
I found this very interesting article (linked to, by the Gentoo people, of course). Without going into details; clearly a Raspberry Pi with an SD card root filesystem needs a filesystem and block device implementation that works well with actual SD cards. This is not trivial and this means doing things very differently from rotating media.

I did the same unpacking of the node.js source on Raspbian (I installed Raspbian on exactly the same SD card as I used for NetBSD): 22 seconds (tar: 18s, sync 4s), compared to 3h for NetBSD.

Conclusion
In theory, NetBSD would be a beautiful fit for the Raspberry Pi. The ARMv6 is not supported by standard Debian. Raspbian comes with a little “too much” for my taste (it is not a real problem), and it does not have the feeling of “Debian stable”, but more some “inoffical Debian test” (sorry Raspbian people – I really appreciate your job!).

I have wondered why Noobs does not come with NetBSD… but I think I know now. And, sometimes I am surpised that Linux seems to work better than Mac OS X, perhaps now I know why.

My romantic idea that NetBSD would be perfect for the RPI was just plain wrong. Installing NetBSD today made me remember installing Slackware on a Compaq laptop in 1998.

Perhaps I will give Arch a try. Or put OpenWRT on the RPi.

Faking a good goto in JavaScript

There are cases where gotos are good (most possible uses of gotos are not good). I needed to write JavaScript functions (for running in NodeJS) where I wanted to call the callback function just once in the end (to make things as clear as possible). In C that would be (this is a simplified example):

void withgoto(int x, void(*callback)(int) ) {
  int r;

  if ( (r = test1(x)) )
    goto done;

  if ( (r = test2(x)) )
    goto done;
 
  if ( (r = test3(x)) )
    goto done;
 
  r = 0;
done:
  (*callback)(r);
}

I think that looks nice! I mean the way goto controls the flow, not the syntax for function pointers.

JavaScript: multiple callbacks
The most obvious way to me to write this in JavaScript was:

var with4callbacks = function(x, callback) {
  var r

  if ( r = test1(x) ) {
    callback(r)
    return
  }

  if ( r = test2(x) ) {
    callback(r)
    return
  }

  if ( r = test3(x) ) {
    callback(r)
    return
  }
  r = 0
  callback(r)
}

This works perfectly, of course. But it is not nice with callback in several places. It is annoying (bloated) to always write return after callback. And in other cases it can be a little unclear if callback is called zero times, or more than one time… which is basically catastrophic. What options are there?

JavaScript: abusing exceptions
My first idea was to abuse the throw/catch-construction:

var withexceptions = function(x, callback) {
  var r

  try {
    if ( r = test1(x) )
      throw null

    if ( r = test2(x) )
      throw null

    if ( r = test3(x) )
      throw null

    r = 0
  } catch(e) {
  }
  callback(r)
}

This works just perfectly. In a more real world case you would probably put some code in the catch block. Is it good style? Maybe not.

JavaScript: an internal function
With an internal (is it called so?) function, a return does the job:

var withinternalfunc = function(x, callback) {
  var r
  var f

  f = function() {
    if ( r = test1(x) )
      return

    if ( r = test2(x) )
      return

    if ( r = test3(x) )
      return

    r = 0
  }
  f()
  callback(r)
}

Well, this looks like JavaScript, but it is not super clear.

JavaScript: an external function
You can also do with an external function (risking that you need to pass plenty of parameters to it, but in my simple example that is not an issue):

var externalfunc = function(x) {
  var r
  if ( r = test1(x) )
    return r

  if ( r = test2(x) )
    return r

  if ( r = test3(x) )
    return r

  return 0
}

var withexternalfunc = function(x, callback) {
  callback(externalfunc(x))
}

Do you think the readability is improved compared to the goto code? I don’t think so.

JavaScript: Break out of Block
Finally (and I got help coming up with this one), it is possible to do:

var withbreakblock = function(x, callback) {
  var r
  var f

myblock:
  {
    if ( r = test1(x) )
      break myblock

    if ( r = test2(x) )
      break myblock

    if ( r = test3(x) )
      break myblock

    r = 0
  }
  callback(r)
}

Well, that is at close to the goto construction I come with JavaScript. Pretty nice!

JavaScript: Multiple if(done)
Using a done-variable and multiple if statements is also possible:

var with3ifs = function(x, callback) {
  var r
  var done = false

  if ( r = test1(x) )
    done = true

  if ( !done ) {
    if ( r = test2(x) )
      done = true
  }

  if ( !done ) {
    if ( r = test3(x) )
      done = true
  }

  if ( !done ) {
    r = 0
  } 
  callback(r)
}

Hardly pretty, I think. The longer the code gets (the more sequential ifs there is), the higher the penalty for the ifs will be.

Performance
Which one I choose may depend on performance, if the difference is big. They should all be fast, but:

  • It is quite unclear what the cost of throwing (an exception) is
  • The internal function, is it recompiled and what is the cost?

I measured performance as (millions of) calls to the function per second. The test functions are rather cheap, and x is an integer in this case.

I did three test runs:

  1. The fall through case (r=0) is relatively rare (~8%)
  2. The fall through case is very common (~92%)
  3. The fall through case is extremely common (>99.99%)

In real applications fallthrough rate may be the most common case, with no error input data found. The benchmark environment is:

  • Mac Book Air Core i5@1.4GHz
  • C Compiler: Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn)
  • C Flags: -O2
  • Node version: v0.10.35 (installed from pkgsrc.org, x86_64 version)

Performance was rather consistent over several runs (for 1000 000 calls each):

Fallthrough Rate           ~8%      ~92      >99.99%      
---------------------------------------------------------
     C: withgoto           66.7     76.9     83.3  Mops
NodeJS: with4callbacks     14.9     14.7     16.4  Mops
NodeJS: with exceptions     3.67     8.77    10.3  Mops
NodeJS: withinternalfunc    8.33     8.54     9.09 Mops
NodeJS: withexternalfunc   14.5     14.9     15.6  Mops
NodeJS: withbreakblock     14.9     15.4     17.5  Mops
NodeJS: with3ifs           15.2     15.6     16.9  Mops

The C code was row-by-row translated into the JavaScript code. The performance difference is between C/Clang and NodeJS, not thanks to the goto construction itself of course.

On Recursion
In JavaScript it is quite natural to do recursion when you deal with callbacks. So I decided to run the same benchmarks using recursion instead of a loop. Each recursion step involves three called ( function()->callback()->next()-> ). With this setup the maximum recursion depth was about 3×5300 (perhaps close to 16535?). That may sound much, but not enough to produce any benchmarks. Do I need to mention that C delivered 1000 000 recursive calls at exactly the same performance as the loop?

Conclusion
For real code 3.7 millions exceptions per second sounds pretty far fetched. Unless you are in a tight loop (which you probably are not, when you deal with callbacks), all solutions will perform well. However, the break out of a block is clearly the most elegant way and also the most efficient, second only to the real goto of course. I suspect the generally higher performance in the (very) high fallthrough case is because branch prediction gets more successful.

Any better ideas?