Category Archives: OpenWrt - Page 2

Build OpenWRT Toolchain on Mac OS X

A very quick guide to building the OpenWRT buildroot or toolchain on Mac OS X (10.10).

1. Install Xcode
Install Xcode from App Store (it is free).

2. Install pkgsrc
I have used fink, macports and homebrew, but now that I have tried pkgsrc I don’t think I will consider any of the others in a while. Install pkgsrc the standard way. Note: there is an x86_64 version for Mac OS X – it is probably what you want – just replace i386 with x86_64 in the download link.

Using pkgsrc, install these packages required by OpenWRT:

$ sudo pkgin install getopt coreutils gawk gtar findutils

3. Case sensitive filesystem
Your root filesystem on your Mac is probably case insensitive, and that is supposed to cause problems to building OpenWRT. Get yourself a USB disk or make a disk image an format it as case sensitive HFS+. If you do it from the command line you can avoid making it journaling:

$ diskutil eraseVolume hfsx OpenWRTdisk /dev/disk3s2

4. Building
This assumes you want the toolchain from that current stable build (14.07):

$ git clone git://git.openwrt.org/14.07/openwrt.git
$ cd openwrt
$ scripts/feeds update -a
$ scripts/feeds install -a
$ make menuconfig

In menuconfig I made just two changes: 1) setting my target platform, 2) asking toolchain to be built. Make all settings you want. I then ran:

$ make toolchain/install

You now find your toolchain is now in staging_dir 🙂
If you instead would have run just “make” the entire OpenWRT firmware would have been built.

Working OpenVPN configuration

I am posting my working OpenVPN server configuration, and client configuration for Linux, Android and iOS. First a little background.

I have an OpenWRT (14.07) router running OpenVPN server. This router has a public IP address and thanks to dyn.com/dns it can be resolved using a domain name (ROUTER.PUBLIC in all configuration examples below).

My router LAN address is 192.168.8.1, the LAN network is 192.168.8.*, and the OpenVPN network is 192.168.9.* (in this range OpenVPN-clients will be given an address to their vpn/dun-device). I run OpenVPN on TCP 1143.

What I want to achieve is
1) to access local services (like ownCloud and ssh) of computers on the LAN
2) to access internet as if I were at home, when I have an internet access that is somehow restricted

The Server
Essentially, this OpenWRT OpenVPN Setup Guide is very good. Follow it. I am not going to repeat everything, just post my working configurations.

root@breidablick:/etc/config# cat openvpn 

config openvpn 'myvpn'
	option enabled '1'
	option dev 'tun'
	option proto 'tcp'
	option status '/tmp/openvpn.clients'
	option log '/tmp/openvpn.log'
	option verb '3'
	option ca '/etc/openvpn/ca.crt'
	option cert '/etc/openvpn/my-server.crt'
	option key '/etc/openvpn/my-server.key'
	option server '192.168.9.0 255.255.255.0'
	option port '1143'
	option keepalive '10 120'
	option dh '/etc/openvpn/dh2048.pem'
	option push 'redirect-gateway def1'
	option push 'dhcp-option DNS 192.168.8.1'
	option push 'route 192.168.8.0 255.255.255.0'

It is a little unclear if the last three options really work for all clients. I also have:

root@breidablick:/etc/config# cat network 
.
.
.
config interface 'vpn0'
	option ifname 'tun0'
	option proto 'none'

and

root@breidablick:/etc/config# cat firewall 
.
.
.
config zone
	option name 'vpn'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option output 'ACCEPT'
	list network 'vpn0'
.
.
.
config forwarding
	option src 'lan'
	option dest 'vpn'

config forwarding
	option src 'vpn'
	option dest 'wan'
.
.
.
# may not be needed depending on your lan policys (2 next)
config rule
	option name 'Allow-lan-vpn'
	option src 'lan'
	option dest 'vpn'
	option target ACCEPT
	option family 'ipv4'

config rule
	option name 'Allow-vpn-lan'
	option src 'vpn'
	option dest 'lan'
	option target ACCEPT
	option family 'ipv4'
.
.
.
# may not be needed depending on your wan policy
config rule
	option name 'Allow-OpenVPN-from-Internet'
	option src 'wan'
	option proto 'tcp'
	option dest_port '1143'
	option target 'ACCEPT'
	option family 'ipv4'

iOS client
You need to install OpenVPN client for iOS from the app store. The client configuration is prepared on your computer, and synced with iOS using iTunes (brilliant or braindead?). This is my working configuration:

client
dev tun
ca ca.crt
cert iphone.crt
key iphone.key
remote ROUTER.PUBLIC 1143 tcp-client
route 0.0.0.0 0.0.0.0 vpn_gateway
dhcp-option DNS 192.168.8.1
redirect-gateway def1

This route and redirect-gateway configuration makes all traffic go via VPN. Omit those lines if you want direct internet access.

Android client
For Android, you also need to install the OpenVPN client from the Store. My client is the “OpenVPN for Android” by Arne Schwabe. This client has a GUI that allows you to configure everything (but you need to get the certificate files to your Android device somehow). You can watch the entire Generated Config in the GUI and mine looks like this (omitting GUI and Android-specific stuff, and the certificates):

ifconfig-nowarn
client
verb 4
connect-retry-max 5
connect-retry 5
resolv-retry 60
dev tun
remote ROUTER.PUBLIC 1143 tcp-client
route 0.0.0.0 0.0.0.0 vpn_gateway
dhcp-option DNS 192.168.8.1
remote-cert-tls server
management-query-proxy

Linux client
I also connect linux computers occationally. The configuration is:

client
remote ROUTER.PUBLIC 1194
ca ca.crt
cert linux.crt
key linux.key
dev tun
proto tcp
nobind
auth-nocache
script-security 2
persist-key
persist-tun
user nobody
group nogroup
verb 5
# redirect-gateway local def1
log log.txt

Here the redirect-gateway is commented away, so internet traffic is not going via VPN.

Certificates
The easy-rsa package and instructions in the OpenWRT guide above are excellent. You should have different certificates for different clients. One certificate can only be used for one connection at a time.

Better configuration?
I dont say this is the optimal or best way to configure OpenVPN – but it works for me. You may prefer UDP over TCP, and may reasons for running TCP are perhaps not valid for you. You may want different encryption or data compressions options, different logging options and so on.

Read OpenWRT reject log (with fwreject)

I configured the firewall on my OpenWRT router to reject outgoing traffic (LAN to WAN) by default, and then explicitely allow protocols and ports as needed. By configuring the firewall to log rejected packages I could identify what legitimate traffic was blocked, and open up the firewall. However, the default logging to the syslog is not particularly easy to read (neither using command line or a web browser). Also, the log is mostly full of other log lines, the log lives very short time (just a few minutes) to not waste memory on the router, and the log lines contain information not needed.

I understand there are powerful products to gather logs on central log servers and analyze them there. I did not want that, but rather a simple web interface directly on the router.

I asked for a simple tool on the OpenWRT forum, no result.

So, I wrote my own tool, fwreject, and published documentation and binaries on DropBox.

Upgrade OpenWRT and reinstalling packages

I just upgraded my OpenWRT router from Barrier Breaker RC2 to RC3. The upgrade guide is excellent, but it only mentions: “You do need to reinstall opkg-packages”… well, it sounds like there should be a smart way to do that.

Before upgrade:

# opkg list-installed > /etc/config/packages.installed

Two things to note: 1) This will take a few kb, that you must have available, and 2) since the file is in /etc/config it will be automatically restored after sysupgrade.

Now the sysupgrade itself (see the upgrade guide):

# sysupgrade -v /tmp/someimage-sysupgrade.bin

The system will restart, and you should be able to ssh into it, just as before the upgrade. Now reinstalling packages:

# opkg update
# opkg install $( cut -f 1 -d ' ' < /etc/config/packages.installed )

You will want to delete the new config files (or manually merge config files, or delete your old and use the new files). The new files have the "extension" -opkg, and can be found with

# cd /
# find | grep -e -opkg\$

That should be it.

Update 20151115
If you upgrade from 14.07 to 15.05 you will probably get this:

 * opkg_install_cmd: Cannot install package kmod-ipt-nathelper

I think this package has just been removed/replaced. Nothing to worry about, it seems.

Buying a router for OpenWRT

Update 2015-07-24: Avoid buying any device with 4Mb flash. Current Chaos Calmer 15.05 typically don’t fit in 4Mb.

Update 2015-05-28: I have been enjoying myself a while with improving the OpenWrt table of hardware. I am temporarily maintaining a table of hardware containing all devices. There are a few challenges that I do not mention below in my original article (things I have discovered later):

  • The definition of “supported” is a bit questionable: if it boots OpenWrt it can be regarded as supported even if things like USB and Wifi does not work.
  • Some devices listed as “supported” does not even boot.
  • Some devices listed as WiP may work better than “supported” ones.

Some specific suggestions:

  1. Avoid Broadcom: Supported devices with Broadcom wifi may never have working WiFi
  2. Buy a TP-Link WDR3600! Or WDR4300.
  3. The Linksys WRT1900AC is quickly improving. It is still not 100% supported, but in the long run this will be a very fine device for OpenWrt. Note that there is a cheaper WRT1200AC coming up. It shall also be well supported, but right now it is neither available nor supported.
  4. The Archer C5 and Archer C7 are also good OpenWrt devices. However, there are different versions here and you can go wrong with some bad luck. And C5/C7 users occasionally report problems. Not so with WDR3600/WDR4300.

Update 2015-01-10: about AC routers (see below).

Original Article
For a while I was thinking about buying a new wireless router for my home network, and I had already decided I wanted to run OpenWRT on it. I spent (wasted) quite some time reading the OpenWRT list of supported hardware, and searching for available routers. With this post, I hope to help you focusing on the essentials, to make a good decision quicker.

I presume, if you buy a new router to run OpenWRT, that you want to run the current stable version of OpenWRT (soon Barrier Breaker 14.07), and that you will want to be able to upgrade in the future.

I think it is a good idea to first decide the need for Flash and RAM, and then work from a much shorter list of hardware.

Flash
Most routers available have the following amounts of Flash (storage for kernel, files, configuration).
4Mb: is just enough, barely, to run OpenWRT.
8Mb: is enough for OpenWRT. You will be able to install packages, and even if future versions should be slightly larger, you should be fine.
16Mb: is more than enough for OpenWRT, but if you want to install many packages or put applications on it, then 16Mb gives you much more flexibility than 8Mb.

If you want to store files (backups, a web site, images, whatever), do that on a separate USB-storage (just make sure the router has USB ports). Too little Flash means you can not install packages, or you get errors when changing configurations. This is bad, but something you can handle in a controlled way.

RAM
Most routers available have the following amounts of RAM:
16Mb: is too little to run OpenWRT beyond version 10.03.1, except for special cases. Don’t buy!
32Mb: can run OpenWRT. But my new router is making use of more RAM than that (see below), running 14.07 RC2 and a few packages.
64Mb: should be enough for running several extra packages.
128Mb: is possibly going to be more than you need, but RAM never hurts, especially if you install extra packages or make heavy use of your router.

Too little RAM makes OpenWRT crash and restart, is my personal experience. Even if would kill processes (instead of crashing) in some cases, it is going to be brutal and disruptive – not the kind of service you want. Adding swap to a USB-storage is perhaps possible, but if you really need it you should probably have gotten another router, or you are using the router for the wrong task.

Flash / RAM conclusion
Chances are you will want 8/64Mb or more when buying a router to run OpenWRT. That will disqualify perhaps 80-90% of all supported routers, making your list shorter and your choice easier.

I really like getting the most out of simple hardware. You may very well have a situation where a 8/32Mb (or even a 4/32) router will be just perfect for you (or your parents or some other friends you are helping out). But if adding packages is important to you, I would not settle with 32Mb RAM.

Chipset / CPU
In the supported hardware table, there are three columns: Targets, Platform, CPU-speed. This is most likely not very relevant information to you. The CPU-speed will be of much less importance than the Flash/RAM when it comes to what you can do with the router. Of course higher CPU-speeds are better, and if you want to compare performance, have a look at the OpenSSL performance page (perhaps the RSA sign/verify columns are most useful for deciding CPU performance, since the numbers are not so big, and since there is probably no hardware support for RSA).

Network speed
Unless you have Internet connection faster than 100Mbit/s, chances are your router will be much faster than you connection, even for a cheap router. Some routers have 100Mbit-switch, some have GBit-switch – this may make a real world difference to you, if you often copy big files between your computers (or if your Internet is faster than 100Mbit, of course).

When it comes to Wireless you will find that most routers support B/G/N (2.4Ghz) and many also support A/N (5Ghz). Of course dual-band is nicer, but chances are it will not make any real difference to you whatsoever.

AC Routers 2015-01-10
The situation is getting better. You have the TP-Link Archer C5/C7 and the Linksys WRT1900AC to chose from. For the TP-Link, note that version 1 of C7 does not work (at least the AC does not work). And for the Linksys, it is still not supported by the official and stable OpenWRT version, but there are several options and builds, and since the WiFi sources were finally published the WRT1900AC situation is quickly getting better. I have no personal experience with any AC router, and perhaps they are not the safest choices at this time.

Status
There is a Stutus-column in the supported hardware table. You want it to say a stable version number: 7.09, 8.09, 10.03, 10.03.1, 12.09 or 14.07. Note that old 4/16Mb routers were supported, but are no longer supported with 12.09 and 14.07, so if it says 0.9 you should probably be careful. If it says “trunk” or “rXXXX” it means that it should work if you use the latest bleeding-edge builds: avoid this for production systems, and avoid this if you dont know how trunk works.

Version
The version column is nasty. Manufacturers release different versions of routers under the same name. The specification may vary a lot between versions, and quite often one is supported and the other is not. Have a look at the Netgear WNDR3700 which is very nice if you manage to get v2 or v4, while v3 does not even run OpenWRT.

Bricking and Fail Safe Mode
It can happen that a firmware installation/upgrade fails and the router is “bricked” (does not start). Different routers have different capabilities when it comes to recovering from a failed installation. Before buying a router, you might want to read about its recovery capabilities. I have never bricked a router with OpenWRT (or any other firmware), but you are more likely bricking it with OpenWRT, than just using OEM firmware.

Not getting paid to write this
I suggest, start looking at the TP-link routers. They are available in differnt price/performance segments, they have good price/performance ratio, they are not hard to find and TP-link seems to have a reasonable FOSS strategy/policy making their routers quickly supported by OpenWRT.

Years ago I liked ASUS routers (the WL-500g Premium v2, I bought several of that one for friends) and of course the Linksys WRT54GL. Buffalo seems to have good models, but I have problems finding the good ones where I live. Dlink is not one of my favourites, and when it comes to OpenWRT I find that the models that I can buy do not run OpenWRT, and the models that run OpenWRT are not available for sale here. And Netgear, I already mentioned the WNDR3700 mess above. Ubiquiti seems to be popular among OpenWRT people.

I bought a very reasonably priced TP-Link WDR4900 with 16Mb flash and 128Mb RAM, and it has a 800MHz PowerPC processor which I believe outperforms most ARMs and MIPS based routers available. Note that in China the WDR4900 is a completely different router.

Memory sitation on my WDR4900
On 14.07 RC2, I have installed OpenVPN and stunnel (currently no connections on neither of them) as well as uhttpd/Luci. This is my memory situation on my WDR4900. I dont know if the same amount of memory would be used (ignoring buffers and caches) if the same processes were running on an ARM or MIPS router with 32Mb RAM. But I think it is clear that at least 64Mb or RAM is a good idea for OpenWRT.

# top -n 1

Mem: 46420K used, 80108K free, 0K shrd, 1744K buff, 15872K cached
CPU:   0% usr   9% sys   0% nic  90% idle   0% io   0% irq   0% sirq
Load average: 0.06 0.10 0.07 1/41 31845
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
25367     1 root     S     5316   4%   0% /usr/sbin/openvpn --syslog openvpn(my
25524     1 nobody   S     2944   2%   0% stunnel /etc/stunnel/stunnel.conf
 2672     1 root     S     1740   1%   0% /usr/sbin/hostapd -P /var/run/wifi-ph
 2795     1 root     S     1736   1%   0% /usr/sbin/hostapd -P /var/run/wifi-ph
 2645     1 root     S     1544   1%   0% /usr/sbin/uhttpd -f -h /www -r ??????
 2340     1 root     S     1528   1%   0% /sbin/netifd
 3205     1 root     S     1516   1%   0% {dynamic_dns_upd} /bin/sh /usr/lib/dd
 2797     1 root     S     1460   1%   0% /usr/sbin/ntpd -n -p 0.openwrt.pool.n
31709 31690 root     S     1460   1%   0% -ash
 2450  2340 root     S     1456   1%   0% udhcpc -p /var/run/udhcpc-eth0.2.pid
31845 31709 root     R     1456   1%   0% top -n 1
31293  3205 root     S     1448   1%   0% sleep 3600
    1     0 root     S     1408   1%   0% /sbin/procd
31690 24443 root     S     1204   1%   0% /usr/sbin/dropbear -F -P /var/run/dro
 2366     1 root     S     1168   1%   0% /usr/sbin/odhcpd
24443     1 root     S     1136   1%   0% /usr/sbin/dropbear -F -P /var/run/dro
 2306     1 root     S     1028   1%   0% /sbin/logd -S 16
24148     1 nobody   S      976   1%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq
 1715     1 root     S      876   1%   0% /sbin/ubusd
 2582  2340 root     S      792   1%   0% odhcp6c -s /lib/netifd/dhcpv6.script

Using WRT54GL with OpenWRT 14.07 in 2014

The Linksys WRT54GL was a very successul product for its time, not the least because sources were available and people could make their own firmwares for it. There are today firmwares like Tomato, DD-WRT and OpenWRT that runs on WRT54GL. My interest is with OpenWRT (as it gives me a full Linux system, not just a firmware, a web gui and parameters to set). The last OpenWRT version that was good and fully supported on the WRT54GL was 10.03.1. But that firmware is four years old, and not so fresh in 2014. I like to keep my Linux systems updated.

I tried to build (from scratch/sources, using buildroot) a very minimal version of Barrier Breaker (OpenWRT 14.07) Release Candidate 1 for WRT54GL. I removed everything that I could possibly live without, and ended up with an image that used less than 3MB of flash. Still, it was impossible to get WiFi running for more than a few minutes, and the router got very slow. Most likely the 16MB of RAM is just not enough (that is the generally accepted explanation).

Update: Someone seems to have been successful running BB with WLAN.

However, without WiFi, the WRT54GL runs Barrier Breaker just fine. That rules out using it as a WiFi router, but it leaves other options open. It is still a functional switch, that is also a Linux-machine with dropbear (ssh) and it can run software (opkg-packages) like nginx, openvpn, stunnel, uhttpd (cgi capable web server), iptables, and many more.

Image Builder
I have used the Image Builder (also called Image Generator) from BB RC2 to generate my own OpenWRT firmware for my WRT54GL. The process can seem scary at first, but it is quite simple. You can choose exactly the packages you want and build an image with just those packages, making the most of your hardware.
Update 2014-10-09: BB 14.07 final works fine, just as RC2.

After downloading and extracting the Image Builder (on my x64 Ubuntu machine), I ran the following command:

make image PROFILE=Broadcom-None
           PACKAGES="-dnsmasq -ppp -ppp-mod-pppoe -libip6tc -kmod-ipv6
           -odhcp6c -ip6tables -odhcpd uhttpd netcat openvpn-easy-rsa
           openvpn-openssl openssl-util stunnel libwrap"

Note, there should be no line breaks when you run this command.

  • Available profiles are found in target/linux/brcm47xx, and gives a starting point for selecting packages
  • Packages can be added using the PACKAGE option to make (ex stunnel)
  • Packages can be removed by using – in front of their name (ex odhcp6c)

I read the massive output which indicated that some packages (netcat, stunnel) failed to install. Also, if there are missing dependencies (libwrap) you will see it in the output.

Not all packages that are available are also included in the Image Builder download, but just download the packages you want and put them in the package folder in the image builder.

Rerun make image until you see no dependency errors, failed packages or other problems.

When make image has finished, have a look in your bin/brcm47xx folder, and you will find your new firmware. The firmware I generated above was less than 3.5Mb, leaving several 100kb for configuration and more packages. Flashing the router the normal way and logging in to the system I find that memory usage is about 10Mb and available disk is about 500kb (the openssl-util alone is 477kb on the filesystem and its opkg is 182kb).

As a benchmark I decided to use the easy-rsa package and build certificates. Particularly the build-dh takes very long time: on this machine 30min. However, on my new TP-Link WDR4900 it took 60 minutes, so obviously this was a silly unpredictable benchmark.

Conclusion
With some fantasy, curiosity and enthusiasm, you can turn your old WRT54GL into a useful component of your home or work network. It can provide some, or a few, useful services. Often it is wise to separate different functions to different hardware, because it makes your network more stable. And not the least, this is a good way to experiment with OpenWRT, without risk of breaking your production WiFi and broadband router.

TP-Link WDR4900, OpenWRT and OpenVPN

My old Linksys WRT54GL has worked fine for long. I want OpenWRT and I want some VPN solution, and the last OpenWRT version that runs properly on WRT54GL is 10.03.1, which is 4 years old by now. That means old versions of all software, and possible security issues.

I gave OpenWRT Barrier Breaker RC1 a chance on my WRT54GL. I even rebuilt OpenWRT from scratch (buildroot and all) to produce a minimal image (no IPv6, among other things). It was well below 3MB big, so I had a whole MB of flash drive available, but the Wireless network just won’t run (for more than a few minutes, then the router almost halts completely). 16Mb of RAM is simply too little.

My thoughts about a new router were ultimately inspired by the announcement of the Linksys WRT1900AC. However, the WRT1900AC was supposed to be FOSS-friendly but now several months later the driver sources required by OpenWRT has still not been released. It seems Belkin has done a really ugly thing. So no thanks Linksys (owned by Belkin).

OpenWRT works on many different routers, but many of them are not very powerful (8Mb flash/32Mb RAM is common). Since I have been struggling with my WRT54GL for years I wanted to get a powerful router this time, that gives me flexibility. As soon as you want something more powerful than 8/32Mb, that is also supported by OpenWRT, you do not have so many choices. And many routers that work are hard to actually buy. To confuse things further, many routers come in different versions with different support, (Netgear WNDR3700 being perhaps the worst example).

TP Link WDR4900
I ended up with a router that is classified as Work in Progress by OpenWRT: the TP-Link WDR4900 (European version 1.3). It has 16Mb of flash and 128Mb or RAM, and a 800MHz PowerPC CPU – who could resist such amazing hardware?

As it was listed as Work in Progress I expected problems, but none so far! I have installed OpenWRT Barrier Breaker RC2 (generic, not the p1020), and I use it as most people would. Both 2.4Ghz and 5GHz WiFi works fine (although, I have read that 40Mhz band does not work, so I have not tried that). The router has 10MB available flash (compared to just 2MB, for a similar 8MB router) and 96Mb available RAM.

OpenVPN
And I have installed OpenVPN. It would be very nice with a VPN solution that works out of the box with iOS and Android, but that is PPTP (which I don’t want) or L2TP/IPSEC-solutions, which all seem very complicated and without really good support anyway. OpenVPN requires a separate App for iOS and Android, but whatever.

I followed the OpenWRT OpenVPN instructions. At first they seem complicated, but it was actually easy and successful. My only issue was that the router would not route VPN traffic to the internet. Turns out a Firewall rule was missing in the instructions (at least for me):

#/etc/config/firewall
config forwarding
	option src 'vpn'
	option dest 'wan'

And, the command “build-dh” which is supposed to take long time, took just over 60 minutes on this router.

I found that each client should have their own certificates (otherwise they get the same IP, which is bad).

Next practical question was what port and protocol should OpenVPN run? For me it is more important to always be able to connect, than to get most optimal performance. So I thought TCP/443 should be a safe choice (no one blocks that). Well, turns out there are firewalls that see the difference between HTTPS/SSL and OpenVPN/SSL and block the later. Some options could possibly be UDP/123 or UDP/53. OpenVPN over ICMP would be VERY nice.

I tried to configure a port forward on the OpenWRT router, to itself, to make OpenVPN operate on several ports. This seems to work very fine with TCP but not with UDP. Don’t really know why but I might find out some day. Alternatively I could run several instances of OpenVPN on different port/protocol combinations, but that would require more configuration (and resources) than just a port forward.

OpenWRT
I first installed BB RC1 (the full 16MB image) and later upgraded to BB RC2 (the smaller, upgrade image). Obviously, the configurations are kept, but all packages are lost – a good thing to know 😉 Well, all I need to do is exporting a list of all installed packages, before upgrade, and then run that list after upgrade. Guess I am not the first.

Also, release builds come with Luci (the web GUI) included, but trunk builds do not. I thought RC meant release candidate, implying that Luci would be included, but not so. However, very easy to find information on how to install Luci.

Benchmarking WDR4900
Benchmarks are tricky. I have a little C-program that is mostly CPU-dependent, not using floating point numbers and that outputs computation time (using clock_gettime so it should be accurate) for the problem given to it. I ran this program for problems small enough to run on my old WRT54GL, and compared the results of WRT54GL (200MHz Mips), WDR4900 (800MHz PowerPC) and an Apple PowerBook G4 (866MHz PowerPC) running Debian. In all cases I compliled with GCC and -Os (optimize for small binary). The 866MHz G4 is marginally faster than the 800MHz P1014. The PowerPCs are about 6 times faster than the old 200MHz Mips of the WRT54GL.

I have also found that the WDR4900 is about twice as fast as a Raspberry Pi for the above application.

Compile program for OpenWRT

I felt a strong desire to compile something, anything, for my WRT54GL running OpenWrt. As is often the case, in the end it is very simple, but finding the simple solution is not very easy. Ironically, the best instructions were not on the OpenWRT site (but I downloaded the Toolchain from openwrt.org).

The program I wanted to compile was a pure C program that I have written myself. Almost clean C89/ANSI code, with a few C99/Posix dependencies. No autoconfigure, no makefile.

Solution/Conclusion
I first downloaded the Toolchain for OpenWRT 12.09, brcm47xx from OpenWRT (despite I run OpenWRT 10.03.1 brcm-2.4). I unpacked it to ~/openwrt/.

Second, two environment variables:

$ PATH=$PWD/OpenWrt-Toolchain-brcm47xx-for-mipsel-gcc-4.6-linaro_uClibc-0.9.33.2/toolchain-mipsel_gcc-4.6-linaro_uClibc-0.9.33.2/bin:$PATH

$ STAGING_DIR=$PWD/OpenWrt-Toolchain-brcm47xx-for-mipsel-gcc-4.6-linaro_uClibc-0.9.33.2/toolchain-mipsel_gcc-4.6-linaro_uClibc-0.9.33.2
$ export STAGING_DIR

Third, compile

$ mipsel-openwrt-linux-uclibc-gcc program.c

Finally, send it to openwrt and test

$ scp a.out root@192.168.0.1:.
$ ssh -l root 192.168.0.1

# ./a.out

That was all I had to do, really. Now some comments on this.

Choosing a compiler/toolchain
OpenWRT gives you three download options, that all seems to be relavant if you want to compile stuff:

The Toolchain is a lot smaller than the others. At least for the brcm platform the SDK is named “for-linux-i486” while the Toolchain is named “for-mipsel”. That confused me because I was not sure the Toolchain was actually a cross compiler.

To confuse things more, as soon as you start reading about how to compile stuff for OpenWRT, everyone talks about the “Buildroot”. No cheating with downloading SDK or Toolchain and get an already compiled compiler! Real men compile their compilers themselves? No disrespect here… the buildroot is fantastic technology, and perhaps I will have my own one day, but right now a C compiler is all I want.

Also, it seems the Toolchain that comes with 10.03.1/brcm-2.4 (my platform) is broken (ok, I am not smart enought to make it work, cc1 complains about unknown parameter). However, in my case the toolchain for 12.09/brcm47xx also worked. I wasted much time with the 10.03.1/brcm-2.4 Toolchain. If my simple steps above dont work quite quickly for you, and you get weird errors from the compiler, download another Toolchain (perhaps even for another platform, and perhaps 12.09/brcm47xx that works for me) just to see if you can get that to work (you may not be able to use the compiled binary, but you can at least confirm that you can generate one).

I suppose it is preferred to use the Toolchain for the OpenWRT platform/version you are actually targeting, but newer toolchains for compatible platforms can also work. Perhaps the newest toolchain is always preferable.

Linking and optimizing
Compiler flags affect the size of your binary, and for OpenWRT you typically want a small binary. I guess the “-Os -s” options to the compiler is the best you can do. The binary itself is static. No dynamic linking. I think that means it only communicates with the rest of the world via Linux system calls, and as long as those have not changed in a non-compatible-way, you can compile your program with a different toolchain than was used to build OpenWRT image and packages (of course, the binary format must be good too).

The C library
What about standard library compliance? My Toolchain came with “uClibc” (although others should be possible). My program uses two things that are not C89/ANSI-C compliant (I know because Visual Studio complained):

1) snprintf(): Worked fine, I believe this is C99 standard.

Update 20140814: The CLOCK_MONOTONIC problem is fixed with BB 14.07
2) clock_gettime(): Compiled without errors or warnings, but did absolutely nothing. The input timespec struct was not modified at all when the function was called. This should be a POSIX function (not Linux specific). I guess it is either not implemented in uClibc, or I should use another clockid_t (I used CLOCK_MONOTONIC), or there is a system call behind it that does not work properly when toolchain is different from the one that build the kernel.

So, generally the compiler and uclibc worked very nicely, but some testing is required.

Build machine
I run the toolchain/cross compiler on a x64 machine running Ubuntu. The toolchain itself seems to be statically linked (ldd tells me), and built for x86 (readelf tells me). So most x86/x64 Linux machines should work just fine, and if you are on BSD you probably know how to run Linux binaries.

Toolchain limitations
For my purposes, the Toolchain was just what I needed. I do not know how to build ipk-package files, and I do not know how to build a complete OpenWRT image. Perhaps the Toolchain is not the right tool for those purposes.

QEMU
If you install QEMU you can test your OpenWRT binary on your x86/x64 Linux machine:

qemu-mipsel -L OpenWrt-Toolchain-brcm47xx-for-mipsel-gcc-4.6-linaro_uClibc-0.9.33.2/toolchain-mipsel_gcc-4.6-linaro_uClibc-0.9.33.2/ a.out

The same way, I guess it should be quite possible to run the Toolchain on a non x86-machine as well. I will write a few lines when I have compiled my OpenWRT/MIPS binaries on my QNAP/ARM running the x86 compiler/toolchain with the QEMU.

IPv6 access with 6to4 OpenWRT Backfire

A little while ago I shared some information on getting IPv6 at home, when all you have is a dynamic (but real/public) IP-address and a good old WRT54GL router with OpenWRT Backfire (brcm-2.4 edition).

I have now stabilized my configuration and I will share some details. You are presumed to

  • be comfortable with editing configuration files manually (using vi, or some other editor in OpenWRT)
  • use OpenWRT Backfire 10.03.1 on your router (which can probably be any router capable of running OpenWRT)
  • have some understanding of what you are about to do and why
  • have a public (but not necessarily static) IPv4 address

If you mess up your firewall rules, worst case you can not log in to your router or you expose your entire network to the world. Proceed at your own risk.

At some point you will start trying your IPv6 connectivity. I suggest using test-ipv6.com, ipv6-test.com and ipv6.google.com.

A good start is the OpenWRT IPv6 Article (it contains much information, but it is not very well structured). First follow the 6to4, 6rd instructions (down to the firewall rule, which is probably fine, but I dont need it).

You also need to enable IPv6 forwarding (which is described in the 6in4 section).
edit /etc/sysctl.conf:

net.ipv6.conf.all.forwarding=1

Then

/etc/init.d/sysctl restart

Now you should start testing what works and what does not. Run ifconfig both on the router and on your local machine (ipconfig on Wintendo). If you have a reasonably new OS, you should now at least have an IPv6-address, even if you cant ping6 or connect to anything.

Note: Your 6to4 IP should start with 2002: (both router and clients). Addresses starting with fe80: are private addresses and completely useless.

Firewall
You probably have a Masquerading firewall configured for IPv4, but if you bother with IPv6 at all you probably don’t want to do Masquerade for IPv6 (dont know if it is possible).

I wanted my IPv4 to work just normally. And I wanted all my LAN-computers to be real IPv6 members accessible from the IPv6 internet (and protected by firewall, as needed, of course). That means, all replies from Internet should be fine, but incoming traffic from Internet should be restricted. The most natural thing would be to use connection tracking, but I encountered problems.

This is what my firewall configuration looks like now:
/etc/config/firewall

config 'defaults'
	option 'input' 'DROP'
	option 'output' 'ACCEPT'
	option 'forward' 'DROP'
	option 'syn_flood' '1'
	option 'drop_invalid' '1'
	option 'disable_ipv6' '0'

config 'zone'
	option 'name' 'lan'
	option 'network' 'lan'
	option 'input' 'ACCEPT'
	option 'output' 'ACCEPT'
	option 'forward' 'REJECT'
	option 'mtu_fix' '1'

config 'zone'
	option 'name' 'wan'
	option 'network' 'wan'
	option 'family' 'ipv4'
	option 'masq' '1'
	option 'output' 'ACCEPT'
	option 'forward' 'DROP'
	option 'input' 'DROP'

config 'zone'
	option 'name' 'wan6'
	option 'network' '6rd'
	option 'family' 'ipv6'
#	option 'conntrack' '1' 
	option 'output' 'ACCEPT'
	option 'forward' 'DROP'
	option 'input' 'DROP'

config 'forwarding'
	option 'src' 'lan'
	option 'dest' 'wan'
	option 'family' 'ipv4'

config 'forwarding'
	option 'src' 'lan'
	option 'dest' 'wan6'
	option 'family' 'ipv6'

config 'include'
	option 'path' '/etc/firewall.user'

config 'rule'
	option 'target' 'ACCEPT'
	option '_name' 'IPv6 WRT54GL ICMP'
	option 'src' 'wan6'
	option 'proto' 'icmp'
	option 'family' 'ipv6'

config 'rule'
	option '_name' 'IPv6: Forward ICMP'
	option 'target' 'ACCEPT'
	option 'family' 'ipv6'
	option 'src' 'wan6'
	option 'dest' 'lan'
	option 'proto' 'icmp'

config 'rule'
	option '_name' 'IPv6: WRT54GL "reply" to 1024+'
	option 'target' 'ACCEPT'
	option 'family' 'ipv6'
	option 'src' 'wan6'
	option 'dest_port' '1024-65535'
	option 'proto' 'tcp'

config 'rule'
	option '_name' 'IPv6: Forward "reply" to 1024+'
	option 'target' 'ACCEPT'
	option 'family' 'ipv6'
	option 'src' 'wan6'
	option 'dest' 'lan'
	option 'dest_port' '1024-65535'
	option 'proto' 'tcp'

Some comments on this:

  • I think it makes sense to think about IPv6 Internet as a separate wan6, not as part of wan
  • Incoming traffic is forwarded, as long as it is to unpriviliged ports (1024+)
  • ICMP works between everyone
  • The firewall.user script contains nothing of interest for IPv6
  • Masquerade is activated for wan, but conntrack (or masquerade) does not work for wan6
  • I have not needed a rule to allow INPUT protocol 41 to the router itself (the 6to4 traffic over IPv4), perhaps it gets allowed as ESTABLISHED,RELATED

Bridging and Connection tracking problems
I believe my configuration is working properly. But something is not completely right. Loading the firewall…

root@OpenWrt:~# /etc/init.d/firewall restart
Loading defaults
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
ip6tables: No chain/target/match by that name.
Loading synflood protection
Adding custom chains
Loading zones
Loading forwardings
Loading redirects
Loading rules
Loading includes
Loading interfaces
ip6tables: No chain/target/match by that name.

In the end of OpenWRT IPv6 documentation:
Note: firewall v1 (e.g. still in Backfire 10.03.1-rc4 and up to r25353) has no default rules at all and ip6tables configuration needs to be done from scratch. Insert the rules below to make the packet filter function properly.

ip6tables -A FORWARD -i br-lan -j ACCEPT
ip6tables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
ip6tables -A FORWARD -j REJECT

Well, I should be on a more recent version (10.03.1) but the second line (with conntrack) gives the No chain/target/match by that name error. I don’t know why, and I don’t know how to fix.

Also, in the same document, under the heading Directly forward ISP’s NDP proxy address to LAN there are instructions for “firewalling on ipv6 even for bridged interfaces”. I believe that this is what I want to do, but the ebtables package/module seems to not be available for WRT54GL/Backfire 10.03.1/brcm-2.4, and it also seems to be known to cause performance problems.

Either:

  1. I messed something up when installing/configuring OpenWRT, and now I dont know how to fix it
  2. Something IPv6-related that I want to do is not fully supported on Backfire/brcm-2.4
  3. I am just trying to do the wrong thing, without understanding it

Other config files
In case it is helpful to anyone (and possibly myself in the future) I post a few of my configuration files.

/etc/sysctl.conf (there are more lines)

net.ipv6.conf.default.forwarding=1
net.ipv6.conf.all.forwarding=1

/etc/config/network (all file)

config 'switch' 'eth0'
	option 'enable' '1'

config 'switch_vlan' 'eth0_0'
	option 'device' 'eth0'
	option 'vlan' '0'
	option 'ports' '0 1 2 3 5'

config 'switch_vlan' 'eth0_1'
	option 'device' 'eth0'
	option 'vlan' '1'
	option 'ports' '4 5'

config 'interface' 'loopback'
	option 'ifname' 'lo'
	option 'proto' 'static'
	option 'ipaddr' '127.0.0.1'
	option 'netmask' '255.0.0.0'

config 'interface' 'lan'
	option 'type' 'bridge'
	option 'ifname' 'eth0.0'
	option 'proto' 'static'
	option 'netmask' '255.255.255.0'
	option 'ipaddr' '192.168.8.1'

config 'interface' 'wan'
	option 'ifname' 'eth0.1'
	option 'proto' 'dhcp'

config 'interface' '6rd'
	option 'proto' '6to4'
	option 'adv_subnet' '1'
	option 'adv_interface' 'lan'

/etc/config/radvd (all other configs have option ignore 1)

config interface
	option interface	'lan'
	option AdvSendAdvert	1
	option AdvManagedFlag	0
	option AdvOtherConfigFlag 0
	list client		''
	option ignore		0

And a few packages that you should probably have installed in OpenWRT:

6to4
firewall
ip
ip6tables
kmod-ip6tables
kmod-ipv6
radvd
libip6tc

DHCP & DNS
I have not enabled any (IPv6) DHCP – autoconfigure works fine for me. I have also not configured anything DNS related. My normal DNS resolves IPv6-only hosts ok (i.e. ipv6.google.com).

The day I want to allow incoming traffic to just a few of my local/LAN machines I will have to think about it.

Troubleshooting
The following tools/strategies have proven useful for troubleshooting:

  • ping6 between router and local/LAN machines
  • ping6 to internet hosts (ipv6.google.com)
  • Disable firewall or set policies to ACCEPT
  • Send/receive TCP traffic using ncat (the best nc/netcat) version for OpenWRT.
  • Test ping/ncat to/from an IPv6 host on a different network – I installed miredo on my Lubuntu netbook and let it connect to internet via my iPhone. That way it had no shortcut at all to my router and LAN.
  • I find myself having more success when I unplug my router to restart it; just restarting makes it not come up properly.

ncat
In case you are not familiar with ncat:

On the router (start listening):

root@OpenWrt:~# ncat -6 -l -p 9999

On your local computer (send a message):

$ echo 6-TEST | nc 2002:????:????:1::1 9999

On the router (should have got message):

root@OpenWrt:~# ncat -6 -l -p 9999
6-TEST

This is useful all directions, and on different ports, to confirm that your firewall works as you expect.

OpenWRT, IPv6, VPN and Replacing WRT54GL

After having relied on the router my Internet provider has supplied me with for years, I decided to take back control over my LAN. There were a few factors that inspired me to put some effort into this:

  1. The announcement of the WRT1900AC could open up the door for a new generation of routers
  2. IPv6 is getting somewhere and I want to be able to play with it to learn – so I want IPv6 at home
  3. I want a VPN solution at home, for different reasons, but one of them is to be able to access the Internet more safely when using public Wifis, and another is to access services when I am abroad
  4. My Wifi at home (supplied by my router from my Internet provider) was not 100% stable

Summary
I ended up keeping my WRT54GL, Installing OpenWRT 10.03.1 on it, and configuring it to provide VPN using PPTP and IPv6 using 6to4. I mostly followed documentation on the OpenWRT web page, but there were and are some issues.
Update 2014-04-12: Details about IPv4 using 6to4.

OpenWRT and WRT54GL
The WRT54GL is not supported by the most recent versions of OpenWRT, and the final release with good WRT54GL support was 10.03.1. Everything I write in this article applies to 10.03.1 (the brcm-2.4 edition).

OpenWRT
OpenWRT is very nice. It used to be more hardcore compared to other router firmware. With that I mean that Tomato (and DD-WRT) are 100% Web-GUI-configurable, while OpenWRT was more dependent on the command line. Most things can now be handled using the Web-GUI. But dont attempt to get advanced things (like VPN/PPTP and IPv6) working without using the command line. If you dont feel comfortable with that, just stay with Tomato (which is very nice). This is for OpenWRT 10.03.1 – perhaps more recent version are more configurable without the command line.

IPv6
For end user needs in 2014, IPv6 is not needed. However, if you anyway decide to play with it, IPv6 is in some ways a more simple protocol than IPv4: not needing a NAT (all your clients get to have real IPs) takes away a lot of things that just happens to be complicated with IPv4. However, although NAT was never meant to provide security it did as a side effect – with IPv6 you need to think about really firewalling incoming traffic to your network. Things like port forwarding and VPN (to access internal resources) suddenly are not needed.

There is also no need for DHCP (as the clients can autoconfigure themselves, and there are so many available addresses on each network, that a conflict is very unlikely). But your IPv6 router must advertise the network so the clients know it exist.

IPv6 – How to get it
How can you get IPv6 if your internet provider only provides IPv4? There are different transition mechanisms that you can use (that are designed just to give you IPv6 when you only have IPv4):

  • Teredo needs to be configured on each client computer seperately, but requires nothing of the network (except that the firewall does not block the traffic). Teredo is the easiest way to access IPv6, but it gives you no IPv6 network. In Debain you just #apt-get install miredo, that is all.
  • Tunnel Brokers provide you IPv6 in a VPN-fashion, much like there are VPN-providers who give you an IP-address in another country, or for anonymization purposes. You can set up the tunnel on a single client, or even better on your router. Your IPv4 router does not have to be your IPv6 router, so it is possible to configure for example a Raspberry Pi as an internal IPv6 router behind a (IPv4) NAT. A Tunner Broker is probably the best and most reliable solution if you have real IPv6 needs. I havn’t tried this, but I suggest start looking at SixXS (who provides free tunnels)
  • 6to4 is a very elegant idea. However, in practice it seems to be a not very popular transition mechanism (supposed to be fading). 6to4 requires that you have a real public IPv4 address (it may be dynamic). This is what I tried, and it works well for me.

Note, when you have IPv6 via a transition mechanism, your cliens may still prefer to use IPv4 when accessing services that are available on IPv4 (which might be all the services you can possibly want to use). There are services to test IPv6.

IPv6 – 6to4 – OpenWRT 10.03.1 on WRT54GL
I followed these instructions (the 6to4 part). I ended up with Firewall problems: the internal IPv6 worked, but I had problems accessing the rest of the world. I have not really stabilized my firewall scripts yet (they give some errors), but if you are not too paranoid, you can try to ACCEPT IPv6 FORWARD on lan (allowing IPv6 traffic from Internet to your local network) and ACCEPT IPv4 INPUT on lan (allowing all IPv4 traffic from Internet to get to your router).
Update 2014-04-12: Details about IPv4 using 6to4.

VPN/PPTP – OpenWRT 10.03.1 on WRT54GL
First, before you set up a PPTP server and use it, consider the security problems with MS-CHAP-v2! If you are aware of the risk and the threat, the advantages with VPN/PPTP are:

  • No need for certificates
  • Good client support

I followed these instructions. Again, I ended up with firewall problems, but found a solution. Try:

iptables -A input_rule -i ppp+ -j ACCEPT
iptables -A forwarding_rule -i ppp+ -j ACCEPT
iptables -A forwarding_rule -o ppp+ -j ACCEPT
iptables -A output_rule -o ppp+ -j ACCEPT

Now, the confusing part is the IP-addresses of your VPN. Each VPN-connection will get both a local and a remote IP-address. And none of these will probably be on your LAN. And this is ok! There is a “localip” option for pptpd which is no longer supported, and I wasted some time trying to assign IP-numbers. But the above firewall rules fixed everything if I just didnt think about about IP-numbers at all.

Best router for OpenWRT
So, what happened to my WRT1900AC plans? Well, the WRT1900AC is not available yet, and I decided to play with my old WRT54GL to see how far I could get with it, and it turned out that for now it does everything I want it to.

OpenWRT has a long list of supported routers (they even have a buyers guide). I did some research (only reading on the Internet) and it seems that TP-link provides fine routers for OpenWRT, for example WDR3600, WDR4300 (N750) or WR1043ND. TP-link also seems to have a good Open Source policy. The N750 is probably what I would buy today, if I were to replace that WRT54GL.

So, what about that WRT1900AC? With Dual core CPU, 256MB of RAM, ESata and USB 3.0 port it is clearly a very capable router. And with 128Mb of storage, much more potent firmwares (or OpenWRT versions) are possible. But is it a good idea? Perhaps the router should only be a router, and other services (fileserver, print server, backup, sql, webserver) are better handled by something else (why not a Raspberry Pi), to not ever disturb the critical router function? I like OpenWRT for having a normal editable filesystem (compared to Tomato or DD-WRT) and packages instead of everything in one image. But 128Mb? Perhaps it would make more sense to just use an SD-card and run Debian?

The WRT1900AC is expensive for being a router, and if it ends up providing no more value/function than the TP N750 mentioned above, what is the point? On the other hand it is not very much money – just expensive for a router. For now I will keep my WRT54GL, but the WRT1900AC is still tempting.