Category Archives: Linux

Raspberry PI performance and freezes

On a daily basis I use a Raspberry Pi v2 (4x900MHz) with Raspian as a work station and web server. It is connected to a big display, I edit multiple files and it runs multiple Node.js instances. These Node.js processes serve HTTP and access (both read and write) local files.

I experienced regular freezes. Things that could take 2-3 seconds were listing files in a directory, opening a file, saving a file and so on.

I moved my working directory from my (high performance) SD-card to a regular spinning USB hard drive. That completely solved the problem. I experience zero freezes now, compared to plenty before.

My usual experience with Linux is that the block caching layer is highly effective: things get synced to disk when there is time to do so. I dont know if Linux handles SD-cards fundamentally different from other hard drives (syncing more often) or if the SD card (or the Raspberry Pi SD card hardware) is just slower.

So, for making real use of a Raspberry Pi I would clearly recommend a harddrive.

Best Raspberry Pi Server Linux Distribution

Since I got my first Raspberry Pi have have wondered: how to turn it into a proper server. Options that I have not been entirely satisfied with:

  • Arch Linux: probably a great option if you know Arch… I have been too lazy to learn.
  • Gentoo Linux: is Gentoo still relevant? Building everything on the RPi sounds very painful (slow)
  • OpenWrt: nice, but slightly too minimal for a server
  • Raspbian: nice, but a little bit too big standard installation (perhaps it does not really matter, but every apt-get upgrade takes longer time, and so on)
  • NetBSD: such a disappointment ๐Ÿ™

I now found, and tested, Raspbian Unattended Netinstaller. For me, this is the shit.

If is really this simple:

  1. Format your SD-card with FAT32 (just as usual)
  2. Unpack (unzip) the raspbian-ua-netinst on your SD-card
  3. Connect the SD-card, ethernet and power to your Raspberry Pi
  4. Wait (about 25 minutes, they say, that was ok with me)
  5. SSH into your new lean Raspbian system (root/raspbian).
  6. Read under “first boot” what to do next

Clearly, you need a properly configured network (DHCP, allow fetching of packages, and you need to know what IP address it got).

The entire experience is much enhanced if you connect to your Raspberry Pi with a serial cable during the entire procedure. Jokes aside, I used a serial with my first installation. Second time when I felt confident with the process I did not bother with the serial cable.

First boot quick guide

#dpkg-reconfigure locales
#dpkg-reconfigure tzdata

/boot/config.txt: add the line
gpu_mem=16

Upgrade to jessie
For some reason, Raspbian installation is still based on wheezy, not jessie (you don’t get the latest version of Debian). I suggest, upgrade immediately:

/etc/apt/sources.list (replace wheezy with jessie, two places)

# apt-get update
# apt-get dist-upgrade

It is almost as fast as the installation itself ๐Ÿ˜‰

Conclusion
I think, for the Raspberry Pi V1, Raspbian installed this way is the best server system you can have (perhaps Arch is better if you know it). For a Raspberry Pi V2, perhaps standard Debian is better (I have never used an RPi v2). Everthing I have written applies perfectly to the RPi v2 as well.

Getting your locale right

So, you get this annoying error (in Debian, some Ubuntu, or perhaps any other Linux or even Unix system):

locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory

Perhaps you get it when running apt-get?
You Google for answers and you find nothing useful and/or just contradictory information?
You are connecting over SSH?
You are connecting from another system (like Mac OS X)?
Read on…
In it simplest incarnation, the problem looks like this.

$ man some-none-existing-program
man: can't set the locale; make sure $LC_* and $LANG are correct
No manual entry for some-none-existing-program

$ locale
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
LANG=en_US.UTF-8
LANGUAGE=en_GB:en
LC_CTYPE=UTF-8
LC_NUMERIC=sv_SE.UTF-8
LC_TIME=sv_SE.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=sv_SE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=sv_SE.UTF-8
LC_NAME=sv_SE.UTF-8
LC_ADDRESS=sv_SE.UTF-8
LC_TELEPHONE=sv_SE.UTF-8
LC_MEASUREMENT=sv_SE.UTF-8
LC_IDENTIFICATION=sv_SE.UTF-8
LC_ALL=

What happens here, in the second case is that “UTF-8” is not a valid LC_CTYPE (in Debian) and since LC_ALL is not set it can not fall back properly. That is all. Why is LC_CTYPE invalid? Perhaps because you have used ssh from Mac OS X (or something else):

mac $ locale
LANG=
LC_COLLATE="C"
LC_CTYPE="UTF-8"
LC_MESSAGES="C"
LC_MONETARY="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_ALL=

mac $ ssh user@debianhost

The point is, here LC_CTYPE=UTF-8, and it is valid in OS X. But it is not valid in Debian. That is all there is to it. Try:

# This will fix the problem
mac $ LC_CTYPE=en_US.UTF-8 ssh user@debianhost

# This will produce the problem even from a Debian machine
debianclient $ LC_CTYPE=UTF-8 ssh user@debianhost

# This will (kind of) eleminate the problem when already logged in
$ LC_CTYPE=en_US.UTF-8 locale
-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8)
LANG=en_US.UTF-8
LANGUAGE=en_GB:en
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=sv_SE.UTF-8
LC_TIME=sv_SE.UTF-8
LC_COLLATE="en_US.UTF-8"
LC_MONETARY=sv_SE.UTF-8
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=sv_SE.UTF-8
LC_NAME=sv_SE.UTF-8
LC_ADDRESS=sv_SE.UTF-8
LC_TELEPHONE=sv_SE.UTF-8
LC_MEASUREMENT=sv_SE.UTF-8
LC_IDENTIFICATION=sv_SE.UTF-8
LC_ALL=

So, the problem is that all the guides on the internet presume you have a problem with your locales on your server, which you dont! The locale is probably fine. It is just that the ssh client computer has an LC_CTYPE which does not happen to be valid on the server.

The Debian Locale guide itself is of course correct. And it states that you should not set LC_ALL=en_US.UTF-8 (!). That is however the only “working” answer I found online.

Solution
I guess the easiest thing is to add a line to your .profile file (on the server):

$ cat .profile 
.
.
.
LC_CTYPE="en_US.UTF-8"

If you are not using a bourne-shell you probably know how to set a variable for your shell. Any other method that will set LC_CTYPE to a locale valid on your Debian machine will also work. Clearly, you dont need (and you probably should not) set LC_ALL.

Not happy with en_US.UTF-8?
Not everyone use the American locale. To find out what locales are available/valid on your system:

$ locale -a
C
C.UTF-8
en_GB.utf8
en_US.utf8
POSIX
sv_SE.utf8

Want others?

$ sudo dpkg-reconfigure locales

or read the Debian Locale Guide.

Upgrading Qnap TS109 from Wheezy to Jessie

The Qnap TS-109 runs Debian Wheezy just fine, but it has to be upgraded from Squeeze (no direct install). How about upgrading Wheezy to Jessie? This device is old and slow, so I decided to find out, and if it does not work, so be it.

Debian links:

Below follows a shorter version of the upgrade guide, focusing on what I actually did with my Qnap.

Getting ready
First you should of course backup your stuff if the device contains anything you can not lose.

It can also be a good idea to clean out packages that you dont need (to both avoid problems when upgrading them, and to save time during the upgrade):

$ sudo dpkg -l
$ sudo apt-get purge SOMEPACKAGES

It is also adviced to make sure you dont have packages on hold. You can do this with:

$ sudo dpkg --get-selections | grep 'hold$'

I confirmed that my system was then fully wheezy-updated

$ sudo apt-get update

$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

With that, I decided I was ready for an upgrade to Jessie.

Upgrade itself
I first updated /etc/apt/sources.list.

Old version:

deb http://ftp.se.debian.org/debian/ wheezy main
deb-src http://ftp.se.debian.org/debian/ wheezy main non-free

deb http://security.debian.org/ wheezy/updates main
deb-src http://security.debian.org/ wheezy/updates main non-free

deb http://ftp.df.lth.se/debian wheezy-backports main

New version:

deb http://ftp.se.debian.org/debian/ jessie main
deb-src http://ftp.se.debian.org/debian/ jessie main non-free

deb http://security.debian.org/ jessie/updates main
deb-src http://security.debian.org/ jessie/updates main non-free

I just deleted the backports source.

And begin upgrade:

$ sudo apt-get update
$ sudo apt-get upgrade

That went fine, so I did a reboot. It took longer than usual, but it came up. So I proceeded with:

$ sudo apt-get dist-upgrade

…which went fine, and I rebooted.

System came up and it seems good! If I encounter problems later I will write about it here.

Working OpenVPN configuration

I am posting my working OpenVPN server configuration, and client configuration for Linux, Android and iOS. First a little background.

I have an OpenWRT (14.07) router running OpenVPN server. This router has a public IP address and thanks to dyn.com/dns it can be resolved using a domain name (ROUTER.PUBLIC in all configuration examples below).

My router LAN address is 192.168.8.1, the LAN network is 192.168.8.*, and the OpenVPN network is 192.168.9.* (in this range OpenVPN-clients will be given an address to their vpn/dun-device). I run OpenVPN on TCP 1143.

What I want to achieve is
1) to access local services (like ownCloud and ssh) of computers on the LAN
2) to access internet as if I were at home, when I have an internet access that is somehow restricted

The Server
Essentially, this OpenWRT OpenVPN Setup Guide is very good. Follow it. I am not going to repeat everything, just post my working configurations.

root@breidablick:/etc/config# cat openvpn 

config openvpn 'myvpn'
	option enabled '1'
	option dev 'tun'
	option proto 'tcp'
	option status '/tmp/openvpn.clients'
	option log '/tmp/openvpn.log'
	option verb '3'
	option ca '/etc/openvpn/ca.crt'
	option cert '/etc/openvpn/my-server.crt'
	option key '/etc/openvpn/my-server.key'
	option server '192.168.9.0 255.255.255.0'
	option port '1143'
	option keepalive '10 120'
	option dh '/etc/openvpn/dh2048.pem'
	option push 'redirect-gateway def1'
	option push 'dhcp-option DNS 192.168.8.1'
	option push 'route 192.168.8.0 255.255.255.0'

It is a little unclear if the last three options really work for all clients. I also have:

root@breidablick:/etc/config# cat network 
.
.
.
config interface 'vpn0'
	option ifname 'tun0'
	option proto 'none'

and

root@breidablick:/etc/config# cat firewall 
.
.
.
config zone
	option name 'vpn'
	option input 'ACCEPT'
	option forward 'ACCEPT'
	option output 'ACCEPT'
	list network 'vpn0'
.
.
.
config forwarding
	option src 'lan'
	option dest 'vpn'

config forwarding
	option src 'vpn'
	option dest 'wan'
.
.
.
# may not be needed depending on your lan policys (2 next)
config rule
	option name 'Allow-lan-vpn'
	option src 'lan'
	option dest 'vpn'
	option target ACCEPT
	option family 'ipv4'

config rule
	option name 'Allow-vpn-lan'
	option src 'vpn'
	option dest 'lan'
	option target ACCEPT
	option family 'ipv4'
.
.
.
# may not be needed depending on your wan policy
config rule
	option name 'Allow-OpenVPN-from-Internet'
	option src 'wan'
	option proto 'tcp'
	option dest_port '1143'
	option target 'ACCEPT'
	option family 'ipv4'

iOS client
You need to install OpenVPN client for iOS from the app store. The client configuration is prepared on your computer, and synced with iOS using iTunes (brilliant or braindead?). This is my working configuration:

client
dev tun
ca ca.crt
cert iphone.crt
key iphone.key
remote ROUTER.PUBLIC 1143 tcp-client
route 0.0.0.0 0.0.0.0 vpn_gateway
dhcp-option DNS 192.168.8.1
redirect-gateway def1

This route and redirect-gateway configuration makes all traffic go via VPN. Omit those lines if you want direct internet access.

Android client
For Android, you also need to install the OpenVPN client from the Store. My client is the “OpenVPN for Android” by Arne Schwabe. This client has a GUI that allows you to configure everything (but you need to get the certificate files to your Android device somehow). You can watch the entire Generated Config in the GUI and mine looks like this (omitting GUI and Android-specific stuff, and the certificates):

ifconfig-nowarn
client
verb 4
connect-retry-max 5
connect-retry 5
resolv-retry 60
dev tun
remote ROUTER.PUBLIC 1143 tcp-client
route 0.0.0.0 0.0.0.0 vpn_gateway
dhcp-option DNS 192.168.8.1
remote-cert-tls server
management-query-proxy

Linux client
I also connect linux computers occationally. The configuration is:

client
remote ROUTER.PUBLIC 1194
ca ca.crt
cert linux.crt
key linux.key
dev tun
proto tcp
nobind
auth-nocache
script-security 2
persist-key
persist-tun
user nobody
group nogroup
verb 5
# redirect-gateway local def1
log log.txt

Here the redirect-gateway is commented away, so internet traffic is not going via VPN.

Certificates
The easy-rsa package and instructions in the OpenWRT guide above are excellent. You should have different certificates for different clients. One certificate can only be used for one connection at a time.

Better configuration?
I dont say this is the optimal or best way to configure OpenVPN – but it works for me. You may prefer UDP over TCP, and may reasons for running TCP are perhaps not valid for you. You may want different encryption or data compressions options, different logging options and so on.

Installing Citrix Receiver 13.1 in Ubuntu/Debian

The best thing with Citrix Receiver for Linux is that it exists. Apart from that it kind of sucks. Last days I have tried to install it on Xubuntu 14.10 and Debian 7.7, both 64-bit version.

The good thing is that for both Debian and Ubuntu the 64-bit deb-file is actually installable using “dpkg -i”, if you fix all dependencies. I did:

1) #dpkg --add-architecture i386
2) #apt-get update
3) #dpkg -i icaclient_13.1.0.285639_amd64.deb
  ... list of failed dependencies...
4) #dpkg -r icaclient
5) #apt-get install [all packages from (3)]
6) #dpkg -i icaclient_13.1.0.285639_amd64.deb

Step (1) and (2) only needed in Debian.

selfservice is hard to get to start from the start menu. And selfservice gets segmentation fault when OpenVPN is on (WTF?). So for now, I have given up on it.

npica.so is supposed to make the browser plugin work, but not much luck there (guess it is because I have a 64 bit browser). I deleted system-wide symbolic links to npica.so (do: find | grep npica.so in the root directory).

#rm /usr/lib/mozilla/plugins/npica.so
#rm /usr/local/lib/netscape/plugins/npica.so

Then I could tell the Citrix portal that I do have the Receiver even though the browser does not recognize it, and as I launch an application I have choose to run it with wfica.sh (the good old way).

keyboard settings can no longer be made in the GUI but you have to edit your ~/.ICAClient/wfclient.ini file. The following makes Swedish keyboard work for me:

KeyboardLayout = SWEDISH
KeyboardMappingFile = linux.kbd
KeyboardDescription = Automatic (User Profile)
KeyboardType=(Default)

The problem is, when you fix the file, you need to restart all Citrix-related processes for the new settings to apply. If you feel you got the settings right but no success, just restart your computer. I wasted too much time thinking I had killed all processes, and thinking my wfclient.ini-file was bad, when a simple restart fixed it.

Debian on NUC and boot problems

I got a NUC (D54250WYKH) that I installed Debian 7.7 on.

Advice: First update the NUC “BIOS”.

  1. Download from Intel
  2. Put on USB memory
  3. Put USB memory in NUC
  4. Start NUC, Press F7 to upgrade BIOS

If I had done this first I would have saved some time and some reading about EFI stuff I don’t want to know anyway. A few more conclusions follow.

EFI requires a special little EFI-partition. Debian will set it up automatically for you, unless you are an expert and choose manual partitioning, of course ๐Ÿ˜‰ That would also have saved me some time.

(X)Ubuntu 14.10 had no problems even without upgrading BIOS.

The NUC is very nice! In case it is not clear: there is space for both an mSATA drive and a 2.5′ drive in my model. In fact, I think there is also space for an extra extra small mSATA drive. Unless building a gaming computer I believe NUC (or similar) is the way to go.

Finally, Debian 7.7 comes with Linux 3.2 kernel which has old audio drivers that produce bad audio quality. I learnt about Debian backports and currently run Linux 3.16 with Debian 7.7 and I have perfect audio now.

Grub Boot Error

Update 20150419: This OCZ SSD Drive is now entirely broken.

My desktop computer (it is still an ASUS Barebone V3-M3N8200) sometimes gives me the following error when I turn it on:

error: attempt to read or write outside of disk 'hd0'.
Entering rescue mode...
grub rescue> _

gruberror

My observations:

  • This has happened now and then for a while
  • It seems to happen more often when the computer have been off for a longer period of time (sounds unlikely, I know)
  • Ctrl-Alt-Del: It always boots properly the second time

I have three SATA drives. BIOS boots the first harddrive, where GRUB is installed on the mbr, and where / is the first and only partition, and /boot lives on the / partition.

The drive in question is (from dmesg):

[    1.339215] ata3.00: ATA-8: OCZ-VERTEX PLUS R2, 1.2, max UDMA/133
[    1.339217] ata3.00: 120817072 sectors, multi 1: LBA48 NCQ (depth 31/32)
[    1.339323] ata3.00: configured for UDMA/133
[    1.339466] scsi 2:0:0:0: Direct-Access     ATA      OCZ-VERTEX PLUS  1.2  PQ: 0 ANSI: 5
[    1.339623] sd 2:0:0:0: Attached scsi generic sg1 type 0
[    1.339715] sd 2:0:0:0: [sda] 120817072 512-byte logical blocks: (61.8 GB/57.6 GiB)

That is, a 60GB SSD drive from OCZ (yes, I had another OCZ SSD drive that died).

I can not explain my occational boot errors, but I have some theories:

  • The SSD drive is broken/corrupted (but no signs within Ubuntu of anything like it)
  • All drive is somehow not initialized when GRUB executes (?)
  • Somehow, more than one hard drive is involved in the boot process, and they are not all initialized at the same time (but this does not seem to be the case)

GSmartControl gives me some suspicious output about my drive… but I do not know how to interpret it:

Error in ATA Error Log structure: checksum error
Error in Self-Test Log structure: checksum error

The (Short) self test completes without errors.

Any ideas or suggestions are welcome! I will update this post if I learn anything new.

Installing Ubuntu on Pentium M with forcepae

If trying to install Ubuntu (or Xubuntu, Lubuntu, Kubuntu) 14.04 (or 14.10) on a Pentium M computer, you may get the following error:

ERROR: PAE is disabled on this Pentium M

ERROR: PAE is disabled on this Pentium M

Just restart the computer and when you come to the install menu…

lubuntu-pae-1

…hit F6 to get a menu of kernel parameters. Now, none of those parameters are what you want, so hit ESC. You should now be able to type forcepae at the end of the kernel command:
lubuntu-pae-3

Now, hit Return, and startup/installation of Ubuntu should proceed just normally.

Background
PAE is a CPU feature that has been available on most x86-CPUs since the Pentium Pro days. Since Ubuntu 12.10, it is a required feature. Some Pentium M CPUs have the PAE feature implemented, but the processor does not announce the feature properly. Since Ubuntu 14.04 the above forcepae option is available, to allow Linux to use PAE even if the CPU officially does not support it.

This affects mostly laptops from perhaps 2000-2005. These laptops are often good computers with 1400-2000MHz CPU and 512+ MB of RAM. As Windows XP is now officially unsupported by Microsoft owners of such harware might want to install an Ubuntu flavour on the computer instead.

There have been ways to make this work with Ubuntu 12.10-13.10. I suggest, abandon those versions and hacks completely, and make a fresh install of 14.04.

I have written before about Ubuntu on Pentium M without PAE.

Migrating from Windows XP
I would personally suggest Xubuntu or Lubuntu as a replacement for Windows XP: Both should be lightweight enough for your Pentium M computer, and both are easy to use with only a Windows background. Lubuntu is most Windows-like and the lightest of them. Xubuntu is a bit heavier (and nicer), and also resembles Mac OS X a bit.

I suggest the “Try Ubuntu without Installing” option. You will have an installera available inside Ubuntu anyways, and you can confirm that most things work properly before you wipe the computer.

USB Drives, dd, performance and No space left

Please note: sudo dd is a very dangerous combination. A little typing error and all your data can be lost!

I like to make copies and backups of disk partitions using dd. USB drives sometimes do not behave very nicely.

In this case I had created a less than 2GB FAT32 partition on a USB memory and made it Lubuntu-bootable, with a 1GB file for saving changes to the live filesystem. The partition table:

It seems I forgot to change the partition to FAT32, but it is formatted with FAT32 and that seems to work fine ๐Ÿ˜‰

$ sudo /sbin/fdisk -l /dev/sdc

Disk /dev/sdc: 4004 MB, 4004511744 bytes
50 heads, 2 sectors/track, 78213 cylinders, total 7821312 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f3a78

   Device Boot      Start         End      Blocks   Id  System
/dev/sdc1   *        2048     3700000     1848976+  83  Linux

I wanted to make an image of this USB drive that I can write to other USB drives. That is why I made the partition/filesystem significantly below 2GB, so all 2GB USB drives should work. This is how I created the image:

$ sudo dd if=/dev/sdb of=lubuntu.img bs=512 count=37000000

So, now I had a 1.85GB file named lubuntu.img, ready to write back to another USB drive. That was when the problems began:

$ sudo dd if=lubuntu.img of=/dev/sdb
dd: writing to โ€˜/dev/sdbโ€™: No space left on device
2006177+0 records in
2006176+0 records out
1027162112 bytes (1.0 GB) copied, 24.1811 s, 42.5 MB/s

Very fishy! The write speed (42.5MB/s) is obviously too high, and the USB drive is 4GB, not 1GB. I tried with several (identical) USB drives, same problem. This has never happened to me before.

I changed strategy and made an image of just the partition table, and another image of the partion:

$ sudo dd if=/dev/sdb of=lubuntu.sdb bs=512 count=1
$ sudo dd if=/dev/sdb1 of=lubuntu.sdb1

…and restoring to another drive… first the partition table:

$ sudo dd if=lubuntu.sdb if=/dev/sdb

Then remove and re-insert USB Drive, make sure it does not mount automatically before you proceed with the partition.

$ sudo dd if=lubuntu.sdb1 if=/dev/sdb1 

That worked! However, the write speed to USB drives usually slow down as more data is written (in one chunk, somehow). I have noticed this before with other computers and other USB drives. I guess USB drives have some internal mapping table that does not like big files.

Finally, to measure progress of the dd command, send it the signal:

$ sudo kill -USR1 <PID OF dd PROCESS>

Above behaviour noticed on x86 Ubuntu 13.10.