Monthly Archives: April 2019

macOS 10.14 on unsupported MacBook Pro

Update 20190929: macOS suggested 10.14.6 Supplemental Update 2. I installed it and the computer restarted properly. For best experience I re-applied Mojave Patcher 1.3.3.

Update 20190528: macOS suggested two things today. First it found something strange about EFI and wanted to send a dump to Apple (I rejected). Secondly, it wants to install a Mac Book Pro Supplemental Update. I tried it and it took a while but the computer came up again with no need to repatch.

20190524: All Good With 10.14.5

Update 20190524: With the latest/current version of Mojave Patcher I successfully upgraded to 10.14.5. When the upgrade restarts it will eventually hang. So you need to re-patch from your USB Mojave Patcher after installing 10.14.5. All fine!

Update 20190520: Today I innocently let my computer update itself. Not smart. It crashed during update. It didnt start. I tried to re-patch it, still does not start. It obviously tried to install 10.14.5 which I successfully installed later on another supported computer.

Update 20190519: A few Das ago I installed an update from macOS Mojave Patcher that was supposed to fix Random Kernel Panics. That worked fine and the problems that much of this article covers are fixed.

Update: I upgraded the MacBook Pro 6,2 to 8GB RAM (2×4 bought for purpose). I had 2×8 modules slightly faster RAM: either of them was very unstable and both of them together did not boot at all. I guess if you have 1×8 of the correct speed that it would work, but I would recommend to buy 2×4. I also replaced the hard drive with a cheap SSD. I would say the computer performs nicely for practical purposes.

Original Post

I got custody of a MacBook Pro 6,2 that has seen very little use. It is the first MacBook with an i5 cpu, it has a 320GB replacable hard drive (a long gone feature), 4GB RAM upgradable to 8GB, and nice 15 inch display and a very nice keyboard.

It was running macOS 10.6.8. I realised it supports 10.13 but official Apple support ends there.

So I found out about a project/software called macOS Mojave Patcher that allows you to install mac OS 10.14 Mojave on certain unsupported Macs, including the MacBook Pro 6,2. I gave it a try and I will write about my findings.

Summary

This MacBook Pro 6,2 runs macOS Mojave 10.14 quite perfectly if installed using Mojave Patcher. However, it seems absolutely critical for stable operation to disable Automatic Graphics Switching (System Preferences -> Energy Saver). This may make the computer run warmer, consume more energy and suffer shorter battery time than it would otherwise.

Below follows details of all my findings. If you don’t want the details, you can skip to Is it worth it in the end of this post.

Attempt 1 : Clean install of 10.14

I made a clean install of 10.14 (using a Mojave Patcher USB). That did not start at all. Instead of the familiar Apple and progress bar, I just got a question mark. I think the problem is that some kind of “firmware” upgrade is needed.

Attempt 2: Install 10.12 – upgrade to 10.14

I made a clean standard install of 10.12. I am quite sure it did some kind of firmware update. Then I upgraded to 10.14 (using Mojave Patcher USB). It took very long time (several hours, usually that takes less than an hour). It first appeared to be good – 10.14 started – but it turned out not to be stable. I got kernel panics of different types quite often (after just minutes of use). I also realised the upgrade had not converted HFS+ to APFS as I expected.

Attempt 3: Clean install of 10.14

I made a new clean attempt with 10.14 (on APFS) and this time the system started up as expected. But the kernel panics remain (as I read them, it was about Nvidia some times, Audio one time, Crypto some time, APFS some time).

Back to 10.13

I made a clean install of 10.13 to ensure there isn’t anything wrong with the computer itself. Currently I am writing this blog post while installing Xcode (not from App Store because that is not allowed with 10.13), and the computer has been stable for a few hours.

Attempt 4: Minimal patches

I made a new clean install of 10.14 on APFS. When running the Mojave Patcher, I only selected these two patches:

  • Boot.plist Patch (Disable Platform Check)
  • SIP Disabler Patch

Installation was successful, system came up, I am writing right here right now, and here is a screenshot (installing Xcode).

A notice a few obvious differences from when I had all recommended patches:

  1. Audio does not work (I did not pick Legacy Audio Patch)
  2. GFX uses the NVIDIA GT 330M only. Not the Intel card. This computer has two GPUs, and it is supposed to switch between them depending on load. Now it only uses the more powerful card. Also the display menu (top right) is not aware of resolutions as it used to. But it seems the display is running at 1440×900, so I am happy with that. It may get warmer now, and there are occasional graphics glitches, but not particularly disturbing. (I did not pick Legacy Video Card Patch)
  3. The Install Patch Updater is obviously not installed, since I did not pick it.
  4. Photos (that come with macOS) crashed when I tried to edit the above picture.
  5. It does not go to sleep, neither if you close it or choose Sleep from the Apple menu.

USB seems fine at first glance (despite I did not pick Legacy USB Support Injector). WiFi works (there was no such patch).

Installing Xcode took an eternity… after more than an hour I got impatient, restarted the computer, installed “Software Update Patch”, started the Xcode installation again, and went to bed. Next morning: Xcode installed and computer still running peacefully.

24 hours later

The computer was stable with 10.14 for an entire workday doing programming (mostly Node.js and Safari). So I decided to apply the Audio patch as well, which gives this list of patches installed:

  • Boot.plist Patch (Disable Platform Check)
  • SIP Disabler Patch
  • Legacy Audio Patch
  • Software Update Patch

So far so good: 30 minutes of Audio Play, both locally and streaming over Bluetooth.

48 hours later

After another stable work day I decided to install all patches except the Video patch. That is:

  • Boot.plist Patch (Disable Platform Check)
  • Legacy USB Support Injector
  • SIP Disabler Patch
  • Install Patch Updater
  • Legacy Audio Patch
  • Software Update Patch

When the system started, the Patch Updater wants to install two things (related to Siri and Night mode), and for now I rejected it.

A few hours later

After a few hours I also installed:

  • Night Shift Patch
  • Siri Patch

This is done using Patch Updater, within macOS (no need to boot on Mojave Patcher USB). It appears to cause no trouble.

The only option left

All this leaves me with just the Legacy Video Patch. There is a twist, as this system has two GPUs, and without the patch it is using only the Nvidia GPU (it is not aware of the Intel GPU).

Under System Preferences -> Energy Saver, there is an option Automatic Graphics Switching. With that one set to OFF the computer should only use the Nvidia GPU anyway. Why would I want to do that? Well, there are currently some minor graphics glitches. Those could potentially go away. Also, I have some limited functionality when it comes to video:

  • no screen resolution options (although it runs at best resolution)
  • connecting an external display does not work
  • cannot adjust screen brightness

If I install the Legacy Video Patch I shall get an unstable system, but what if I also disable Automatic Graphics Switching? (to be completely honest, I am not even certain it was ever enabled).

And finally Legacy Video Patch

Its just been a few hours, but I installed Legacy Video Patch (all recommended patches) and disabled Automatic Graphics Switching. The computer seems stable and now sleep, brightness and external display works. Also the occasional graphics glitches seem gone.

Is this worth it?

I suppose, to some people running unsupported 10.14 rather than supported 10.13 makes sense. To me:

  • 10.14 allows latest version of Xcode
  • 10.14 has dark mode
  • 10.13 is still supported (receiving updates) by Apple (as of May 2019), this should change as 10.15 is release (could be end of 2019).

There could be a 10.15 Patcher in the future. And it could work with this computer. Or not. As long as you can run the latest current macOS without too much hazzle, that could be preferred to running an unsupported version of macOS. But today (May 2019) 10.13 is not unsupported. In fact, Apple still seems to release security updates to 10.12.

But I am curious, and I think I want the latest Xcode, so here I am.

Adobe CS4 and macOS 10.14 Mojave

How about using an old unsupported CS application on current macOS?

Today I had a reason to try it out. The short version is that it seems to work with Photoshop and Illustrator, but not with Indesign.

Background

Adobe used to sell “perpetual licenses” with their Creative Suite software. Years ago they stopped doing that and changed to a subscription model with Creative Cloud. People with perpetual licenses could still use them, but support in mac OS is getting more troublesome with every upgrade. A perpetual license could be used on two computers, and there was an activation/deactivation feature.

My Case

I have a friend who use CS4 on two older Macs running unspported version of macOS. My friend now got a brand new MacMini (with macOS 10.14 Mojave) and ideally we wanted to deactivate CS4 on the old MacMini, and activate it on the new. That was obviously a gamble.

Activation Servers Down

It appears the Adobe Activation servers are down. Deactivation on the old MacMini was not possible. Thus proper Activation should also not be possible.

Trying an old workaround

There used to be an old workaround (clearly used for software piracy) described here. We tried installing CS4 on Mojave and that worked fine. We then did the hack in the article, and that worked too. It seems Photoshop and Illustrator worked correctly, but Indesign did not. Indesign ran into a perhaps well known problem about “empty toolbox”. Also, when choosing “New”, an empty dialog window opens. This can perhaps be fixed, but we did not bother.

Restoring CS4 using Time Machine gave the same result for practical purposes: Photoshop worked but Indesign appeared broken and could not even be started.

CS5, CS6

Obviously, I can’t say anything definite about CS5 and CS6. I guess the workaround does not work. And I believe after reading some forums that the activation servers are down as well.

Conclusion

You may be able to install and use CS4 Illustrator and Photoshop on macOS 10.14 Mojave. Rumours indicate that it will not work at all in 10.15 when 32-bit support is finally dropped from macOS.

Performance, Node.js & Sorting

I will present two findings that I find strange in this post:

  1. The performance of Node.js (V8?) has clearly gotten consistently worse with newer Node.js versions.
  2. The standard library sort (Array.prototype.sort()) is surprisingly slow, often slower than a simple textbook mergesort.

My findings in this article are based on running a simple program mergesort.js on different computers and different node versions.

You may also want to read this article about sorting in Node.js. It applies to V8 version 7.0, which should be used in Node.js V11.

The sorting algorithms

There are three sorting algorithms compared.

  1. Array.prototype.sort()
  2. mergesort(), a textbook mergesort
  3. mergesort_opt(), a mergesort that I put some effort into making faster

Note that mergesort is considered stable and not as fast as quicksort. As far as I understand from the above article, Node.js used to use quicksort (up to V10), and from V11 uses something better called Timsort.

My mergesort implementations (2) (3) are plain standard JavaScript. Nothing fancy whatsoever (I will post benchmarks using Node.js v0.12 below).

The data to be sorted

There are three types of data to be sorted.

  1. Numbers (Math.random()), compared with a-b;
  2. Strings (random numbers converted to strings), compared with default compare function for sort(), and for my mergesort simple a<b, a>b compares to give -1, 1 or 0
  3. Objects, containing two random numbers a=[0-9], b=[0-999999], compared with (a.a-b.a) || (a.b-b.b). In one in 10 the value of b will matter, otherwise looking at the value of a will be enough.

Unless otherwise written the sorted set is 100 000 elements.

On Benchmarks

Well, just a standard benchmark disclaimer: I do my best to measure and report objectively. There may be other platforms, CPUs, configurations, use cases, datatypes, or array sizes that give different results. The code is available for you to run.

I have run all tests several times and reported the best value. If anything, that should benefit the standard library (quick)sort, which can suffer from bad luck.

Comparing algorithms

Lets start with the algorithms. This is Node V10 on different machines.

(ms)     ===== Numbers =====   ===== Strings =====   ==== Objects =====
sort() merge m-opt sort() merge m-opt sort() merge m-opt
NUC i7 82 82 61 110 81 54 95 66 50
NUC i5 113 105 100 191 130 89 149 97 72
NUC Clrn 296 209 190 335 250 196 287 189 157
RPi v3 1886 1463 1205 2218 1711 1096 1802 1370 903
RPi v2 968 1330 1073 1781 1379 904 1218 1154 703

The RPi-v2-sort()-Numbers stand out. Its not a typo. But apart from that I think the pattern is quite clear: regardless of datatype and on different processors the standard sort() simply cannot match a textbook mergesort implemented in JavaScript.

Comparing Node Versions

Lets compare different node versions. This is on a NUC with Intel i5 CPU (4th gen), running 64bit version of Ubuntu.

(ms)     ===== Numbers =====   ===== Strings =====   ==== Objects =====
sort() merge m-opt sort() merge m-opt sort() merge m-opt
v11.13.0 84 107 96 143 117 90 140 97 71
v10.15.3 109 106 99 181 132 89 147 97 71
v8.9.1 85 103 96 160 133 86 122 99 70
v6.12.0 68 76 88 126 92 82 68 83 63
v4.8.6 51 66 89 133 93 83 45 77 62
v0.12.9 58 65 78 114 92 87 55 71 60

Not only is sort() getting slower, also running “any” JavaScript is slower. I have noticed this before. Can someone explain why this makes sense?

Comparing different array sizes

With the same NUC, Node V10, I try a few different array sizes:

(ms)     ===== Numbers =====   ===== Strings =====   ==== Objects =====
sort() merge m-opt sort() merge m-opt sort() merge m-opt
10 000 10 9 11 8 12 6 4 7 4
15 000 8 15 7 13 14 11 6 22 7
25 000 15 35 12 40 27 15 11 25 18
50 000 35 56 34 66 57 37 51 52 30
100 000 115 107 97 192 138 88 164 101 72
500 000 601 714 658 1015 712 670 698 589 558

Admittedly, the smaller arrays show less difference, but it is also hard to measure small values with precision. So this is from the RPi v3 and smaller arrays:

(ms)     ===== Numbers =====   ===== Strings =====   ==== Objects =====
sort() merge m-opt sort() merge m-opt sort() merge m-opt
5 000 34 57 30 46 59 33 29 52 26
10 000 75 129 64 100 130 74 63 104 58
20 000 162 318 151 401 290 166 142 241 132
40 000 378 579 337 863 623 391 344 538 316

I think again quite consistently this looks remarkably bad for standard library sort.

Testing throughput (Version 2)

I decided to measure throughput rather than time to sort (mergesort2.js). I thought perhaps the figures above are misleading when it comes to the cost of garbage collecting. So the new question is, how many shorter arrays (n=5000) can be sorted in 10s?

(count)  ===== Numbers =====   ===== Strings =====   ==== Objects =====
sort() merge m-opt sort() merge m-opt sort() merge m-opt
v11.13.0 3192 2538 4744 1996 1473 2167 3791 2566 4822
v10.15.3 4733 2225 4835 1914 1524 2235 4911 2571 4811
RPi v3 282 176 300 144 126 187 309 186 330

What do we make of this? Well the collapse in performance for the new V8 Torque implementation in Node v11 is remarkable. Otherwise I notice that for Objects and Node v10, my optimized algorithm has no advantage.

I think my algorithms are heavier on the garbage collector (than standard library sort()), and this is why the perform relatively worse for 10s in a row.

If it is so, I’d still prefer to pay that price. When my code waits for sort() to finish there is a user waiting for a GUI update, or for an API reply. I rather see a faster sort, and when the update/reply is complete there is usually plenty of idle time when the garbage collector can run.

Optimizing Mergesort?

I had some ideas for optimizing mergesort that I tried out.

Special handling of short arrays: clearly if you want to sort 2 elements, the entire mergesort function is heavier than a simple function that sorts two elements. The article about V8 sort indicated that they use insertion sort for arrays up to length 10 (I find this very strange). So I implemented special functions for 2-3 elements. This gave nothing. Same performance as calling the entire mergesort.

Less stress on the garbage collector: since my mergesort creates memory nodes that are discarded when sorting is complete, I thought I could keep those nodes for the next sort, to ease the load on the garbage collector. Very bad idea, performance dropped significantly.

Performance of cmp-function vs sort

The relevant sort functions are all K (n log n) with different K. It is the K that I am measuring and discussing here. The differences are, after all, quite marginal. There is clearly another constant cost: the cost of the compare function. That seems to matter more than anything else. And in all cases above “string” is just a single string of 10 characters. If you have a more expensive compare function, the significance of sort() will be even less.

Nevertheless, V8 is a single threaded environment and ultimately cycles wasted in sort() will result in overall worse performance. Milliseconds count.

Conclusions

Array.prototype.sort() is a critical component of the standard library. In many applications sorting may be the most expensive thing that takes place. I find it strange that it does not perform better than a simple mergesort implementation. I do not suggest you use my code, or start looking for better sort() implementations out there right away. But I think this is something for JavaScript programmers to keep in mind. However, the compare function probably matters more in most cases.

I find it strange that Node v11, with Timsort and V8 Torque is not more of an improvement (admittedly, I didnt test that one very much).

And finally I find it strange that Node.js performance seems to deteriorate with every major release.

Am I doing anything seriously wrong?

JavaScript Double Linked List

JavaScript has two very powerful and flexible build in data structures: [] and {}. You can program rather advanced JavaScript for years without needing anything else.

Nevertheless I had a conversation about possible advantages of using a linked list (instead of an array). Linked lists are not very popular, Stroustrup himself has suggested they should be avoided. But what if you mostly do push(), pop(), shift() and unshift() and never access an item by its index? Higher order functions as map(), reduce(), filter() and sort(), as well as iterators should be just fine.

I decided to implement a Double Linked List in JavaScript making it (mostly) compatible with Array and do some tests on it. The code both of the DoubleLinkedList itself, and the unit tests/benchmarks are available.

Disclaimer

This is a purely theoretical, academical and nerdy experiment. The DoubleLinkedList offers no advantages over the built in Array, except for possible performance advantages in edge cases. The disadvantages compared to Array are:

  • Lower performance in some cases
  • Less features / limited API
  • Less tested and proven
  • An extra dependency, possible longer application loading time

So, my official recommendation is that you read this post and perhaps look at the code for learning purposes. But I really doubt you will use my code in production (although you are welcome to).

Benchmarks

Benchmarks are tricky. In this case there are three kinds of benchmarks:

  1. Benchmarks using array[i] to get the item at an index. This is horrible for the linked list. I wrote no such benchmarks.
  2. Benchmarks testing map(), reduce(), filter(), that I wrote but that show consistently no relevant and interesting differences between built in Array and my DoubleLinkedList (my code is essentially equally fast as the standard library array code, which on one hand is impressive, and on the other hand is reason not to use it).
  3. Benchmarks where my DoubleLinkedList does fine, mostly that heavily depends on push(), pop(), shift() and unshift().

The only thing I present below is (3). I have nothing for (1), and (2) shows nothing interesting.

The machines are in order an Hades Canyon i7-NUC, an old i5-NUC, a newer Celeron-NUC, an Acer Chromebook R13 (with an ARMv8 CPU), A Raspberry Pi v3, and a Raspberry Pi V2. The Chromebook runs ChromeOS, the i7 runs Windows, the rest run Linux.

My benchmarks use Math.random() to create test data. That was not very smart of me because the variation between test runs is significant. The below numbers (milliseconds) are the median value of running each test 41 times. You can see for yourself that the values are quite consistent.

The tested algorithms

The push(), pop(), shift(), unshift() tests use the array/list as a queue and push 250k “messages” throught it, keeping the queue roughly 10k messages.

The mergesort() test is a mergesort built on top of the datastructures using push()/shift().

The sort() test is the standard Array.sort(), versus a mergesort implementation for DoubleLinkedList (it has less overhead than mergesort(), since it does not create new objects for every push()).

Benchmark result

                    Node8   ============== Node 10 =====================
(ms) NUCi7 NUCi7 NUCi5 NUC-C R13 RPiV3 RPiV2
unshift/pop 250k
Array 679 649 1420 1890 5216 11121 8582
D.L.L. 8 13 10 20 40 128 165
push/shift 250k
Array 37 31 31 49 143 388 317
D.L.L. 10 12 10 19 44 115 179
mergesort 50k
Array 247 190 300 466 1122 3509 3509
D.L.L. 81 88 121 244 526 1195 1054
sort 50k
Array 53 55 59 143 416 1093 916
D.L.L. 35 32 42 84 209 543 463

What do we make of this?

  • For array, push/shift is clearly faster than unshift/pop!
  • It is possible to implement a faster sort() than Array.sort() of the standard library. However, this may have little to do with my linked list (I may get an even better result if I base my implementation on Array).
  • I have seen this before with other Node.js code but not published it: the RPiV2 (ARMv7 @900MHz) is faster than the RPiV3 (ARMv8 @1200Mhz).
  • I would have expected my 8th generation i7 NUC (NUC8i7HVK) to outperform my older 4th generation i5 NUC (D54250WYK), but not so much difference.

More performance findings

One thing I thought could give good performance was a case like this:

x2 = x1.map(...).filter(...).reduce(...)

where every function creates a new Array just to be destroyed very soon. I implemented mapMutate and filterMutate for my DoubleLinkedList, that reuse existing List-nodes. However, this gave very little. The cost of the temporary Arrays above seems to be practically insignificant.

However for my Double linked list:

dll_1 = DoubleLinkedList.from( some 10000 elements )
dll_1.sort()
dll_2 = DoubleLinkedList.from( some 10000 elements )

Now
dll_1.map(...).filter(...).reduce(...) // slower
dll_2.map(...).filter(...).reduce(...) // faster

So it seems I thought reusing the list-nodes would be a cost saver, but it turns out to produce cache-misses instead

Using the Library

If you feel like using the code you are most welcome. The tests run with Node.js and first runs unit tests (quickly) and then benchmarks (slower). As I wrote earlier, there are some Math.random() in the tests, and on rare occations statistically unlikely events occur, making the tests fail (I will not make this mistake again).

The code itself is just for Node.js. There are no dependencies and it will require minimal work to adapt it to any browser environment of your choice.

The code starts with a long comment specifying what is implemented. Basically, you use it just as Array, with the exceptions/limitations listed. There are many limitations, but most reasonable uses should be fairly well covered.

Conclusion

It seems to make no sense to replace Array with a linked list in JavaScript (Stroustrup was right). If you are using Array as a queue be aware that push/shift is much faster than unshift/pop. It would surprise me much if push/pop is not much faster than unshift/shift for a stack.

Nevertheless, if you have a (large) queue/list/stack and all you do is push, pop, shift, unshift, map, filter, reduce and sort go ahead.

There is also a concatMutate in my DoubleLinkedList. That one is very cheap, and if you for some reason do array.concat(array1, array2, array3) very often perhaps a linked list is your choice.

It should come as no surprise, but I was suprised that sort(), mergesort in my case, was so easy to implement on a linked list.

On RPiV2 vs RPiV3

I have on several occations before written about that the 900MHz ARMv7 of RPiV2 completely outperformes the 700MHz ARMv6 of RPiV1. It is about 15 times faster, and not completely clear why the difference is so big (it is that big for JavaScript, not for C typical code).

The RPiV3 is not only 300MHz faster than the RPiV2, it is also a 64-bit ARMv8 cpu compared to the 32-bit ARMv7 cpu of RPiV2. But V3 delivers worse performance than V2.

One reason could be that the RPi does not have that much RAM, and not that fast RAM either, and that the price of 64-bit is simply not worth it. For now, I have no other idea.

References

An article about sorting in V8: https://v8.dev/blog/array-sort. Very interesting read. But I tried Node version 11 that comes with V8 version 7, and the difference was… marginal at best.

Micro service separation

Lets say we have a simple service that we can start:

$ ls
my-service my-data
$ ./my-service -d my-data -p 8080

As we interact with the service over HTTP on 8080 it stores data in the my-data folder. This may seem simplified but it is basically how any network (web, file, directory, database and so on) service works.

Micro Services

Lets say you have 4 different such services responsible for different things (like html/UI, storage, log and authentication) and they work together: well you basically have a Micro Service architecture.

All your different services can have a binary and a data folder. They can all exist in the same folder. You can start, stop and update them independently. If a service is “heavy” you can can run several instances of it. The services need to know about each other and listen to different ports, but that is just a matter of configuration and it can be automated.

Separation of micro services

While the simple approach above works (and it works surprisingly well), you may run into issues such as:

  1. you want to be sure two services can’t mess with each others data
  2. a service may be heavy and should run on separate hardware
  3. the services have common dependencies but it must be possible to update them separately (dll hell)
  4. the services may not even run on the same operating systems
  5. two services may use some resource that would cause a conflict if they shared it (say both use the same windows registry keys)
  6. different teams may be responsible for different services, and they shall neither be able to mess with each other, or blame each other

Clearly, running all services on the same computer, as the same user, in the same folder is not good enough in these scenarios. What options do we have?

Separate Hardware

Traditionally, especially in the Windows world, each service got its on computer (I refer to custom application services, clearly Windows itself comes with multiple standard services running).

In Windows, you typically install something with an install Wizard. It updates and stores stuff in different places in the system (Registry, Windows folder, Program Files and perhaps more). Multiple services may not coexist completely safely. So each get a Windows server that you backup entirely in case of disaster. This is expensive, wasteful and complicated.

Virtual Machines

VMWare was founded in 1998 and VMWare workstation was released in 1999. It changed everything, especially on Windows. Instead of having multiple computers you could have one physical computer running multiple virtual computers. Such a virtual computer “thought” it was a real computer. It needed special device drivers for the virtual hardware it ran on.

This is great. But you end up duplicating megabytes, perhaps gigabytes of system files. Installation and configuration of a computer is not trivial. Even if you automate it there are many details that need to get right. And a virtual computer may need to swap/page, and does so to what it thinks is a physical drive, but it is just a file on the host computer.

Just different users

You may run your different services in the home directories of different users. In theory that could work in Windows, but it is a quite unlikely setup.

In *NIX it would mostly work fine. You can have multiple terminals and log in as multiple users at the same time. If you are root you can easily write scripts to become any user you like, and execute whatever you want in their home directory.

Also, in *NIX most software can actually be built, installed and run in a regular user home directory. Often you just build and install as the regular user:

$ ./configure --prefix=/home/service01
$ make
$ make install

Basically, *NIX is already Micro Service-ready.

Chroot

For some purposes, running in a home directory may not be optimal. Instead, you may want to run the application in an environment where everything it needs, and nothing else, is in / (the root). There is a command called chroot that allows you to do this.

But chroot is not perfect. First it is not entirely safe (there are ways to break out of it). Second, you need to populate /bin, /lib, /etc with everything you need, and that may not be obvious. And you will only run the service in the chroot – the administrator or devops team need to access the computer normally, and they are not restricted to or don’t just see the chroot.

Containers

While all the above methods can be made to work for a microservice architecture, much effort has been made to come up with something even better, especially for deploying applications to the cloud: containers.

Containers and the tools around them focus much on development, deployment and automation. They are made for DevOps. It is cheap to configure, create, run and discard containers.

Application containers (Docker) are quite similar to a chroot, but they exist on Windows too. It is about putting everything an appliation needs, and nothing else, into the container so you can easily move, reconfigure it, duplicate it, and so on without touching your operating system itself. The issue of having exactly the right OS version, with the right patches, and the right versions of the right dependencies installed is much simplified when you can create a stable container that you can run on any machine capable of running containers.

System containers (LXC) are quite similar to a virtual machine. But while a virtual machine emulates hardware and runs a complete kernel, a system container just emulates a kernel (that may require some contemplation). It has all the advantages of a Linux virtual machine on Linux, but less of the costs.

Conclusion and Summary

Containers are popular, and for good reasons. But they are also a bit hyped. In the end of they day, you have your service code, and when you run it, it (perhaps) works on local data. That is it. You can separate, scale, isolate, secure, manage, deploy, monitor and automate this on different levels:

  1. Run your services in different folders (most light weight)
  2. Run your services as different users
  3. Run your services in chroots
  4. Create appliation containers for your services
  5. Create system containers for your services
  6. Create virtual machines for your services
  7. Get physical machines for your services (most heavy weight)

You can clearly mix and match strategies (you may very well need to).

And the price of a Raspberry Pi is so low, that even #7 can be very cheap for many purposes.