Category Archives: Uncategorized

Micro service separation

Lets say we have a simple service that we can start:

$ ls
my-service my-data
$ ./my-service -d my-data -p 8080

As we interact with the service over HTTP on 8080 it stores data in the my-data folder. This may seem simplified but it is basically how any network (web, file, directory, database and so on) service works.

Micro Services

Lets say you have 4 different such services responsible for different things (like html/UI, storage, log and authentication) and they work together: well you basically have a Micro Service architecture.

All your different services can have a binary and a data folder. They can all exist in the same folder. You can start, stop and update them independently. If a service is “heavy” you can can run several instances of it. The services need to know about each other and listen to different ports, but that is just a matter of configuration and it can be automated.

Separation of micro services

While the simple approach above works (and it works surprisingly well), you may run into issues such as:

  1. you want to be sure two services can’t mess with each others data
  2. a service may be heavy and should run on separate hardware
  3. the services have common dependencies but it must be possible to update them separately (dll hell)
  4. the services may not even run on the same operating systems
  5. two services may use some resource that would cause a conflict if they shared it (say both use the same windows registry keys)
  6. different teams may be responsible for different services, and they shall neither be able to mess with each other, or blame each other

Clearly, running all services on the same computer, as the same user, in the same folder is not good enough in these scenarios. What options do we have?

Separate Hardware

Traditionally, especially in the Windows world, each service got its on computer (I refer to custom application services, clearly Windows itself comes with multiple standard services running).

In Windows, you typically install something with an install Wizard. It updates and stores stuff in different places in the system (Registry, Windows folder, Program Files and perhaps more). Multiple services may not coexist completely safely. So each get a Windows server that you backup entirely in case of disaster. This is expensive, wasteful and complicated.

Virtual Machines

VMWare was founded in 1998 and VMWare workstation was released in 1999. It changed everything, especially on Windows. Instead of having multiple computers you could have one physical computer running multiple virtual computers. Such a virtual computer “thought” it was a real computer. It needed special device drivers for the virtual hardware it ran on.

This is great. But you end up duplicating megabytes, perhaps gigabytes of system files. Installation and configuration of a computer is not trivial. Even if you automate it there are many details that need to get right. And a virtual computer may need to swap/page, and does so to what it thinks is a physical drive, but it is just a file on the host computer.

Just different users

You may run your different services in the home directories of different users. In theory that could work in Windows, but it is a quite unlikely setup.

In *NIX it would mostly work fine. You can have multiple terminals and log in as multiple users at the same time. If you are root you can easily write scripts to become any user you like, and execute whatever you want in their home directory.

Also, in *NIX most software can actually be built, installed and run in a regular user home directory. Often you just build and install as the regular user:

$ ./configure --prefix=/home/service01
$ make
$ make install

Basically, *NIX is already Micro Service-ready.

Chroot

For some purposes, running in a home directory may not be optimal. Instead, you may want to run the application in an environment where everything it needs, and nothing else, is in / (the root). There is a command called chroot that allows you to do this.

But chroot is not perfect. First it is not entirely safe (there are ways to break out of it). Second, you need to populate /bin, /lib, /etc with everything you need, and that may not be obvious. And you will only run the service in the chroot – the administrator or devops team need to access the computer normally, and they are not restricted to or don’t just see the chroot.

Containers

While all the above methods can be made to work for a microservice architecture, much effort has been made to come up with something even better, especially for deploying applications to the cloud: containers.

Containers and the tools around them focus much on development, deployment and automation. They are made for DevOps. It is cheap to configure, create, run and discard containers.

Application containers (Docker) are quite similar to a chroot, but they exist on Windows too. It is about putting everything an appliation needs, and nothing else, into the container so you can easily move, reconfigure it, duplicate it, and so on without touching your operating system itself. The issue of having exactly the right OS version, with the right patches, and the right versions of the right dependencies installed is much simplified when you can create a stable container that you can run on any machine capable of running containers.

System containers (LXC) are quite similar to a virtual machine. But while a virtual machine emulates hardware and runs a complete kernel, a system container just emulates a kernel (that may require some contemplation). It has all the advantages of a Linux virtual machine on Linux, but less of the costs.

Conclusion and Summary

Containers are popular, and for good reasons. But they are also a bit hyped. In the end of they day, you have your service code, and when you run it, it (perhaps) works on local data. That is it. You can separate, scale, isolate, secure, manage, deploy, monitor and automate this on different levels:

  1. Run your services in different folders (most light weight)
  2. Run your services as different users
  3. Run your services in chroots
  4. Create appliation containers for your services
  5. Create system containers for your services
  6. Create virtual machines for your services
  7. Get physical machines for your services (most heavy weight)

You can clearly mix and match strategies (you may very well need to).

And the price of a Raspberry Pi is so low, that even #7 can be very cheap for many purposes.

Doublethink 2019

Doublethink is: the acceptance of contrary opinions or beliefs at the same time, especially as a result of political indoctrination

As I listen to politicians and influencers in 2019 there are very many contradictory things that they seem to believe, and that I am expected to also believe.

I publish the list below not to have a discussion about every single item. Perhaps you agree about some and think I got others wrong (which is of course possible). I publish this list to show that it is common that politicians and other influencers hold and express contradictory opinions at the same time, and that we are supposed to accept and follow this.

I am convinced that each and every one of us should ideally hold zero contradictory beliefs at any given time. So even if you only agree about some of my items, we should be able to agree that we have a systemic problem with doublethink.

Much of this doublethink happens in the name of Political Correctness. Welcome to 1984.

We are supposed to believe…

that Nazism is worse than Communism despite the number of dead tell a very different story

that taxation is useful to restrict things like tobacco, alcohol and emissions, but not causing any harm to real businesses and employment

that they care about reducing CO2-emissions, despite they don’t want any nuclear power

that nuclear power is very dangerous, despite coal cause more deaths monthly on the planet than nuclear accidents ever caused

that the unequal distribution of power and money under capitalism will disappear when even more power is centralized with politicians to control not only government, but also companies under socialism

that Israel is a terrible apartheid state, despite there is a significant minority of arabs and muslims in Israel, enjoying a good living standard and having more democracy and human rights than arabs and muslims anywhere else in the middle east

that they stand up against all racism – nazism most of all, except when it comes to antisemitism among people from the middle east

that the hijab (or variants of same purpose) is a symbol of womens freedom when many women in and from the middle east tell a very different story

that the white European man brought slavery and slave trade to the world, when the arab slave trade had been going on for 1000 years (both from Africa and Europe) and the market and infrastructure was already there in Africa when Europeans for a few hundred years did slave trade (a horrible thing)

that capitalism is the worst threat to the environment, when the atrocities when it comes to destroying the environment in the Soviet Union are unprecedented

that Europeans brought all cruelty and misery to the american continent, despite human sacrifice and very authoritarian societies were massively widespread among the natives before the continent was discovered (the Europeans also did horrible things)

that socialism is in strong opposition to nazism, when clearly nazism is a form of socialism, and Marx coined the term “the jewish question” himself

that the left is against hatred and violence, when their own rhetoric against capitalists, white men and other opponents is very aggressive

that all criminals deserve another chance and can be good citizens and humans, despite the fact that psychopathy exists there is no known working treatment

that anti-fascism is a democratic mindset, however the Berlin Wall was stated to be an anti-fascist measurement and it had very little to do with fascism and everything to do with oppressing people in a socialist state

that capitalism is bad to people, despite it is practically unheard of people leaving free market nations for socialist nations, while millions of people have fled socialism

that Islam is a religion like others, and a religion of peace, despite it has been at war with itself, at war with its own people, at war with its women, and at war with its neighbors since the days of Mohammed

that the free market is to blame for lack of housing, when the housing market is one of the few markets that are heavily regulated, and while most free markets show no lack of options for relatively poor people: cheap clothes, food, fast food, entertainment, air travel, furniture are provided by global brands like HM, MacDonalds, Lidl, Netflix, Ryanair and IKEA (how about letting those companies work with lack of housing?)

that ISIS is not representative of Islam despite they just follow the footsteps of Mohammed which is clearly a virtue in Islam

that Islamism is a separate thing from Islam, despite the Koran considers the law, Sharia, central to the life of a muslim, and despite Erdogan has said that the separation of Islam and Islamism is an ugly western construction

that Islam is not a threat to freedom, democracy and human rights, despite there are virtually no muslim majority countries that are free, democratic and respect human rights

that Mohammed is a respectable prophet, despite he was one of the most cruel humans in history

that the environmentalists care about science when it comes to human caused climate change, but when it comes to GMO and studies finding disadvantages with organic farming they ignore scientific results

that later muslim aggressions have been a reasonable response to the crusades, when the crusades are minor isolated events compared to the many centuries of war, imperialism and slavery that muslims had brought upon the Middle East, Europe and Africa before the crusades

that women are structurally and systematically oppressed by men in the west, despite women live 5 years longer, spend less than 20% time in prison, commit far fewer suicides, experience far few workplace accidents, and generally have better mental health and more friends than men do

that feminists care about women who have a hard time, despite they don’t listen at all to what prostitutes say or want

that very few women go into tech jobs because of discrimination and sexist attitudes, despite women a century ago were restricted from most careers and (very impressively) managed to make it into medicine, teaching, law, economics, politics, and so on (feminists today are implying that tech men are worse than other men which is speculative, and also injury-on-insult since women tend to reject nerds in the first place)

that democratic socialism has anything to do with democracy, despite all socialist states have ended up seriously authoritarian, many of them with the word democratic in their names

that the west, USA and modern Europe, are the only evil empires, despite the history of the world is the history of empires (chineese, persian, roman, muslim, aztec, ottoman, russian, and so on) that conquered nations and enslaved people.

that we are very certain about the catastrophic effects of climate change, despite previous threats: peek oil, HIV, the ozone layer, nuclear power, nuclear weapons, world wars, new ice age, turned out to be exaggerated (in fact humans have always thought they live just before the end of time)

that men and women are practically the same, when the biological differences (both physical, psychological, and when it comes to abilities and talents) are clearly significant (and this is also proven scientifically in numerous studies)

that equality of outcome is desirable, when the way to achieve it necessarily is very authoritarian and restrictive to individual freedom and choice

that anxiety of climate change is a virtue, while anxiety of terrorism and criminality is a sin

that nationalism is inherently evil, despite no welfare state has ever existed beyond the scope of a nation

that women are for all purposes equal to men, except when they do horrible things like joining ISIS, then they are to be understood as passive victims and they can play the woman-card

that the anti-racist left are anti racist at all, when they often hold and spread very antisemitic attitudes and support antisemitic (muslim) organisations

that feminism is about fighting the patriarchy, when it is mostly indifferent to outright oppression of women in/from non-western cultures

that feminists have valid reasons to embrace people like Linda Sarsour (an advocate of Sharia in USA), when they reject people like Ayaan Hirsi Ali (an ex-muslim standing up for human rights)

that political correctness is more important (when it comes to respect of minorities and other cultures) than the words of Martin Luther King: History will have to record that the greatest tragedy of this period of social transition was not the strident clamor of the bad people, but the appalling silence of the good people.

that cows, chicken and pigs will be happily liberated by vegetarians, when the animals will instead be practically extinguished

that christianity and islam are mostly the same, despite christianity have been reformed on several occasions and christians mostly accept and respect different interpretations of their religion, while Islam is inherently resilient to reformation, has failed to reform but rather tend to fall back into fundamentalism, and muslims mostly don’t accept the interpretation of other muslims.

that socialism is a fine idea, which it perhaps was just like Thalidomide (Neurosedyn in Swedish), but after seeing the catastrophic effects of applying it, it is nothing less than heartless, cruel and evil to make another attempt (but socialism has, unlike capitalism, science and most other activities, absolutely no feedback loop so it is applied over and over again with the same horrible result – only the propaganda is refined)

intolerant extremists to the right (fascists and nationalists) are considered an absolute evil and threat and they must not even be talked to, but the returning murderers and rapists of ISIS are supposed to be respectfully integrated (getting more support than any other refugee or ISIS victim could dream of)

that the idea of socialism, which ultimately is to reward those who do wrong and punish those who do right, can ever lead to a good society despite any workplace, relationship, raising of children or keeping pets would fail horribly following the same inferior idea (I dare you to start encouraging bad behavior and punishing good behavior with people or pets around you)

that the problem is that some people are very rich, when in fact poverty globally is being effectively pushed back (except mostly in a few unfortunate authoritarian and/or socialist countries); caring about improving the situation for the poor is emphatic, caring about taking from the rich is only jealousy (and shows ignorance of basic economics)

that gender theory and political correctness are good things, when they are just the racial biology and racial hygiene of our time (the purpose is to classify and group people based on origin and physical attributes, and then to create a conforming population for the utopia, not to respect human rights or diversity)

that women have absolute right to their own bodies (abortion now legal up to day of birth in some places), unless they want to make profit on sex or sexuality

that there is anything reasonable about transgender women competing with other women in elite sports, despite the obvious fact that they have a massive unfair advantage

that gender is a social construction, although we are born with a sexual orientation

that Che Guevara is a fighter of freedom and equality for the left, when he was a homophopic, rasist, chauvinistic murderer

that while abortion is an absolute right (my body my choice), surrogacy is unacceptable

that criticism of Islam is Islamophobic, while criticism of Israel is not antisemitic

In Sweden, we are supposed to believe…

that it will reduce emissions to build high speed railroads to the north to replace a few daily flights (emissions from construction will exceed flights for foreseeable future)

that criminality can not be reduced by longer prison sentences, despite criminals answer that they did not even recognize they got a punishment at all for their crime

that there is evidence longer prison sentences don’t work to reduce crime, when it has never been tried in modern time in Sweden (references are usually to studies from the US where already draconian prison sentences were made even longer, which obviously says very little about what happens if criminals start being punished at all, or from a very low level)

that socialism made Sweden rich, when in fact the foundation of wealth in Sweden was built before the second world war by a low-tax, laissez-faire economy (that later came to slow down as taxes rises under socialism)

that socialism brought social security to the working class in Sweden, despite the workers organized it themselves as private insurances (that the socialist government later nationalized and took credit for)

that we have world class healthcare, despite people who have been living abroad are often shocked with Swedish health care

that they care about the well-being of animals, when they want wolves to hunt freely causing massive cruelty to wild and domestic animals wherever they come

that they care about vulnerable people, when they are mostly unwilling to protect ordinary people – even abused women – from criminals with a long record of offences

that the left (Swedish V) is inherently against racism, when they were the only party who did not oppose Hitler at the time Nazi Germany conquered Norway and Denmark.

that the left (Swedish V) is democratic and against imperialism, when they were the only party not support the Polish Solidarity movement at the time

that restricting access to weapons to honest citizens and hunters even more will make the country any safer, despite these legal weapons are virtually unheard of in crime cases (except rare self defense cases)

that they care about ethnic diversity and indigenous people, when Sweden blatantly ignores UN Declaration of the Rights of Indigenous People when it comes to the samis.

About Venezuela, we are supposed to believe…

that USA is to blame for the catastrophe in Venezuela, despite Russia and China has much more dealings with the socialist regime

that it is about USA wanting the oil, when USA is rather self sustained

that Russia has some moral ground when they object to interfering in the internal affairs of other nations, like Venezuela, given their recent history in Georgia, Ukraine and Chechnya, and a longer tradition of occupying neighbour states

that they care about potential climate refugees in a distant futures, despite they don’t give a shit about refugees of socialistic Venezuela

that the crimes the socialist regime in Venezuela commit against human rights and its own population (censorship, torture, oppression, socialistic market regulation and currency regulations resulting in lack of everything, blocking humanitarian aid, ignoring the constitution, establishing collectivos to harass and brutalize people, and so on) are somehow reasonable given a narrative that USA is working against the chavism and socialist movement

that the socialist regime had inherently good intentions towards its own population when it ran welfare programs in the past, despite the same regime now is completely indifferent to the suffering of its own people, even rejecting international emergency humanitarian aid (the regime only cares about its loyal supporters, everyone else who don’t support the socialistic cause can suffer)

that things are mostly fine in Venezuela despite UNHCR tells that more than 3 million people have fled the country in a few years (3 million people had left GDR before the Berlin Wall was built to prevent it)

ArchLinux on RPi with USD Harddrive

I have found that one of the weakest parts of a Raspberry Pi server or workstation is the SD card: it is slow and it will break sooner rather than later. There may be industrial SD cards or better SD cards, but a good old USB hard drive is just better.

With RPi v3 it shall be possible to boot straight off a USB drive! That sounded great so I got a brand new RPi v3 B+, a USB hard drive, and I installed ArchLinux on the hard drive, just as if it was a memory card. Fail. That did not work (with ArchLinux, Raspbian may be another story).

But there are levels of pain:

  1. All SD-card
  2. SD-card, but /home on USB harddrive
  3. USB harddrive, but /boot on SD-card
  4. All USB harddrive

I decided to try #3.

It turns out that when the RPi boots it runs u-boot (its like the BIOS of RPi, and many other embedded devices). At one point u-boot reads boot.scr (from the first VFAT partition of the SD card). It had the lines:

part uuid ${devtype} ${devnum}:2 uuid

setenv bootargs console=ttyS1,115200 console=tty0 root=PARTUUID=${uuid} rw rootwait smsc95xx.macaddr="${usbethaddr}"

I figured that I could do this instead:

# part uuid ${devtype} ${devnum}:2 uuid

setenv bootargs console=ttyS1,115200 console=tty0 root=/dev/sda2 rw rootwait smsc95xx.macaddr="${usbethaddr}"

However, boot.scr has a checksum so you cant just edit it. But it tells you what to do: run ./mkscr. But it is dependent on mkimage, so the procedure is:

  1. Install uboot tools
    1. ARCH: pacman -S uboot-tools
    2. Ubuntu/Debian: apt-get install u-boot-tools
  2. Edit boot.txt (not boot.scr) to your liking
  3. Run: ./mkscr

Now only /boot is on SD-card. That is quite ok with me. There is very little I/O to boot so the SD-card should survive. If I want to I can make a regular simple backup by simple file copy of /boot to a zip-file or something, and just restore that zip-file to any SD-card.

There seems to be no need to edit anything else (like fstab).

Well, the bad thing is it did not work out 100% as I hoped. The good thing is that this should work with any RPi, not just the RPi v3 that supports USB boot.

Best Train Simulator 2019

I have some personal enthusiasm for trains, and last years part of that has been playing Train Simulator on PC. That is the game that used to be called Railworks and that currently is named Train Simulator 2019. While I have spent much time with it I also have mixed feelings.

In 2019 there are two alternatives to TS2019 that I have tested and that I will write about: Train Sim World and Trainz Railroad Simulator 2019.

My experience with Dovetail Train Simulator (2019)

I got Train Simulator because I wanted to try to drive trains. The game has developed over the years but there are some annoyances.

The game has some quirks and bugs. The physics, engines, wagons, signals, AI and scenario conditions sometimes don’t work in a way you would expect.

The game is also rather unforgiving. One little mistake can ruin a scenario so you can’t even continue. If that was passing a red light, ok. But sometimes I am just a little late, a little early, I connected or disconnected the wrong wagon, I went into the wrong siding or something like that.

The combination of bugs and being unforgiving is rather frustrating. When there is a little inperfection in the game I can perhaps accept the lack of good simulation experience, but if it ruins the scenario completely, it is worse.

The game has a competitive aspect (it is a game) where you drive scenarios and get scores. This is particularly unforgiving. Decouple a wagon and for some reason (bug?) I get “operational error”, being penalised with -750 points (1000 is max), and I have no choice but aborting the scenario. Also, speeding is penalised heavily. This is annoying for two reasons: first the time table is often ridiculously tight, second it is not uncommon that maximum allowed speed changes unexpectedly.

You can read about the outdated graphics of TS2019 and that is true, but it does not ruin my experience. You can read about all the expensive DLC, but that is your choice (I bought some, but most everything on sales). What I find more annoying is that I buy a nice piece of DLC and it comes with very few scenarios. That is where the (Steam) workshop comes in and there are quite many scenarios (of varying quality) do download.

I found that creating scenarios was often more fun than driving myself and I have contributed some 44 scenarios on Steam Workshop. If driving is quirky, creating scenarios is kind of black magic (the problem is I need to test it, and when it fails after 40 minutes, I need to guess whats wrong and drive again for 40 minutes until I know if it works – a horrible development and debugging experience).

It seems to me it would be very possible to deliver a better Train Simulator game!

On Realism

It is easy to talk about realism. But is it really what we want. My experience…

  • Some routes allow for long eventless sessions. That is the realistic truth about driving a train, but how entertaining is it?
  • A real challenge when driving a train is breaking and planning your breaking. The weight and length of the train matters, as long as other factors. In the real world a train engineer makes calculations about breaking distances. They are not going to be driving a new train, with unknown weight, on a new track on a tight time schedule. Yet in a train simulator this is what we do, because we want (much) variation (it is a somewhat boring game anyways).
  • A real engineer knows the line well, and has special physical documentation about the line available. And he has studied this before. You don’t do that in a train simulator.
  • A real engineer spends much time checking things like breaks and wheels. And there is much waiting.
  • You can have a realistic “regulator”, that you can operate in the locomotive cab. That will look realistic in one way. But a real engineer would not point and look at it with a mouse, he just happens to have his hand there in the first place. User-friendly, where man-machine becomes one, is good simulation to me.
  • Real(istic) timetables are good, but not when it is almost impossible to arrive on time in the simulator.

My point is that I don’t want a realistic simulator. I want a simulator that gives me the feeling I am driving a train. And I want the time I spend with my computer to be more eventful, entertaining and challenging that the average work hour of a train engineer. And also somewhat more forgiving and I want support with things that are easier in the real world.

Train Sim World

Train Sim World is produced by the same company (Dovetail) as Train Simulator 2019. It appears they thought of it as a replacement for Train Simulator 2019, but it also appears that for now the games exist side by side. It is not clear that Train Sim World will ever replace, or even survive, Train Simulator 2019.

The good:

  • It looks (the graphics) better than the alternatives.
  • It may be the most “polished” option (also available for Playstation and Xbox, which gives you a hint).
  • If you get a “package” at discounted price on Steam (EUR25 for 4 routes) it is quite good value.

The bad:

  • It does not look that good; it is still computer graphics with obvious artifacts and problems. Also, the sound is not too convincing and the surroundings are pretty dead.
  • Walking around (in the scenarios) does not appeal to me, and it is not well made enough to add to the realism of the game.
  • Menus are a bit messy.
  • Quite limited number of scenarios, but plenty of “services”, but I think that contributes to (even) less events, action and storytelling.
  • The routes seem small, and very little action or room outside the mainline (like very linear).
  • Occational glitches like “what do I do now”, “what happens next” or “how do I do that”? (driving a service, I was done, told to get off, the train drove away by itself with no visible driver or no comments, and then nothing… had to just quit).
  • It lacks something. Like its not a bit dirty, noisy and rough… but just too smooth and clean.
  • So far, no possibility for user generated content. It is promised, and it will be based on Unreal, so it seems to be very technically demanding. I myself would prefer to be able to make scenarios with a story easily, without changing anything about the route or the other assets at all.
  • Unreal (which is to thank for the better graphics) seems to be a more complex (expensive) development environment, and perhaps this will limit in the future the availability or routes and assets, and make the price high (pure speculation).

I did give Train Sim World a first try, wrote a very negative review, refunded it, but after a few weeks I gave it a new try, and now I have a more balanced opinion about it.

Trainz Railroad Simulator 2019

Years ago I obviously did research and opted for Train Simulator rather than Trainz. Now that I was a bit disappointed with Train Simulator and rather disappointed with Train Sim World I felt I had to give Trainz 2019 a try.

My expectations based on marketing and what I read was:

  • Better graphics than Train Simulator, but perhaps not as good as Train Sim World.
  • More creator-, community- and sharing oriented (which appealed to my preference to making scenarios).
  • It’s a railroad simulator, rather than a train driver simulator.

I must say right away that I am quite disappointed. I ended up paying EUR 70 for Trainz, and EUR 25 for Train Sim World, and that does not reflect the value of what I got.

Download Station

Trainz comes with “its own Steam Workshop, Download Station”. This is the worst part of it. Hundreds of assests, organised alphabetically, with virtually no filtering and no community/feedback/rating function. Unless I completely missed something, this is shit. My use case is that I want to see if someone created a nice 30 min session for one of the premium routes that came with my purchase (and that has no extra dependencies). Trainz seems to live in the world where people download zip-files from ftp-servers and spend the effort of maintaining their virtual asset library like the stock portfolio. I am tempted to make a few sessions myself, and sharing them here, on my blog, but why?

Graphics

There is something idyllic, picturesque, beautiful and friendly about Trainz that is missing in Train Simulator and Train Sim World. There are gorgeous screenshots from Trainz out there. But when it comes to actual game performance on my actual computer (a NUC Hades Canyon) Trainz is the worst. I have been spending not so little time optimizing my graphics settings (and there are many settings to play with).

Quality

To my disappointment the routes come with quite few sessions. The beatiful route from Edinburg to Aberdeen (perhaps just to Dundee) has two sessions: a passenger service with the same Deltic locomotive going both ways. These two scenarios both take 1h30min each to drive. And the one I did try did not work in CAB (realistic) drive mode, because for some reason the Deltic can not pull those wagons with any speed whatsoever. Isn’t it reasonable to expect when a new EUR 70 release is made after 7 years, that the sessions are tested at least once, and working?

Then there was another beautiful session on the Cornish mainline where a 2MT steam locomotive pulls ~25 freight wagons and it just can’t make it up the grades. I asked in the forum and I had managed to get further than most people, but the suggestion was to just try another locomotive (edit the session). Why release a session with the wrong locomotive in the first place?

If driving steam locomotives in realistic mode can be a challenge in Train Simulator (often a frustrating one), in Trainz it feels… not realistic. Perhaps I need more practice, but it is very… unsmooth.

Other things

There is no support for a Gamepad (although I found a little software called AntiMicro) which works decently well for my purposes.

I really miss the look-out-throw-the-side-window camera view.

I appreciate that I can see the status of the next signal in the HUD.

When I have completed a session it does not remember (marked as completed) so I made my own list

A good thing about Trainz is that it is more forgiving than Train Simulator. I ran out of boiler pressure, but then I could switch to simple driving mode and at least complete the session.

I get the feeling that for people who already own and love the old Trainz this is an upgrade. But for a new player it is a rough experience.

Conclusion and recommendation

Unfortunately I think none of the games I have written about live up to the expections you should allow yourself to have in 2019. And I am not aware of a better game in the genre.

Clearly this genre appeals to enthusiasts who want to make their own assets and modify the game, and clearly Train Simulator and Trainz are based on old technology that have not aged too well (and people are reluctant to abandon their assets). Train Sim World, being based on Unreal, has not been able to deliver a workshop- or sharing-experience at all, yet.

If you are curious about how it is to drive a train, get Train Sim World (and an Xbox controller if you get it on PC, I know nothing about the Playstation/Xbox experience). Sit comfortably, turn up the volume, have some coffee (or whatever you drink) and do your best to enjoy the experience. Spend time with the tutorials and dont get too frustrated if you get stuck.

If you want to have your own digital train layout, and play with it (dispatch and control multiple trains), get Trainz, and make sure to have a powerful enough computer.

If you think that Steam workshop is a nice idea where you can share scenarios (and other assets) and communicate with other people about them get Train Simulator 2019. Cost/price aside, there are very many routes (and extra locomotives) available for Train Simulator 2019.

Train Simulator 2019 now supports 64-bit mode. Technically its not… hot… but it is being improved. Train Sim World looks better, but it is not that much better. Honestly, folks who make a living reviewing computer games say: “TS2019 looks so old, but TSW is built on Unreal like all the other cool games, much better.” But for your total train simulation experience, the difference is… marginal.

I would not be too surprised if the Train Sim World Editor never happens. If it is released I would not be surprised if it is too complex and a critical portion of contributors and enthusiasts never switch. The advice to enthusiasts to “Download the UE4 Editor from Epic and start learning”, I am sceptical about it. I doubt I will contribute scenarios if I have to get into a real 3D studio to place some trains and make some timetables/rules.

I would hope that Trainz gets a real workshop experience where you can easily share assets in a social way and where you don’t need to worry too much about dependencies. And I would hope that Trainz manages to polish their game, test it properly, and provide a solid graphics experience.


Trainz Log

I decided to try Trainz (now that 2019 is out) and I think it is useful to keep track of my progress, so I will do it here.

Gamepad

I prefer to drive my train with a gamepad. I found a little free software called AntiMicro which allowed me to map my Xbox gamepad to relevant Trainz keys. It was easy and I recommend it.

Graphics

My computer is a NUC Hades Canyon. It has an Intel Core i7 CPU, AMD Radeon RX Vega M GPU, 16GB or RAM and 512GB SSD. This computer can not at all play Trainz 2019 in “Ultra”, instead I have to tune down the performance settings quite much. I have found (and I may change opinion when I have experimented more) that I want to run Trainz in full resolution (1920×1080) and I think the lowest anti aliasing (2x) makes most sense. I allow myself to use “Clutter + Turf FX” and High details. Different routes are supposed to be demanding on the GPU.

Cornish Mainline and Branches

A trip to Falmouth: Completed with 2 stars.
Freight Delivery: This session is tricky and I started a thread on the Trainz forum. I dont think the 2MT is capable of pulling 23 wagons all the way, I got completely stuck after 7 miles in a 2%+ grade. I modified the session and used a 4200 tank locomotive instead but I got two new problems: 1) AI trains going in the other direction all stand still, 2) When I have finally arrived the session never ends (or continues). Perhaps I broke something when I modified it.
Helston freight run: Completed with 5 stars.
Helston passenger run: Completed with 4 stars.
Mainline passenger service: Completed with 1 star (!), on time, gorgeous!
St Ives passenger run: Completed with 5 stars.

ECML Edinburgh – Dundee

09-10 Dundee – Kings Cross: Completed with 5 starts (but can’t drive in realistic CAB mode)

Sebino Lake

Maintenance Service: Completed with 5 stars (in DCC, can’t get the train rolling in Cab mode)

Lambda Functions considered Harmful

Decades ago engineers wrote computer programs in ways that modern programmers scorn at. We learn that functions were long, global variables were used frequently and changed everywhere, variable naming was poor and gotos jumped across the program in ways that were impossible to understand. It was all harmful.

Elsewhere matematicians were improving on Lisp and functional programming was developed: pure, stateless, provable code focusing on what to do rather than how to do it. Functions became first class citizens and they could even be anonymous lambda functions.

Despite the apparent conflict between object oriented, functional and imperative programming there are some universally good things:

  • Functions that are not too long
  • Functions that do one thing well
  • Functions that have no side effects
  • Functions that can be tested, and that also are tested
  • Functions that can be reused, perhaps even being general
  • Functions and variables that are clearly named

So, how are we doing?

Comparing different styles
I read code and I talk to people who have different opinions about what is good and bad code. I decided to implement the same thing following different principles and discuss the different options. I particularly want to explore different ways to do functional programming.

My language of choice is JavaScript because it allows different styles, it requires quite little code to be written, and many people should be able to read it.

My artificial problem is that I have two arrays of N numbers. One number from each array can be added in NxN different ways. How many of these are prime? That is, for N=2, if I have [10,15] and [2,5] i get [12,15,17,20] of which one number (17) is prime. In all code below I decide if a number is prime in the same simple way.

Old imperative style (imperative)
The old imperative style would use variables and loops. If I had goto in JavaScript I would use goto instead of setting a variable (p) before I break out of the inner loop. This code allows for nothing to be tested nor reused, although the function itself is testable, reusable and pure (for practical purposes and correct input, just as all the other examples).

  const primecount = (a1,a2) => {
    let i, j;
    let d, n, p;
    let retval = 0;


    for ( i=0 ; i<a1.length ; i++ ) {
      for ( j=0 ; j<a2.length ; j++ ) {
        n = a1[i] + a2[j];
        p = 1;
        for ( d=2 ; d*d<=n ; d++ ) {
          if ( 0 === n % d ) {
            p = 0;
            break;
          }
        }
        retval += p;
      }
    }
    return retval;
  }

Functional style with lambda-functions (lambda)
The functional programming equivalent would look like the below code. I have focused on avoiding declaring variables (which would lead to a mutable state) and rather using the higher order function reduce to iterate over the two lists. This code also allows for no parts to be tested or reused. In a few lines of code there are three unnamed functions, none of them trivial.

  const primecount = (a1,a2) => {
    return a1.reduce((sum1,a1val) => {
      return sum1 + a2.reduce((sum2,a2val) => {
        return sum2 + ((n) => {
          for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
          return 1;
        })(a1val+a2val);
      }, 0);
    }, 0);
  };

Imperative style with separate test function (imperative_alt)
The imperative code can be improved by breaking out the prime test function. The advantage is clearly that the prime function can be modified in a more clean way, and it can be tested and reused. Also note that the usefulness of goto disappeared because return fulfills the same task.

  const is_prime = (n) => {
    for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
    return 1;
  };

  const primecount = (a1,a2) => {
    let retval = 0;
    for ( let i=0 ; i<a1.length ; i++ )
      for ( let j=0 ; j<a2.length ; j++ )
        retval += is_prime(a1[i] + a2[j]);
    return retval;
  };

  const test = () => {
    if ( 1 !== is_prime(19) ) throw new Error('is_prime(19) failed');
  };

Functional style with lambda and separate test function (lambda_alt)
In the same way, the reduce+lambda-code can be improved by breaking out the prime test function. That function, but nothing else, is now testable and reausable.

  const is_prime = (n) => {
    for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
    return 1;
  };

  const primecount = (a1,a2) => {
    return a1.reduce((sum1,a1val) => {
      return sum1 + a2.reduce((sum2,a2val) => {
        return sum2 + is_prime(a1val+a2val);
      }, 0);
    }, 0);
  };

  const test = () => {
    if ( 1 !== is_prime(19) ) throw new Error('is_prime(19) failed');
  };

I think I can do better than any of the four above examples.

Functional style with reduce and named functions (reducer)
I don’t need to feed anonymous functions to reduce: I can give it named, testable and reusable functions instead. Now a challenge with reduce is that it is not very intuitive. filter can be used with any has* or is* function that you may already have. map can be used with any x_to_y function or some get_x_from_y getter or reader function that are also often useful. sort requires a cmpAB function. But reduce? I decided to name the below functions that are used with reduce reducer_*. It works quite nice. The first one reducer_count_primes simply counts primes in a list. That is (re)useful, testable all in itself. The next function reducer_count_primes_for_offset is less likely to be generally reused (with offset=1 it considers 12+1 to be prime, but 17+1 is not), but it makes sense and it can be tested. Doing the same trick one more time with reducer_count_primes_for_offset_array and we are done. These functions may not be reused. But they can be tested and that is often a great advantage during development. You can build up your program part by part and every step is a little more potent but still completely pure and testable (I remember this from my Haskell course long ago). This is how to solve hard problems using test driven development and to have all tests in place when you are done.

  const is_prime = (n) => {
    for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
    return 1;
  };

  const reducer_count_primes = (s,n) => {
    return s + is_prime(n);
  };

  const reducer_count_primes_for_offset = (o) => {
    return (s,n) => { return reducer_count_primes(s,o+n); };
  };

  const reducer_count_primes_for_offset_array = (a) => {
    return (s,b) => { return s + a.reduce(reducer_count_primes_for_offset(b), 0); };
  };

  const primecount = (a1,a2) => {
    return a1.reduce(reducer_count_primes_for_offset_array(a2), 0);
  };

  const test = () => {
    if ( 1 !== [12,13,14].reduce(reducer_count_primes, 0) )
      throw new Error('reducer_count_primes failed');
    if ( 1 !== [9,10,11].reduce(reducer_count_primes_for_offset(3), 0) )
      throw new Error('reducer_count_primes_for_offset failed');
    if ( 2 !== [2,5].reduce(reducer_count_primes_for_offset_array([8,15]),0) )
      throw new Error('reducer_count_primes_for_offset_array failed');
  };

Using recursion (recursive)
Personally I like recursion. I think it is easier to use than reduce, and it is great for acync code. The bad thing with recursion is that your stack will eventually get full (if you dont know what I mean, try my code – available below) for recursion depths that are far from unrealistic.  My problem can be solved in the same step by step test driven way using recursion.

  const is_prime = (n) => {
    for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
    return 1;
  };

  const primes_for_offset = (a,o,i=0) => {
    if ( i === a.length )
      return 0;
    else
      return is_prime(a[i]+o) + primes_for_offset(a,o,i+1);
  }

  const primes_for_offsets = (a,oa,i=0) => {
    if ( i === oa.length )
      return 0;
    else
      return primes_for_offset(a,oa[i]) + primes_for_offsets(a,oa,i+1);
  }

  const primecount = (a1,a2) => {
    return primes_for_offsets(a1,a2);
  };

  const test = () => {
    if ( 2 !== primes_for_offset([15,16,17],2) )
      throw new Error('primes_with_offset failed');
  };

Custom Higher Order Function (custom_higher_order)
Clearly reduce is not a perfect fit for my problem since I need to nest it. What if I had a reduce-like function that produced the sum of all NxN possible pairs from two arrays, given a custom value function? Well that would be quite great and it is not particularly hard either. In my opinion this is a very functional approach (despite its implemented with for-loops). All the functions written are independently reusable in a way not seen in the other examples. The problem with higher order functions is that they are pretty abstract, so they are hard to name, and they need to be general enough to ever be reused for practical purposes. Nevertheless, if I see it right away, I can do it. But I don’t spend time inventing generic stuff instead of solving the actual problem at hand.

  const is_prime = (n) => {
    for ( let d=2 ; d*d<=n ; d++ ) if ( 0 === n % d ) return 0;
    return 1;
  };

  const combination_is_prime = (a,b) => {
    return is_prime(a+b);
  };

  const sum_of_combinations = (a1,a2,f) => {
    let retval = 0;
    for ( let i=0 ; i<a1.length ; i++ )
      for ( let j=0 ; j<a2.length ; j++ )
        retval += f(a1[i],a2[j]);
    return retval;
  };

  const primecount = (a1,a2) => {
    return sum_of_combinations(a1,a2,combination_is_prime);
  };

  const test = () => {
    if ( 1 !== is_prime(19) )
      throw new Error('is_prime(19) failed');
    if ( 0 !== combination_is_prime(5,7) )
       throw new Error('combination_is_prime(5,7) failed');
    if ( 1 !== sum_of_combinations([5,7],[7,9],(a,b)=> { return a===b; }) )
       throw new Error('sum_of_combinations failed');
  };

Lambda Functions considered harmful?
Just as there are many bad and some good applications for goto, there are both good and bad uses for lambdas.

I actually dont know if you – the reader – agrees with me that the second example (lambda) offers no real improvement to the first example (imperative). On the contrary, it is arguably a more complex thing conceptually to nest anonymous functions than to nest for loops. I may have done the lambda-example wrong, but there is much code out there, written in that style.

I think the beauty of functional programming is the testable and reusable aspects, among other things. Long, or even nested, lambda functions offer no improvement over old spaghetti code there.

All the code and performance
You can download my code and run it using any recent version of Node.js:

$ node functional-styles-1.js 1000

The argument (1000) is N, and if you double N execution time shall quadruple. I did some benchmarks and your results my vary depending on plenty of things. The below figures are just one run for N=3000, but nesting reduce clearly comes at a cost. As always, if what you do inside reduce is quite expensive the overhead is negligable. But using reduce (or any of the built in higher order functions) for the innermost and tightest loop is wasteful.

 834 ms  : imperative
874 ms  : custom_higher_order
890 ms  : recursive
896 ms  : imperative_alt
1015 ms  : reducer
1018 ms  : lambda_alt
1109 ms  : lambda

Other findings on this topic
Functional Programming Sucks


Fix broken Marshall Stanmore

First I want to be clear, this post is about fixing a broken Marshall Stanmore speaker by turning it into an active loudspeaker. It is not about repairing it to its original functionality.

My Marshall Stanmore died after about two years of little use. One day it simply did not turn on. Completely dead. It seems to be a common fate of those loudspeakers and there seems to be no easy fix. I opened up the loudspeaker and quite quickly decided that I would not be able to repair it.

I felt very certain that the loudspeaker elements themselves were not broken. The loudspeaker looks and sounds quite good and it is against my nature to just throw such a thing away. So I started looking for ways to make a working active loudspeaker of it (allowing to use it with an iPhone or as a computer speaker). Since I thought this was a fun project I was willing to put some time and effort into it. But a brand new Marshall Stanmore is 200 Euros so the fix had to be significantly cheaper than that.

2.1
The Stanmore is a 2.1-loudspeaker. It has two tweeters and one woofer. The cutoff frequency is 2500Hz meaning that the tweeters are responsible for higher than 2500Hz frequencies and the woofer for the lower frequencies. There are different ways to properly produce 2.1 audio from a 2.0 signal. If I remember correctly the tweeters are rated at 2x20W and the woofer at 40W. I don’t know the impendance (Ohm).

The thing not to do
It is not a good idea to just simply connect L+R and connect it to the woofer. Regardless whether you do this before or after the amplifier you will drive current into components that are only supposed to produce a signal and this can destroy your equipment (your smartphone or computer pre-amp, or your amplifier).

Cutoff filters
There are special cutoff filters to split a signal into a lower and a higher part. I looked into this first, but it seemed a bit to advanced (expensive and complicated) for my project, and the problem with mixing L+R remains.

2.1 Amplifiers
There are 2.1 amplifiers to buy. The problem is that they are designed for use with a subwoofer (very low frequencies), not our 2500Hz woofer. This may or may not be a problem.

Mono
If I had a mono amplifier (that accepts stereo input and produce mono output) I could connect all the three loudspeakers to the same output. Since the distance between the tweeters is less than 25cm I don’t think the lack of stereo-tweeters will matter. However, it was not very easy to find suitable mono amplifiers (or “bridged amplifiers” that can be used as a mono amplifier).

Two-trick-solution
In the end I decided to go for a simple solution based on two parts.

First, pre-amp, it is very easy to convert stereo to mono. The only thing needed is two resistors (470 Ohm, or something close to that).

Second, a 2.0 amplifier can drive the tweeters on one channel and the woofer on the other (that is 40W on each channel).

Cleaning out the Stanmore
I removed (unscrewed) the back of my Stanmore. When I was done with it, the only thing that remained (and in place) was:

  • The box itself (except the back of it).
  • The three loudspeaker elements, and as long cables as possible.
  • The top golden colored control panel (because removing it would not make anything pretty) and the board attached to it (because it was hard to remove).
  • The cable (black+white+red) from the 3.5mm connection on top of the loudspeaker.
  • The 4 red cables from the on/off-switch.

What I also needed
This is a list of other components I used

Assembly
I neatly connected everything in a way that it fits nicely inside the Stanmore.

  • DC-power to two red cables connected to Stanmore power switch
  • The other two red cable to Jtron board (make sure to no reverse!)
  • One Jtron channel connected to yellow+black of woofer
  • One Jtron channel connected to red/blue+black of tweeters
  • Black of 3.5mm connector to Jtron input (middle)
  • Red/white of 3.5mm connector connected via two 470 Ohm resistors
  • From between the resistors, connect to Jtron input (left and right)

This is what I got:

As you can see the Jtron is pretty small.

For now my laptop DC supply is outside the Stanmore and there is just a little hole in the back for the cable.

Operating
The power switch on top is operational and I connect my audio source to the 3.5mm connection on top. The Jtron knobs work as expected (there is no balance).

About the Jtron
The Jtron was very good price and I thought 2x50W was kind of optimal for me. Also, it is a digital amplifier with high power efficiency (little excess heat). There are obviously many other options.

Serial vs Parallell
I connected my tweeters in parallell. I suppose they could have been used in series instead. Perhaps serial would have been more safe: impendence would be 4x higher, which would be less demanding on the Jtron.

Review
Well, I shall not review my own work. To be honest I have not fixed a new back plane yet and I think not having it in place is far from optimal for audio quality. Despite that, the Stanmore sounds very decent. It plays loud enough for me (perhaps louder than before). You probably want to experiment with bass/treble until satisfied. The way I use it (with an iPhone) I will set preset volume to loud, and mostly use the iPhone to control volume.

What I have lost compared to the original Stanmore is RCA-input, bluetooth and volume/treble/bass on top of the unit. I can live with it.




SonarQube 6.5 – Mac OS X – Installation problems

It seems SonarQube 6.5 does not work with JDK 9. Java 8 upgrade 144 works fine.

There is no automated way to uninstall JDK 9 properly, but you can have different Java side by side. This is how to pick Java version.

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_144.jdk/Contents/Home

After that you can run

sonar.sh start

as usual.

Can not log in to SonarQube 6.5 on Mac OS X with default password
With JDK 9 authentication with default user/pass admin/admin fails. It turns out the error HTTP code is 500, not 401, which made me suspicious.

Syncthing: breaking upgrades

Syncthing is usually very easy to upgrade: it upgrades itself silently or via apt-get. Problem is, when it changes version (lastly from 0.12 to 0.13) it is not compatible anymore. It is not a big problem since it is easy to upgrade, but you kind of have to upgrade your entire Syncthing cluster at the same time, otherwise you get an undesired fork.

This was the quite confusing error message i got on the newly updated system that failed to connect to non upgraded systems:

[THC2C] 19:17:52 INFO: Failed to exchange Hello messages with <ID> (<ADDRESS>): EOF

So, be mindful when upgrading syncthing so you dont get a non breaking upgrade when you dont have time to upgrade everything.

Syncthing on Android

I installed Syncthing a few weeks ago. Now I found it was time to connect my Android mobile to it. Installing Syncthing via Google Play was easy. Configuring it, not that easy. The amount of useful error messages… close to zero.

I found:

  1. When I manually write the address to my other syncthing unit (like my NAS), only IP address works (with :port after it). Writing a domain name fails.
  2. When sharing a folder, I can not share a folder on the SD card: I get something like “Error (100%)”. To me, this is a pity, because I could put a big SD card (32-64GB) and have synchronised music there… but it seems not possible.

Update 2015-11-14: Upgraded to new syncthing version (0.12.2). Syncthing (for Android) now does not start properly. It just keeps “Loading”. No error message. No way to interact with it.