The deal with me and linux (or Why I don't use Linux unless someone is paying me)

I take quite a bit of flack for not being a Linux zealot. In fact I'm actually not particularly fond of any linux distributions out there and for some reason this invokes the ire of many a devotee who feels personally affronted by my lack of enthusiasm for using the operating system to which they feel a very deep and personal attachment. Here I endeavor to clarify my position and de-escalate the BS.

NOTE: If I sent you to this links it's probably because you're acting like a GNUb and I don't have time to keep re-typing the responses to the questions/challenges you pose. You can either read and learn or not. In the words of Dres from Black Sheep "...the choice is yourrrrrrrrssss...".

[update: 9/6/2012]: So, today Randal L. Schwartz found my post and shared it on Google+.  I saw more traffic in the past 3 hours than in the past 3 months. A few people have commented on my use of the term GNUb so I wanted to, here, clarify what I mean especially since more eyes are landing here and I genuinely do not endeavor to offend.

First I do not use GNUb to refer to all Linux, GNU/Linux or GNU enthusiasts. I have way too much respect for the likes of Linus, Simon Phipps, Alvaro Lopez (Cherokee Project), Thomas Hatch (Salt) and many others for that to be the case. The term is meant to apply to those who have very little experience with unix systems yet presume to know everything based on their (usually limited) exposure to GNU/Linux. These are the sorts of people that most of the hacker community can't stand but because their ire is pointed in the right direction (Redmond or Cupertino) no one challenges or corrects them.

The term GNUb is a bit callous and dismissive so I understand if you're offended. My hope, though, is that you realize I have no desire to disparage the GNU/Linux community as a whole, only those among you to whom the term truly applies.
[/update]

Where to begin... 

I've been sitting on this post since early in June 2012 and have finally re-read and tweaked it enough times to think that maybe it's OK to post. Maybe I'll make this a living document and keep updating it over time (not likely but let's say maybe). The problem is that there is entirely too much ground to cover quickly and succinctly. One thing I should probably do is qualify what I'm about to say : 

    I've been doing some form of systems, software and/or security engineering since 1993 (yes I was in high school then). Bearing this in mind I have tremendous respect for people who choose to create software for the rest of us to use, especially those who do so without any expectation of monetary compensation (Free as in beer). Also, I admire those who fight to make sure that the rights of users of software are respected and upheld (Free as in freedom). 

I often use pretty strong language to express my dislike of or for various systems/tools. It is my hope that, despite that fact, you understand that I applaud all of the contributors who act with goodwill and in good faith to improve the landscape of Free and Open Source Software for all of us.

A (byte/8) about me (specifically my history with Unix-like systems)

Short version: I started programming in 1993 while in high school, mostly TI-BASIC (on my TI-81) and ASSEMBLY and BASIC on my 8088 at home. I was into the BBS scene and a buddy of mine had 'hacked' into a BSD/386 system and let me poke around a bit. I was smitten. Then in 1995 while a student at Northwestern University I got a job in the technology center (Vogelbach). The team I was on  was responsible for both servicing the technology needs of various departments across all supported platforms (Windows, Iris, DGUX, HP-UX, Novell Netware, Mac OS 7, and more) and running a helpdesk phone line for faculty and students seeking resolution on various internet service issues. I learned much.

A buddy I worked with, named Julian Koh, had lent me a copy of Slackware linux when I decided that I wanted to run a unix-like system at home. Fastforward a few months and I had about 8 computers at home with 4 ISDN lines and rezidew.net was up and running powered by a combination of Linux (slackware) and FreeBSD (thanks to another coworker, Daniel Johanson).

Since then I've been staunchly entrenched in the Unix world. I've supported just about every type of unix system you can name, at some point. I've been in small companies and large, always doing systems, software, or security engineering on unix (small 'u'). I've managed my own servers (actual hardware that I own) since 1995 and have used them in real world scenarios for various services and applications. One such service has been hosting the longest running US based mirror for LinuxMint since 2007. Giving back to the open source and free software communities is something about which I feel strongly.

The meat of the matter

The bottom line to me is this: using most linux systems is annoying to me. I know... that sounds mean ... it's okay... take a second.

Now, I don't mean the above to be offensive or provocative. It's simply a statement of fact. I don't hate your beloved OS of choice it's just annoying as hell to me and I choose not to use it. I'm not saying "Linux sucks!", just that I choose not to use it on any of my personal systems for the reasons listed below. 

I'm not dissing the kernel

I've stated many times in various posts that I think the linux kernel just awesome. I truly admire the kernel team and all of the work they have done. When I talk about linux systems being annoying to use, very little of that (if any) is the fault of the kernel.

Well there is NetFilter / IPTables

Ok IP Tables is a mind f**k that I wouldn't wish on my worst enemies. I have visions of a future where enough people realize that IP Tables was such a bad idea that it's synonymous with 'tried your best but completely screwed it up' and that get's used in sitcoms and vernacular:

 "Boy you really IP Table'd that one."

 "I know. I wish I were dead. Someone shoot me."

 "Heyyy..bud cheer up. Who knows next time you could totally PF it!"

OS not Distro

I like the cohesiveness of a system that's well put together. OS X, FreeBSD, OpenBSD, Solaris they all have one thing in common they are unix based operating systems (that I happened to run at one point on one of my desktop machines). Redhat, Debian, and Ubuntu are distributions where they take the kernel from one source the utilities from another source and cobble it all together. This isn't a fit of my pedantic nature shining through, there's actually a point here. 

It takes a considerable amount of energy to put together a system and get it working properly, there are many moving parts. If you control the kernel and the user land utilities you can make changes as you see fit. When (as in the case of Linux distributions) you get the kernel from X and the user land from Y (most often GNU) and paste them together you have to juggle pieces you don't control. You can patch things for your system and hope they get adopted upstream or you can fork and take over maintainer ship for the version that ships just for your system but either way it's a crapshoot and a bunch of work.

GNUbs say: ... but FreeBSD, NetBSD, and OpenBSD are all just distributions of BSD...

Yes and no. Yes because, let's face it, the D in BSD is, in fact, for "Distribution". All of the popular open source BSDs are based on on the 4.x BSD releases from the University of California at Berkley. No is where it get's interesting and sometimes too subtle for the GNUbs to grasp. Each of the BSD systems takes responsibility for the evolution of it's own kernel, user land tools, utilities, packaging system, etc. For example OpenBSD forked from the NetBSD project. When it happened they took the kernel and everything, modified it the way they wanted and it has since grown separately from NetBSD. The kernels share a common origin but are not compatible in the way you might think of Ubuntu being compatible with Debian.

Base System & Add on Packages

One thing I think most linux distributions get wrong is the whole idea of a base install or core system. Call it what you will but the idea is that I should know what's installed as part of my operating system versus what's an add on package. This concept appears to be lost on nearly every major linux distro.

To make matters worse, on some distributions it's unclear where certain files will be installed. For example, If I install the apache web server on distro-xyz. It may throw configs under /etc/httpd and actual content under /var/httpd/html. On the same distribution, if I install the lighttpd web server it might decide to install configs under /var/lighttpd/etc and html under /var/lighttpd/html. Stuff like that is pretty annoying and exceedingly commonplace in the GNU/Linux world.

File System Layout

On the BSD systems it's generally /usr/local/; on Solaris and HPUX it might be /opt/ on most linux distros it's /. WTF????

When I install a third party package it should be put someplace where I can easily identify whether a file is part of the operating system or not. On Debian, RHEL, Ubuntu, etc. when a third party application is installed it goes into the root of the system along with core utilities and tools like grep, awk, ed, vi, and such. Some things get thrown into  /var and /etc polluting the system. 

(Not) surprisingly the Linux distributions that take some pretty heavy cues from BSD systems have relatively sane ways to deal with this even if it's not set by default. Arch, Gentoo, Slackware and others fall into this category. Sometimes their solution is to just stay out of the way and let you run pkgsrc alongside their system package manager.

GNU/Linux User Land

Let's not forget that most Linux distributions are GNU/Linux. The GNU shell (BASH) is just annoying to me. I find it's conventions like declare and local to be useless bloat. The same goes for many GNU utilities like netstat which (re)appropriates -p to show process names despite the fact that on HPUX, Solaris, *BSD, and many other unix systems -p is used for protocol. In general the GNU user land utilities are pretty bloated and have odd options when compared to their non-gnu counterparts in both proprietary and free systems. Of course there are exceptions and, in some cases, justifications.

GNU grep is blazingly fast when compared to other implementations. Unfortunately if you were expecting it to respect the bounds of POSIX basic regular expressions (as is required by the standard) you'd be out of luck. Because it's so fast it doesn't need to discriminate between basic and extended regular expressions so be careful. 

Remember that GNU stands for GNU's Not Unix. In some very notable cases the FSF and GNU have fought hard to make sure that open standards are published and implemented in their tools. Often GNU tools were the first (if not only) utilities to actually be fully compliant with said standards (namely OpenGroup/POSIX). Where there has been room to implement outside the bounds of the standards they have in many cases chosen the most unconventional approaches. As someone who at one point supported 6 different flavors of unix at the same time working with the GNU versions of things like sed, awk, bc, make, tar, ifconfig, find, and roff was vexing to no end.

Add to the above, the fact that each distribution has it's own custom blend of GNU utilities plus some other random stuff they found somewhere to either supplement or substitute the parts from GNU.

With Ubuntu 12.04 Canonical has decided that it will no longer include the traceroute utility as part of the operating system. You can add it later if you decide that you want it but it doesn't come as part of the system (WTF?!?!?).  

GNUBs / GNUBes / GNUBies

Now that GNU/Linux is so prominent in the industry there is an issue of collateral damage springing up. That of GNU users being ignorant of things like standards and portability. I'm sick of seeing 'shell scripts' that are bash specific without noting as much. CLI extensions like pythonbrew and rvm while not trivial to implement could have been trivially portable to shells other than bash. It takes very little time to write portable code but now no one thinks about it because they are convinced that everyone is running GNU/Linux tools or should be. The same goes for other things like vi. note to all of the GNUBies out there: VIM != vi && GAWK != awk; and I say that as someone who uses VIM most of the time and loves gawk. 

I sit in IRC channels, browse stackoverflow.com, and watch forums where users ask UNIX questions and get GNU/Linux answers. In some cases this works out alright for that user sometimes they come back and state that something did not work and no one has any idea why. In either case I think it's bad for the unix community as a whole. 

Bloat

Beyond the utilities from GNU, most linux systems come with far more turned on than I want or think is reasonable. In may cases just installing something (like a web server) results in the that same something starting automatically when you reboot the system. 

And come on colorls; really?!?!?

I need a system that just works

Randal Schwartz host of FLOSS weekly, international traveler, speaker, and world renowned Perl hacker get's flack from GNUbs and Open Source zealots for using a mac instead of some Linux distribution. In several talks he's mentioned that he needs a system that just works, he doesn't have time to keep tweaking his OS because he actually needs to get work done. I'm pretty much in the same boat. While some linux distributions offer a fare degree of customizability none of them come pre-built with the user land I like, a layout I can stomach, and a packaging system that doesn't make me want to pull my hair out. 

True I could cobble together my own system or endlessly tweak Arch, Gentoo, Funtoo, Slackware, DSL, etc but guess what.. FreeBSD, OpenBSD, DragonflyBSD and Mac OS X already work for me. Ok admittedly with OS X I have to go through the process of installing Homebrew and PKGSRC but that's trivial compared to just preparing to install Funtoo or Gentoo (which I've done a number of times).

Despite all of this

I'm still a fan of Linux; both the kernel and the GNU/Linux distributions. While I have no desire to use GNU/Linux on my personal systems I do work with Linux systems more often than any other single unix like OS except Mac OS X. Most of my client work is on Linux and I rarely ever suggest to a client that they should deploy anything other than linux for their applications. 

I know linux systems very well and over the past, decade and a half,  have come to appreciate them in the larger scheme of unix (like) systems. It's because of this that I support and contribute to projects like LinuxMint.  I wouldn't to be doing what I do today without linux in the picture. I just wish the GNUbs out there realized that diversity is good and portability is important.