The Exigent Duality
Transparency is my friend - 14:08 CST, 4/11/17 (Sniper)
I've been using DOS since 1990, Windows since 1992, OS/2 between 1992 and 1995, and GNU/Linux since 1996. Further, in the past fifteen years, I've professionally developed with: Bash, PHP, Python, Ruby, Java, Scala, VB.Net, SAP ABAP, and C#, using source control systems such as Git and Team Foundation Server, and utilizing way too many frameworks and assisting tools to even name, across a gamut of environments, including Windows servers and a GNU/Linux-based Hadoop cluster.

For those counting, that's between fifteen and twenty seven years, depending on the technology, of hardcore computing experience.

I'm establishing my credentials because what I'm about to say is going to be hard to swallow for some people, even though it shouldn't be controversial: some technologies suck way more than other technologies. And for me, that dividing line has been: on a spectrum, is a given technology transparent, or opaque, in how it operates?

To the degree with which technologies are opaque is the degree to which I find myself spending large percentages of my time fighting with said technology, versus doing productive things. The inverse is also true.

Windows is opaque. When you change a setting in its "control panel", good luck figuring out what the hell it just did. Probably, if you devote, oh, a year or two of your life, you could find which flag was tripped using their crusty old "regedit" tool. Good luck with that. By contrast, when you issue a "systemctl enable someservice.service" in GNU/Linux, it creates a fucking symlink. And it even tells you exactly which one it created! It absolutely could not be more transparent.

I spent the past year at work doing full-time Python and Scala development, using Git, against a Spark installation on a GNU/Linux cluster. In that entire year, I fought with the technology zero times; everything either "just worked", or when it didn't, it was trivial to figure out what went wrong, because everything was built on super transparent, super simple building blocks, like "files", "symlinks", or text-based config files. Just like my "systemctl" example above.

Now, I'm back to doing Microsoft .Net work using C#, NuGet, Not-Very-Visual-Studio, and Microsoft's dreaded "I like to attribute files as read-only" Visual Source Safe Team Foundation Server, running on Windows and Microsoft's Apache competitor, "Internet Information Services". These past few months have been a stone cold refresher on why I wanted to move to Hadoop development in the first place, as I've spent enormous amounts of time doing nothing other than arm wrestling with this monolithic stack of junk, versus actually providing value to the company.

Did you know: when .Net's compiler says "predefined type is not defined or imported", hours of searching will reveal that a DLL which a project depends upon was compiled against a different version of the .Net framework libraries? Did you know: when NuGet complains, "if your PowerShell execution policy setting is set to AllSigned...", that means you need to hack an cryptic Windows registry key? Or try this one: when "SQL Server Express" says "only part of a ReadProcessMemory or WriteProcessMemory request was completed" and won't start, that means you need spend hours figuring how to load the database in single user mode so you can update utterly obscure file paths to "mdf" files?

And these are just a few of the character building exercises I've had the joy to resolve in the past couple of months. I've been writing them all down, and the list is ever-growing. Hilariously, in that final example, I'd just installed MongoDB-- total install time for MongoDB was 45 seconds, versus "SQL Server Express", which was four hours.

And it's not even like opaqueness is proportional to complexity. Spark is an enormously complicated piece of software. Arch Linux and Ubuntu are equally complex, yet Arch is much more transparent than Ubuntu. IntelliJ IDEA has way more features than Visual Studio, yet is significantly simpler to use, and never seems to break or crash. And so on.

I remember when I was brand new to .Net, coming from professional PHP development. In PHP land, people talk about files, and directories, and source code. In .Net, I immediately noticed that everyone spoke in these super abstract, higher-level concepts, like "providers", and "facades". I recall having to break down the strange language for myself, just to track the conversations without getting totally lost: "Ok, a 'provider' is just a class in a source code file". I think that was an immediate glimpse into the quagmire.

And here I am, nearly fifteen years later, writing this blog post...