Frameworks are fundamentally broken

This post was origionally written in 2012, and later revised in 2014. At the time, I refrained from posting it due to concerns about how certain topics were articulated, and how it might be recieved in the community. After accidentially publishing this in 2016, the positive feedback I recieved encouraged me to release this officially. The article is an opinion piece that I hope resonates with other functional programmers. Certainly, not everyone will agree with what’s written here, and that’s absolutely fine with me. All I ask is that you read the article for what is, and recieve it with the good intentions it was written.

I’ve been thinking about writing this post for a while - several years in fact - but its reached a point where I have to get this out of my head and onto the screen: Frameworks are the worst thing to happen to software development in the last decade (and possibly ever).

For the purpose of this article, I shall define a Software Framework as this: one or more pieces of software that are designed to work in tight unison, with the aim of smoothing / easing / hastening / or otherwise “improving” the development of a given application development cycle in a particular domain. The software in question is typically bundled together and binary modules of the project are typically not used outside the intended framework usage or the framework itself. Examples of software frameworks include AngularJS, Play!, Ruby on Rails etc. At this point in my software engineering career, I have used a wide range of software frameworks and have been involved in writing more than one, and I even wrote a book about Lift. With this frame good reader, please appreciate that one does not come to such a decision to criticise frameworks as a programming paradigm lightly. The following sub-sections outline what I see as the primary issues that make frameworks fundamentally flawed.

Lack of Powerful Abstraction

Business domains are often inherently complex, and this impacts the engineering that needs to take place to solve a problem within that business domain in a very fundamental way. In this regard, frameworks tend to be intrinsically limiting because they were written by another human-being without your exact, complex business requirements in mind - you are programming inside someone else’s constraints and technical trade-offs. More often than not, those trade-offs are not documented explicitly or encoded formally, which means users encounter these limitations through trail-and-error usage in the field.

Many framework authors take the approach that they are solving a general problem in a given engineering sector (web development, messaging, etc), but typically they end up solving the problem(s) at hand in a monolithic way. Specifically, authors have a “outside in” approach to design, where they allow “plugin points” for users of the framework to write their own application logic… the canonical example here can be found in MVC-style web framework controllers. In all but the most trivial applications, this is a totally broken approach as one often observes users either writing all their domain logic directly in the controller (i.e. inside the framework constraints), or alternatively, observing parts of the domain logic or behaviour “leaking” into the controller. Whilst it could be argued that this is simply an education problem with users, I would disagree and argue that it takes a high-degree of discipline from users to do the right thing… The easy thing is most certainly not the right thing. Instead of the root cause being an education issue, I would propose that a fundamental problem exists with the mindset of the frameworks themselves - which often encourage this kind of poor user behaviour - in short, frameworks do not compose. Frameworks make composition of system components difficult or impossible, and without composition of system components there can never be any truly powerful abstraction… which is absolutely required to build reasonable systems. To clarify, the lack of composability exists both in the micro and macro levels of a system; components should plug together like lego bricks, irrespective of which lego pack those bricks came from (here’s hoping you follow that tenuous analogy, good reader). Users don’t wish to extend some magic class and be beholden to some bullshit class hierarchy and all its effects; users wish to provide a function that is opaque to the caller, provided the types line up, naturally. When frameworks do not do this, its a fundamental issue with the design of these software tools.

An obvious supposition might be that these kinds of monolithic, uncomposable designs occur because framework authors are trying to optimise for certain cases - more often than not, a case high on the list to satisfy is to make the framework “easy” to get started with. An interesting side-effect of this is that authors usually assume that users won’t know too much about what they are using and that the code they write needs to be minimal. Whilst i’m all for writing less code, assuming that users won’t know how to use framework APIs only applies when the system is not based on any formal or well-known abstractions. The ramification of this lack of formalisation is two fold:

1. Enhanced burden on the framework author(s) as the lack of formalisation requires them to “teach” the user how to do everything from scratch. In practice this means writing more documentation, more tests and examples and more time spent on the community mailing lists helping users - ad infinitum.

2. Users have to invest their time fairly aggressively in a technology without truly understanding it. This typically means getting up to speed with all the framework-specific terminology (e.g. “bean factory”, “interceptor”, “router”) and programming idioms. As an interesting side-note, I believe this aggressive investment without understanding is actually what gives rise to a lot of “fanboism” in technology communities at large: people get invested quickly and feel the need to evangelise to others simply because they invested so much time themselves, and subsequently need to ensure that the tool they selected gains critical mass and long-term viability / credibility… that is no doubt a subject for another article though.

Let’s consider for a moment what would happen if a framework component were implemented in terms of a formally known concept… For example, if one knows that a given component is a Functor, then one can immediately know how to reason about the operations and semantics of that component because they are encoded formally as a part of the functor laws. This immediately frees framework authors and users from the burdens listed in points one and two above. However, what if framework users don’t know what a Functor is? and they are not familiar with the relevant laws? Well, there is no denying that to learn many of the formal constructs will require effort on the part of users, but critically, what they learn is fundimentally useful when it comes to reasoning about problems in any domain. This is wildly more beneficial than learning how to operate in one particular solution space inside one particular framework. They will have learnt something fundamental about the nature of solving problems - something that will serve them well for the rest of their careers.

Concepts such as Functor should not be scary. Many engineers in our industry suffer from a kind of dualism where theory and practice are somehow separate, and formal concepts like Functor, Monad and Applicative (to name a few) are often considered to “not work in practice”, and users of such academic vernacular are accused of being ivory tower elitists. Another possible explanation might be that engineers (regardless of their training: formal or otherwise) are simply unaware of the amazing things that have been discovered in computer science to date, and proceed to poorly reinvent the wheel many times over. In fact, I would wager that nearly every hard problem the majority of engineers will encounter in the field has had its generalised case be the subject of at least one study or paper… the tools we need already exist; its our job as good computer scientists to research our own field, and edify ourselves on historical discoveries and make best advantage of the work done by those who went before us.

Short-term Gain

All software is optimised for something; sometimes its raw performance, sometimes its type-checked APIs and sometimes its other things entirely. Whatever your tools are optimised for, some trade-offs have been made to achieve said optimisation. Many, many frameworks usually include phrases like these listed below:

• “Increased productivity”
• “Get started quickly!”
• “Includes everything you need!” for XYZ domain

These kinds of benefits usually indicate the software is optimised for short-term gain. Users are hooked on the initial experience building “TODO” applications. More often than not, these users then later become frustrated when they hit a wall where the framework cannot do exactly what the business needs, and they have to spend time wading through the framework internals to figure out a gnarly work-around to solve their particular problem.

The real irony here is that optimising for the initial experience is such a wildly huge failure: the majority of engineers will not spend their time writing new applications, rather, they will be maintaining existing applications and having to - in many cases - reason about code that was not written by them. On large software projects, there are usually a myriad of technologies being employed to deliver the overall product, and having each and every software tool have similar concepts with different names and annoying edge cases is frankly untenable. Once again, the lack of formalisation or composability causes havoc in many areas (lest we forget taking the time to figure out work arounds is usually painful and time-expensive).

Community fragmentation

For the vast majority of frameworks, they usually have a particular coding or API style, or a set of conventions users need to know in order to produce something - disastrously, these conventions are often not enforced or encoded formally with types. Whilst these conventions are probably obvious for authors of the framework, it makes moving from one framework to another a total mind-fuck for users - essentially giving users (and ergo, companies) a vendor lock-in long-term. Whilst vender lock-in is clearly undesirable, there is another more important aspect: frameworks create social islands in our programming communities. How many StackOverflow questions have you seen with a title along the lines of “what’s the Rails way to do XYZ operation?”, or “How does AngularJS do ABC internally?”. Software is written by people, for people, and it must always be consumed in that frame. Fragmenting a given language community with various ways to achieve the same thing (with no formal laws) just creates groups with arbitrary divisions that really make no sense; these dividing lines usually end up being taste in API design, or familiarity with a given practice.

Whilst the argument could be made that branching, competing and later merging of software projects is beneficial, when it comes to the people and the soft elements related to a technical project, the mental fallout from the fork/compete/merge cycle is extremely heavy and usually the merge process never occurs (if it does, it usually takes years). Moreover, if a given framework community island fails, its incredibly hard on the engineers involved. I have both experienced this personally, and witnessed it happening in multiple other communities - which is a worrying trend (again, lots of material for a later post to lament about that).

Looking forward

It is imperative to understand that the need for composability in our software tools is an absolute requirement. If we as an industry have any hope of not repeating ourselves time and time again, we have to change our ways. In conclusion, dear reader, if you’re wondering what you can do to make the industry a better place going forward: study the past and read as many releevant academic papers as you can reasoanbly consume… be curious and continually ask questions. Demand lawlful programs and excellent tools. Engage in software communites in a meaningful and positive way, and always look to improve the world around you :-)