Fifteen months ago, I decided to learn OCaml by working through the 2019 Advent of Code problems and (painfully) blogging about it.

The first article was enthusiastic; the second one was glum. As I mulled what I was going to write for the third installment, it dawned on me that I was traveling a well-known trajectory. To wit:

Gartner’s hype cycle

Garner’s hype cycle is a staple of the corporate innovation ecosystem. It thrives in solution provider white papers, “thought leadership” blog posts, and corporate R&D seminars. You’ve certainly run into it at one time or another. If not, I envy you.

The hype cycle posits that all new technologies go through the same phases as they make their way from hot new stuff to boring old stuff.

The cycle begins with an innovation trigger, when a new technology enters the market and begins to find recognition. The initial excitement grows, and reaches a peak of inflated expectations. When the technology inevitably fails to live up to the hype, it falls into the through of disillusionment. As the technology matures, it climbs the slope of enlightenment and finally reaches the plateau of productivity.

A side project’s hype cycle

It turns out that side projects (at least, my side projects) follow the same curve.

It all starts with an idea for a new side project. My imagination starts running loose; I mentally list all the features and the cool stuff I could fit in there. Before I know it, this innocent idea has turned into an overblown pipe dream.

Then I actually get started, and reality catches up to me: this project is going to be much harder than I expected. This is not something I can handle with a couple of hours of work each week; it’s going to require real work. My initial motivation plummets and I fall into the chasm of despair.

By now I have a much better idea of how much I was underestimating the complexity of the project; I lower my expectations accordingly. The initial enthusiasm is gone. Guilt becomes the prime motivation to push on: I have entered the phase of glacial progress.

This phase is often strewn with side quests, which sprout their own tiny hype cycles. An example: Hey, I was really into learning OCaml but now it’s become painful and boring. I wonder why that is? Ooh, this idea of “side project hype cycle” sounds interesting! Let’s write a post about *that* instead of getting on with the initial project.¹

Fitful progress, lowered expectations, occasional side quests and long periods of shutdown. Lather, rinse, repeat and after a few rounds I eventually reach the plateau of good enough. Time has played its part. It has eroded the initial goals of the project, watered down the guilt, distracted me with new ideas. The project stopped being fun a long time ago and I’ve achieved enough progress that I can stop with a clear conscience. So I shelve the project, never to touch it again.

Sound familiar?

But wait, there’s more!

I don’t claim that this is a very deep insight; I’m hardly the first person to have trouble finishing side projects (seriously, don’t read all those links! I barely skimmed the first two myself).

But the parallel to Gartner’s hype cycle is interesting; it suggests that this individual behavior more or less holds out in the aggregate. Sure, the drivers are different, or at least described differently; the guilt and stubbornness that push an individual project forward are recast as “strategic vision” or whatever. But the overall shape is the same.

And although I’ve described it through the lens of my personal projects, I’ve seen very similar trajectories in “real” projects at work². The initial enthusiasm for a new product, driven by the promise of large market shares; the grand kick-off with the ExCom sponsor… And then the cold shower of the scoping phase and the initial work estimate; the versioning of features into v1 (we’ll give it our best shot), v2 (mayyyybe…), and v3 (never gonna happen); the long slog to ship the v1, while the backlog grows and further arbitrages are made to stick to the initial schedule. The v2 that devolves into a v1.5, then a v1.1… Until finally the ExCom sponsor moves on to the next hot thing and the various stakeholders shuffle awkwardly away from the program, which is labeled a resounding success and shelved.

It looks so easy from the top of Mount Stupid

There’s a third curve that looks like the ones above, if you squint hard enough. It describes the Dunning-Kruger effect.³

Mount Stupid, courtesy of SMBC

From Wikipedia: the Dunning–Kruger effect is a cognitive bias in which people with low ability at a task overestimate their ability.

Very roughly, the less you know about a topic, the likelier you are to overestimate your ability. The effect also works the other way: competent people tend to underestimate their ability. Hence Mount Stupid, a summit of misplaced confidence and as-yet undiscovered incompetence.

The parallel with the hype cycle seems clear. When you embark on a new innovative project⁴, you only have a vague idea of what the technology can do. You tend to overestimate what you can achieve and underestimate the problems you’ll discover along the way: Dunning-Kruger is taking you on a scenic trip to the peak of inflated expectations. It’s only after you gain some competence and domain knowledge (i.e. after the first few pitfalls) that you start to perceive how much you don’t know; you dial confidence all the way back, overcompensating for your previous cockiness, and plunge towards the through of disillusionment. It’s only once you’ve hit the high and low, calibrated your confidence so to speak, that you can hope to make sustainable progress.

Where does this leave us?

I must admit I don’t really have an ending for this post. It started, oh so long ago, as a jaded observation about my then-current side project. I thought it would make for a quick, witty post, maybe something that would pique the interest of HN (do you recognize the overblown pipe dream?).

But as I fleshed out the initial idea, I kept thinking there was something larger lurking behind.

I picked up and dropped off this draft multiple times over the past year, trying to tease something meaningful out of disparate concepts. I’m not convinced that I followed these ideas to their logical conclusion, nor even that they are related; we humans are good at recognizing patterns, sometimes where there are none.

Nevertheless, I’ve achieved enough progress that I can stop with a clear conscience. I’ve reached the plateau of good enough.