This is a tricky one to navigate and think through. I’ll try my best to articulate my thoughts on Scrum, how it relates to GTD, and some new epiphanies I had about the roles they play in successfully creating software.
Org-mode and GTD are awesome tools for the individual because they clearly lay out all potential paths you could walk and has you start managing them via externalized memory and regular check-ins. In other words, these tools have you write down and outline all possible tasks and projects you may want to pursue. And in doing so, you directly face your list of all possible paths you can take, all possible action steps you can execute on.
And then you perpetually manage this system in regular intervals, always moving forward with your projects in some way.
A well implemented project management cycle is the same in any organization. Scrum tries to package itself as the project management solution to do this. Scrum attempts to be the organizational-level version of GTD. It attempts to externalize memory from the individuals on the team via wikis, issue trackers, and kanban boards. Sprint meetings and retrospectives are the Scrum analogues to GTD’s weekly reviews.
Some organizations are wildly successful with Scrum, while others are 100% mediocre.
Once you understand the value-add of both systems, you can begin to pick apart how users interact with both methodologies, why the methodologies fail in some instances, and why they succeed in others. Quick aside: Part of it is because GTD targets the individual, while Scrum targets the organization. And comparing the two reveals the values and weaknesses of each, particularly of Scrum (I’m not gonna delve too much into the dynamics here- but organizational level solutions almost always fail if the individual isn’t properly taken into account, which is why Scrum is more prone to weakness and failure than GTD when implemented).
So my epiphany is actually pretty simple- project management is probably a fractal pattern. It applies to the individual, the organization, and beyond.
While simple, there are some interesting conclusions on what this means for overall strategy.
At an ideal level, a developer is executing their own individual instance of project-management (eg. org-mode + GTD), the product-development department is executing their own organizational instance of project management (eg. project documentation + Scrum), and the business is doing it’s analogue as a whole (ie. the CEO + board of directors are managing their work as ideally as possible).
The major point is that the strategies align at every level of analysis. From the individual, to the organization, to the entire entity as a whole. The individual GTD-strategy can’t be compromised for the organization. But the organization can’t be compromised for the individual. All strategies have to be in harmony or it fails.
And thus, the key takeaway is that Scrum (or any organizational project management tool) must allow for the individual to implement their project management processes freely. The individual must be given the discretion to make seemingly arbitrary decisions. They must be given creative freedom. In short, they must allow the developer to execute on the tasks that they think are the highest value.
A couple of components of Scrum stick out as bottlenecks to this philosophy, particularly estimates and sprints. Creative freedom means developers aren’t locked into completing a particular subset of tasks, aka the sprint backlog, because they will naturally tackle the highest value tasks out of their own interest and affinity. Estimates are also a similar bottleneck in that they often establish an implicit contract for what gets completed and when. Which immediately eliminates the ability for the developer to work on necessary tangents in the project. AKA it removes the necessary discretionary and creative decisions developers must make and, as a result, will likely introduce it’s own form of technical debt, which leads to time and money costs and even further project delays.
Ultimately, a strategy aligned at the organizational and individual levels means that Scrum estimates don’t matter. If it means the developer doesn’t complete their assigned stories at all, then so be it.
Of course, it’s reasonable for stakeholders to know what was done instead of the assigned stories. If developers went off-the-road and worked on non-story tasks, it should result in exponential returns, since they are deliberately choosing to pursue higher-value tasks than the assigned stories. No one wants to spend their time on make-work, and if the assigned stories are later revealed to be make-work, then it’s better for developers to focus on higher-value aspects of the project. That’s the key value in giving developers creative freedom. That immediate ability to identify tasks that will be a waste of time, cull them, and move on to bigger and better project work.
Forecasting and estimates can play a role in giving semi-realistic projections that allow businesses to make decisions in other parts of the organization, so long as making these estimates is not more time-consuming than it needs to be and so long as all parties understand the volatility inherent in the forecast. (And if you’re interested in the non-linear dynamics of organizational strategy, this volatility can be used to your advantage. We don’t know what the reality of the marketplace is. We don’t know with 100% precision how the market will respond to our products. We don’t have an accurate picture as to the moves our competitors are going to make. By embracing volatility in our own product development, we are allowing our products to be developed in such a way that it will ultimately be a more accurate reflection of the marketplace than by the top-down imposition of what we guess it is).
Thus, based on what I’ve described here, as well as some of my observations of Scrum in general, I have a new attitude towards the methodology- Scrum isn’t inherently bad, but it seems to propagate mediocre dynamics just like the cubicle did. Instead of being leveraged for it’s strengths, and instead of culling out it’s weaknesses, Scrum is often used to impose predictability on an inherently unpredictable process. The results speak for themselves.
A good organization with quality project management is good in spite of Scrum. Because they’d be good anyway without it. The product leader “knows” the spirit of GTD even if he doesn’t know of GTD. AKA the same attitude and values that would drive him to adopt GTD if he knew of it, at least drive him to cull out make-work and to focus on high-priority tasks. He has the same values and therefore would inevitably implement a successful strategy, or find one that aligns with his values. Those same values would lead him to interfacing with the higher and lower levels of the organization with an acute awareness of the limits of forecasting. And if tasked with using Scrum as a methodology, he’d pick and choose the useful components of Scrum relative to the project’s needs and cull the weak parts without hesitation.
Turns out I don’t have an aversion to Scrum and to the idea of project management (hell, I love GTD). I just have an aversion to Scrum when it’s done in a nonharmonious fashion. AKA when it gets in the way of doing useful work rather than being a catalyst for it.
The effects of a dysfunctional Scrum project management cycle are worse for the individual in the short-term because it means the individual worker’s day-to-day life sucks and is closer to a psychological prison- no creative freedom, completion of backlog tasks that provide little-to-no real world value, inability to develop his skills as a developer (which otherwise would be compounding interest for the organization’s product development in general), etc. The organizational effects are worse in the long-term- while they might survive for 4-8 quarters on budgeting alone by “forcing predictability,” the effects will be more severe. The individuals in the organization learn to not complete the higher value work (since they don’t have the discretion to override the assigned tasks that in-development reveal themselves to be useless or make-work), and thus not build a product that more accurately reflects usefulness in the real-world. As a result, even if the project gets done on time, it will not be a product worth using. And thus, the organization, or at least that product, will probably die in the long-term.
In other words, by not embracing unpredictability and allowing for real-time micro-changes in the backlog, the organization will fail to get ahead of the real-world threats they do not perceive (competitors, market changes, etc.). They’ll force a model that grows stale and more wrong with each passing day. And they’ll do this to their peril.