The reason architecture was brought into question in Fowler’s paper is because when you are doing evolutionary-based design (ie. bottom-up design, where you articulate first-class entities as they reveal themselves, a lot like how I build my own org-mode headlines), then you aren’t designing a whole gestalt system of components all in one go, what an architect typically does. Instead, you are guiding the growth of the system, identifying boundary points as they reveal themselves. You are more a gardener than an architect.

Clearing the weeds and design or system calcifications, defining BoundedContexts and ApplicationBoundaries, refactoring as necessary.

The reason architects are needed in buildings is because they’re not structures that can be trivially changed. Most likely, you won’t be able to change your mind later, so you get it right the first time.

For software, refactoring is always possible, though it has a cost. So it’s more sculpting and closer to art than traditional engineering disciplines.

The value-add of the “architect” is refactoring direction.

Design == refactoring/gardening/cultivation.

Intermediating between chaos|entropy and order.

Defining the taxonomies as they grow, and managing them sanely such that they are well articulated to the people who use them.

Image by buwaneka | CC BY 2.0

This is a tricky one to navigate and think through. I’ll try my best to articulate my thoughts on Scrum, how it relates to GTD, and some new epiphanies I had about the roles they play in successfully creating software.

Org-mode and GTD are awesome tools for the individual because they clearly lay out all potential paths you could walk and has you start managing them via externalized memory and regular check-ins. In other words, these tools have you write down and outline all possible tasks and projects you may want to pursue. And in doing so, you directly face your list of all possible paths you can take, all possible action steps you can execute on.

And then you perpetually manage this system in regular intervals, always moving forward with your projects in some way.

A well implemented project management cycle is the same in any organization. Scrum tries to package itself as the project management solution to do this. Scrum attempts to be the organizational-level version of GTD. It attempts to externalize memory from the individuals on the team via wikis, issue trackers, and kanban boards. Sprint meetings and retrospectives are the Scrum analogues to GTD’s weekly reviews.

Some organizations are wildly successful with Scrum, while others are 100% mediocre.

Once you understand the value-add of both systems, you can begin to pick apart how users interact with both methodologies, why the methodologies fail in some instances, and why they succeed in others. Quick aside: Part of it is because GTD targets the individual, while Scrum targets the organization. And comparing the two reveals the values and weaknesses of each, particularly of Scrum (I’m not gonna delve too much into the dynamics here- but organizational level solutions almost always fail if the individual isn’t properly taken into account, which is why Scrum is more prone to weakness and failure than GTD when implemented).

So my epiphany is actually pretty simple- project management is probably a fractal pattern. It applies to the individual, the organization, and beyond.

While simple, there are some interesting conclusions on what this means for overall strategy.

At an ideal level, a developer is executing their own individual instance of project-management (eg. org-mode + GTD), the product-development department is executing their own organizational instance of project management (eg. project documentation + Scrum), and the business is doing it’s analogue as a whole (ie. the CEO + board of directors are managing their work as ideally as possible).

The major point is that the strategies align at every level of analysis. From the individual, to the organization, to the entire entity as a whole. The individual GTD-strategy can’t be compromised for the organization. But the organization can’t be compromised for the individual. All strategies have to be in harmony or it fails.

And thus, the key takeaway is that Scrum (or any organizational project management tool) must allow for the individual to implement their project management processes freely. The individual must be given the discretion to make seemingly arbitrary decisions. They must be given creative freedom. In short, they must allow the developer to execute on the tasks that they think are the highest value.

A couple of components of Scrum stick out as bottlenecks to this philosophy, particularly estimates and sprints. Creative freedom means developers aren’t locked into completing a particular subset of tasks, aka the sprint backlog, because they will naturally tackle the highest value tasks out of their own interest and affinity. Estimates are also a similar bottleneck in that they often establish an implicit contract for what gets completed and when. Which immediately eliminates the ability for the developer to work on necessary tangents in the project. AKA it removes the necessary discretionary and creative decisions developers must make and, as a result, will likely introduce it’s own form of technical debt, which leads to time and money costs and even further project delays.

Ultimately, a strategy aligned at the organizational and individual levels means that Scrum estimates don’t matter. If it means the developer doesn’t complete their assigned stories at all, then so be it.

Of course, it’s reasonable for stakeholders to know what was done instead of the assigned stories. If developers went off-the-road and worked on non-story tasks, it should result in exponential returns, since they are deliberately choosing to pursue higher-value tasks than the assigned stories. No one wants to spend their time on make-work, and if the assigned stories are later revealed to be make-work, then it’s better for developers to focus on higher-value aspects of the project. That’s the key value in giving developers creative freedom. That immediate ability to identify tasks that will be a waste of time, cull them, and move on to bigger and better project work.

Forecasting and estimates can play a role in giving semi-realistic projections that allow businesses to make decisions in other parts of the organization, so long as making these estimates is not more time-consuming than it needs to be and so long as all parties understand the volatility inherent in the forecast. (And if you’re interested in the non-linear dynamics of organizational strategy, this volatility can be used to your advantage. We don’t know what the reality of the marketplace is. We don’t know with 100% precision how the market will respond to our products. We don’t have an accurate picture as to the moves our competitors are going to make. By embracing volatility in our own product development, we are allowing our products to be developed in such a way that it will ultimately be a more accurate reflection of the marketplace than by the top-down imposition of what we guess it is).

Thus, based on what I’ve described here, as well as some of my observations of Scrum in general, I have a new attitude towards the methodology- Scrum isn’t inherently bad, but it seems to propagate mediocre dynamics just like the cubicle did. Instead of being leveraged for it’s strengths, and instead of culling out it’s weaknesses, Scrum is often used to impose predictability on an inherently unpredictable process. The results speak for themselves.

A good organization with quality project management is good in spite of Scrum. Because they’d be good anyway without it. The product leader “knows” the spirit of GTD even if he doesn’t know of GTD. AKA the same attitude and values that would drive him to adopt GTD if he knew of it, at least drive him to cull out make-work and to focus on high-priority tasks. He has the same values and therefore would inevitably implement a successful strategy, or find one that aligns with his values. Those same values would lead him to interfacing with the higher and lower levels of the organization with an acute awareness of the limits of forecasting. And if tasked with using Scrum as a methodology, he’d pick and choose the useful components of Scrum relative to the project’s needs and cull the weak parts without hesitation.

Turns out I don’t have an aversion to Scrum and to the idea of project management (hell, I love GTD). I just have an aversion to Scrum when it’s done in a nonharmonious fashion. AKA when it gets in the way of doing useful work rather than being a catalyst for it.

The effects of a dysfunctional Scrum project management cycle are worse for the individual in the short-term because it means the individual worker’s day-to-day life sucks and is closer to a psychological prison- no creative freedom, completion of backlog tasks that provide little-to-no real world value, inability to develop his skills as a developer (which otherwise would be compounding interest for the organization’s product development in general), etc. The organizational effects are worse in the long-term- while they might survive for 4-8 quarters on budgeting alone by “forcing predictability,” the effects will be more severe. The individuals in the organization learn to not complete the higher value work (since they don’t have the discretion to override the assigned tasks that in-development reveal themselves to be useless or make-work), and thus not build a product that more accurately reflects usefulness in the real-world. As a result, even if the project gets done on time, it will not be a product worth using. And thus, the organization, or at least that product, will probably die in the long-term.

In other words, by not embracing unpredictability and allowing for real-time micro-changes in the backlog, the organization will fail to get ahead of the real-world threats they do not perceive (competitors, market changes, etc.). They’ll force a model that grows stale and more wrong with each passing day. And they’ll do this to their peril.

Here is a quick software development methodology cheat sheet.

Note: There are many specific processes defined for some of the methodologies that are beyond the scope of this article. The main goal of this article is to give very high-level definitions of each methodology.

  • Waterfall - You do ALL of one phase, then ALL of the next, then ALL the next (eg. design -> coding -> testing).
  • Agile - Agile is a philosophy, not a step-by-step methodology1. It is a set of attitudes about software development with the goal of improving communication and the overall development process for all parties involved. You are “agile” if you align yourself with the agile philosophy and make your actions in development reflective of that.
  • Scrum - A template of steps to follow for the development cycle. It is an implementation of agile- it is supposed to follow the principles and attitudes of the agile philosophy. At the same time, it actually gives you the steps that you can follow. In this methodology, you have a prioritized todo list, and you incrementally and iteratively complete chunks of the todo list. By performing incremental and iterative development, you allow requirements to be flexible and changeable since you are not committing to completing one entire phase before moving to another one (waterfall). Thus, the development process can better cope with inevitable changes.
  • Extreme Programming (XP) - XP is a methodology which implements a set of rules that you follow to optimize software development and make it easier to collaborate and work with a team while sustaining a high level of software quality. XP is an agile methodology as it aligns with agile principles.
  • Kanban - A software development process that aims to manage the flow of items being worked on (for example, the flow of completion of each item on the project’s todo list). Kanban processes seek to identify and eliminate bottlenecks in the software development lifecycle.
  • Lean - Product development principles that have been translated to the software development domain from lean manufacturing. There are 7 principles- eliminate waste, build in quality, create knowledge, defer commitment, deliver quickly, respect people, and optimize the whole.
  • Cowboy Coding - Developers have complete control of software development. There is no formal structure that guides the project along. The easiest way to think of this is as a “free for all” where you can do whatever part of the project you want whenever you want.
  • Iterative Development - The software is built knowing that future versions (iterations) will be used to refine the project.
  • Incremental Development - A software project is divided into chunks and each chunk is developed by the team, one at a time.
  • No Silver Bullet - A paper written in 1986 by Fred Brooks, in which he argues that /”there is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity.”/

Let me know if I’ve missed anything and I’ll put it up!

Footnotes:

1

In the real world, a lot of people mix up agile and scrum. Sometimes they mix up agile and some other completely unrelated methodology. Just remember that agile is the philosophy, scrum is a methodology, and people will probably get them mixed up.

So you’ve decided you want to be a developer. You want to understand code and be a master of computers. You want to tell them what to do instead of struggling to figure out what they may or may not be doing. But you don’t know where to begin.

I’ll help you out.

Pick a Language…any language

First, pick a language. Any language, ideally one that interests you. But if you don’t know which language would interest you, just pick anyone.

  • Python
  • Ruby
  • Java
  • C#
  • PHP
  • C
  • C++
  • Etc. etc. etc.

PHP, Python and Ruby are languages typically used to power dynamic web sites. Java and C# are powerful languages that are used to power web sites that are heavy on traffic and are often used in game development and in many other applications. C is the mother of all languages and is the source for a lot of linux packages.

There are A LOT of languages. Pick one and learn it’s syntax. A lot of what you learn in your language of choice is transferrable (you won’t have to learn your second or third language “from scratch”).

Learn How To Write And Execute Code In That Language

The best way to learn syntax is by writing programs yourself. This means defining a problem you want to solve and writing a program that solves it. However, everyone has to start from scratch, and if you don’t know ANY syntax, you have to start somewhere. Learn the basic core concepts of the language of your choice:

  • How to write code by following syntax rules
  • How to run the written code

Different languages have different syntax rules (how to write code properly so that the computer can understand it), different data types (eg. number data, letter data and word data), and different tools to help you create or manipulate that data. There are lots of free books and resources online that will teach you. In fact, here’s a whole list of them. Awesome!

There are also lots of great Youtube Channels that have full courses on how to program. I love thenewboston. This was the channel that helped me get my feet wet with mobile application development when I began learning how to program. You can also search YouTube for “[language name here] tutorial” and look for programming tutorials. Before you know it, you’ll be writing all kinds of programs!

Different languages are executed differently. PHP is interpretted, while C is compiled and then the resulting binary can be executed. Java uses the Java Virtual Machine (JVM) and C# uses the Common Language Runtime (CLR) to ensure compiled code can run on almost any machine. It’s okay if you don’t know what any of this means for now. But learn how the language you are choosing is executed- how the computer knows to run the code.

Start Writing Programs

Once you have a basic grasp of the syntax as well as how the language is executed, start writing programs. Here is a key lesson that may not be obvious.

It is okay to research when you write your programs.

It is okay to google. Developers google a LOT. Googling saves time. Googling cuts cost, effort and increases efficiency. And there are lots of things you will not know on your own without googling.

You will almost never write non-trivial programs off the top of your head without consulting external resources.

There are many situations in software development where you will not be able to derive a solution on your own. You will need to use an external resource to solve a problem. For example, you would not be able to intuitively deduce how to make an app work on Android, even if you perfectly knew the syntax of the language Android apps use (Java).

That previous link is to a page of the Android developer documentation. The documentation contains a complete listing of methods and functionality you can use when you write apps for Android. This functionality is collectively known as an Application Programming Interface (API).

There is no way you can know what the API allows, what functions or methods you can use, without doing research. This is a common theme in programming. You need to research. Get good at researching. Get good at understanding your research. And based on what you learn, you can come up with a solution for the problem you are trying to solve.

While we’re at it, here is another key lesson.

Documentation is a developer’s best friend.

Developers are constantly writing code and consulting external resources, especially API documentation.

If you check out the Android API, you’ll see it’s HUGE. How is a developer supposed to learn all of that?

Languages and APIs typically have A LOT of tools and functionality. You will not be able to learn them all without devoting a ton of time, and this will be a wasted effort. APIs frequently change. Functions and methods get deprecated, new ones get introduced, bugs are found and fixed, exploits are discovered and patched, software rots. And thus, APIs are constantly changing, evolving and even growing. This is the nature of the beast.

Here’s how to make sure you don’t waste your time.

The trick to efficient programming is to learn what you need when you need it.

You don’t have to know every single function or method a language library offers. Just focus on learning and understanding what you need in order to solve your problem. As you write more programs and solve more problems, you’ll begin to see what is commonly used, what is not, and how to learn to do what you need to do should you ever be confronted with a new problem you’ve never solved before.

Conclusion

So to conclude, a quick recap. In order to start learning how to develop, you must:

  1. Pick a language
  2. Learn how to write and execute code in that language
  3. Start writing programs!

And it’s okay to use external resources! In fact, you should get good at researching and understanding what you find.

That’s it for now!

Documentation was sparse regarding how to run Robolectric unit tests on a local database file stored on your computer. So here is what I managed to find.

The Problem

Robolectric is a great framework for running Android unit tests without having to deploy the application to your device or to an emulator. As a result, this significantly speeds up unit testing during Android application development.

However, SQLite databases become tricky when you have a premade database on which you wish to operate and invoke tests. This is because Robolectric uses an “in-memory” database for all SQLite operations. This works great, if you are running database creation queries and building your tables directly within the testing environment. Otherwise, you may find yourself looking for a way to test on a premade database.

Don’t worry. If you want to use a pre-made database and run your tests on that SQLite file, Robolectric’s got your back.

The Solution

First, ensure that the Robolectric dependency in your pom.xml file is version 2.1 or later. Support for local disk-based SQLite testing was introduced in version 2.1.

<dependency>
  <groupid>org.robolectric</groupid>
  <artifactid>robolectric</artifactid>
  <version>2.2</version>
  <scope>test</scope>
</dependency>

Once you have that done, place a copy of your database file in the ${project_root}/src/test/resources directory.

When you compile your tests, files in this directory will get copied into an “output directory” containing all the compiled classes and files. On my machine, the directory is located at ${project_root}/target/test-classes. Thus, the file you place in the ${project_root}/src/test/resources directory will not be modified in any way. The copied file in the “output directory” will be modified. Any database operations performed in your unit tests will be operating on that “output directory” database file.

Now create a new Java test file. We’ll call ours DaoTest.java:

@RunWith(RobolectricTestRunner.class)
public class DaoTest {

    // This path is relative to ${project_root}/src/test/resources
    // This path is used in building the absolute path for the database
    private static final string DB_PATH = "/database/MyDbFile.db";
    // This will contain the absolute file path to the database
    private string dbPath;

    @Before
    public void setUp() throws Exception {
        String path = getClass.getResource(DB_PATH).toURI().getPath();
        File dbFile = new File(path);
        assertThat(dbFile.exists()).isTrue();
        dbPath = dbFile.getAbsolutePath();

        // Perform any other necessary set-up operations...
    }

    @After
    public void tearDown() throws Exception {
        // Perform any necessary clean-up operations...
    }

    @Test
    public void testGet() throws Exception {
        SQLiteDatabase db = SQLitedatabase.open(dbPath, null, OPEN_READWRITE);

        // Perform database operations...

        // Perform assertions on query results...

        db.close();
    }
}

That’s it. It really is that simple!

dbPath will contain the absolute path to the database file in that “output directory.” From there, you can invoke SQLiteDatabase.openDatabase() and perform any necessary operations on the disk-based SQLite database file.