This is the second of a two-part article by Robert Peake, CTO of the David Allen Company. Be sure to start with yesterday's first part, "Why GTD Matters To Programmers."

Part II: GTD and Extreme Programming

by Robert Peake, David Allen Company

I have to admit that I'm not a perfect adopter of Extreme Programming. We don't program in pairs, for example -- quite the opposite, our coders are flung far and wide, tethered together only by a broadband connection. However, as much as GTD is "advanced common sense", so to my mind is Extreme Programming a form of "best practices on steroids" -- and for this reason, there are not only many parallels, but great crossover when it comes to managing programming projects.

Software didn't used to have to be very adaptable. Mainframe software systems were only slightly less mutable than the hardware they ran on, and dependability -- not responsiveness, new features, or interoperation with some other cool, new application -- was key. Along came the internet, and the ability to patch and change software online. All that changed.

Likewise, while even forty years ago dependability seemed to exist both in terms of job definition and job security, adapting to change may well be the single most important trait of the knowledge worker today. And increasingly, the ability to manage large volumes of information, focusing on what's important and ignoring the dross, has become key. For this reason, a few critical concepts in programming apply to getting software done with maximum efficiency and also suggest some important directions for any kind of project -- they are: regression testing, refactoring, and documentation.

Regression testing is, quite simply, spelling out a list of tests to perform so that you can verify that what you have created works as expected. The "regression" part of it is that you often want to run the same rudimentary tests you ran in the beginning of the project much later in the evolution of the product. This is to make sure you didn't break something. Regression testing goes hand in hand with documentation, which stems from the same inspiration: describing how the thing should work. Once you have described how the thing should work, and made tests you can perform later to prove it, you need to make sure that documentation exists within the source code. In our case, we use automatic tools to pull out the documentation we mix in with our source code, and have created for ourselves an automatically updated manual of how various functions and parts of our system work. We even set up shortcuts so that with a couple keystrokes, we can access documentation for not only the core language, but our own collection of application-specific functions, right from within the source code we are working on at the moment. The result? Six months after writing something critical, I don't have to go crawling back through file after file of source code to figure out how to use that particular building block or tool.

All of this relates to GTD when you bring in the concept of outcome-based thinking. Every time you pick up an artifact of your own thinking (be it a scribble on a note-taker wallet page or a mind map on butcher paper) and then decide that what you've really notated for yourself is a project, the very next question is: what is the desired outcome? In other words, how will I know I've finished this project? What will be the result? Likewise, all the activities in Extreme Programming such as gathering user stories, writing regression tests for each unit and component of the software, and documenting the intended behavior inside the code itself so someone else can later understand what you were trying to do -- embody outcome-oriented thinking for software. Jumping to the end first works consistently and well to line up your thinking in the right direction. Getting that thinking captured appropriately into your trusted system -- and not over-thinking into all the possible ramifications and rabbit trails that will likely only get blown away when new real-world input arrives -- is key to staying agile, flexible, and dynamic.

Another key to programming in an agile way is refactoring. This is going back to what you've written -- not to add features or fix bugs per se, but to clean up your thinking and make it line up with your new understanding of the project. The truth is that thinking takes time, and it evolves over time. So, revisiting the code on a regular basis, purely for the sake of shoring things up and reworking things into a more effective paradigm, may seem like a lateral shift on the surface -- but makes a huge difference in the overall maintainability of the code. Likewise, the maintainability of your life and your trusted system that runs it can also be equated to how often you perform a weekly review. The weekly review is refactoring for life -- a time to take stock, reevaluate your understanding of and progress on projects, make sure all actions are tied to projects and projects to actions, and to scrub out all the stuff that's been either completed or rendered irrelevant by time (sort of like getting rid of the variables you thought you'd need in your code but didn't -- preventing memory leaks and tightening things up for later). Refactoring is as important in life as it is in software.

We used these three branches of Extreme Programming to great success in building GTD Connect. One other critical technique or skill involved was being able to flip from a very high-level creative brainstorming session with our senior people into sussing out the specific outcomes that would demonstrate we've done our job on the technology side. None of this precluded writing up documents spelling out our understanding of what's important and what's nice to have, the timing involved, as well as some of the mechanics of what we are doing and the business side of how much it would cost. But ultimately, being able to differentiate between and translate from the right-brain mind mapping sessions into left-brain outcomes led to just enough of a specification to get our coders excited. Trusting this process gives you the freedom to be wildly creative in the brainstorming session (because you're not nailing anything down yet). It also empowers programmers with the flexibility to find a smart, dependable, architecturally sound approach to solving the problem on the other end -- because when you feed programmers outcomes rather than dictating precise specifications in pseudo-code, you give them the freedom to work within the context of the whole system to come up with the best solution. And you leave room for constructive, creative surprises -- the unexpected killer features that can only thrive in a healthy, outcome-oriented relationship.

One of the tenets of Extreme Programming Merlin mentioned in our panel is the idea that, "You ain't gonna need it." While I never overtly said this to The David in our sessions, the spirit of this approach is something that has pervaded the process of leaning into the great (and so far highly successful) experiment of launching Connect. That spirit of the idea is to build to a need, rather than a nice idea. In balance with this, we also tried to cast our net wide enough in terms of variety of features so that we can test and see what people gravitate toward, and what might fall away. This natural process of meeting customer feature demand on its own terms is highly analogous to the GTD approach of getting through each work day on its own terms, using action lists tied to projects rather than other forms of "best laid plans."

A big key to the mechanics of getting a project as big as Connect done (and then maintaining and improving it incrementally) has been a shared project list accessible by all our developers. First of all, we trust our people. I have never seen the approach of trying to enforce people's job constraints through software permissions really work when it comes to project management. Instead, anyone can add, edit or mark a project complete in our system. This means that our development people can keep their complete project inventory in a shared format and can edit projects assigned by others to keep notes and add important details. We use a Lotus Notes database for this, but this approach could also be easily implemented via one of many web-based applications or even a shared whiteboard if everyone works in the same room.

Another key with all of this, and a somewhat revolutionary approach, is that we don't track bugs and feature requests in separate places. It all has to get done anyway. And because maintaining a comprehensive inventory of everything that needs to get done in one place, accessible by all the relevant people, we can very quickly re-calibrate to focus on what's critical. Sometimes that's a bug. Sometimes it's a great new feature. Either way, we use the system to support us like a giant radar screen, and good communication in person, by phone, email, and IM to determine what's critical and go after it. So far, it's worked great to keep us engaged with our members and our own staff and be responsive to their input.

Connect has been a lot of fun to build, and it's great to watch it start to really thrive. Hopefully this little peek under the hood has got your own gears turning about software development with GTD and perhaps even other areas of your work or life where these analogies hold. If you're feeling particularly geeky, more platform-specific details on the best practices we drew from when pulling Connect together are here.

And if any of this has rung your bell and you want to chat, please drop me a line at robert at davidco dot com, or catch me on the forums. Until next time, happy coding, happy GTD'ing, and be well.

Robert Peake used to teach programming languages to computer science students at Berkeley before earning his degree in poetry. These days he is the CTO of The David Allen Company, where he reads, writes, and thinks about many things in many languages.

Update 2006-10-18 10:05:52 - For your convenience, here's an easy-to-print HTML version of both of Robert's posts.