On Agile Tools

“When we create Agile teams, we create teams of real, live people. That means our control and communication systems must be built around the qualities and capabilities of people, not robots. We are doing much more in a team room than simply transferring abstract pieces of metadata around about a project.”

Daniel B. Markham’s “Tyranny of the Tools” (via HN, which has good comments too). In short, for co-located teams, simple tools that are as open as our communications beat structured tools which focus the team around inputting data

Posted in Quotes | Comments closed

Institutional memory and deciphering documentation

Some of the methods and notation are familiar, but others are long obsolete. Even where nothing has officially changed, cultural assumptions about what should be documented explicitly or can be assumed have changed, making interpretation difficult. And it would really be nice to have a big-picture overview book. At the end of the project someone should’ve been commissioned to write a book, “What This Goddamn Plant Is: And, How It Works”. That book is effectively being written now, only by archaeologists.

From an engineer’s wonderful account of coming out of retirement to consult on a plant he helped design (via @danmil/@kellan).

Posted in Quotes | Tagged , , , | Comments closed

Burn Out

What I learned is that burning out isn’t just about work load, it’s about work load being greater than the motivation to do work.

Keith Yost, from part 3 of (1 2 3 4) on being a consultant in Dubai

Posted in Quotes | Tagged , , | Comments closed

Making Good Days

From the moment I opened my inbox, I knew things were bad. An overnight process had found SERIOUS PROBLEMS in my latest project, which was supposed to be finished. I jumped straight into the fix, kicking myself for not finding it sooner, despite all my efforts to check. My nerves were fried, my confidence shaken – it was going to be a bad day.

Which reminds me of this blog post I’ve been meaning to write, and the very catchy headline in a recent Harvard Business Review which inspired it: “What really motivates people“. Motivation is one of those Big Questions – how can we be more exited and interested in what we’re doing. A classic view of motivation says that people go to work to make money, so offering incentives like more money motivates people. While this proves relatively true for mechanical tasks, research has shown that for knowledge/thinking tasks, it is intrinsic motivators that work. Dan Pink has some great presentations on this, and he further points to three types of intrinsic motivators: autonomy (being able to set your own course), mastery (the opportunity to develop a significant skill), and purpose (working towards a higher cause). But these are all longer term motivators: month, year, career things, and with the spectre of a ruined day in front of me, I’m thinking on a shorter scale: days or weeks.

Which brings us back to this HBR article, which suggests a different motivator: making progress.

The authors, Teresa M. Amabile and Steven J. Kramer, did a multi-year project to analyze regular workspace factors and how they influence creativity, productivity, and so on. From several hundred participants, they collected daily motivation and emotion rankings, then compared them against daily diaries. They found that good days – days with high motivation rankings – were very correlated with days when the participants felt like they had made good progress1. In an earlier article, the authors gave some details of their analysis with a series of examples from a software team in a crunch.

“Achieving a goal, accomplishing a task, or solving a problem often evoked great pleasure and sometimes elation. Even making good progress toward such goals could elicit the same reactions…Not surprisingly, there is a flip side to this effect. Across our entire database, the worst days—the most frustrating, sad, and fearful days—were characterized by setbacks in the work. Again, the magnitude of the event is not important: Even seemingly small setbacks had a substantial impact on inner work life.” 2

So I went back to work that bad day looking for a way to feel like I had made progress. I tightened up comments, refactored some code. I sent long overdue e-mails, and sat down to properly read about some company initiatives. I broke my biggest task into smaller pieces and got most of those pieces done. And while it still wasn’t a great day, it turned out okay.

Since then, when my starts to go south, I try to resist my urge to stay focused on just one thing until it’s put to rest. I start looking for quick wins – things I can put to rest easily and feel like I’ve made progress in the day. Most of the time, this works. It’s hard to keep up a good set of tasks that give quick wins; my best source is getting down on paper stuff that I’ve been thinking about in the back of my head. And sometimes I’ve got deadline pressure, and I can’t just put work down. For the times when I can use it, it’s great to have something to try to salvage the day.

  1. Amabile and Kramer are not the first to link good days with productivity. In 1959, Frederick Herzberg also did a study on what correlated with good days. Herzberg came up with something called the Two Factor Theory: One set of factors cause job satisfaction – these are called motivators, and are things like achievement and recognition. Another set of factors cause dissatisfaction – these are called hygiene factors, and include things like your salary or relationship with your boss. Motivators are internal to the work being done, while hygiene factors are external – more about working conditions and policies. Herzberg’s study supported his theory that these sets of factors are distinct – so to increase satisfaction involves aiding the intrinsic motivators, while decreasing dissatisfaction involves keeping up with the extrinsic hygiene factors. While Herzberg’s study covered many areas, it still reinforces that idea that the most powerful motivator is our own sense of achievement, or our sense of progress. 

  2. Page 10, Inner Work Life, Teresa M. Amabile and Steven J. Kramer. _Inner Work Life* expands on the analysis with examples from the diaries of a software team working on a short-schedule, high-visibility project. 

Posted in The Profession | Tagged , , , , | Comments closed

Every Technique I Know to Understand Code

Figure out how others understand it

Form a mental model of the system by exploring the UI:

When I don’t really know what a piece of code is trying to do, it helps to poke around the user interface – clicking on buttons, finding screens, trying to see the effect of my inputs. Often forming a model of what’s happening through playing with the interface helps me understand where the code under the hood fits in. And often, I’ll start tracing bugs from the last button or dialog I touch to recreate them.

Get someone to explain a subsystem and its design decisions to you:

When I’m looking at some subsystem and I know what it does in general, but I’m having trouble knowing what’s involved and how it fits together (often because the implementation is more complex than I thought it would be), I try to find someone that wrote it or has worked with it. I’ll admit to not having this down pat, but I try to get them to explain how it works, and what design decisions or thinkings influenced how they did it. This often illuminates the most puzzling convolutions in the code.
[How Coders at Work Understand Code: Everything: {Zawinski - p34, Eich - p158, Bloch - p182, Norvig - p317} Top Down: {Ingalls - p407, Thompson - p479, Allen - p489, Knuth - p600} Refactor: {Crockford - p106, Cosell - p547, Fitzpatrick - p70} Traces: {Steele - p334} Not sure about: {Armstrong, Deutsch, Peyton Jones}]

Read about it

Read the high level driving code/documentation to get an idea of the major structure of the application:

When I’m having trouble figuring out what the major components are or I want to know which parts of the application I can safely ignore, I like to start with the driving code. Sometimes there’s a good core function or class, but sometimes I head for the design documentation to get an idea. That documentation isn’t always up to date, so reader beware, but it can sometimes give a direct answer to questions about the application in-the-large. The nice thing with understanding an application from the top down is that it’s organized hierarchically like a tree in your head, so it’s easy to search and dig in and find relations between things.

Strip out everything but the frameworks and study the lifecycle of the program:

When the application is more coupled with a framework than its own driver, I like to focus on the framework and see where the code gets invoked. I haven’t tried it myself, but I’ve seen recommendations to actually just rip out all the code except API calls in/out to see what the major parts of the application are (reverting it afterwards, of course).

Read and understand the most low-level data structures:

A great way to understand an instant messaging program is to look at the protocol and the message objects. When there are a few core data structures, understanding them and all their possibilities can be a key to understanding the rest of the program. This is when its worthwhile to read through each bit of the low-level objects, and maybe make up some quick-reference sheets.

Add a breakpoint and look through the trace:

When I want to figure out how the application gets from one point to another, I’ll add a breakpoint somewhere, then look at each level of the call stack and understand what each function does to arrive at the next function down. This can especially help when there are a lot of small functions or tricky dependency injections – it gets you right to the code that’s actually being executed.

Pick somewhere important-looking and figure out every single line:

When I’ve run out of places to try to zero-in on what I’m looking for, I fall back on the ever-reliable standby: pick the best thing you’ve got, and read it line-by-line until you understand it. It helps when you’ve skimmed enough to know that it can be read and understood in a reasonable amount of time.

Read the support:

When I’m looking at code and the implementation is still too tricky to figure out easily, I’ll go for some support. Help pages? Unit Tests? Documentation? Source control check-in notes? Development tickets/notes? I take whatever I can find, as long as it’s useful.

Use tools to understand inheritance hierarchies and find patterns:

When I’m dealing with things that are deep in an inheritance tree or spread around several classes, I turn to some automated support. There are several tools out there which will generate UML or, at the very least, show inheritance trees and all the methods defined for a class. Often I’ll find a base class responsible for one thing, and a subclass responsible for extending it to add one thing, neither of which are clear from reading either one alone.

Write about it

Attempt to draw the architecture of the system:

When I’m trying to come to grips with a high level view of the system, I like to try to draw out an architecture and explain the major components and connections in the system. This helps me succinctly describe and remember what the system does, and also provides something that I can hand off to other people, more acquainted with the system for review. And that leads to a more accurate understanding of it.

Write documentation:

When I run across code that seems important but doesn’t have good comments, I’ll often organize myself around writing some good documentation for it. Maybe I run into a class and don’t know what it does, so I read it enough to come up with a quick explanation. Then I read it in depth to see if that explanation is actually true. I find similar things helpful for sussing out functions, especially ones with undocumented inputs. I explain the expected constraints and meanings of each of the inputs, and then the meanings and assurances on the outputs. I like JavaDoc (or, as I work on C++ code, Doxygen) comments for these – they live in the code and are easy to update together, but are also extracted into nice formatted documentation.

Effect Sketches:

When I want to see how a change propagates through a system, effect sketches help me diagram and keep track of what leads to what. I learned about these from Feathers’ Working Effectively with Legacy Code. I start with a change, and then draw out a network of what that affects, so on down the line until I see its impact.

Change it

Add a test case:

When I want to know for sure something works the way I think it does and it looks decently testable, I’ll just check out the right files and add a new test case. And, as an added bonus, I know more about how to use it. As a triple added bonus: more tests!

Change something – anything – and see its impact. Then revert the change:

When I just want to know that I’m changing the place that I think I am, I make a small change and run the program to see it. Maybe it’s just changing a window title, or adding 42 to a calculated value, or printing out some logging statements with background calculations. It all gets me more comfortable with changing and rearranging the code.

Refactor some code to make it clearer:

When I’m reading code and think it could be better organized some other way, I’ll sometimes refactor it so that it’s clearer and more self-expanatory. I don’t just refactor everything I don’t agree with, but I do try to make right things which were particularly hard for me to figure out. Sometimes I’ll also just leave a note: // TODO: these five classes do the same thing – they could be refactored to remove duplication

Find a small bug and fix it:

When I have a whole big system to acquaint myself with, and I just don’t know where to start, I go with something useful: a bug. Figure out a subsystem enough to see where it goes wrong in the given case, and figure out the design enough to see where the fix should go. It’s a good idea to find someone else who can judge whether things are small enough for a starter project.

Find a small enhancement and do it:

When the bugs are too complicated or not in the right spot, find a little enhancement and implement it. Maybe it’s something as small as adding some keyboard shortcuts or more robust UI behavior, or better error messages. There’s probably someone on the project who knows a few common requests you can start with.

Posted in The Craft | Tagged , , , , | Comments closed

Working Effectively with Legacy Code: The Best & Worst

[Working Effectively With Legacy Code, 2004, Michael C. Feathers][1] 3/5 [Stars](http://givesahoot.com/book-ratings/)

When I first read this book, I was quite disappointed. I’d expected a book that follows the title, but Feathers turns this notion on its head before the book even begins. He says, “legacy code is simply code without tests. … It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests,1 we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse” (p xvi).

I didn’t agree with that when I first read it, and I don’t agree with it now, but the entire book is predicated upon this assumption. Feathers encourages doing anything – mangling code, breaking encapsulation, making too complex code even more complex – just to get code under test. On reflection, this is actually a good book about unit testing anything – even those things you thought were impossible.

[[An alternate cover for the book: 'Unit Test Anything: A Guide for Java, C++, and C']][2]

A hypothetical alternate cover

But it’s not a book about working effectively with legacy code.

I wrote last week that I think this definition of legacy code as code without tests is a subset of a larger, more encompassing definition. I also have to take issue with the idea that unit tests are the only way to determine code quality. There are a lot of different ways of doing this, and their suitability depends on your situation: static analysis, spot code reviews, pair programming, code metrics, automated system and integration tests, randomized tests, hell; even manual tests. I love unit tests and I love TDD, but it’s no silver bullet, it’s not the One True Way.

Outside expectations from the title, this is exactly the book that I would expect from someone of Feathers’ background. Feathers was developer and maintainer of CppUnit, the C++ port of JUnit. He’s also been a consultant with Object Mentor for several years, with a focus in legacy systems. Working Effectively with Legacy Code is clearly written by a very smart guy with a lot of success working with legacy systems and by someone who is an expert at unit testing and refactoring.

“The book is highly entertaining and comes across as a conversation with a really sharp, really patient guru developer. Often, it’s a chore to slog through code-heavy books. But Feathers manages to keep my attention with interesting stories, loads of examples, and well-written text.” – J.J. Langr

“The high level goal of the book is show you how to write good unit tests for code that wasn’t designed with unit tests in mind.” – Kevin

The Best

Seams. Seams allow you to change part of the behavior of existing code without changing that code itself. With them, you can add features or alter the behavior enough to test a method. Different types of seams (preprocessing, link, and object) allow you to get in and alter different parts of the code, and each has its own tradeoffs. Thinking in terms of seams makes you recognize these opportunities, or enabling points.

Effect/Feature Sketches. I was immediately drawn to these two techniques for understanding code. Effect sketches help you see the impact of things by drawing out what what they impact. This helps you find a method further down the line that will still let you check the behavior of the first method. Feature sketches are a way of seeing the responsibilities in a class by drawing out the relationships between functions and the member functions or member variables they use.

Scratch Refactoring. This reminds me a lot of experiments – prototyping. But while experiments are usually done in simplified, quick environments outside the code base, scratch refactoring is about just branching the code, refactoring a bunch to see what’s possible, then throwing out the changes and keeping only the lessons. I had never thought of doing this, and I’m using it more now. It’s hard to avoid the prototyping trap (“Looks like you’ve already got it working, let’s just use that”) because it seems like such a big step forward to have gotten it where it is in the refactoring.

”We Feel Overwhelmed”. (p 319) After twenty-three chapters of talking about working with existing code, Feathers has a brilliant two page pep talk. It’s here that his “sharp, really patient guru developer” voice is strongest.

The Worst

Low construction quality. So it’s strange to comment on the construction of the book, but it matters. My copy is falling apart, and I’ve only read it once. The glue on the cover and keeping the page groups together is almost completely gone. My boss had a copy of the book but stopped reading it because it was missing a chapter and had duplicate pages.

Simple book examples that don’t scale. This is a common problem for all software books, but it makes it especially hard when you talk about dealing with huge classes and monster methods. The examples methods are only a page or two long, when my monster methods would double the size of the book. I’m not sure how some of the techniques scale. I tried to use a feature sketch on a huge class of our own, so big that I had to write a script to find all the relationships. But all it told me was that I had a big, giant mess on my hands.

You, and everyone on your team, and all your bosses have to agree: you cannot work without unit tests. It’s certainly not a given that you all agree, or that you can easily sell them on it. This is not really something that Feathers is concerned with, for all that it’s the foundation of the book.

But still…

Writing unit tests for existing code is still important, and I think Working Effectively with Legacy Code is a good book if you want to start using TDD on the legacy areas of your code. I think what I would really love to see is a book of case studies about teams which have done well with legacy code. Case studies are really good for thinking about “how do you approach this situation?” and legacy systems present a lot of situations that are beyond what we know how to deal with.

This slide deck from Orbit One Internet Solutions condenses the book into a 35-slide powerpoint. One from Feathers (I think?) describes the ‘steps’ of working with legacy code using an example. Michael Feathers has a website and a blog of his own, updated a half dozen times a year, and is on twitter.

  1. A side note here from me: he means unit tests, not just any tests. Good unit tests are quick, descriptive, and local. Tests that run for minutes or hours don’t let you change code quickly. 

Posted in Book Reviews | Tagged , , , , , , , , | Comments closed

A good programmer with great habits

I’m not a great programmer; I’m just a good programmer with great habits

Kent Beck, as quoted in Refactoring

Posted in Quotes | Tagged , , | Comments closed

What is Legacy Code?

Is legacy code any “old” code?
Is it code I can’t fix because it might break?
Is it code without documentation?
Or code without unit tests?
Is it code I can’t throw away because it works?
Is legacy code just any code I didn’t write?

While it’s hard to define legacy code, we know what it feels like to work with. It’s messy, opaque. It’s frustrating, confusing. It doesn’t let us work the way we want to. I know something else, too: I’ve got thousands of lines of it, and they’re not going away any time soon.

So I’ve set out on a mission: to get better with legacy code. To learn how to deal with it and make it manageable, not frustrating, and not confusing. To learn how to clean it up and to clear it up.

But to do that, it’s good to have a definition. And in researching my forthcoming review of “Working Effectively with Legacy Code” (coming soon; a spoiler: it’s not the solution to my mission its title implies), I ran across an insightful comment at Slashdot:

> Legacy code is anything developed under a different process than you’re using now. The only thing that remains constant in the recognition of difficult maintenance is this: “We didn’t plan to maintain it the way we’re maintaining it now. – [devnullkac](http://news.slashdot.org/comments.pl?sid=979493&cid=25196045)

Most of our techniques for dealing with software are at their best when we can rely on their simplifying assumptions. Unit tests simplify our development by quickly verifying existing program behavior. Coding styles simplify the task of reading code. Architectures simplify development by making the dependencies between major modules few and clear. When we can rely on these things, it reduces our mental overhead and makes it easy to maintain.

Legacy code is those parts of the system that don’t conform. *Legacy code is things developed without the simplifying assumptions that you’re using now.*

That’s part of what makes legacy code so frustrating – it’s the wild, untamed spaghetti. Its got bones in – traps and pitfalls that you aren’t used to dealing with or which throw your techniques for a loop. It may even be unforkable (have I stretched the metaphor too much?) – so different that the tools you’d like to bring to bear on it, from automated builds to static analysis, become major projects just to set up.

I like this definition a lot because it encompasses most of the other definitions I’ve heard. Defining legacy code as code without documentation or tests is just stating a preference on simplifying assumptions. Defining it as “old code” or code “I didn’t write” is trying to get at that feeling that things are done differently now, and the code from before is *missing* something. It even covers the definitions which describe legacy code as unknown quantities – code you’re afraid to break or code that has to stay because it’s worked for too long to throw away. These ones define legacy code by it’s complexity, by its lack of even basic simplifying assumptions.

And for me, thinking about legacy code like this casts it in a new light. It is no longer a pit of doom, but a new frontier where I’ll need new strategies and I’ll need to put new spins on old ones. I know I’m starting to pay attention to what assumptions I can reasonably make, and which ones I would most like to make to simplify my tasks. Instead of thinking, “augh! This all needs to be rewritten,” I’m thinking, “What can I do to make this workable, to let me make assumptions and fit it in my head?”

Posted in The Craft | Tagged , , | Comments closed

The Pragmatic Programmer: The Best & Worst

“The Pragmatic Programmer: From Journeyman to Master,” 1999, By Andy Hunt & Dave Thomas
4/5 Stars

The Pragmatic Programmer is chock full of useful advice, well written and helpfully framed around 70 tips. Those tips touch on most aspects of software engineering: from developing yourself and coding/design practices to advice for the whole team. At 260 pages, it’s a very good introduction to a lot of software best practices.

And while most of the advice has stood the test of time well, a few bits seem outdated. Having the advice spread out among 70 nuggets can make it hard to remember, and some of the tips are simply reminders of allegories from the book, which won’t refresh your memory if you don’t already remember the book. For a book so easy to read and dense with information, though, it’s easy to recommend to those friends and to revisit yourself – probably several times.

“This book is a hard copy of the distilled knowledge of a couple of exceptional programmers, not some lengthy academic monologue about software engineering theory or the latest methodology.” – Darren Collins

“Honestly, though, I worry that the book is somewhat futile. It may be one of those books that you either understand and agree with already, or you never will.” – Steve Yegge

(Some of) The Best

Don’t Repeat Yourself (DRY): if you find yourself writing the same code or logic in more than one place – stop! Writing it in one place means that you have fewer hidden dependencies. But this goes beyond refactoring code – when you have revision histories in a file and in your source control, that’s repetition too. When you have information in documentation and comments, that’s repetition too.

Don’t Live With Broken Windows: The broken windows theory says that when things are already in a state of disrepair, people tend to allow more and larger infractions. Not living with broken windows means not letting bad code pile up – it will only lead to worse code in the future.

Separate business policies from requirements and interface details: While not one of the tips, in the requirements part, Dave and Andy mention separating the details of interface and business policy. So a request that the system have a tab which lets the user see uncompleted tasks past their due date has both interface details (the tab) which could change based on system convention, and business policy details (overdue -> incomplete tasks past their due date) which may change based on policies (maybe tasks without a due date can be overdue after a few months).

Care about your craft: One of Dave and Andy’s first tips is to care about your craft, and I think it’s an essential bit of wisdom. When you care, you’re always trying to understand and improve. When you care, your heart is in it and you want to do things well. When you care, you do things like read the rest of the book, and go on to read more books or talk about it with people. Caring about your craft is the fuel which drives your future. It was this tip which inspired me to name this blog “Gives a Hoot.”

The Worst

Use a single editor well: Of all the tips, the recommendation to become proficient in your editor by using it for everything seems to have aged the worst. While this might make sense in a land of vi and emacs, where one text editor is used for editing all text, it’s impossible to ignore the benefits of an IDE and the code-specific extensions that it can do (static analysis, integrated debugging, code search, etc…).

Write code that writes code & Don’t use wizard code you don’t understand: My unease with Dave and Andy’s advice on code generators probably comes from my own experience working with generated code. I definitely think there are times when it works well – Dave and Andy describe the generation process as part of the build, and I think that it works especially well when you need to use one model to generate code in multiple languages (Dave and Andy give the example of a database schema and object file, and it is certainly invaluable in things like Google’s protobuf or Facebook’s Thrift). However, code generators make it easy to produce huge amounts of code that isn’t well understood – maybe you understand it because you wrote the generator, but will people who pick up the project from you understand it? I like generators as a last resort tool – it’s very easy for them to go wrong.

Iceberg Tips: The Pragmatic Programmer is about breadth, not depth. There are many, many more things that could be said about each tip. Since it was published, there have been great, solid treatments of these subjects, and that leaves The Pragmatic Programmer as a handy introduction, perhaps The Original, but just a start. There is no further reading, though maybe that would age what is a timeless book. Maybe what I’m really looking for is some online, up-to-date guide with references for learning more.

Beyond the Book

Atwood reproduced The Pragmatic Programmer‘s Quick Reference Guide on CodingHorror. The quick-reference or the table of contents serve as good overview of what the book covers. If you’re curious, but not enough to put down money, Dave and Andy did an series of 10 interviews with Artima Developer in 2003, covering/repeating most of the book (The link goes to the last in the series, which links to the others). Dave and Andy revisit the book 5 years after its publication in a 2004 interview with O’Reilly’s OnLamp, and talk about what they’ve been doing since. If you still want more, Dave has a blog, though it’s quiet at the moment, and Andy does too, though the topics of The Pragmatic Programmer have passed from their front pages.

It is worth mentioning, and perhaps a fitting encapsulation of this review, that I received my copy of The Pragmatic Programmer as a gift from a colleague of mine who used to have two copies, but decided he didn’t need them anymore and gave them both away to the newest members of his team.

Posted in Book Reviews | Tagged , , , , , , , , , | Comments closed

Three Critical Steps

Q: How have you tried to achieve excellence in the work you do?
A: Three critical steps: 1. Show up. 2. Do it. 3. Repeat. But next time, do it better.

Posted in Quotes | Tagged , , , | Comments closed