Missives from the 6th Floor

Software development process often evolves in a similar way. A group starts out eschewing process - "we're more effective without it," "it just gets in the way," or "we don't have time for all that bureaucracy."

Over time, as the group grows, or the projects get more complex, or they blow past a deadline or two due to poor estimation and lack of planning or risk analysis, someone will suggest introducing a bit of process, to avoid failing the same way twice.

If the process works, it's seen as a rational way to remediate some of the failings in the development cycle. A bit more process is added, always pragmatically - to address a specific painful experience.

While the process is evolving, the group is evolving as well. People leave the group - taking with them the "why" of the process, and leaving only the artifact. New people add more process to address new failings - either experienced in other groups, anticipated, or observed as failings in the current group. Looking at the group a few generations after this cycle starts, there are parts of the process no one can explain - they're just steps that must be taken, without understanding. Even if the "why" is written down (an advantage that software types have over monkeys, though they're often not good at it), the visceral "why" is gone, replaced with a more abstract "why."

It's not just the process the large that evolves this way - code does as well. A body of code evolves - it changes to meet changing requirements. It changes as defects are fixed, features are added, and the subtle nuance of the problem are better understood. A long-lived enough project undergoes the same sort of lossy evolution as the process as a whole. Code is changed (hopefully) by people who have a deep and immediate understanding of the "why" behind the change. The good ones even document the "why" in the code, but the next person who has to make changes to the code-base often doesn't have a direct link to the reason for the change. It all starts to merge together until "it's just the way it's always been."

At that point, in a complex system, making changes is scary - at least, it's scary to a developer who actually thinks about what they're doing. They don't understand the system the way the original authors did. That's not to say they understand it worse, just that they understand it different.

This is a large part of what's meant when people say that "reading code is significantly harder than writing it." The "what" and "how" are typically relatively easy to understand. It's the "why" that's difficult.

That's why neophytes have such a hard time learning to comment code well - they focus on the "what" or the "how," - because to them, at that moment, the "why" is obvious.

If only it stayed that way.