End Point

News

Welcome to End Point's blog

Ongoing observations by End Point people.

Review: Practices of an Agile Developer

Practices of an Agile Developer
Venkat Subramaniam and Andy Hunt
Pragmatic Bookshelf, 2006

Practices of an Agile Developer is a fast, enjoyable, often-amusing read. It's also full of wisdom and excellent advice for engineers of all levels, technical managers, and just about anyone with an interest in the software development process. The book makes for a great companion to The Pragmatic Programmer (also co-authored by Andy Hunt), but is more concise and focuses on bigger-picture development practices than does the often-implementation-oriented latter. The authors clearly outline a number of agile methodologies, with attention given to tasks ranging from managing teams, managing time, and keeping estimates realistic, to integration automation, release management, and test-oriented development.

Of particular interest are topics that relate to developer/client interaction, which could apply to a variety of professional relationships (consultant and client; development team and sales team; etc.). The practices outlined present a path to greater success for all parties, with a greater sense of ownership for the client and a greater sense of satisfaction for the developer. Sounds pretty good, doesn't it?

A common scenario in developer/client requirements gathering and project planning involves the production of elaborate specifications of enormous scope; the client is supposed to approve the requirements, effectively saying "yes, this represents everything I want, the way I want it", while the developer must then both estimate the time for completion and, assuming the estimate is accepted, move forward with implementation. In a consulting situation, the client may require a fixed price for the work, rather than just an estimate. This scenario seems perfectly rational, as the requirements represent a point of consensus around which work will be done, bills paid, and so on.

Or so it may seem, until one has been down that road.

Software development is really the business of change; the authors take great pains to illustrate this repeatedly, and they make their point effectively. While requirements and project scope are established, neither the client nor the developer possesses perfect information or a complete understanding of the possibilities. Usability studies may reveal a better way to approach certain problems; market research may indicate more profitable ways of selling goods; any such discovery may necessitate a requirements change on the client's part. Furthermore, during project development, all parties may observe new possibilities implicit in the growing system -- possibilities that would never occur to developer or client without the benefit of actual working examples. Given tightly-specified requirements and strict budgetary constraints, these new discoveries and possibilities cannot be pursued or easily integrated into the project. Software, and its theoretical ability to accomodate real-world change, is stifled by this development approach.

The Agile Developer authors instead favor a model that could be described as a "feedback loop"; the development team works in short, manageable iterations lasting between a few days to perhaps a couple of weeks, and at the end of each iteration provides tangible deliverables to the client. This provides the client with the opportunity to review the results and play a part in steering where the next iteration will go. While a master set of goals or desirables may guide the entire process, the individual iteration is the largest chunk of development work ever pursued at any one time; estimates can be based on each iteration, and each successful iteration brings the software closer to the final goals. Most critically, the process allows all parties to account for change as new information and experience inevitably brings new desires and ideas.

Such an approach fits nicely into other development methodologies advocated by the authors, guided by axioms like "keep it releasable" (indicating that a developer should never go home for the day leaving code in a broken state), "integrate early; integrate often" (effectively requiring developers to work in small chunks that continuously get merged back into the main project code base), and a variety of suggestions regarding test-driven development practices (develop test cases before implementing actual systems as a means of establishing implementation/interface requirements, automate execution of test cases as part of the continuous build process, treat failed tests as bugs, etc.).

Ultimately, Practices of an Agile Developer illustrates just how subject to change all aspects of the software development process can be, and offers compelling ways to build the management of change into your systems and methodologies. Such methods become all the more valuable as companies and individuals increasingly look to the web as a means of delivering a service, and services increasingly appear as unique web-based applications which need to organically grow with their user community, rather than in discrete, versioned releases to be consumed and re-consumed by users over time. Check out this book if you haven't already; it highlights a great approach to which anyone involved in the software development cycle can aspire.

Trouble with MySQL 4.1 under heavy load

Two of our customers running high-traffic websites backed by MySQL (one using PHP, the other using Perl and Interchange) recently ran into serious problems with their MySQL server getting extremely slow or ceasing to respond altogether. In both cases the problem happened under heavy load, with MySQL 4.1 running on Red Hat Enterprise Linux 4 (i386) with the MySQL RPMs built by Red Hat, installed from Red Hat Network. In both cases, we were unable to find any log output or traffic patterns that indicated the cause of the problem.

When this happened on the first server, we tried numerous MySQL configuration changes, and wrote scripts to monitor the MySQL daemon and restart it when it failed, to give us time to investigate the problem fully. But eventually, out of expediency we simply upgraded to MySQL 5.0 with RPMs provided by the creators of MySQL. Doing so immediately fixed the problem. About a month later when another client encountered the same problem, we went straight for the upgrade path and it fixed things there too.

We haven't had trouble like this with MySQL before, from any source. I filed a bug report in Red Hat's bug tracker and Tom Lane quickly pointed me to another similar bug.

Apparently the latest RHN update of MySQL 4.1.20 came just a little too late for our first encounter with this problem, and MySQL 5.0.21 that we upgraded to had the fix in it. It sounds like using the latest MySQL for RHEL 4 from RHN now should work. If you're seeing similar problems, we hope our experience will be of use to you.

At least we can report that the upgrades to MySQL 5.0 were trouble-free. Using nonstandard MySQL client libraries requires you to build new php-mysql, perl-DBD-MySQL, and other dependent RPMs to match, but that's not too hard and is worth the effort if you want to use features from the newer version.