80/20 Rule in Software Development
Most people know of the concept behind the 80/20 Rule. In context here, it generally states that 20 percent of a population or sample consumes 80 % of the resources. Attributed to a 19th century economist by the name of Vilfredo Pareto, who realized that 80% of the wealth in a given population was concentrated in the hands of 20% of the population, the observation has found its way into many disciplines.
Why this may be a general rule is often debated. Current research seems to suggest that variables that make up the population are time dependent, and follow a power distribution of exponential decay. William Reed, from the Department of Mathematics and Statistics at the University of Victoria, BC. offers a formal explanation. In his paper, "The Pareto, Zipf, and other Power laws", he states that phenomena exhibiting power-law behavior, of which the Pareto law is part of, can be modeled by a differential equation for geometric Brownian motion. The details of his research are beyond the scope this article, but interesting points to note are that the variables being studied each have unique starting values, and vary over time, and that the starting time for each contributing member is unique. If we choose our variables correctly, and if the growth rate of the contributing variable is negative, then each member of the distribution spends most of its time in an insignificant state. In essence, 80% of the time they don't count.
How a Pareto Distribution Maps to Software Development
In software development the typical process is that of requirements gathering, requirements review, design, design review, coding, code review, component testing, integration testing, release, and maintenance. In short, projects start with a set of requirements, and then go through a series of refinements where the original specifications are compared with the end product for completeness and correctness and against real-world usage for additional refinement. During each revision, if the requirements of the development process follows a Pareto distribution, then a few key issues will manifest themselves and need to be addressed at each new revision. Old requirements will fade into the background, and new ones will arise with their own surrounding context and measure of importance.
Practical experience tells us that at the beginning of a software project those problems may include such issues at platform of choice, language of implementation, database, data model, mode of access and key functionality of the application. Over time, the continued refinements may introduce requirements that address how the application works for a specific group of users, or in concert with other external applications.
The Added Benefit of Open Source
Engineers often use the Pareto principle in the management of engineering design and development. The age old adage is that the first 80 percent of a project is easy compared to that last 20 percent. More specifically, in a formal process of prioritization of requirements using Pareto analysis, each unique requirement is given a ranking as a percentage of the entire project. Individual requirements are then presented in a bar chart from most significant to least significant. To apply an 80/20 Rule of requirements management, a line would be drawn from left to right as the cumulative percentage total of all requirements to the left of the point. A line is then drawn at the 80 percent mark, with those to the left, the significant requirements, and those to the right, the insignificant requirements.
Software companies like Microsoft understand this, and design for a product that addresses 80 percent of the requirements, leaving the last 20% as customizations by the end user. Most applications today have been designed to include some form of customization or extensibility including user preference settings, scripting languages or APIs for custom extensions using more traditional languages.
Open Source applications add one further benefit, not present in proprietary, closed source applications. Not only is the software customizable via APIs and preferences, but because source code itself is accessible, the actual application can be customized to fit the needs of a particular environment or continually refined over time.
A Practical Example
Plone is a well known Open Source application designed as a customizable content management system.
If we take a way-back tour of the plone.org website via archive.org, we can get a sense of the development history of the application. (Note that the wayback machine allows you to visit a website in a point in time based on the crawl history of archive.org.) The notes abstracted from the plone.org homepage over time (shown below) hint at a project that evolved from a set of skins for an underlying application server (zope) content framework, to a full fledged content management system, complete with its own domain specific customizations:
Jan 12 2002: (Skins)
plone.org is the website for the Plone visual style and skins for the Zope CMF in general. More content will be added soon, we are finishing up the first real release of Plone at the moment.
Jan 26, 2002 (Framework)
Plone is a theme for the Content Management Framework aka CMF. CMF is an add-on for Zope. It's intended to make the out-of-the-box experience of the CMF as pleasurable and easy as possible.
May 23, 2003 (CMS)
This is the community site for Plone, a userfriendly and powerful Content Management System.
On this site you will find information regarding Plone - documentation, news, discussions and mailing lists - and you will also be able to download the latest release of Plone to try it out yourself.
Plone is built on top of the open source application server Zope and the accompanying Content Management Framework which has thousands of developers around the world.
The web site you are looking at right now is built with Plone - in fact this is the way it looks like out-of-the box, with little to no customization. Plone is easily adapted to very different needs, and comes default with several professionally designed layouts that you can base your site on.
Plone is ideal as an intranet server, as a document publishing system and as a groupware tool for collaboration between separately located entities.
Nov 24, 2002 (The dominant Open Source CMS application)
[in addition to the content above, form May 23, 2003, the following additional text was added]
I won't do the complete rundown of all the "competing" open source content management frameworks. I'll cut to the chase: the winner is Plone. The package with the most tools, the most professionalism, the most traction, and above all, the most buzz. - Mike Sugarbaker, from the Open Source Content Management Conference 2002
April, 21, 2003 [demo site created]
Demo site created
June 9, 2003
The Plone Team has released Plone 1.0, and is currently working on the next release, see the roadmap for more information.
While the above historical outline lacks the details associated with specific requirements of each revision of Plone, the message is clear in its evolving target market and goals. What's interesting to note is that Plone started as a simple customization for an existing open source application, and then became a project in its own right. With a little effort, one could imagine graphing the set of all possible requirements at each revision and applying the Pareto principle to each revision.
If such an exercise was performed, I would imagine that the resulting product would be the Plone we have today and furthermore, that most good software development naturally follows the Pareto process.
I will also go one step further and conjecture that even with today's maturity of Plone, we are still only 80 percent done, and there is a lot of work left to do. The proof, however, is left as an exercise for the reader. For now, I will be customizing Plone for my own application. If all goes well, I'll share how it went in the next Link.
Bob Kaehms - Media Net Link