“The product shall.” Market requirements typically define the problems your product will address using a formal, stilted language known to all technology people. For some reason, the verb shall be “shall”—not “should” or “will” or “must” or “it’d be neat if.” Maybe it goes all the way back to the Ten Commandments: You Shall Honor Thy Father and Mother; You Shall Not Murder; You Shall Not Steal. You Shall Not Write Requirements Without the Verb Shall.

I’ve completely given up on formal requirements. We’re just terrible at writing them; and, still, the development group wants more detail—and more and more and more.

The requirement is the problem

Product managers write horrid requirements that are littered with buzzwords, ambivalent language, and non-specific performance parameters. They read like somewhat-technical marketing hype. And developers have to make sense of the requirements. They complain, “I cannot program to these requirements.”

And they’re right.

So product managers try to become more specific by writing what I call ReqSpecs. Part problem, part implementation—and impossible to use. Developers then complain, “Don’t tell me how to do my job,” because the requirement now explains how the feature should be implemented.

And they’re right again.

Developers say they cannot program to Product Management’s requirements. And they’re right. What they need is a functional specification. ReqSpecs don’t solve the problem.

Let’s clarify some terms.

A requirement is a short statement of the problem.

A specification is a detailed description of how to solve the problem.

Alan Cooper, often called “the father of Visual Basic” and an expert in interaction design, argues that most projects fail because they do not have a spec at all. This is because most companies do not have product designers or architects. Product managers create desired feature lists; developers write code. But neither the product managers nor the developers are designing anything. Would you hire a carpenter to design a house? Of course not. You would hire an architect. Yet programmers often write complex software products without a design.

Cooper suggests we add a role that is new to most Development organizations: the product designer. The designer analyzes the problems described in the requirements and then creates a specification, at a functional level, to describe how the problem can be solved with the product. The designer delivers a recommended approach to solving the problem, serving as the bridge between Product Management and Development.

What’s a requirement?

In eXtreme Programming (XP), a requirement fits on an index card and is delivered in the form of a story. This requirement also comes with an implicit “agreement to discuss,” so that developers fully understand the problem.

Rather than being in the requirement, implementation details must be in the specification. A specification is the designer’s intended implementation to solve the problem.

In his excellent series on “Painless Functional Specifications” (www.joelonsoftware.com), Joel Spolsky, CEO of Fog Creek Software, writes:

“…on any non-trivial project (more than about 1 week of coding or more than 1 programmer), if you don’t have a spec, you will always spend more time and create lower quality code.

“A functional specification describes how a product will work entirely from the user’s perspective. It doesn’t care how the thing is implemented. It talks about features. It specifies screens, menus, dialogs, and so on. A technical specification describes the internal implementation of the program. It talks about data structures, relational database models, choice of programming languages and tools, algorithms, etc.”

So, a requirement states the problem; a specification states the solution. Spolsky defines two kinds of specification: functional, which is the intended approach to solving the problem; and technical, which is a detailed internal implementation.

In the end, we all have a critical role to play:

• The product manager finds and quantifies market problems, articulating them in the form of requirements.
• The product designer writes a functional specification describing the approach to solving the problem.
• The product developer creates a technical specification that fully describes how the functional specification will be implemented.

Why the waterfall fails

Every company and every consultant has a laundry list of acronyms for the documents that must be produced: BRD, MRD, PRD, SRS. Enough already! There are only a few documents required: the problems (the basis of the market requirements document) and how we will solve them (a product specifications document).

In the old days, tech companies followed the “waterfall” model: Documents moved downstream one department at a time, from requirement to specification to build to test. But the final result—the product—looked nothing like the requirements. Product managers were always surprised in the end.

The waterfall approach fails. What’s missing from the waterfall is collaboration and iteration.

Imagine going to a college seminar and not being able to ask the professor any questions! No matter whether or not you understand it, you’re being tested on the lecture without being able to clarify any of the professor’s points. How likely are you to do well in that course?

And now imagine that college ends with a single exam—without any tests along the way. In four or five years, you sit through 60 or more courses–ranging from Western Civilization to a foreign language to biology to women’s studies to macro economics to business law to volleyball. Now take a single exam to receive your diploma. No one could pass such an exam!

Isn’t it the same for products? Imagine a software development effort that takes multiple years to deliver—without any opportunity to test your progress along the way.

Just say “No”

Years ago, I joined a product team that was entering its third year of programming without a product release. Everyone on the team was despondent. Over the course of two or three years, every role had been filled more than once. Long ago, they had lost any hope of succeeding, and all of them were looking for jobs elsewhere.

I took one look at the requirements and saw the problem: The executives were constantly adding new requirements. Every week brought new features to the list—and features were being added to the list faster than they could be programmed into the product. The solution was simple: Someone—and that someone was me—had to stand up to the executives and tell them to stop! We were no longer accepting requirements for the next release. Here’s how we did it.

I worked with the development lead to make a comprehensive list of requests from our customers, executives, and sales people. We estimated the effort for each request and put them all into a spreadsheet. It showed we had requests for more than 10 years worth of programming to do. I noted the contractual commitments that were pushing the strategic priorities to later in the roadmap. I showed the executive team the ramifications of their decisions.

We needed someone to say “No” to change. And I accepted the role.

The Capability Maturity Model Integration (CMMI) defines the level of discipline in a company on a scale of 1-5—from anarchy to nirvana. Wikipedia describes CMMI Level 1 this way:

At maturity level 1, processes are usually ad hoc and the organization usually does not provide a stable environment. Success in these organizations depends on the competence and heroics of the people in the organization and not on the use of proven processes. In spite of this ad hoc, chaotic environment, maturity level 1 organizations often produce products and services that work; however, they frequently exceed the budget and schedule of their projects.

Sound familiar? Does your company rely on the “heroics of the people in the organization” to deliver products? Most companies I encounter need a little more discipline. And they need it in the executive team—not the rank and file. Someone needs to be there to reject all change requests.

I frequently take the Acela train from Washington, DC to New York City. It leaves every hour, on the hour, and arrives in New York three hours later. Every time. Because once the train leaves the station, no one changes its destination. Like it or not, you’re going to New York.

Building products is like moving a train. It takes a long time to get everyone organized; it takes a long time to get started; and, most of all, it is virtually impossible to change direction once the train has left the station. And the good news is that there’s another train in an hour.

Agile is happening

The brilliance of agile development is that we break the job into small chunks, work for two weeks (or four), and deliver something. New requirements can always be added to the next chunk of work since we’re starting a new iteration soon.

Agile teams are designed to deliver small amounts of functionality with rapid iterations. Every two weeks, they deliver another product feature that could be released to the customers. Agile teams are designed to help companies be more responsive to market needs.

Or are they?

Agile is often an attempt to manage our executives rather than to be more responsive to the market. The executives keep changing their minds, adding new feature requests, flip-flopping on issues. The agile approach of development is a technique in forcing executives to choose.

Agile development isn’t so much a new approach to programming as it is a response to bad management. In the past, we spent too much effort documenting exactly what would be delivered before a line of code was actually written. We wasted too much energy getting precise estimates long before those estimates could be considered reliable.

Everyone in Development knew these methods weren’t working. But management didn’t know. So developers became increasingly frustrated with the planning process. Management enforced dates that no one believed; management required detailed documentation and schedules long before the details were known. No wonder developers were frustrated.

Long ago, we used to be agile. A customer would ask for a report, and we’d show up with 132-character wide grid paper to design the report. “Company name? Okay, 40 characters plus a space is 41. Invoice date? Six characters plus one space. What else?”

Then computing became a key part of every part of the business world. And the head of Development wanted to know how long you’d be working in Accounting because the Manufacturing project needed your skills, too. And customers wanted to know how much the report would cost before they committed to the cross-department billing. And then HR wanted to know how many actual hours were spent in each area for internal billing of your time. These are all legitimate requests, aren’t they?

Working as a developer in a business involves working as a business—which means sourcing and scheduling highly skilled workers.

The big problem agile development is trying to address is not so much that management is bad; it’s really that management is early. They want to know too much too soon…long before the development team actually knows. They ask for estimates, get our guesses (and they are guesses), and then announce delivery dates for the project.

Management mandates rigor and precision before the scope of the work is truly understood. “How long would it take you to build it?” “Well, that depends on what it is, doesn’t it?” “Yes, but give me a date anyway.” Management over-commits Development all the time.

Developers see the product managers as senior management’s police force. And I have to admit that this is somewhat legitimate. Haven’t many product managers imposed dates on projects they didn’t truly understand? Haven’t many product managers enforced process and documentation beyond what is necessary?

So finance people and sales people and marketing people are making scheduling decisions for development projects they don’t understand based on inadequate information. Sounds like a recipe for disaster. And it is. Thus, agile development was born!

Where does product management fit in agile?

One of the key aspects of agile programming is an onsite customer. While this makes sense in a custom programming environment where customer and developer are in the same building, it’s unworkable for vendors. How do you have an onsite customer in a vendor model?

Answer: The product manager serves as the customer representative in planning and requirements definition.

Building a repeatable product requires feedback from many customers, not just one. So someone needs to aggregate the requirements from many sources into a single coherent set of requirements. Answer: The product manager defines the requirements and the product roadmap for a market of customers.

Delivering a product to a market of customers requires synchronizing the software, hardware, services, documentation, promotional programs, and sales tools to present a complete product to the marketplace. Answer: The product manager must integrate all schedules, so we can deliver and support the product in the market.

As product managers, we should support the ideals of agile development. We want some process, but not too much process. Small iterations give us more flexibility to adapt to change. Team collaboration means less time is spent documenting, leaving more time for doing.

We want to assist the team in delivering a complete product to a market and creating a product that people want to buy. Whether the product manager uses an Excel spreadsheet or a bunch of index cards wrapped in a rubber band, it’s imperative to maintain a prioritized set of market problems to be solved in the next product iteration, as well as a vision for future product generations.

Successful product teams are agile, combining collaboration with small iterations. The key to any agile team is building products that people want to buy. To do that, an agile team needs a messenger for the market, a product manager who thoroughly understands the problems facing today’s customers. In agile programming—and frankly in any programming model—the effective product manager serves as representative of a market of customers.

Steve Johnson is an expert in technology product management. He works for Pragmatic Marketing as an instructor for the top-rated seminars Practical Product Management, Requirements That Work and Pragmatic Roadmapping. Steve is a frequent presenter at technology marketing forums throughout the United States and Europe, author of many articles on technology product management, and the writer of the ProductMarketing.com blog. Contact Steve at sjohnson@pragmaticmarketing.com