Was There Process Before Agile?

Reader June Kim recently wrote:

On p.328 of Volume 4 of your Quality Software Management series, the first line goes:

"Here's another example, mixing incremental development and hacking:"

In this Chapter 18, the process models you mention and describe are Waterfall, Cascade, Iterative Enhancement, and Prototyping (with Hacking and Rapid Prototyping as its variants). There isn't incremental development.

"incremental development" as you wrote on that page, doesn't appear before that line in the chapter.

Is Agile An Example of Incremental Development?

QUESTION: Which process model are you referring to when you wrote "incremental development"?
Is it one among the four process models that you mentioned earlier in the chapter, or Is it something different?

ANSWER: First, you have to remember that when QSM was written, there was no "Agile" development craze. We were doing various development process, some of which were given capital letters, some which were not, and some which were "owned" by certain advocates. I was trying to be descriptive then, not favor anybody's pet process.

There were some "owned" processes (using the hated word, "methodology," and charging tens of thousands of dollars for shelves full of notebooks which nobody read). I suppose some of them are still around, but most of the organizations I work with are now smarter than to fall for that fallacy. For the most part, every organization custom-tailored its own process (or, in most cases, processes, plural).

Of course, that's still true today. I don't find many organizations using some "pure" version of agile.

Where Do You Place Agile?

QUESTION: I am interested where you would put Agile process. I think Agile(XP and Scrum, for example) is closest to Rapid Prototyping as you described.

ANSWER: Historically, the people who first named "Agile" processes were borrowing the best of all these methods. You could also say that any agile process is a cascade (or iterative enhancement if you're actually putting each iteration's output into use). Agile is much more than these processes, making explicit many team practices to support the iterative nature of the development.

What Happens When the Customer Won't Participate?

QUESTION: If so, it has the same danger when the customer isn't willing to be the integral part of the process.

ANSWER: That's always the case, no matter what the method, if the customer is reluctant to participate. (Until the end, when they whine, "But that's not what we wanted.")

QUESTION: What could you do in this case? Drop Agile process?

ANSWER: It's not an Agile process if the customer (or surrogate) isn't participating. In fact, I would drop any customer who doesn't participate. That's the rule I use in all my consulting, too. I don't believe you can help people who aren't willing to help themselves.

QUESTION: Or, make the customer be the part of the process? Then how? This is still a hard question to me, even with 10 years of experience in agile.

ANSWER: It's definitely one of the hardest questions with Agile or any method. Hard for most technical leaders because they lack the training and skills to work with reluctant customers.

So, I train them in these skills (a major goal of the AYE Conference and the PSL workshop), but primarily everything starts by simply pausing the work unless and until the customer has been identified and persuaded to participate.

Session Based Test Management Advice

One of the challenges of a blog on consulting is the difficulty of showing what actual consulting looks like. From time to time, though, I receive a consulting request by email which can be answered with a brief email. Brent Plumley recently sent me a request and had given me permission to blog it and my reply, so others may share, and comment.

Brent's Situation

I am currently researching methods to improve our (Session Based Test Management) SBTM debrief/review process in an attempt to make the process more efficient and scalable. Our current process has the Test Manager reviewing 100% of the testing sessions. As the testing team grows, the test manager no longer has the capacity to review all the sessions and as a result a significant backlog session list has accumulated.


- I have several areas that I was hoping you would be able to provide some insight

(1) Testing Session Debrief Trade off

- When a test session is reviewed we consider the following and attempt to achieve a 10/10 score on each.

     (a) Quality of testing performed

          - Did all the risk areas get covered

          - Were the correct test techniques implemented

          - Are there anything missing(different combinations, order of events etc...)


     (b) Quality of Testing Notes

          - Do the notes clearly outline the testing

          - Is the testing reproducible using the notes

          - Does the test properly convey the thoughts and observations

          - etc...



- As a result, the time required to perform the debriefs is quite long.



QUESTION: When performing a session debriefs should we consider a trade off between 'time', 'Quality of Testing Performed' and 'Quality of Testing Notes'. (ie. spent less time, ensure 10/10 on 'quality of testing performed' and 6/10 on 'quality of notes') or, do you think that both (a) and (b) above are important?

(2) Debrief Process

- Even if we can reduce the time spent on session reviews, as the team continues to grow, we will once again be faced with the problem of scalability (more testers = more test sessions = more time needed to review)



- We are currently considering several ways to address the scalability issue


(1) Test Sessions are reviewed by other team members (peer-review process). This will help relieve the strain on the Test Manager of having to review all test session. Also, this will provide opportunity to all test team members to learn from each others unique styles.


ANSWER: This is what I usually suggest to clients in similar situations. The number one job of any manager is to develop the people who work under their mentorship. This is an excellent way to do so.


You can introduce this leadership training opportunity incrementally, with the early leads being supervised by the manager until she finds each person prepared to lead on their own.



(2) Test Manager continues to review the test sessions but depending on the experience level of the tester, only a certain percentage of test sessions will be reviewed (ex: 100% of sessions are reviewed for junior/New testers, and 15% of test sessions for more experienced testers)


ANSWER: Absolutely not. This simply encourages people to game the system by not having their test sessions reviewed. DO NOT DO THIS.


- I have identified several problems with both of the above options. For both options, the test manager may feel out of the loop as much of the testing is not reviewed by them. Ultimately it is the test manager that is responsible for signing off on the testing that is performed. If the session notes are reviewed by peers, or not reviewed at all (only 15% of experienced testers sessions are reviewed), the test manager personally cannot guarantee perfect coverage of testing.


ANSWER: Managers must know how to delegate. If you cannot delegate, you should not be a manager.


Option 2, however, is not a matter of delegating, but of abdicating.


Again, DO NOT DO THIS.


- We are currently leading towards the Peer-Review process. This will ensure that all test sessions are reviewed by someone, which will help guarantee proper test coverage.


ANSWER: I totally agree.


QUESTION: Do you know of any other debrief processes which have been successful for other companies?


ANSWER: Not successful ones.


QUESTION: What method would you suggest for keeping the test manager informed and in the loop? We have thought of bi-weekly team meetings where key points and high level outline of testing performed is presented. Of another option, the test manager will debrief a small sample of the test sessions to get a measure of the quality of testing and debriefing performed by their team.


ANSWER: Forget the sampling. Each person who leads a review should fill out a simple report that everyone, including the test manager, can see. (For more on such reports, see my book, The Handbook of Walkthroughs, Inspections, and Technical Reviews.)


I hope this helps, Brent. It represents half a century of observations on my clients, what works and what doesn't work.



My blog readers, of course, know that no advice works in every situation, though certain mistakes seem to repeat themselves with every new generation. Brent responded by telling me the advice was helpful to him, but let me know how this advice works for you. And ask for answers and clarification.