Wednesday, April 4, 2018

Processes And Plans: Some Thoughts

     Before I retired from wage employment – you know, I got a lot more rest back then – I worked at a company that had gone all the way around the bend on the subject of plans. The walls were festooned with posters exhorting us to “plan your work.” “Failing to plan is planning to fail” was one of the most common of the mantras. There was also “Plan the work. Then work the plan.” It was tied in with the craze – sorry, ISO, there’s no better word for your lunacy – for rigid development processes. Such a process, we were told, would guarantee the absolutely repeatable reproduction of whatever garbage we’d already fabricated, such that we could learn how to do it better...and incorporate the lessons into the process, of course. We were regularly subjected to assessments of how well we conformed to the process...even if no “process” had ever been defined beyond a few windy generalities.

     It was a chore to remain even-tempered in the face of all that.

     I was a first-tier supervisor back then. I ran a small group dedicated to the production of laboratory simulations and other support software. My group had the highest rating for productivity and quality of any software group in the company. Moreover, it was considered Mecca for the software engineer: every software engineer who thought well of himself wanted to be part of it.

     Process? We had none – at least, I declined to enforce one. Plans? They were at the individual engineer’s discretion. So were design reviews, code reviews, metrics, and all the other trendy practices of the day.

     “Come to me when you have a problem,” I told my engineers. “If you need help interpreting a spec, I’ll help. If you’re struggling to understand an algorithm or design an implementation, I’ll help. I’ll gladly help with your debugging and validation. But understand me, please: I’m not going to push myself, my approaches, or my opinions on you. If I were to insist on being an active participant in every effort in our group, we’d never get anything done. I have to trust you.

     Talk about this heretical approach was, of course, widespread. So was the bafflement of “upper management” at our results. My engineers averaged six times as productive as the company norm. How could it be, when we had no process and didn’t plan?

     Yet there it was. The numbers were plain. It pissed off the Evangelists of Process and Planning to no end. They couldn’t endure it. They were determined to put an end to it. But we had an impregnable defense: our customers, the groups that relied upon our output to accomplish their tasks, would not stand for any interference in our methods.

     Maybe my chortling at the memory means I’m a bad person.

     A plan is not a solution to a problem. It’s a pathway to follow in the hope of arriving at a solution. If the problem is soluble, there are many possible pathways by which a solution can be found and tested against the relevant specification and constraints. Therefore, unless the plan leads the worker to an acceptable solution, it has no value whatsoever.

     A plan must respond to the needs of the worker. Given how often the requirements for a project change between inception and completion, the plan must remain open to changes as well. But if the plan can be changed in mid-stream, is it really a “plan” in the conventional sense, or is it something else – possibly something we haven’t yet got a name for?

     Viewed from a height, “processes” and “plans” tend to merge into one another. For example, the classical “waterfall” development process for software engineering:

  1. Assemble requirements;
  2. Design an approach to the problem;
  3. Implement the approach;
  4. Debug and test against the requirements;
  5. Recycle the knowledge from the testing phase into the design and implementation. actually the one and only technique for making anything:

  1. Requirements: “What am I trying to do?”
  2. Design: “Do I have a clear idea about how to do it?”
  3. Implementation: “Give it a whirl.”
  4. Debug / test: “Test the product. Is it avocado dip or some other mess?”
  5. Recycling: “OMG! I forgot to add the mustard!”

     The late Gregory Bateson called this deutero-learning: the leap of abstraction and comprehension that takes place when one succeeds in taking the specific solutions to previous problems and generalizing from them to an all-encompassing procedure. Human beings, as far as we know, are the only creatures capable of it.

     If you don’t have a subconscious grasp of this process, you can’t abstract general characteristics from solution to solution. Every problem you face will look absolutely unique, such that nothing you’ve learned or achieved previously is applicable to it. What you’ve learned about slicing ham will teach you nothing about slicing cheese.

     The above quasi-diatribe is significant, not in that it told you anything you didn’t already know – I have more respect for my Gentle Readers than that – but in that it illuminates the cleavage between those who can make new things – products, previously unknown solutions to important problems, original works of art, scientific breakthroughs – and those who cannot. Mastery of any one of those fields indicates the mastery within that field of the great and insuperable Uber-Process: the procedure for making new things regardless of their particular characteristics.

     A polymath is one who masters more than one field. The great polymaths of history – Leonardo da Vinci, Isaac Newton, Johannes Zukertort, Thomas Jefferson – performed the grandest generalizations of the Uber-Process. Oftentimes we admire them without appreciating what lay at the core of their achievements.

     Every one of us possesses the potential for such achievements. But beware the Evangelist of Processes and Plans! His mission is to rigidify your approach, to remove its flexibility by subjecting you to his scrutiny of your personal process and his demands that you justify your variances from what he considers proper procedure. He has a distressing number of confreres throughout Corporate America. Your greatest need as a Maker of Fine Whatevers is to keep him at a healthful distance – for the sake of your health, not his.

     Food for thought.

1 comment:

HoundOfDoom said...

Love this and it's 100% spot on. I'm a Sr. PM that's contracting b/c I could not take the process uber alles mindset of a lot of companies, and determined that if I was stuck in any one company for over a coupe of years, it would be violently detrimental to the people I worked for.

Am obsessed with getting requirements, but once they are in, the teams need to get the work done as best they can. I have some great programmers, but also have had some spectacular flame outs. I love firing those posers.

My only metric is progress. "Show me your results" is the most review they can expect to see. I have been a programmer and DBA, and can usually sniff a rat, but mostly I am pleasantly surprised with what I see.

No a growing problem is that the business can't figure out what they want. Gathering specs turns into a lengthy process, with the business leads retreating into obvious CYA when they find out that THEY are responsible for knowing what they want, and that depending on precog skills from the IT team is not going to fly.

I'm nearing the end of my career in ID, and am glad for it.