Prepared to Launch: Growing up PM at NASA

This looks like it could be interesting!  Sorry for the late posting. 

--- Dave

These are things that my mind drifts to occasionally.   They are real, they happened to me while managing projects, but had a larger life-changing effect.   They affected my personality, my outlook on life and the way my brain works.

A couple I’ve already blogged about, but in the light of “how we managed projects.”  These blogs are a series of how managing projects at NASA changed my life.  I’m sure the same thing has happened to you.  You learn from managing projects and those lessons apply to more than your next project.  They go into your brain and are “compiled” into your personality

Background

The issue to be solved was the Shuttle orbited upside down, backwards – and with ZERO fuel remaining – and it needed to de-orbit and land on a runway.   The shuttle did have maneuvering fuel and a *LOT* of potential energy.   Knowing those limitations, create a model for the safe return of the orbiter to one of several runways.   This problem became known as TAEM (pronounced: as TAME) which stands for Terminal (stopping) Energy Management.  The potential energy of the orbiter needed to be converted to kinetic energy – precisely.

To find a solution to this problem involved aerodynamics, physics, mathematics, engineering and software. A large and highly skilled team worked on TAEM.

A computer program onboard the shuttle made the last adjustments of speed and altitude as the spacecraft started to land. The vehicle began re-entry by firing the Orbital Maneuvering System (OMS) while flying upside down, backward, and in the opposite direction to orbital motion about six minutes before touchdown.  The TAEM software constantly checked the shuttle's position, attitude, velocity, and descent angle, and automatically guides the craft into its final approach to the runway.

If you're really into this stuff, you can read the NASA report on TAEM here 

Working together

There were a LOT of people involved in the formulation of the problem and the solution.  The group had what is called: “Informational Diversity.”  Informational diversity is variations of skills, abilities, and knowledge among team members. This type of diversity is based on different functional, educational and industry backgrounds that constitute information and knowledge resources upon which the team draws.

Conflict

Naturally, there was conflict within the team.  But team conflict can be a good thing.   The traditional view of conflict was developed in the 19^th century, prevalent through the 1940s and still exists today

• Conflict is bad
• Always has a negative effect on projects
• Performance declines as conflict increases
• Conflict must be avoided!

It was the manager’s job to reduce, suppress or eliminate any type of conflict! 

But, there is another approach!

In the “Interactionist View” Conflict is natural and inevitable in all organizations, it may have either a positive or negative effect, and Project Managers should focus on managing conflict rather than eliminating it. 

During the TAEM development project, conflict was encouraged by:

• Asking tough questions
• Invite members with different views to speak
• Appointing a “Advocatus Diaboli” (Devil’s Advocate)
• Consider alternatives.

Overall rules for managing a diverse team

1. Establish a Sense of Mission

One my most favorite stories that illustrates a sense of mission is when President John F. Kennedy was visiting NASA headquarters for the first time, in 1961. While touring the facility, he introduced himself to a janitor who was mopping the floor and asked him what he did at NASA. The janitor replied, “I’m helping put a man on the moon!”  The janitor got it. He understood the vision, and his part in it, and he had purpose.

Project Mission Thoughts..

• Why does this project exist?
• Why is it important?
• Will It will be used?
• Did I had a part in fulfilling the mission?
• Believe it, Say it, Do it, Repeat

2. Establish a Communications Framework

The old-fashioned way of doing this is by posting things on a wall.  It works!  And it’s still used today.

Today, there are many electronic dashboards that serve the same purpose – but, to me – lack the same feel as paper pinned to a wall.

3. Create and live by a set of Team Guidelines

This sounds corny, but it works!  As a group, we would invent, add, remove and modify a set of team guidelines.  Yes, even adults with super-egos can abide by a set of guidelines.  But you can’t just write them down once and distribute them.  You, as a group, need to keep them in mind and remind others of a “rules violation.”

Here the rules we developed from a different team, I wasn’t smart enough to capture other project’s rules.

1. Neatness doesn’t count; accuracy does
2. If in doubt people know about it, write it down
3. Bad news is good; good news is great!
4. Full truth is permitted
5. Keep your charts up to date at all times
6. Don’t roll over or give up
7. Read other’s charts!
8. Stay focused
9. All meetings are held here – on time!

The Project Teams worked Well and TAME worked well.

The Smallest Thing Can Cause Major Problems, and take a long time to find & fix

Life Lessons I learned as a Project Manager at NASA. 

These are things that my mind drifts to occasionally.   They are real, they happened to me while managing projects, but had a larger life-changing effect.   They affected my personality, my outlook on life and the way my brain works.

A couple I’ve already blogged about, but in the light of “how we managed projects.”  These blogs are a series of how managing projects at NASA changed my life.  I’m sure the same thing has happened to you.  You learn from managing projects and those lessons apply to more than your next project.  They go into your brain and are “compiled” into your personality

BACKGROUND

The primary caution and warning system is designed to warn the crew of conditions that may adversely affect orbiter operations. The system consists of hardware and electronics that provide the crew with both visual and aural cues when a system exceeds predefined operating limits.

The primary system's visual cues consist of four master alarm lights, a 40-light array on panel F7 and a 120-light array on panel R13.  The caution and warning system interfaces with the auxiliary power units, data processing system, environmental control and life support system, electrical power system, flight control system, guidance and navigation, hydraulics, main propulsion system, reaction control system, orbital maneuvering system and payloads.   

THE PROBLEM

One day one of the indicators on the caution and warning panel was “stuck on.”  I can’t remember which one it was, but it didn’t take long to discover that it SHOULDN’T be on.  It was a false indication of a problem.  Think of a Project Management dashboard that shows a problem – that doesn’t really exist.  ANNOYING.  And, in this case it was VERY annoying to the crew.  Everyone wants to fly in something that has all its parts working.  I know I do!  The false indication seemed to suggest there was something in the Avionics or Glass displays that was wrong.  That’s how I got involved. 

What could cause a false indication? 

The shuttle used MANY MIL-standard 61 pin connectors for the cables.  A GREAT many of them.   They are very rugged, reliable and only semi-difficult to work with.  And the electrical systems on the orbiter are complex with lengthy runs, packed cables and difficult to get to.  Unplugging a cable is something like standing on your head while trying to unscrew a garden hose.   Meanwhile schedules were slipping, deadlines being missed, testing delayed, EVERYONE was aware of the problem and asked me about it all the time.  It moved to the top of the “squawk” list.  Again, I was encountering a “Significant Emotional Event.” 

Finding the problem took  a long, long time.  Scope here, meter there, inspect visually here.  That doesn’t sound too difficult but take a look at the wiring we had.  And there’s lots of connectors!!  There was a problem in there someplace.  The wires were neatly laced, the pins were crimped on and the 61 pin connectors were all in the right places.   Every time a lacing was cut, it had to be re-done and re-inspected.

My Project:  THE LIGHT WAS ON.  FIX IT!

The charter: Manage a group of about 12 people to scientifically and methodically look for the problem.  

We laid out a plan, devised ways to “split the problem” succcussively into halves until we could isolate it. 

This was a very boring task.  Not thrilling, not doing engineering,  not math…  But it was a problem and it had to be fixed.   After about a week I found it.

IT WAS A BENT PIN

The life lesson learned from this rather simple but important project was that sometimes it’s the littlest thing that can set your project back and ruin your schedule.  But you can’t give up – you must “press ahead.”   I think this is  true of nearly every project.

In other words: “Notice the small things. The rewards are inversely proportional.” -  Liz Vassey

By the way, I can still tie the NASA standard lacing knot.  I use it for everything from tying up bags of leaves to fixing my lawn mower.

Here's what I consider to be the "standard" lacing knot -- a single (not running) lacing knot. 

These  stories are  are real, they happened to me while managing projects at NASA and were a life-changing event.   They affected my personality, my outlook on life and the way my brain works.

A couple I’ve already blogged about, but in the light of “how we managed projects.”  These blogs are a series of how managing projects at NASA changed my life.  I’m sure the same thing has happened to you.  You learn from managing projects and those lessons apply to more than your next project.  They go into your brain and are “compiled” into your personality

BACKGROUND

Early in the Shuttle Program, (the third launch or STS-3) an important mission was for the astronauts to fly Space Shuttle Columbia for its longest stay in space thus far (7 days).  In preparation for that mission, the astronauts planned to spend 40 hours – without stop - in the Shuttle Mission Simulator (SMS) practicing on-orbit operations.

The very large simulator was a mixture of flight hardware and commercial equipment.  The commercial equipment (several very large main frames and 20 or so “Front End Processors” – we’d call them servers today) had the job of creating the environment to make the flight computers believe they were, indeed on-orbit. 

The primary objectives of the mission were to continue testing the "Canadarm" Remote Manipulator System (RMS), and to carry out extensive thermal testing of Columbia by exposing its tail, nose and top to the Sun for varying periods of time.   And, we were running an important and useful simulation using the SMS.  The simulation tested the flight software and hardware and helped the crew establish their procedures, processes and help generally “ring out” the system. There other purposes as well.  Experiments included a plasma diagnostic package, a mono-disperse latex reactor experiment, the first study of insects in space, and many more.  

The Crew in the Simulator During the 7 Day Trial

Everyone had a role in the simulation, Mission Control, Remote tracking stations, experts – I was told there were 800 people world-wide participating in the 40-hour simulation.  Many were “on the loop” or plugged into the audio network we’ve all heard many times on TV.  I had a headset and could hear the network communications.

THE PROBLEM

We wanted to run the simulation for the full 7 days.  But, the computer systems failed just before 40 hours each time.  Not good!  The clever single flight computer voting scheme, didn’t work, the system simply came to a complete HALT – always at about the same time.  Maybe 38 hours, maybe 39 hours…   Emotions flared, crew members were not amused and would leave the building and everyone would yell about something.  This happened about 4 or five hours. Imagine 800 + angry people, all with a headset!

The work-around the first few times was to pause the simulation, set the event “clocks” to 40+ hours and continue.  This worried several of us a great deal.   Why couldn’t get past 40 hours?  Was it the simulator or the flight hardware, the complex simulated environment we invented (the main frames and servers).

AN APPROACH

I built a system (cleverly called “Test data”) so I could monitor the traffic from our main frames to the flight computers.  The idea was to catch whatever was causing the problem.  My little system had probes all over, wires running around to each of the 12+ (large) computers, a disk drive, monitor and a nice hidden spot in the corner of the SMS complex.    For days, I watched the data flow from the main frame to the flight computers. 

After a few more failed simulations, I was convinced I spotted the problem.  (not to get too nerdy, but the main frames were 60-bit single precision words and the servers were 32 bit words.  We had designed hardware to do the “stack and pack” to and from 32 bits and 60 bit words.   I looked like it failed after 30+ hours, but only once!  Once was enough to crash the system.  A real needle in a haystack.

I wanted to stop the simulation right then so I could try and see what caused it.  So…. I did.   I pulled emergency stop.    800 people went off line because young Dave shut down everything. 

GOOD NEWS / BAD NEWS

The good news is that I captured the data and could point to the cause (heat!).  We fixed it easily and within a day or so.   The bad news is that this was one of many of what we used to call “A significant Emotional Event” for me.   I was instantly very, very unpopular with a LOT of folks.  Remember, everyone was already on edge when I pulled the stop.  I imagine people around the world (all the tracking stations) were using my name in short sentences. 

THE AFTERMATH

About a week later, my wife and I were shopping for a new vacuum cleaner at Sears, and one of the crew (pictured above) came up to me and said that I did the right thing. “It was a decisive and bold move well worth the risk.  Thanks Dave!   Later, I was given a hand-made award by the astronaut office for what I did.   It’s very crudely made, and fading now, but it’s my favorite award.  Signed by the crew and John Young (head astronaut at the time).

AWARD FOR REACHING 60 HOURS, 6 MINUTES AND 12 SECONDS.  :) 

THE LIFE LESSON LEARNED

If you believe something is the right thing to do, and the time is right – DO IT.  Don’t wait, don’t hesitate, don’t worry about repercussions.  However, think about your action ahead of time, make sure you’ve considered other options, make sure you know what you’re doing, but DO IT! Have the courage of your convictions.