Project Management at 30 Below — the AGO project cycle, Part 3

So what are the lessons learned that project managers can take for a project like this? When I asked Andy this question, he had a list of lessons right at hand, learned through hard, cold, exhausting experience. I had expected to hear a list of unique planning and management practices to accommodate the unique conditions of the project activities. In retrospect, I should have realized that the practices he and his teammates have learned to apply would be pretty much the same as those we ordinary project managers apply, just taken to a more extreme degree around the theme of contingency planning.

Planning, priorities, procurement. There are limited opportunities for tinkering, iterative development, incremental improvements, and procurement in Antarctica, much less 300 miles from the South Pole at the AGOs, where the nearest corner hardware store is 1000 miles or so away and phone contact can occur only during a window during the day when the communications satellites are visible. In addition, the amount of time the team can spend at each AGO, like the time of the flights to them, is a variable not in the field team’s control. The duration of their visit that year depends on weather conditions and availability of pilots and planes. Hence it’s essential to have a clear plan with clear scope, priorities and outcomes. That way the team can maximize the use of whatever time they do have available at the AGOs; and beforehand they can prepare and pack all of the supplies and tools they could possibly need while they are there. Along with planning the work, the team has learned to do as much of the work as they can before even leaving for Antarctica. On early missions they spent time assembling mechanical and circuit systems in the field, but no more. They now design and preassemble components while Stateside and get them ready to drop in as much as possible.

“Build it 5-10 times bigger than you think it needs to be. Make it redundant. Test it twice as much as you think it needs.”The operating environment is so extreme and unusual that equipment can fail in unexpected ways (like fine blowing ice crystals in the power turbine bearings). Because the systems have to operate continuously under extreme conditions that are not commonly experienced, Andy and Bob perform extra testing under even more extreme conditions in order to allow for margins of uncertainty.

The team can visit each sensing station only once per year, and depending on weather conditions, possibly not at all or not for a full visit. It can cost $20K in fuel alone to get to one AGO. Therefore even though it may seem excessive to spend hundreds or thousands of dollars extra to overdesign, overbuild, overtest, and overplan, it is all a small expense in comparison to the expense of trips into the field followed by a year’s station and multiple science experiments’ failure. In fact, due to uncertainties in the weather and transport availability, it is a real possibility that they might not be able to visit a particular sensing station at all the following year. So building in extra margins for fault-tolerance and redundancy could save more than a year’s data from a station.

Design for reliability and standardization. Planning and packing has become more streamlined as the team has figured out how to streamline designs and standardize the equipment at the sensor stations. Ad-hoc repairs, when required, are that much easier because it’s now more likely that they will have the right tools and supplies on hand.

Bring everything you could possibly need. All supplies and equipment need to be packed and shipped for planned work and unplanned contingencies. During early missions the team relied on some supplies and tools from South Pole Station, but learned that it was much more reliable to bring their own. Doing so required more planning on the front end, but ultimately saves time and improves quality of work in the field.

The result of these lessons is that over several years’ visits the sensing stations have become more reliable and easier to maintain. Four years ago, when Andy started participating, none of the AGOs were reliably producing their own power and functioning for a full year. Now all five are functioning and transmitting data over the course of a year. In fact, Andy thinks it would now be “reasonable to set a goal of getting the power systems at the stations to a point where they could be left alone for 10 years.” (There are still data transmission issues, which the team have traced to modem problems – a simple fix, but one that will have to wait until another onsite visit. The power systems, which have been the main focus for reliability improvements, are now robust enough that they are confident they will run unattended for several years, and the team can turn to other issues.)

The right people. It should be apparent at this point that the success of the yearly AGO field project cycle ultimately depends on people who have a broad set of planning, physical fitness, electrical and mechanical engineering skills as well as a willingness to improvise under tough conditions. This project could not succeed without the right people participating on the field team, along with the scientists who conceived the original AGO idea and design the theoretical experiments from which the sensor stations derive their value. Andy and Bob are both engineers and long-time amateur radio enthusiasts. Andy, for example, has a 30-foot moonbounce antenna with remote tuning and positioning equipment that he periodically erects in his backyard, and a small machine shop in his basement along with an electrical lab and several small spare jet engines. Andy and Bob are also cyclists and hikers and work to stay fit during the year, in part to pass the physical exam and make shovelling snow and lifting structures at altitude easier. (Fitness trainers know that it is easier to stay fit if you have a goal!)

Project management under extreme conditions. When I first set out to write this article I thought it would be interesting to discover the ways in which the success factors for a project conducted under extreme circumstances differs from the everyday hardware and software development projects that I am typically involved in. Instead, I learned that the success factors are largely the same but that certain best practices are amplified. The team is especially emphatic about scope and contingency planning, overdesign, overtesting, and incorporating margins for inherently uncertain operating and scheduling conditions. By doing so they have been able to greatly improve reliability of the systems in their purview and drive down their project and operating costs, and enable a more reliable stream of scientific data for the experiments from which the program derives its value.