Demonstrating Compliance with a Performance Standard

Performance standards are the new* “in” thing in the regulatory world. Everyone wants them. Everyone notes how they offer greater flexibility, greater opportunity for innovation, and greater speed in adopting new designs. Everyone wonders why they didn’t do them sooner.  When considering them, however, rule writers should keep in mind that the regulated people need to have some certainty as to what an acceptable demonstration of compliance might be.

What are these great things? In the regulatory world, performance standards are requirements that mandate satisfaction of a goal instead of requiring a specific design as the solution to a problem. Rather than saying that your time machine must carry six ounces of dilithium crystals for every fifty pounds of weight, the performance-based regulation says “Each time machine must possess sufficient power to return all occupants to the present.” The first version of the requirement envisions only one way of getting everyone back to the positive now, only one ratio, and only one acceptable power source for achieving that goal. The second approach focuses on the underlying goal, and allows any power source, any method, and any ratio so long as it works. Returning time travelers to the present has to work, because you can’t walk into today from yesterday but one second at a time, and if the time machine dumps you 24 hours earlier than now, you’ll stay there, possibly run into yourself, and create a temporal vortex in the time/space continuum. No one wants that.

The mandatory design solution is not without its admirers.  It’s very easy to administer:  if the time machine weighs 100 pounds, it better have 12 ounces of dilithium crystals powering it.  No one need perform any additional analysis.  It provides certainty.  It’s what the guy who invented the time machine used, and he always brought everyone back, except for the time that wasn’t his fault.  Lastly, the mandatory design solution forces manufacturers to ask for waivers and exemptions, so the Time Stream Administration can look at any deviations individually.

Nonetheless, there is a growing consensus that good government requires performance based standards for purposes of transparency, encouraging innovation, and avoiding unnecessary costs.  These are all virtues.

However, when it adopts performance based requirements, an agency should not lose sight of making all its requirements transparent.  There is one last requirement at issue, and it plays an important role in the whole process, namely, the demonstration of compliance.  At one end of the spectrum, we can be pretty sure the TSA won’t be satisfied with an applicant’s bald statement that his machine has enough power to bring everyone back.  At the very least, the agency will want to know the proposed power source and how it works.  What more would the agency want?  A computer model or actual real-time testing?  How many hours of testing or how many successful journeys would qualify a time machine using chewing gum as its power source?  Should an agency mandate reliability and confidence levels?  Should it put that out as guidance rather than a regulation?  If it does, will it wind up treating similar applicants differently?

Agencies don’t always put their demonstrations in their regulations.  One of the prettier performance standards I’ve seen is for human space flight.  In 14 C.F.R. § 460.5(b), the FAA requires that “[e]ach member of a flight crew [aboard a licensed or permitted launch or reentry] must demonstrate an ability to withstand the stresses of space flight, which may include high acceleration or deceleration, microgravity, and vibration, in sufficient condition to safely carry out his or her duties so that the vehicle will not harm the public.”  “In sufficient condition” to carry out one’s duties shows that a commercial flight crew member need not be a superhuman astronaut.  One might only need to retain sufficient consciousness to work the controls .  That the flight “may” include high acceleration suggests that all flights might not include that particular stress of space flight.  If a capsule gets to space via a balloon, the operator might not need to demonstrate to the FAA that the crew member can withstand high acceleration.  The requirement, in other words, is tailorable to the technology, and an applicant need only demonstrate that the crew can withstand the stresses of his particular vehicle.

However, what the regulation does not say is what a successful demonstration of compliance looks like.  Does the flight crew have to undergo the anticipated stresses?  How many times?  To what level of reliability?  Nine times out of ten?  99 times out of 100?  The answers to these questions may reveal hidden costs of the regulation.  While each applicant gets to make his case for his vehicle, operators of similar vehicles should be treated similarly.  One crew member should not be required to undergo hours of high acceleration while another one is subjected to only minutes.

Consistency and fairness suggest that these unwritten “requirements” be made public. If the FAA finds one method of demonstrating compliance acceptable for certain circumstances, it could let everyone else know by publishing that method in an advisory circular. (The Administrative Procedure Act also requires the publication of an agency’s opinions.) Then, other operators with a similar flight profile would know that they could follow that method without long talks with the agency.  Alternatively, that same operator might have something less costly in mind and could go to the FAA and makes it case for using a different demonstration.  That demonstration could also be shared.  Publication, of course, carries concerns regarding proprietary information, but those can be worked out.

None of this is to suggest that performance standards aren’t awesome.  They are, but no one should lose sight of the question of how much advance notice is necessary regarding the demonstrations of compliance.

*”New” is a relative term.  In the regulatory world, “new” can span a couple decades.