[This article was first published in Army Sustainment Professional Bulletin, which was then called Army Logistician, volume 1, number 2 (November–December 1969), pages 8–11, 24–25.]
“… in the process of development and engineering, reliability, maintainability, and human engineering have, all too frequently, been sacrificed in attempts to obtain the best operational characteristics. Following this road practically guarantees that we will produce something less than the best possible weapons and equipment.” — GEN F.J. Chesarek
To the soldier in battle, reliability and maintainability of his weapon and equipment can mean life or death—and he knows it.
Forged in hard fought battles in Vietnam, this opinion is the result of personal observation and experience.
Although hardware performance in the user environment can be predicated with a reasonably high degree of accuracy, actual field investigations disclose that more positive steps can and should be taken to insure maximum performance. Although some progress has been noted, improvements in the reliability and maintainability of weapons and equipment have not been spectacular. Much remains to be done.
The use of solid state components and printed circuitry improved the reliability of many electronic items. Similarly, permanent lubrication applications and the use of Teflon and nylon in bearing reduced some maintenance requirements. On the other hand, the quality of fastenings, welds, materials, assembly and design continues to pose real challenges in terms of quality assurance for the developer and producer.
Climatic conditions in Vietnam also present an endless variety of problems. The Army mechanic’s problem in Southeast Asia varies with the season of the year—and the level of hostile activity in his vicinity. At one time, he contends with temperatures of 100°F, a relative humidity of 80 percent, and dust to his shoe tops. When the season changes, the fine, gritty dust turns into sticky mud. This, combined with new climatic factors, makes his task more difficult. Something as simple as the replacement of bearings or seals without contamination becomes a major operation.
Reliability and maintainability of material are prime considerations in any military operation. They are identified as principal characteristics of equipment, and the status of these characteristics is assessed throughout all phases of its life cycle.
The U.S. Army Materiel Command (USAMC) has the primary responsibility for insuring the reliability and maintainability of Army materiel. The machinery available to the USAMC commander for assuring, validating, and demonstrating the reliability and maintainability of Army materiel exists largely in the U.S. Army Test and Evaluation Command (USATECOM). The command test structure is well adapted to meeting new or increased requirements posed by technological advances or brought into view by unusual conditions in the field.
The test-oriented workforce includes representatives of almost every technical and scientific discipline, military occupational specialty, and trade.
Relatively new as organizations go, USATECOM was established in 1962 during the overall reorganization of the Army. As a major element of USAMC, it is made up of a headquarters located at Aberdeen Proving Ground, Maryland, plus environmental test centers, service test boards, proving grounds, and ranges. These are located in a dozen states coast-to-coast, and in Alaska and Panama.
Environmental test centers are maintained in the tropics, the arctic, and in the desert where test items are subjected to extreme weather conditions. Also available are test facilities which simulate the environments, both natural and induced. These include temperature, humidity, vibration, nuclear effects and other conditions.
User or operational-type tests are conducted at the service test boards. Military personnel who use the items operate and maintain the equipment in the field. During service testing, the tactical environment is simulated as closely as possible without compromising the soldier's use of the test item. In short, the service test is the "proof of the pudding.” It determines the degree to which an item meets the individual elements of the Qualitative Materiel Requirement, the Department of the Army approved document which describes desired or essential characteristics of new materiel.
The service test is controlled to insure that objective data are obtained. The trend is toward the use of more data collection instrumentation where it can be introduced without compromising the test.
Increasing cost and complexity of modem military equipment, the importance reducing leadtime, and the necessity for accelerating the decisionmaking processes put a premium on the accumulation of sound test data and factual knowledge of test conditions. In the assessment of reliability, for example, it is as important to know why failures occur as it is to know the conditions under which they occurred.
Testing For Reliability
Testing for reliability and maintainability involves development of sound procedures. Determining "how much testing is necessary" is frequently in question. Availability of funds may limit the number of items tested or the amount of testing done. This calls for ingenuity in framing test plans, procedures, and analytical methods that provide acceptable results at minimum cost.
Cost is only one factor to be considered when determining how much testing is to be done. The number of samples needed for a statistically valid assessment must also be taken into account.
The question becomes more complicated in testing for extremely high reliability requirements where failure is likely to result in personnel catastrophe. The probability of an in-bore premature detonation of a high explosive shell is a case in point. The acceptable probability of failure rate might be one failure per million rounds fired. It would be impractical to actually fire one million rounds to test the hypothesis. Therefore, this particular problem is tackled from the design aspect by overtesting components which generate prematures. The overtest in this instance consists of firing a limited number of rounds at higher than normal pressures and applying engineering experience.
Oddly enough, the opposite problem also crops up. It is sometimes necessary to convince a customer that a test sample of one hundred or three hundred is sufficient to satisfy requirements for which the customer had programed several thousand.
"Failure"—A Test Problem
In any discussion of reliability and maintainability testing, a most troublesome definition is that of the term "failure." A variety of definitions are associated with the type of equipment, use of equipment, and equipment components. Stoppage of a machinegun may be considered a failure if it cannot be corrected by manipulation of the charging device. Stoppage of another gun, or of the same gun in another configuration, may be counted a failure if it cannot be corrected in 10 seconds. For other equipment the allowable correction time may vary from 10 minutes to an hour. Some materiel proponents specify that a failed component discovered during a regular maintenance operation will not be included in calculating reliability.
There seems to be no problem with the basic definition of a failure—a stoppage which precludes the initiation of or completion of a mission. However, there now evolves the definition of a mission, at least for some systems. An aircraft having independent functions of observation and fire support offers some interesting possibilities in semantics. Considering fire support only, the mission is aborted when a stoppage of the weapon occurs; however, the aircraft may continue its observation mission. How should we calculate the reliability of the overall system?
We have a similar definition problem with the armament system. For purposes of reliability calculation, should a mission be an event or a collection of events—a specific number of trigger pulls with the expectation the weapon will fire each time it is required to do so, the fire-out of a full complement of ammunition, or the number of rounds in the complement of ammunition?
Statistical Techniques
Reliability studies are performed principally to determine the probability of a device performing its mission adequately for the period of time intended under operating conditions expected to be encountered. Statistics are essential in these studies.
Since many important decisions are based on the results of tests in which statistics play such key roles, the choice of the proper methodology takes on great importance.
Sometimes important decisions are based on tests made on a single test sample of equipment. This is a practical course of action, but it contains certain built in hazards. Considering a mission time of 10 hours, operation for 300 hours should provide a reliability figure at a relatively high confidence level. The difficulty is that the results obtained from a single test item may not be representative.
To the determination of reliability based on test samples, it is important that the mathematical techniques used be fully understood. For example, assume that the tester decides that his conclusion about the equipment, using 29 samples, must reach a 95 percent confidence mark. He may select one of several mathematical models.
The first model requires that a test item pass only if there are no failures. Thus, a random sample of 29 test items, for which the true reliability criterion is specified as 90 percent, will show a confidence factor of 1, or 100 percent. At this point the tester can have 100 percent confidence that the item will fail the test and will continue to fail until it reaches the 90 percent reliability point.
The probability of making the correct decision—to pass the item—begins once the 90 percent true reliability criterion is reached. A 22 percent passing factor is attained if the item possesses a true reliability of 95 percent. Not until the item reaches a true reliability factor of 97 percent does the tester's confidence attain a value of 75 percent. Yet, even at this value the tester can say with confidence only that he is not recommending bad equipment. However, only when the test equipment reaches 100 percent on the reliability scale can the tester declare with 95 percent confidence that he will recommend good equipment.
Such a test model requires the manufacturer to overdesign the equipment in order to meet high reliability requirements imposed on it by the Army. This increases the purchase price.
A second mathematical model available requires a test rule which allows acceptability of no more than five failures per the 29 samples. Although this gives the developer a high degree of certainty that the equipment will be acceptable at the assigned 90 percent true reliability mark, it leaves the tester confident only that he will not buy bad equipment. The reason is that at a 95 percent confidence level, the true reliability of the equipment slips from the assigned 90 percent criterion to 67 percent.
Faced with these extremes, the tester may decide to use a mathematical model with intermediate characteristics. He may choose a model with a test rule of acceptability of only two failures. Such a model would make it possible to begin his confidence statements with at least a 50-50 chance of making the right decision.
Thus, the selection of an appropriate model is one of the critical decisions to be made by the test engineer.
Testing for Maintainability
Many maintenance problems can be traced to maintenance engineers at contractor plants who overlook the limited experience of young soldier-mechanics who maintain or repair Army equipment in the field.
Too frequently, designers of various systems components neglect to coordinate their efforts with maintainability in mind. The problems are myriad, many of them unnecessary.
The accessibility of components for maintenance and servicing is high on the list. Adjustment points are sometimes impossible to reach, batteries are difficult to service, replacement of oil filters becomes un unnecessarily arduous task.
It is impractical for the maintenance soldier to carry many special tools with him. Contract specifications for most military equipment require that provisions be made for performing maintenance chores using tools included in standard kits. This requirement is often ignored and calls for corrective action.
Maintainability is further complicated by extreme environments. At 25° below zero, the soldier must be warmly dressed. He wears heavy arctic mittens most of the time, using light contact gloves for brief periods only. Yet he must adjust and repair all types of equipment. Adjusting knobs which are small or too close together, gelling at tool boxes in places where snow or ice build up, making adjustments or repairs which require the use of bare hands are a few of the maintenance problems we must solve.
Deficiencies in maintenance manuals, circuit diagrams, and similar basic literature which accompanies most new items are a constant source of irritation. Errors creep in when changes are made in the end product after explanatory literature has been prepared. Others occur inadvertently. Sometimes the omission of an important step in an assembly or disassembly procedure produces unnecessary confusion. Too often, what seemed obvious to the maintenance engineer becomes a source of consternation in the remote repair shop. It may remain a complete mystery to the inexperienced mechanic.
The entire maintenance package, pertinent literature included, is checked out and examined carefully by USATECOM test personnel. Deficiencies in this package are considered as important as those in the test item itself. They may cause a delay in the acceptance of a research and development item, or, in the case of a new production item, may result in withholding release of the materiel to the field.
Few of the reliability-maintainability problems encountered are new or cannot be corrected. Surprisingly, little seems to have been done to date. The best opportunities for improving our materiel performance appear to be in the hands of the designer and the developer. But the job is not theirs alone; improving the performance of the Army hardware is everybody's business.
Mr. Benjamin S. Goodwin serves as special assistant and technical advisor on test and evaluation matters to the commanding general, U.S. Army Test and Evaluation Command.
Social Sharing