the relationship between the output (y) or performance metric, and
two inputs, or factors (X1 and X2). one can see the output is maximized
near the center of the space, suggesting m... VIEW ORIGINAL
PICATINNY ARSENAL, N.J.--In case you've forgotten the adage that "uncertainty is the only certainty," Picatinny Arsenal's Statistics Group is prepared to remind you. Its latest initiative known as "Uncertainty Quantification" focuses on studying engineering modeling and simulation uncertainty from a statistical standpoint and leveraging it as a more credible approach to weapon and munition product development.
Uncertainty Quantification, or UQ, is a family of statistical methods which identifies and exploits uncertainties or unknown information in engineering models and simulations. While it potentially could apply to a wide variety of scientific topics being studied (such as weather events), Picatinny's statistics group focuses on military equipment and technologies, exploring uncertainties in system behavior, such as weapon and munition performance and design.
In other words, instead of asking, "What do I know about this system," the team asks, "What don't I know about this system," and then determines possible outcomes, using a series of statistics, mathematics, reliability engineering, probabilistic, and risk analysis tools and techniques.
By combining it with current modeling and simulation (M&S) analytics, the team can achieve more credible modeling and simulation evaluations. In doing so, UQ helps to reduce test costs, make design and manufacture more robust, develop resilient and reliable armament systems, and improve the transition of science and technology initiatives to program managers.
"I think this is one of the biggest advances in modeling in many years," said Don Carlucci, an ARDEC senior research scientist at Picatinny. "It will not only allow us to determine how accurate the models are, but we will be able to use it in combination with reduced order modeling techniques to improve the speed and fidelity of our computations. It is the next big thing in computational mechanics."
Some information that can be gained through UQ includes: how a system performs relative to warfighter needs, what's the probability of system failure, what's the best way to maximize safety, how can we reduce lifecycle and testing costs, where can the design improve and be made more robust to environmental and operational stresses, and the optimal settings to obtain the best performance from a system.
The effort to capitalize on UQ is a science and technology funded project and led by Picatinny's Statistical Methods and Analysis Group, or Statistics Group. The group is part of the Quality Engineering and System Assurance Directorate, which is part of the U.S. Army Armament Research, Development and Engineering Center, or ARDEC. Located in Picatinny Arsenal, New Jersey, ARDEC provides life-cycle support for nearly 90 percent of the lethal Army systems used by warfighters.
The job of the Statistics Group is to consult with a variety of Integrated Product Teams across ARDEC and the Joint Services and specialize in test-design and data analysis, merging their statistical expertise and engineering backgrounds in the development and improvement of armaments and warfighter systems. Current team members are: Eli Golden, Melissa Jablonski, Thorsten Roberts, Kevin Singer, Kelly Martin, Christopher Drake, Douglas Ray, and Paul Roediger (a contractor).
"Our intent is to make this the default approach that ARDEC uses for all the modeling and simulation work that we do here," said Ray, ARDEC's lead statistician, about UQ. "By using these techniques, we can transition into a test phase from modeling and simulation quicker, strengthen predictions and get better information about the system's capabilities. "And, we can get that information much earlier in the cycle, even pre-prototype," added Ray. "Whereas in the past, engineers would use a deterministic model [meaning: no randomness or uncertainty is incorporated] and then build a bunch of items and have to test them a lot, to get usable results."
THE CORNERSTONE OF UQ
In essence, UQ has one cornerstone: 'Design of Experiments'-based emulator. The emulator is a multi-dimensional math model that allows statisticians to combine and adjust different inputs (or factors) that affect the output (or desired response.) These factors can be visually represented as dimensions in a space.
Consider, for a moment, Isaac Newton's second law of motion: Force equals Mass times Acceleration, or F=M*A. This is a simple but useful mathematical model for the physical relationship between two inputs (mass and acceleration) and one output (force). However, this calculation doesn't account for the small deviations that may occur during a live experiment, such as changes in mass or speed. It also doesn't incorporate all the inputs that influence the output. Hence, there may be various results for force when measured.
By inputting these variations into the equation, though, the Statistics Group could get a more realistic picture of what the output, force, will look like. They will also be able to find additional inputs that influence the result which may have not been considered before.
At Picatinny, where much of the work is much more complex than this simple physics model, this is more difficult to execute, requiring the use of an emulator or a more simplified 'surrogate' to replace the original physics model. ARDEC's statisticians will often collaborate with M&S practitioners to adapt UQ techniques to resolve these highly complex problems, such as gun launch dynamics models or aeroballistics simulations, which may have dozens of variables and require a high performance computer with specialized software several hours, or even days, to solve a single iteration. For these difficult problems, the surrogate model is key to running efficient sensitivity analyses.
The resulting emulator is a mathematical representation which could be described visually as a topographic map with contour lines illustrating elevation change, but instead of two input dimensions (such latitude and longitude) it represents dozens or even hundreds of input dimensions. Once this emulator has been generated, it can then be used to study variation in a much more cost- and time-efficient manner than the 'brute force' method we would use to account for and study variation in force measurements in the previous example.
VALIDATION AND VERIFICATION
Another key component to this UQ method are its Validation and Verification (V&V) techniques.
The V&V techniques are statistical calculations that evaluate the accuracy and details of computational M&S predictions relative to live testing. It can generate information about model structure, the techniques used, its design, and additional data needed. Four common techniques are informal, static, dynamic, and formal, which differ based on how the model was tested.
Essentially, though, V&V tells an engineer whether the M&S executed is a proper, accurate and reliable representation of how a technology would perform in the field. If not, it could tell the engineer where to improve the product design to achieve the desired performance.
Suppose an engineer does multiple model runs to find out which factors are key drivers in a rocket assisted projectile's maximum range. Then, the statistician uses UQ to find potential combinations to maximize the range and, based on those results, the engineer may decide to do a few more model runs before narrowing the problem down and develop prototypes to test live in a laboratory or at a proving ground. Live testing results of these prototypes can be used to calibrate and improve our engineering models for future M&S evaluations.
The V&V calculations would take the data from UQ and identify what the engineers can improve, change, or eliminate about the model run to achieve the intended result. This means that the engineer would have to do less model runs and that the engineer would have credible information to support future M&S evaluations.
"Anything that ARDEC can do to get more out of our technical activities, to make our systems better, we should do. In a lot of cases, those changes are very difficult and there's not much payoff," said Ray. "In this case, we've got the people, we've got the motivation and capability to do this, and the payoff is huge. Not just in terms of the dollars and the credibility and reputation of our organization, but on the impact of the systems that we develop for the warfighter."
APPLYING UQ AT ARDEC
While UQ may be new to Picatinny, its techniques aren't unknown. The first applications of UQ happened in the 1940s during the Manhattan Project and development of nuclear weapons, where scientists from Los Alamos National Labs used Monte Carlo Simulation (a computerized technique that performs sensitivity analysis) to better understand radiation shielding.
However, major advancements in computers have encouraged federal organizations, private industry, and academic researchers to incorporate powerful UQ techniques into their engineering processes, indicating a major shift in the statistics and engineering fields. Some organizations that are exploring UQ include the U.S. Department of Energy, NASA, Proctor & Gamble, Georgia Tech, and Pratt & Whitney.
In fact, Pratt & Whitney visited Picatinny Arsenal on June 29 to share information and insight into its work with UQ with the Arsenal's engineers and scientists. The company has been a pioneer in the application of UQ (a formal methodology termed 'Design for Variation') to their systems.
At ARDEC, the Statistics Group has been researching UQ for about five years, partnering with these different organizations, and collaborating with renowned statistics researchers at universities, such as Professors C. F. Jeff Wu and V. Roshan Joseph of Georgia Tech, and Peter Qian or University of Wisconsin-Madison. But, the Statistics Group has only recently received the funding to formalize their initiative and started applying UQ to a greater spectrum of different programs and projects, including a number of science and technology projects.
Two examples of such programs are The Precision Optics Wing Sensing system, or POWS system, and the Future Advanced Squad Technology, or FAST. FAST was a project that occurred within the last year, in which the engineers were looking to characterize the system's design space. The term "design space" refers to the variables which affect system performance, such as its caliber, velocity, propellant burn rate, and environmental conditions, etc.
"They were interested in understanding the design space and where it would be worthwhile to pursue new designs," explained statistics group member Jablonski. "For us, that basically meant which region of that design space, or which combination of inputs, could give us designs of interest that the engineer could pursue further."
"We joined the project late and it [FAST] was such a difficult problem because it had 30 different inputs," added Ray. "So, we had to really rely on numerical optimization techniques to understand all of that info. What it did reinforce, though, was the importance of planning, especially when problems are complex, and incorporating UQ into the early stages of development."
The importance of UQ has made such headway that it is a now major initiative of the Director of Operational Test and Evaluation Office of the Secretary of Defense, Michael Gilmore. He aims to make the UQ methods, especially its Design of Experiments-based V&V techniques, a standard practice for Joint Services organizations evaluating DoD systems where M&S was utilized in their Test and Evaluation.
According to Ray, ARDEC has tackled that mission head-on, demonstrating a leading-edge effort through ARDEC's UQ initiative. As a result, UQ is helping not only to create more reliable armament systems but changing how ARDEC engineers design, test and optimize its technologies for the warfighter.
Related Links:
Armament Research, Development and Engineering Center (ARDEC) homepage
Social Sharing