Advanced Practices for Statistically Determining Flaw Detection Limits

Abstract

<p>For nondestructive examination (NDE) methods, the most common question asked is “What is the smallest flaw detectable?” While common, this has traditionally been of less importance than the largest flaw that might be missed. It is also important to ask “How often is the technique likely to detect a flaw that isn’t really there?” The rejection of sound parts results in needless investigations, rework and expense. Advances in manufacturing techniques and the ever present desire for lighter, higher performing structures have resulted in components that frequently test and occasionally exceed the detection limits of NDE methods. It is the job of engineers, chemists, and scientists to continuously push the boundaries materials and structural designs. It follows that inspectors and statisticians, whose work establishes NDE method detection limits, must also push the boundaries of existing and developing inspection systems, and take additional rigor in assuring the accuracy of their analyses. Because of this, NASA is reexamining Probability of Detection (POD) methodologies the Agency helped pioneer in the late 1970’s in a multi-Center project supported by the NDE Program, NASA Office of Safety and Mission Assurance (OSMA). Methods in use within the Agency [1] and international community include the Full POD method described in MIL-HDBK-1823A [2] and Point Estimate methods [3]. The procedural recommendations of the handbook provide excellent guidance. However, result inconsistencies in the software developed to support Full POD methods has raised concerns regarding configuration control, the underlying proof property of analysis methods, and overall software validation [4]. An alternate method relying on the statistical confidence of flaw size regions has been developed by NASA under the guidance of the following key subject matter experts: Dr. Bill Vesely, Technical Discipline Manager for Technical Risk Assessment and Statistical Analysis Head Statistician; Floyd Spencer, NDT statistician with 37+ years in the industry recommended by NESC Statistician Ken Johnson; Bill Meeker, Professor of Statistics at Iowa State University; and Edward Generazio, Delegated Program Manager for the OSMA NDE Program. The analysis approach developed has been approved by one of the founders of POD, Ward Rummel. Over the life of this project, proof-property validated analysis methodology [5] and Excel algorithms (DOEPOD) [6], an updated analysis [7] of historic data [8] NASA uses to establish detection limits, new method demonstration samples, and volumes of NDE inspection data have been produced. Current efforts [9] focus on completing analyses of 800+ method demonstration samples, updating procedures, and developing a validated, secure, web-based, agency-wide analysis and archival tool for POD data. The resulting DOEPOD software produced by this effort is available from NASA by contacting Kathy A. Dezern (757.864.5704, kathy.a.dezern@nasa.gov).</p>