November 29, 2023

Allegation Regarding Computing Facilities Maintenance and Calibration at the Oak Ridge National Laboratory

UT-Battelle, LLC manages and operates the Oak Ridge National Laboratory (ORNL), which is a multi-program research laboratory in Eastern Tennessee for the Department of Energy.  ORNL provides computing facilities, which include one of the most powerful high-computing centers in the world, providing researchers with leadership-class computers for scientific simulations.  The computing facilities and their support facilities are maintained by UT-Battelle, LLC and multiple subcontractors through the Computational Facilities Complex.

On September 16, 2022, the Office of Inspector General received an allegation regarding computing facilities maintenance and calibration in the data centers related to buildings 5300, 5600, and 5800 at ORNL.  The complainant alleged that operations and maintenance for ORNL’s data center were poorly managed.  Specifically, the calibration program was not adequate, and there was poor or no maintenance on pressure relief valves.  We initiated this inspection to determine the facts and circumstances regarding the allegation concerning computing facilities maintenance and calibration at ORNL.

We substantiated the allegation regarding computing facilities maintenance and calibration at ORNL.  Specifically, we found that the calibration program was inadequate to meet quality assurance requirements.  Further, we found that pressure relief valves were not always maintained in accordance with applicable criteria. 

These issues occurred because of a lack of ORNL oversight.  For example, ORNL was unable to provide the required documentation showing how the quality assurance requirements were met for the relied-upon software. Additionally, ORNL did not always follow its Standards-Based Management System Procedure when maintaining pressure relief valves.  Separately, while addressing the allegations, we found that for one subcontract, ORNL did not properly flow down Department Order 414.1D Change 2, Quality Assurance, as required.  This was due to an oversight on ORNL’s part.

This report contains three recommendations that, if fully implemented, should help ensure that the issues identified in this inspection are corrected.  Management fully concurred with our findings and recommendations, and its proposed corrective actions are consistent with our recommendations.