21st Annual Report, Volume I, Section 4.6, Performance Improvement Programs

4.6.1 Overview and Previous Activities

Termed “Corrective Action Program” in previous reports, this section is now expanded to “Performance Improvement Programs” to include programs included in DCPP’s Performance Improvement Initiatives, such as Corrective Action, Industry Operating Experience, Benchmarking, Self-Assessments, etc. Many consider these to be “learning” programs whereby the organization learns to improve from its and others’ experience.

As have all nuclear plants, DCPP has implemented a Corrective Action Program (CAP). The CAP is a formal, controlled process used to identify and correct problems which occur. A key part of the CAP is root cause analysis which is utilized to ascertain the real cause of a problem or event such that corrective action can be taken to prevent its recurrence. During the previous reporting periods, the DCISC has reviewed the DCPP CAP and numerous events which were identified and resolved using the CAP. NRC refers to this type program as Problem Identification and Resolution (PI&R).

The events, analyses and corrective actions reviewed during the previous several reporting periods included the following:

DCPP’s Corrective Action Program (CAP) has continued to undergo significant reviews, assessments and audits by both internal and external organizations. Actions are being taken to respond to the reviews. The NRC has identified a substantive crosscutting issue in the DCPP Problem Identification and Resolution area, indicating unsolved problems with the CAP. The DCISC will continue to monitor the CAP.

DCPP’s Operating Experience (OE) Program appears to be in jeopardy of becoming resource-limited if the function is reduced to just a single person performing OE duties. A similar situation had led to a high OE backlog in 2005. The DCISC will monitor this situation.

4.6.2 Current Period Activities

The DCISC reviewed the following in DCPP’s Performance Improvement Program during the current reporting period:

Line Use of Operating Experience (Volume II, Exhibit D.2, Section 3.5)

The DCISC Fact Finding Team met with the OE Program Assessment (OEA) Manager, to discuss the Line Organization use of the OEA Program. DCPP has only one person assigned to the OE program implementation. When he is on vacation or out of the plant, there is no one to fill in for him and the backlog will just increase and he is not too sure they can continue to effectively perform the requirements of the OE program with just one person performing the duties.

A Daily Report is sent out to about 800 to 1,000 employees, which results in 70% to 80% feedback from Line Organization users. OEA performs a quick screening using priority list criteria to screen out the ones not applicable to DCPP. After screening, the OEs that need an evaluation are sent to the applicable system engineer to perform a formal evaluation. The system engineer has 14 days to either accept or reassign the OE. A total of 50 days or less is allowed for the Manager to agree with the evaluation. The total time allowed for the completed evaluation is 60 days except 90 days for a Yellow Significant Operating Experience Report (SOER) and 150 days for an Red SOER.

The schedule is checked by sending out a reminder that the OE evaluation is due in the next 30 days and another reminder is sent out that the evaluation is due in 7 days. A notice is sent out if the evaluation is not received in 57 days. The average age for completion of the evaluation is 38 days and no evaluations are currently due and over 60 days. About 90% of OEs come from INPO and the remaining 10% from other sources. The OEA Manager is the only one at DCPP who can close out the OEs after checking that the Manager has agreed with the evaluation. About 60% of OEs requiring formal evaluation require no action.

DCPP is trying to put together an OE review team of the Line Organizations to assist in the screening process and evaluations.

DCPP’s screening of industry Operating Experience (OE) information appears to be continuing to function well. DCPP’s decision to reduce screening staff for its incoming OE to one person could hinder the entire OE function at the station. The DCISC will follow up on this issue to evaluate whether this cutback has an impact on DCPP’s use of the Operating Experience Program.

Status of Performance Improvement Action Plan (Volume II, Exhibit D.6, Section 3.6)

The DCISC Fact Finding Team (FFT) met with the Manager of Problem Prevention and Resolution. The Performance Improvement Action Plan was focused on the nature of and methods used by the station’s performance improvement activities rather than focusing on specific improvements that are needed in aspects of plant operation and performance. The Plan’s Problem Statement reads as follows:

“DCPP’s use of Performance Improvement (PI) programs lags the industry with the result that performance shortfalls continue to occur and performance relative to the industry is declining.”

The Plan focuses on improving methods, techniques and tools for identifying, measuring, and assessing gaps between actual DCPP performance and desired performance. Specific methods, techniques and tools discussed in the Plan include: benchmarking, self-assessing, performance indicators, gap analysis, Corrective Action Program (CAP) procedures, Apparent Cause Evaluations (ACE), Root Cause Analyses (RCA), reviews of plant and industry operating experience, and reviews by external groups. It does not discuss specific actions that were felt to be needed to actually improve specific areas of plant performance but rather addresses the performance improvement process in general.

The Performance Improvement Action Plan stemmed from the 2009 plant evaluation conducted jointly by the Institute of Nuclear Power Operations (INPO) and the World Association of Nuclear Operators (WANO). Another performance improvement tool is a Health Report for self-assessments–a single sheet template that a department would fill out quarterly and would be submitted to the Self Assessment Review Board.

The station should have about 20 personnel who are trained in and are capable of performing causal analyses. Currently there are six such individuals, two of whom are in Problem Prevention and Resolution. Further, departments have Performance Improvement Coordinators, whose focus is supposed to be on self-assessments, benchmarking, RCAs, and ACEs. However, they are devoted largely to managing the corrective action backlogs and performing other departmental duties.

The monthly Plant Performance Indicator Report (PPIR) highlights those Performance Indicators that have improved during the past month and those that have declined. What is not shown are those indicators that have remained in Red and/or Yellow Status from month to month.

DCPP has had difficulties with evaluating and addressing station problems, including the area of engineering evaluations. The NRC has issued DCPP a significant cross-cutting aspect for deficiencies in its Corrective Action Program, a major program included in DCPP’s Problem Prevention and Resolution area. The DCPP Performance Improvement Action Plan is an appropriate vehicle for helping to correct and improve DCPP’s performance. DCISC concludes from this review that some causal factors related to this problem may be due to an inadequate number of trained and qualified personnel as well as to a lack of clarity in personnel responsibilities.

Corrective Action Program (CAP) (Volume II, Exhibit D.7, Section 3.7)

The DCISC met with the CAP Manager, for an update of the CAP. The CAP Index, an overall measure of plant CAP health has declined from White (acceptable) to Yellow (not acceptable) in the last month as shown in the chart below. The main causes were due to a high average age of Root Cause Analyses (RCAs), failed RCA evaluations, high Apparent Cause Evaluation (ACE) evaluation times, and a high number of open Condition Reports. DCPP expected the CAP Index to be Green in the first quarter of 2011. The DCISC believes that the most significant CAP measure is the CAPR (Corrective Action to Prevent Recurrence), which is indicative of how well the problems were identified and resolved to prevent them from happening again. This is defined as the number of unsatisfactory effectiveness evaluations for three months running.

In 2008 DCPP received four NRC Green Findings with the cross-cutting theme involving the lack of thoroughness of problem evaluations. DCPP performed a Root Cause Evaluation (RCE) focused on inadequate thoroughness of engineering evaluations and established a controlled process for better documenting engineering evaluations as corrective action to prevent recurrence (CAPR). This evaluation thoroughness cross-cutting theme continued throughout 2009 with six additional Green Findings, and in March 2010 NRC determined that a DCPP a Substantive Cross-Cutting Issue (SCCI) existed in the area of Problem Identification and Resolution (PI&R) related to the thoroughness of problem evaluations.

DCPP initiated another RCA, “Adverse Trend in Thoroughness of Problem Evaluation.” The RCA was impressive in its depth, scope, extent, and straightforwardness. The DCISC concluded that the evaluation was extensive and thorough in that the Root Cause Team reviewed 14 evaluations that had been identified as lacking thoroughness, interviewed 23 personnel from director level through individual contributor level, assessed the existing training and indoctrination to perform evaluations, and identified the extent to which plant personnel understood what a good evaluation looked like. Review of the 14 deficient evaluations led the RCT to conclude the following key factors contributing to the deficiencies:

  • Assumptions not validated
  • Narrow focus of RCAs (misalignment between the cause and the problem statement)
  • Narrow focus of corresponding corrective actions (e.g., reliance on training per se as a corrective action)
  • Poor understanding of plant design and licensing bases
  • CAP not entered when deficiencies were identified

Additionally, reviewing five of the 14 evaluations using the Kepner-Tregoe methodology, the RCT identified these underlying reasons for inadequacy of evaluations:

  • Mind set/mental model (past bad behaviors were considered adequate)
  • Incorrect interpretation of design and licensing bases requirements
  • Inadequate independent technical review (time pressure)
  • Focus on process rather than on the issue

Root Cause:

Contributory Causes

  1. The licensing bases were not well documented nor easily retrievable
  2. Weaknesses in causal evaluations prevented earlier resolution of the SCCI
  3. Loss of proficiency in performing evaluations contributed to less than adequate evaluations.

Recommended Corrective Actions To Prevent Recurrence:

  1. Establish generic governance for evaluation programs in order to establish the right standards
  2. Train Program Sponsors (Director level) and Program Owners (Manager level) on the structure of an effective Program Governance
  3. Execute a Program Implementation Matrix to ensure evaluation programs incorporate the essential elements for their sustainability

DCPP combined these recommendations and the WANO CAP Areas for Improvement from its most recent evaluation into a comprehensive “2010-2011 Operating Plan–Performance Improvement Focus Area Integrated Action Plan.” The Plan recognizes that “DCPP’s use of Performance Improvement (PI) Programs lags the industry with the result that performance shortfalls continue to occur and performance relative to the industry is not improving.” This is an extensive, far-reaching initiative involving not just evaluation thoroughness but the following PI areas:

The Plan appears to “leave no stone unturned.” It is assessing and questioning DCPP’s programs and processes in the following specific PIP/CAP areas:

DCPP completed a comprehensive performance improvement benchmarking visit to Byron Nuclear Station in August 2010. INPO performed a Performance Improvement Assist Visit at DCPP in August 2010. Results from these activities have been factored into the Plan.

Following determination and implementation of corrective action (CA) to prevent recurrence (CAPR) for ACEs and RCAs, the line organization performs an Effectiveness Evaluation of the cause evaluations. The DCISC reviewed CA Effectiveness Evaluations for five problems. The EEs appeared to be well thought out, well prepared and fact-based. In all cases the CAs were determined to have been effective. Some went beyond minimum requirements in that in two cases additional actions were recommended, and in another more time was needed to accumulate additional operational data. The evaluations followed the guideline described above and appeared to thoroughly assess the effectiveness of corrective action.

In October 2010 the Quality Verification (QV) Department performed an assessment of the CAP for the period January–October 2010. The assessment concluded that implementation of the CAP is effective, except that problems previously identified by QV in the evaluation area have not been resolved. The assessment noted that in response to a June 2010 QV Audit Finding for the plant’s inability to identify and implement sustainable corrective actions for conditions adverse to quality, DCPP had established a qualification and training program for CARB Members, ACE performers and approvers, root cause analysts and team leaders.

DCPP’s Corrective Action Program (CAP) appears to be generally effective overall; however, there is a major deficiency in the thoroughness of problem evaluations such as Apparent Cause Evaluations (ACEs), Root Cause Analyses (RCAs), Licensing Basis Impact Evaluations (LBIEs), etc. This has been a continuing problem since NRC identified its original Substantive Cross-cutting Issue in 2004, culminating with NRC again identifying a Substantive Cross-cutting Issue in 2010. In response, DCPP performed an extensive RCA, which concluded that, despite multiple warnings and corrective attempts over the years, management has not provided adequate standards, nor effectively demonstrated or reinforced behaviors, nor established sustainable programs in the area of evaluations. DCPP has crafted a comprehensive Performance Improvement Focus Area Integrated Action Plan to address these and other deficiencies and gaps and has begun its implementation. The DCISC will follow DCPP’s progress and success in implementing the Plan, specifically with respect to problem evaluation adequacy.

Responses to Industry Operating Experience (Volume II, Exhibit D.8, Section 3.7)

The DCISC Fact Finding Team met with the Operations Performance Manager. All plants in the nuclear industry have programs established to enable them to learn from the operating experience of other plants in the industry. Various mechanisms used by DCPP Operations to benefit from industry operating experience include:

DCPP employs an extensive array of tools and methods for examining the applicability of industry events to DCPP. Further review of this topic by the DCISC should be conducted on a case basis whenever DCPP experiences a significant event similar to one experienced earlier by another plant.

Overview of the Performance Improvement Review Board.

A relatively new process in DCPP’s Performance Improvement Program includes the Performance Improvement Review Board (PIRB), which is based on industry benchmarking. The function of the PIRB is to review the different line organizations’ Performance Improvement Integration Matrix (PIIM) reports. PI monitoring includes performance monitoring; analyzing, identifying and planning solutions; and implementing those solutions. Mr. David reviewed the PI tools being used at DCPP including:

The PIRB works to achieve and maintain performance excellence by ensuring effective use of PI tools to improve personnel and plant performance; fostering effective performance results from a strong partnership between the line and PI coordinators in the areas of trending and work group evaluations; using OE to enhance learning and work products; and providing effective use of feedback and observation to improve PIP quality and assure employees are familiar with DCPP standards and the expectations for meeting those standards.

The PIRB uses of Performance Improvement Information Matrix (PIIM), a tracking tool which provides a graphical representation of the PI tools in use. The line organizations are brought into the process to discuss their current level of performance and the PIIM is used to assess and determine if a DCPP organization is self-critical; seeks excellence in performance; is diverse in approach and not reliant on a single process or program to identify gaps in performance; prioritizes appropriately and effectively; develops effective corrective actions; and implements those actions well and with rigor; and represents broad organizational involvement. The matrix includes entries for: tracking numbers for the Notification and associated due dates; identification of the performance gap or issue; other stakeholders; whether the issue is part of the 2011 Station Initiatives; how the performance gap was identified; the method to be used to resolve the issue; and resolutions being monitored.

4.6.3 Conclusions and Recommendations

DCPP’s Performance Improvement Program continues to be strengthened with the addition of the Performance Improvement Review Board, a management board which monthly meets to review the program and specific items, which are lagging, and the Performance Improvement Action Plan, a multi-faceted plan to integrate the results of several assessments and reviews of the program.
DCPP’s Corrective Action Program (CAP) has continued to undergo significant reviews, assessments and audits by both internal and external organizations. Actions are being taken to respond to the reviews. The NRC has identified a substantive crosscutting issue in the DCPP Problem Identification and Resolution area, indicating unsolved problems with the CAP. The DCISC will continue to monitor the CAP.
DCPP’s Operating Experience (OE) Program appears to be in jeopardy of becoming resource-limited if the function is reduced to just a single person performing OE duties. The DCISC will monitor this situation.
None; however, see Recommendation R11-1 in Section 3.5 of this report, which states:
Due to the substantial increase in the numbers of NRC Non-cited Violations and Severity Level IV Violations over the last two reporting periods and because the NRC Substantive Crosscutting Issue in Problem Identification and Resolution still exists, the DCISC recommends that DCPP re-examine its earlier Root Cause Analysis for effectiveness and consider an independent review of its corrective actions by Quality Verification, the Nuclear Safety Oversight Committee, or the Institute of Nuclear Power Operations.

For more information contact:

Diablo Canyon Independent Safety Committee
Office of the Legal Counsel
857 Cass Street, Suite D, Monterey, California 93940
Telephone: in California call 800-439-4688; outside of California call 831-647-1044
Send E-mail to: dcsafety@dcisc.org.