Report on Fact-finding Meeting by Diablo Canyon Independent Safety Committee (DCISC) at Diablo Canyon Power Plant (DCPP) on June 28 & 29, 2007 by Per F. Peterson, Member and Jim E. Booker, Consultant [18th Annual Report, Exhibit D.1]
1.0 Summary
The results of the June 28 & 29, 2007 Fact-finding Trip to the Diablo Canyon Power Plant in Avila Beach, CA are presented. The subjects addressed and summarized in Section 3 include:
- Environmental Qualification (EQ) Program Overview
- Operating Experience (QE) Program Overview
- Review NCVs (Non Cited Violations) and Trend of NRC Identified Issues
- Review with Director of Engineering Services – Plans for Transfer of Knowledge and Plans for New Engineers
- Overview of Online Maintenance
- Attend Corrective Action Review Board (CARB) Meeting
- Review all Yellow and Red Mechanical Systems and Recovery Plans
- DCISC Member Meeting with DCPP Management
- Attend Nuclear Safety Oversight Committee (NSOC) Meeting
2.0 Introduction
This Fact-finding Trip to the DCPP was made to evaluate specific safety matters for the DCISC. The objective of the evaluation was to determine if PG&E’s performance is appropriate and whether any areas revealed observations which are important enough to warrant further review, follow-up, or presentation at a public meeting. These safety matters include follow-up and/or continuing review efforts by the Committee, as well as those identified as a result of reviews of various safety-related documents.
Section 4 – Conclusions highlights the conclusions of the Fact-finding Team based on items reported in Section 3 – Discussion. These highlights also include the team’s suggested follow-up items for the DCISC, such as scheduling future Fact-finding meetings on the topic, presentations at future public meetings, and requests for future updates or information from DCPP on specific areas of interest, etc.
Section 5 – Recommendations lists specific recommendations to PG&E proposed by the Fact-finding Team. These recommendations will be considered by the DCISC. After review and approval by the DCISC, the Fact-finding Report, including its recommendations, is provided to PG&E. The Fact-finding Report will also appear in the DCISC Annual Report.
3.0 Discussion
3.1 Environmental Qualification (EQ) Program Overview
The DCISC Fact-finding Team met with Paul Johnson, Environmental Qualification (EQ) Program Manager (recently moved to I & C Electrical Supervisor), for an update on the program. The DCISC last reviewed this program in September, 2006 (Reference 6.1) at which time it concluded the following:
The DCPP Equipment Qualification Program has made major progress in addressing weaknesses identified in 2003. Corrective actions are essentially complete, and the program appears healthy. Equipment replacements are planned along with the Steam Generator (SG) replacements because of changed temperature profiles as a result of the new SG design.
This review was performed to review the status of the EQ Program since the September, 2006 Fact-finding Report. The EQ Program was rated “Green” overall with the following specific cornerstone ratings:
- Program Personnel – Green
- Program Infrastructure – Green
- Program Implementation – Green
- Equipment/Plant Performance – Green
DCPP has completed reviewing 47 of the EQ files in response to the non-conformance report (NCR) initiated in 2003 and the Action Requests (AR) documenting discrepancies found. They hired external staffing to assist them in this review. They have trained and qualified 6 DCPP employees and 1 consultant (still on staff) to conduct this review. They now have 11 DCPP employees qualified for EQ review.
They intend to have a new EQ Program Manager soon. They are incorporating the new Steam Generator (SG) criteria into their review and Mr. Johnson furnished a copy of the SG replacement list and the EQ file revision schedule. They have identified the items that need to be replaced. These items are:
- All of the level transmitters on the steam generators (wide range and narrow range).
- Three Limitorque motors were identified as not meeting the temperature requirements of the HELBOC. During the DCP process walkdown it was identified that only two required change because one had been replaced with an RPE and the CDB had not been properly updated.
- NAMCO valve position switches in the GW 115” area did not meet the HELBOC temperature requirements. These are being insulated to protect them during the High Entergy Line Break transient. This affects 16 switches.
- ASCO solenoid valves in the GE/GW 115” area have been analyzed for performance of their safety function. Previously the function was stated to be 10 minutes following initiation of a post-accident severe environment for isolation of several containment penetrations. The new HELBOC duration is lower in temperature but longer and other similar breaks also have time/temperature transients that challenge the ASCO valves. An analysis was conducted to demonstrate that the Safety Valves (SV) perform their safety function for every event with margin and will perform the required safety function. The ASCO pilot operated SVs were analyzed to assure that once commanded to the fail-safe position, the valves will not change positions and will therefore continue to provide adequate protection.
The new calculations were based on actual qualified life instead of over 40 years. They are also reviewing the EQ files for life extension. They have about 10 files remaining to be reviewed for life extension and for radiation.
DCPP has resolved all ARs and AEs and have about 4 files to be completed. The design change procedure requires that any equipment change be coordinated with the EQ Engineer. The EQ Program files are now in the PIMS system and will have to be transferred to the NEXIS system when it is completed.
A self-assessment is planned for October 2007, and Mr. Johnson will send a copy of the report to the fact Finding Team when it is completed.
- Conclusion:
- The DCPP Environmental Qualification (EQ) Program has continued to make progress in addressing weaknesses. Corrective actions are essentially complete, almost all EQ files have been reviewed, and the program appears healthy. Equipment replacements have been identified as a result of the Steam Generator (SG) replacements because of changed postulated accident temperature profiles. DCISC should review the EQ program as it is transferred from PIMS to NEXIS and the completion of all files being reviewed and updated.
3.2 Operating Experience Program Overview
The DCISC Fact-finding Team met with Chris Joyce, Senior Engineer & Program Manager for the Operating Experience (OE) Program to discuss the status of the Program. He also had Jessica Eggleston (an intern from Cal Poly) and Walter Sims (Senior Engineer OE Program at Plant Farley) with him for the presentation. The DCISC last reviewed this program in January 2006 (Reference 6.2) at which time it concluded the following:
“The Operating Experience Assessment Program (OEAP) appeared to be operating satisfactorily overall. Substantial improvements are being made to the process as a result of an INPO evaluation, Quality Verification assessment, benchmarking of other plants’ programs, and alignment to the new SAP system. Staffing, a prior DCISC concern, has improved to adequate with a second person. However, there is still no specific measure for program effectiveness.”
The OE program incorporates a formal system to collect and evaluate operating experience reports from within DCPP as well as from other nuclear plants, INPO/WANO, NRC, vendors, and the Electric Power Research Institute (EPRI). The reports are evaluated to assess the applicability of others’ experiences to DCPP to identify actions to help prevent or mitigate them at DCPP. The OE Program documents each step in the process. The Program is controlled by a plant procedure which was updated in December 2005.
Mr. Joyce used NRC inspections, QV audits, INPO evaluations and DCPP self-assessments to discuss the overall OE program. The INPO evaluation was used to review how the program was organized. . They are now evaluating what the line organization knows about OE and how they are using OE information. Mr. Joyce believes that DCPP is using the OE information effectively but could improve. They now have Mr. Joyce and one Senior Engineer in the OE Department to screen the information to determine its applicability to DCPP and distribute it to the proper department for review.
Their goal for responding to OEs is 7 days for a 10CFR50 Part 21 review, 14 days for the OE group to distribute it to Component Engineer, and 60 days for the Component Engineer to evaluate if DCPP needs to take action. DCPP has received about 1500 OEs during the first 6 months of 2007. They normally receive about 3000 OEs per year with about 80% of them screened out as not applying to DCPP. About 5% of the OEs generate an Action Request (AR). The INPO guideline for submitting OE from power plants is 50 days after event. DCPP is submitting their OE information to INPO in about 30 days and sends about 35 to 40 OE to INPO per year.
Mr. Joyce has developed an OE Action Plan which included findings and recommended improvements from the QV audits, self-assessments and INPO evaluations. The Action Plan, which is now complete (except for the items involving NEXIS), includes the following attributes:
- Defining the OE Program
- Roles and Responsibilities
- Action
- Implementation
- Change Management
They still do not have any method to measure effectiveness of the overall program. Mr. Sims reported that the Southern Company is developing a corporate program to measure the effectiveness of the OE Program for their three nuclear plants. Mr. Joyce stated that self-assessments appear to be the most effective tool for measuring how well the employees are implementing the OE program. DCPP is performing a self-assessment of the OE Program every two years. When other departments are performing self-assessments, they are looking at how well the employees are using OE in preventing or solving problems. Mr. Joyce has been on one benchmarking trip and one assist visit to review how other nuclear plants are using the OE Program.
- Conclusion:
- The Operating Experience (OE) Program appears to be satisfactory overall. Improvements continue to be made as a result of completion of the action plan. However, other than self-assessments, there is still no specific measure for program effectiveness as to how well line organization employees are using the program in preventing or solving problems. DCPP believes that self-assessments appear to be the most effective tool for measuring how well the employees are implementing the OE program.
3.3 Review NCVs and Trend of NRC-Identified Issues
The DCISC Fact-finding Team met with Steve Hamilton, Acting Supervisor – Regulatory Assessment, to discuss DCPP review of NRC issued Non Cited Violation (NCVs) and the trend of NRC identified issues. The DCISC last reviewed this issue at the September 2005 Fact-finding Meeting (Reference 6.3) at which time it concluded the following:
“DCPP is taking actions to reduce its numbers of NRC-issued Non-cited Violations (NCVs). This is appropriate; however, to a large degree, dependent on the NRC. The DCISC notes that all DCPP NCVs are of very low safety significance (Green) and has not seen any adverse trends in their content or causes.”
Mr. Hamilton stated that an NCV is a severity level IV violation for which the NRC has exercised enforcement discretion and not issued an NOV (Notice of Violation) because the plant has already identified the issue and put it into its Corrective Action Program. No formal response to the NRC is required from DCPP for NCVs. NCVs are usually of very low safety significance. All NCVs are entered into the DCPP Correction Action Program, and an Action Request (AR) is issued. The NCVs are reviewed for their safety significance, and cross-cutting issues. DCPP will perform an Apparent Cause Evaluation (ACE) for the NCV as determined by the Directors.
There have been nine NCVs issued for the period January-May, 2007. Seven of these were from 2006 NRC inspection reports and two from 2007 NRC inspection reports. Of these nine NCVs , two were self-revealing and seven NRC identified. All of these NCVs were classified by the NRC as having very low safety significance. DCPP has issued an AR for an evaluation of whether or not there was an opportunity for DCPP staff to have identified some or all of the NRC NCVs during this period. Mr. Hamilton reported that there was an NRC data base for review of NRC inspections and violations at other nuclear plants.
The number of DCPP projected violations in 2007 is trending below the Region IV plant annual average. The Region IV plant average is 16. Two NCVs have been issued for 2007 NRC inspections thus far. DCPP had 16 NCVs issued in 2006.
- Conclusions:
- DCPP expects to have fewer NCVs issued in 2007 than 2006 and fewer than the plant average for NRC Region IV. Seven out of the last nine NCVs issued by the NRC were NRC-identified and two were self-revealing. All of these NCVs were classified by the NRC as having very low safety significance. DCPP has issued an Action Request (AR) for an evaluation of whether or not there was an opportunity for DCPP staff to have identified some or all of the NRC NCVs during this period.
3.4 Review with Director of Engineering Services
The DCISC Fact-finding Team met with Ken Peters, Director of Engineering Services to review Plans for the Transfer of Knowledge from engineers retiring and the hiring of new engineers. The DCISC last reviewed this topic at the September 2005 Fact-finding Meeting (Reference 6.4) at which time it concluded the following:
“The training rotation for new engineers at DCPP seems to be very good, and junior engineers that were interviewed thought that it was beneficial in that they were able to learn more broadly about the various departments. These engineers seemed to know their jobs very well and knew what they wanted to do next. They also have a very good attitude and enjoy working at DCPP. Engineering plays a key role in sustaining plant safety, and an important issue is whether experience from senior engineers is transferred effectively to junior engineers. The practice of hiring “classes” of engineers has proven to be highly effective in promoting mentoring and effective education and training of junior engineers. This appears to be the best approach to sustaining long-term engineering capability at DCPP.”
Mr. Peters reported that their 2007 business plan included adding 10 new engineers in 2007 and 10 new engineers in 2009. These plans have been put on hold while staffing for the entire plant is being reviewed. New plans should be developed in about one month. The staffing study was performed by consultants. They will probably hire some new engineers in 2007, but not 10, and they hope to hire some new engineers in 2009.
There are presently 176 employees in Engineering Services and about 55% to 60% will be eligible to retire in the next 5 years. They are eligible to retire at 55 years of age and 20 years of service. Mr. Peters noted it is very difficult to predict when employees are going to retire.
Mr. Peters also discussed the issue of knowledge transfer from experienced engineers to new engineers because of the possible retirement of so many employees. PG&E Corp. will be conducting a survey in the next few weeks on this issue. The survey will include 65 system questions and 20 key program questions. All Engineering employees will be asked to participate in the survey. They will be looking at key gaps from the responses and the output should help determine how they can cross-train.
The DCISC Fact-finding Team discussed with Mr. Peters how the staffing of design, construction and operation of the next generation of nuclear plants could impact staffing at existing plants such as DCPP. They also discussed how DCPP can increase relations with Cal Poly for future engineers.
The large number of NRC NCVs being identified by NRC instead of by DCPP Engineering was discussed. Mr. Peters stated that in the past a questioning attitude and human performance were not as good as they should have been. They were also not performing self-assessments. As they are now performing self-assessments and doing a better job with them, and as they improve at performing them, they should show improvements in identifying these issues (NCVs).
Mr. Peters discussed the Human Performance Simulator process developed to check how the Engineers are proceeding through a task. The Human Performance Simulator process develops the Scenario, Sets Expectations, Identifies Training Aids, and then performs the Skit. He handed out an example which was:
- Step 1
- Conduct an Engineering Tailboard/Pre Job Brief on revising a STP during the outage on backshift.
- Step 2
- Discuss the requirements of AD1.1D2, “Procedure Review and Approval” and AD1.1D7, “Editorial Corrections and On-the-Spot Changes” with assigned engineer.
- Step 3
- Check qualifications of performer and/or ITR
- Step 4
- Practice 3 way communications when assigning task, including when to complete.
- Step 5
- Check OE for this task
- Step 6
- Procedure Quality, procedure in error previous outage
- Step 7
- Use of 2 minute rule when work is interrupted
- Step 8
- Safety, ladder use by the test team
- Step 9
- Questioning Attitude, Supervisor not clear on due date/time
- Sept 10
- Post Evolution Critique
All steps are observed and missed opportunities identified. The Human Performance Simulator is used on various tasks performed by engineers to observe how they normally perform their jobs.
- Conclusion:
- DCPP has delayed the hiring of new engineers for the Engineering Services Department until the staffing study for the entire plant is completed. PG&E Corp. will conduct a survey to identify knowledge transfer needs. The DCISC Fact-finding Team is concerned that the DCPP current hiring program may not be providing enough experienced personnel to replace employees retiring or leaving due to new industrial competition. DCISC should continue to review Engineering Department Staffing and retirement at future fact finding meetings.
3.5 Overview of Online Maintenance
The DCISC Fact-finding Team met with Jack Purkis, Director Maintenance Services, to review on-line maintenance at DCPP. The DCISC last reviewed this topic at the September 2005 Fact-finding Meeting (Reference 6.5) at which time it concluded the following:
“DCPP uses Probabilistic Risk Analysis (PRA) and Outage Risk Analyzer-Maintenance (ORAM) to determine when to do on-line maintenance. They perform all Technical Specification (TS) required Preventive Maintenance (PM) and testing on-line and in general, they plan to perform work on a piece of equipment that takes only 50 % of out-of-service time allowed by TS. It appears that PG&E is using the proper selection process to perform on-line maintenance of equipment.”
Mr. Purkis stated that the maintenance backlog is higher than normal (about 380), but this is not too unusual after completing outages such as 1R14. They should be at about 300 (usual backlog) by October 2007. Their guidelines for on-line maintenance are to do only maintenance that required 50% or less of Technical Specification (TS) allowed outage time. They can use more than 50% only with management review and approval. They can also use a one time NRC TS exception if necessary. They have only used one one-time exception in the last two years. Their plans are to take one Emergency Diesel Generator (DG) (EDG) out of service for maintenance during an outage and perform maintenance on the other two with the unit on-line.
Maintenance is setting the on-line maintenance schedule and Probabilistic Risk Analysis (PRA) performs the risk assessment using PRA and Outage Risk Analyzer-Maintenance (ORAM) techniques. Their method of prioritizing work is functioning very well with Operations and Maintenance. Mr. Purkis thinks the amount of work on-line vs. during outages is about right. DCPP may be doing a little more work during outages than other plants. DCPP maintenance is walking clearances and confirming them by name tag to be sure they are not working on wrong component, wrong unit, wrong train or wrong equipment. They also require dual verification. Mr. Purkis stated that the discipline maintenance organization (mechanical, electrical, etc.) is working a lot better than the previous organization with all disciplines being assigned to areas.
The Fact-finding Team also discussed with Mr. Purkis how maintenance is using the Operating Experience Program to plan their work. He reported that when they do a self-assessment, they look into how the employees are using OE. Their supervisors should be using the most current OE information in their morning meetings. He thinks the OE organization or QV should do a sampling of work to see how the employees are using OE.
- Conclusion:
- DCPP continues to use Probabilistic Risk Analysis (PRA) and Outage Risk Analyzer-Maintenance (ORAM) techniques to determine when to do on-line maintenance. They plan to perform work on-line only on a piece of equipment that takes 50 % of out-of-service time allowed by TS. It appears that PG&E is using the proper selection process to perform on-line maintenance of equipment. DCPP thinks the amount of work on-line vs. during outages is about right. Their method of prioritizing work is working very well with Operations and Maintenance.
3.6 Attend Corrective Action Review Board (CARB) Meeting
The DCISC Fact-finding Team attended the Corrective Action Review Board (CARB) meeting to observe the conduct of the meeting.. The DCISC last attended a CARB Meeting at the January, 2007 Fact-finding Meeting (Reference 6.6) at which time it concluded:
“An agenda had been distributed and all participants had reviewed the actions to be taken at the meeting. The members of Corrective Action Review Board (CARB) discussed each of these items in detail and reached an agreement as to what actions needed to be taken. The members were well prepared to discuss in detail each of these agenda items. It appeared that on some of these items, the CARD went into too much detail before coming to a conclusion.”
Ken Peters, Director of Engineering Services, was chairman of this CARB meeting. The members in attendance were K. Peters, D. Peterson, P. Roller, J. Purkis, and K. Langdon. Also in attendance were Jack Martin, R. West, and L. Fusco. An agenda for the meeting had been distributed and all participants had reviewed the actions to be taken at the meeting. The CARB meets every week. Five regular members or alternate members of which no more than two may be alternates must be in attendance for a meeting. The items to be discussed at this meeting were:
- NCR N0002213 “Unit 2 manual trip on high RCP Stator Temperature”. Review/Approve NCR corrective action change request.
- AR A0691366-01 “Area for Improvement (RP.1-2) Review/Approve Apparent Cause Evaluation”
- NCR N0002210 “Personnel Performing Work Without Required Qualifications” A0680302 “Perform Effectiveness for NCR N0002210” Review/Concur Interim Effectiveness Evaluation.
The chairman of the CARB first discussed nuclear safety, questioning attitude and reviewed minutes from the last meeting.
The members of CARB discussed each of these items in detail and came to unanimous agreement as to what actions needed to be taken. These actions were:
- CARB agreed to the corrective action change request
- This ACE was revised after the CARB package was assembled and was requested to be removed from the agenda. CARB agreed to this request
- CARB did not approve this interim effectiveness evaluation and asked it be returned to CARB in two weeks.
The CARB members were well prepared to discuss in detail each of these agenda items, and went into much detail before coming to a conclusion to approved or disapprove the agenda items. The CARB requested feedback for each action item to be sure it was understood and what was to be done.
After agenda items were complete, Jack Martin, Member of NSOC, reviewed feedback from his last meeting. He has reviewed about 32 ACEs and thinks they are improving. He would have sent back 4 of the ACEs that involved INPO AFIs, to be reworked. There were also 10 downgrades which Mr. Martin thought were like a lot to be sent back.
- Conclusion:
- An agenda had been distributed and all participants had reviewed the actions to be taken at the Corrective Action Review Board (CARB) meeting. The members were well prepared to discuss each of these agenda items. CARB members discussed each of these items in detail and reached an agreement as to what actions needed to be taken. The CARB appeared to be performing their function well.
3.7 Review all Yellow and Red Mechanical Systems and Recovery Plans
The DCISC Fact-finding Team met with Susan Westcott, Manager Mechanical Systems Engineering, to review Mechanical Systems Health Reports that are rated Yellow and Red (unsatisfactory). The DCISC last reviewed the Mechanical Systems Health Report at the November 2006 Fact-finding Meeting (Reference 6.7) at which time it concluded the following:
“DCPP have changed the System Health rating on many systems from Green or White (both satisfactory) to Yellow or Red (both unsatisfactory) to identify those that must get funding and schedule for corrective action activities in order to remedy many of the long term equipment problems. They have identified corrective actions necessary to return the systems to White or Green rating and have scheduled repair or replacement actions, as necessary. Some of the schedules seem to be excessively long term. The System Health Reports are a valuable tool to identify the systems that need attention. DCISC should continue to review the status of the System health Reports at Fact-finding Meetings.”
The System/Component Health is represented by a color:
- Green (satisfactory) indicates the structure, systems, and components (SSC) has no performance issues
- White (satisfactory) indicates the SSC has completed all actions to correct major performance/health issues and performance is trending toward the goal or target
- Yellow (unsatisfactory) indicates the SSC has major performance/health issues with actions scheduled for implementation
- Red (unsatisfactory) indicates the SSC has major performance/health issues and actions are being developed
Ms. Westcott reviewed each of the Mechanical systems in Red or Yellow status, how long they have been in that status, what needed to be done, and the schedule for returning them to White or Green rating. The ECD, estimated complete date, for each of the systems is listed after each system. The systems in the Yellow status are:
- Condensate Polishing System-Yellow (U1)
- Steam Generator Sulfates –Adverse Trend-ECD 9/2007
- Waste Recovery System- Operator Burden-ECD 8/2007
- Auxiliary Feedwater System – Yellow (U1 & U2)
- MDAFW pump LCV Control System – MR (a)(1) Status-1R16/2R16
- LCV-106, 107, 108, 109 MOV Actuators-MR (a)(1) Status-ECD 10/07 (Return to (a)(2) after monitoring period)
- LCV- 110, 111, 113, 115 EH Actuators – Adverse Trend- ECD 2R14
- Residual Heat Removal System – Yellow (U2)
- Check valve RHR-2-8740A – Critical Component Failure- ECD 2R14
- Service Cooling Water System – Yellow (U1 & U2)
- Chemistry Control – Adverse Trend-ECD 5/2008
- SCW Heat Exchanger Tubes – Adverse Trend- ECD 12/2007
- Makeup Water System – Yellow (U1 & U2)
- Condensate Storage Tank Liner – “POA” –ECD 2R14/1R15
- Auxiliary Saltwater System – Yellow (U1 & U2)
- ASW Pump Packing Leakoff – Critical Equipment Failure – ECD 12/2008
- Fire Protection System – Yellow (U1 & U2)
- Firewater to Containment –Single Failure Vulnerability – ECD 12/09
- Cardox System Control Valves – Adverse Trend- ECD 5/2009
- Firewater Storage Tank and Piping – Adverse Trend- ECD 12/2009
- Deluge System Piping – Adverse Trend- ECD 1R15/2R15
- Emergency Diesel Generators (DG) – Yellow – (U1 & U2)
- EDG 2-3 Unavailability – MR (a)(1) Status- ECD 2R14 (Return to (a)(2) after Monitoring period)
- Heating, Ventilation, & Air Conditioning – Yellow (U1 & U2)
- CFCU Anti-Rotation – Adverse Trend-ECD 2R14/1R15
- CFCU Cooling Coil Framework – Adverse Trend- ECD Perform Study 12/2007
- POV Panel- Material/Equipment Condition –ECD 12/2009
- Post LOCA Sample Room – Adverse Trend- ECD 11/2007
- Aux/FH Bldg. Supply Fans- Material/Equipment Condition- ECD Present to PRC 8/2007
- Control Room A/C – Material/Equipment Condition – Present to PRC 8/2007
- ASW 1-2 Exhaust Fan – Critical Equipment Failure- ECD 9/2007
- Backup Air/Nitrogen System – Yellow (U1 & U2)
- Instrument Tubing and Fitting – MR (a)(1) Status-ECD 1R15
- Accumulator Sizing – MR (a)(1) Status- ECD 1R15/2R15
There are no Mechanical Systems rated Red. DCPP has identified all of the Mechanical Systems that need actions taken to return the System Health reports to White or Green. They have also determined what the actions need to be taken and have scheduled the estimated completion dates.
- Conclusion:
- There are no DCPP mechanical systems in the Red (unsatisfactory health) category. DCPP has identified corrective actions necessary to return the mechanical systems rated Yellow (unsatisfactory) to White or Green, (both satisfactory) and have scheduled repair or replacement in a timely manner for most of the systems. It appears to the DCISC Fact-finding Team that some of the repairs are scheduled too far in the future. DCISC should continue to follow systems rated yellow and the completion dates scheduled for each.
3.8 DCISC Member Meeting with DCPP Management
DCISC Member Per Peterson met separately with Jim Becker, VP Diablo Canyon Operations and Station Director, to discuss items reviewed in this Fact-finding meeting and other items of interest to the Committee
3.9 Observe Nuclear Safety Oversight Committee (NSOC) Meeting
The Fact-finding Team observed the June 29, 2007 NSOC meeting. The DCISC typically attends two NSOC meeting each reporting period and last attended one in December 2006 Fact-finding Meeting (Reference 6.8) when it concluded the following:
“DCPP Nuclear Safety Oversight Committee (NSOC) meetings are satisfactory and continue to improve. Three of the four external member positions are filled. Having a non-member facilitator appeared useful in allowing all members to focus on content instead of meeting dynamics and timing. The Committee reviewed each previous and new meeting action item to assure clarity, understanding, schedule, and responsibility. Agenda items were pertinent to nuclear and personnel safety. There was good attendance and participation by all with effective questioning and three-way communication. The NSOC sub-committee system works well, but the DCISC Fact-finding Team believes it would be beneficial for the sub-committee chairs and members to be from different disciplines for an outside viewpoint.”
PG&E Members in attendance were: Jim Becker- VP Diablo Canyon Operations and Station Director, John Conway- Site VP Diablo Canyon Power Plant , Donna Jacobs - VP Nuclear Services, Jack Keenan - PG&E Senior Vice-President Power Production and Chief Nuclear Officer, Ken Langdon- Director Operations Services, Ken Peters- Director Engineering Services, Jack Purkis- Director Maintenance Services, Paul Roller- Director Performance Improvement, and Dave Taggart - Manager QV. All four of the External Member were present and included Darrell Eisenhut- former Nuclear Regulatory Commission (NRC) Administrator, Ed Hux- formerly with INPO, Dhiaa Jamil- Duke Energy Group VP, and Jack Martin- former NRC Regional Administrator. Also present from PG&E was Bill Manheim- Chief Counsel Generation and DCPP employees Dennis Peterson, Kent Oliver (NSOC Secretary) and Lynn Walter, meeting facilitator.
The agenda for the meeting was
- Safety Review for the Meeting
- Review Action Items from 3/9/07 Meeting
- Plant Performance Indicators, NRC, PIs, MSPI, and ODMI
- 1R14 Outage Results and Lessons Learned
- QV’s Assessment of Performance
- QPAR Improvement
- QV Rotation Assignments
- Operations/Production Subcommittee
- Chemistry Equipment/INPO Mid-Cycle Issues
- New Chemistry Performance Indicators
- Maintenance Subcommittee
- 1R14 Human Performance Review
- Engineering Subcommittee
- Review Status of Critical Components, RPEs, and Single POINT Vulnerability
- Containment Fan Cooler Motor Damage
- Pressurizer Welds
- Oversight & Self Assessment Subcommittee
- Organization Effectiveness Subcommittee
- Performance Improvement
- ACE Quality Improvements
- Observation Program Improvements
- 2007 Business Plan Comments
- Plus/Delta
Safety Review for the Meeting
Donna Jacobs discussed personnel safety for the conference room in case of fire or other emergency and also actions if someone were choking and how to apply the “Heimlich Method”.
She then reviewed Nuclear Safety and read from INPO’s “Principles for a Strong Nuclear Safety Culture,” Principle 6: “A questioning attitude is cultivated. Individuals demonstrate a questioning attitude by challenging assumptions, investigating anomalies, and considering potential adverse consequences of planned actions. Group-thinking is avoided through diversity of thought and intellectual curiosity. Opposing views are encouraged and considered.”
Review Action Items from 3/9/07 Meeting
All Action Items from 3/9/07 meeting have been completed except one which is scheduled to be complete by 7/30/07.
Plant Performance Indicators, NRC, PIs, MSPI, and ODMI
Jim Becker reviewed Plant Events and Operational Status. He reviewed the 18-month lost-time injury rate and recordable injuries for PG&E employees only. It was suggested that they also track all injuries, not just PG&E employees. He also discussed collective radiation exposure and the INPO performance index. One of the external members recommended they consider outside assistance in evaluating how they estimate dose during outages. He reported all NRC Performance Indicators are rated Green. The plant events and operation of both units was reviewed for the time since the last meeting.
The failed fuel in Unit 2 was discussed and an external member recommended that one of the subcommittees look into what DCPP is doing to identify fuel failure before 2R14.
There were no nuclear safety concerns identified and three Action Items developed.
1R14 Outage Results and Lessons Learned
Dennis Peterson, Director Outage Management, reviewed 1R14 outage results. He discussed major outage scope items and performance areas goals and actual results. There was a lot of discussion pertaining to safety practices and injuries during the outage. It was suggested that Near Misses are not getting enough attention and should be considered for inclusion in clock resets.
Scope growth for the outage was 12 percent, while 10 percent was expected. This growth impacted the outage schedule and the ALARA exposure goal. The Shutdown Safety Risk Profile actual closely reflected the planned schedule and resulted in best Nuclear Safety results ever. The Containment Recirculation Sump modification went as scheduled, but vendor quality control needs to be monitored much closer.
Loss of off-site power during the outage will be further investigated and preventive actions will be implemented. There were also a many questions and discussions about Turbine lube oil cooler failure, and it was reported that DCPP is performing an Apparent Cause Analysis (ACE) on this failure.
There were no nuclear safety concerns identified and six Action Items developed.
QV’s Assessment of Performance
Dave Taggart presented Quality Verification (QV’s) assessment of overall plant performance. Overall First Quarter 2007 YELLOW station rating was based on previously identified issues that remain unresolved. A common theme is organizational effectiveness and follow-through:
- Emergency Response Organization (ERO) issues
- Industrial safety behaviors/standards reinforcement
- Low initial operator training through-put
- Lack of consistent, rigorous pre-job walkdowns
- Engineering DPO on RSG level setpoints
RED performance was noted in the areas of Emergency Preparedness/ERO and the Containment Recirculation Sump Project.
1R14 Issues:
- FME Readiness & Support; deletion of specified QV inspection points in issued WOs
- Industrial Safety & Significant Near Misses re: Polar Crane
- Continuing lack of consistent, rigorous pre-job walkdowns
- Late Parts Identification – Questions rigor of Outage Milestone Ratings
- Sump Project Challenges (engineering, procurement, supplier performance – GE & Fabrico, construction)
- By-passed QV Hold Points – Worst Performance in over 20 years (22 occurrences)
- Schedule/Cost versus Quality/Safety Emphasis & Organizational Perceptions
Mr. Taggart also reviewed QV rotation and staffing with Operations, Maintenance, RP/Chemistry, Operations/Training, and Engineering. Future rotation and staffing included Maintenance two year rotation, Manager development rotation (6 months to 1 year), and QV Supervisor attrition.
An action item was to review how they look at QPAR and determine if ARs are needed to address “responses” requested. Evaluate if, using Benchmarking, the QPAR could be improved as far as effectiveness. Report back next NSOC. Due 08/24/07.
There were no nuclear safety concerns identified and one action item (listed above) developed.
Operations/Production Subcommittee
Ken Langdon & Darrell Eisenhut reported on the Operations/Production Subcommittee. Key issues from interviews by Darrell Eisenhut with Ops, RP, and Chemistry personnel were discussed. These were:
- Those interviewed believed that overall station performance was about average.
- Most were not aware of benchmarking results and have not participated in those efforts.
- Too many are accepting substandard equipment performance.
- Workers are concerned about staffing shortages, knowledge transfer, and hiring new personnel. They are particularly concerned about the amount of overtime that is required because of the staffing shortages.
- Operations personnel indicated they do not see senior personnel in the plant often enough and that senior management attendance at training sessions has declined.
- Operators believe that pre-job briefs continue to improve and they appreciate being involved in the improvement process.
- Operators believe that Training is declining and that line management support is lacking.
- RP personnel voiced a concern that RP procedures need to be improved.
- Chemistry workers are concerned about staffing issues and the aging workforce.
- Chemistry technicians and supervisors are very concerned about plant material conditions that affect Chemistry. These include the ongoing fuel leak, condenser leakage, closed cooling water issues, service water issues, Unit 1 SG sulfates, etc.
Current Operational Decision Making (ODM) reports were reviewed and overall decision making was determined to be adequate but lacked details explaining why a particular decision was made. In many cases, that explanation never reached plant personnel. Several examples were sited and are included in the complete subcommittee report.
One of the external members asked QV why they did not further pursue difficulties Operations appears to be having relative to training throughput.
There were no nuclear safety concerns identified and three Action Items were developed from the issues above.
Maintenance Subcommittee
Jack Purkis and Dhiaa Jamil presented the NSOC Maintenance Subcommittee report. Mr. Jamil began by sharing observations he made during the execution of a Task Performance Evaluation (TPE). This was the first task the trainee was attempting to get qualified on and the evaluator was a seasoned journeyman. Mr. Jamil felt the technician did a great job during the evaluation but that the evaluator missed several learning opportunities for the new trainee. For example, the evaluator did not validate many examples of good behaviors and did not coach the trainee when he added a step that was not in the procedure. Mr. Jamil emphasized that opportunities to reinforce standards should not be missed. Discussion followed around that fact that DCPP utilizes Bargaining Unit employees for TPEs and that the standard practice, for the plants that want a step change in performance, is to utilized management employees only.
The FME Program was then discussed as well as the improvements that have been implemented by the new FME Coordinator. The recent 1R14 outage did experience some problems with FME and those are being addressed and additional benchmarking is scheduled so that improvements can be implemented before the 2R14 outage in early February, 2008. Observations made by an outside NSOC member on the FME program will also be incorporated into lessons learned prior to 2R14.
Human performance practices during 1R14 were improved from 2R13 but have not reached industry-leading performance. Safety performance declined in 1R14 and it was noted that most injuries were received by supplemental personnel. It was also noted that only about 50 percent of the supplemental workers received safety training at DCPP. An NCR on 1R14 safety practices as been initiated to resolve these issues.
There were no nuclear safety concerns identified and two Action Items were developed.
Engineering Subcommittee
Ken Peters reported on the Engineering Subcommittee Report. The DCISC fact finding team noted that there is not an external member on this subcommittee.
Preventative Maintenance Optimization (PMO) efforts are on track and will meet the 2007 Business Plan objectives. Single Point Vulnerability (SPVs) reviews are on track with seven being scheduled for completion in 2007. DCPP has not experienced any Critical Equipment clock resets through June of this year.
A High Impact Team (HIT) consisting of Operations, Maintenance, Engineering, and Work Control has been formed to identify actions to improve DCPP’s performance regarding critical safety systems and their impact to the INPO Index ratings.
All Containment Fan Cooler (CFCU) motors have been inspected and repaired for cracked welds in the stator bars to motor frame mountings first identified around 1997. In 2R13, evidence of stator/rotor rubbing was identified in CFCU 2-4. The motor was sent out for inspection and some cracked welds and slight rubs have been identified, but are not an operability concern at this time. It is planned to have two additional spare motors ordered if the Plant Health Committee approves the request.
Mr. Peters stated that the Pressurizer Dissimilar Metal Weld problems that applies to Unit 2 only has A82/128 welds on six pressurizer nozzles. DCPP is committed, through EPRI, to the Industry Program MRP-139 to complete inspections and formalize their plans to mitigate any weld cracking by the end of 2007 or have CNOs concur on a later date. The possibility remains that the NRC could require plants to complete all the inspections by the end of 2007, which would impact DCPP’s Unit 2 Outage schedule.
There were no nuclear safety concerns identified and three Action Items were developed.
Oversight & Self Assessment Subcommittee
Dave Taggart and Jack Martin reviewed the Oversight and Self-Assessment Subcommittee Report. They reported that major issues are investigated through root cause analysis. These types of analysis continue to be done well. Second level problems are evaluated by Apparent Cause Evaluations (ACEs), which are improving but do not meet expectations. The subcommittee reviewed 20 recent ACEs and found 16 to be satisfactory.
Two ACEs addressed INPO AFIs concerning problems with maintaining adequate chemistry due to equipment issues. Lack of ownership and lack of visibility to the issues show up in both evaluations. The subcommittee will continue to follow this long standing issue.
The subcommittee found the actions taken to address the USA assessment of Human Performance to be satisfactory. The DCPP 2007 Business Plan was also found to properly address Human Performance and Industrial Safety issues. Concern was expressed with the performance indicators for human performance in 1R14. Two near misses during the 1R14 outage involving the Polar Crane could have resulted in a fatality but were not captured as clock resets since there was no actual injury. A significant learning opportunity was missed.
After expectations are issued and training provided, the human performance effort is most effectively implemented through assessment of behaviors and reinforcing the desired behaviors. This is normally done through the use of the observation program to systematically reinforce expectations for desired safety practices and use of human error tools. The observations can be periodically refocused on areas where performance data indicated behavior weaknesses. The following comments were offered after the June observations were reviewed:
- Many of the observations were only 15 minutes in length. Few were as much as an hour or longer. The 15 minute observations were of insufficient time to assess industrial safety behaviors or use of error prevention tools.
- Most observations were of meetings, job briefs or training as opposed to work out in the plant where human behavior and safety behavior problems can best be observed and coached.
- Most observations do not seem to be focused on human behavior or safety practices.
- A small number of the observations were outstanding and could be used as models.
There were no nuclear safety concerns identified and one action item developed.
Organizational Effectiveness Subcommittee
Ed Hux, the new external member of NSOC and the subcommittee, reviewed his recent observations on the plant and comments from his plant tour and employee interviews.
General Comments:
- The number of injuries during the outage should be a significant concern for the station, a root cause evaluation should be conducted and aggressive actions taken to prevent recurrence.
- Clear plastic was noted wrapped on a component in the Unit 1 Spent Fuel Pool area.
- During plant tours several personnel were noted as not carrying gloves as required.
- It does not appear that the organization fully believes in CAP as a core business and a valuable tool to improve and maintain excellent performance.
- QV observations tend to be related to required audits and assessments. Most high performing plants fully utilize the oversight organization to help conduct observations and write ARs that will help with continuous improvement efforts.
Interview Results:
- Some workforce personnel felt there is less empowerment of the workforce than in the past and that the organization is more directive.
- Engineers are concerned about the loss of talent and future loss due to retirements and they are not aware of any plans for “knowledge transfer.”
- Many utilities have developed plans and are hiring personnel to develop and take the place of those expected to retire and have critical plant system and component knowledge.
- There was a strong accountability focus on schedule adherence for the outage. Personnel were uncomfortable with the change from past outages. Some felt the outage was not well planned, walk-downs were sometimes not conducted, but the schedule was pushed.
Mr. Hux’s general concern was about staffing in Operations, Maintenance, and Engineering with employees retiring or leaving and DCPP not having a plan for adding new employees.
There were no nuclear safety concerns identified and one action item developed.
Performance Improvement
Paul Roller reported on performance improvements. The ACE Quality Improvement Actions were:
- ACE grading was modified to be performed by a group rather than an individual
- New procedure is in place that raises quality standards of ACE’s
- Added CARB review to help assure quality standards are consistently met
- ACE quality grades have improved from 78 percent to 90 percent
The improvements in the Observation Program were:
- PI roll up reviews now include performance gaps in the Observation Program
- New metric in place to measure field observation times
- Human performance tools emphasized in observations
The upcoming Observation Program improvements will be:
- Face-to Face cascading reviews between leaders and their direct reports
- Minimum of 2-hour field observations on designated items
- Crucial Conversations training to help observations coaching
- Conclusion:
- There were no nuclear safety concerns identified. And one action item developed.
2006 Business Plan Results/2007 Business Plan Review:
Due to time restraints this presentation was rescheduled for the next NSOC meeting on August 24, 2007.
Action Items:
Action Items identified during the meeting were repeated to assure all of the Action Items were noted.
Plus/Delta:
Lynn Walter identified the Plus/Delta from the meeting. These were:
Plus
- Jack Martin’s challenge to the Polar Crane “near miss” as not being treated with appropriate programmatic review/corrective action
- Good feedback from outside members
- Strong self critical meeting
Delta
- Need control and time on size of agenda
- They need to increase the focus on lessons learned from past experiences in order to improve present and future performance. Learning from their mistakes and near misses will help them improve performance and the effectiveness of their leadership.
- Conclusion:
- DCPP Nuclear Safety Oversight Committee (NSOC) meetings are satisfactory and continue to improve. All of the four external members were present. The Committee reviewed previous Action Items that were not completed and new meeting action item to assure clarity, understanding, schedule, and responsibility. Agenda items were pertinent to nuclear and personnel safety. There was good attendance and participation by both DCPP and external members was strong with effective questioning and three-way communication. The External Members were very effective in asking questions and making comments. The NSOC sub-committee system and reports works well, but the DCISC Fact-finding Team did not observe identification of action plans to address concerns or problems identified in the Subcommittee Reports. The DCISC should continue attending NSOC Meetings each year.
4.0 Conclusions
- 4.1
- The DCPP Environmental Qualification (EQ) Program has continued to make progress in addressing weaknesses. Corrective actions are essentially complete, almost all EQ files have been reviewed, and the program appears healthy. Equipment replacements have been identified as a result of the Steam Generator (SG) replacements because of changed postulated accident temperature profiles. DCISC should review the EQ program as it is transferred from PIMS to NEXIS and the completion of all files being reviewed and updated.
- 4.2
- The Operating Experience (OE) Program appears to be satisfactory overall. Improvements continue to be made as a result of completion of the action plan. However, other than self-assessments, there is still no specific measure for program effectiveness relative to how well line organization employees are using the program in preventing or solving problems. DCPP believes that self-assessments appear to be the most effective tool for measuring how well the employees are implementing the OE program.
- 4.3
- DCPP expects to have fewer NCVs issued in 2007 than 2006 and fewer than the plant average for NRC Region IV. Seven out of the last nine NCVs issued by the NRC were NRC-identified and two were self-revealing. All of these NCVs were classified by the NRC as having very low safety significance. DCPP has issued an Action Request (AR) for an evaluation of whether or not there was an opportunity for DCPP staff to have identified some or all of the NRC NCVs during this period.
- 4.4
- DCPP has delayed the hiring of new engineers for the Engineering Services Department until the staffing study for the entire plant is completed. PG&E Corp. will conduct a survey to identify knowledge transfer needs. The DCISC Fact-finding Team is concerned that the DCPP current hiring program may not be providing enough experienced personnel to replace employees retiring or leaving due to new industrial competition. DCISC should continue to review Engineering Department Staffing and retirement at future fact finding meetings.
- 4.5
- DCPP continues to use Probabilistic Risk Analysis (PRA) and Outage Risk Analyzer-Maintenance (ORAM) techniques to determine when to do on-line maintenance. They plan to perform work on-line only on a piece of equipment that takes 50 % of out-of-service time allowed by TS. It appears that PG&E is using the proper selection process to perform on-line maintenance of equipment. DCPP thinks the amount of work on-line vs. during outages is about right. Their method of prioritizing work is working very well with Operations and Maintenance.
- 4.6
- An agenda had been distributed and all participants had reviewed the actions to be taken at the Corrective Action Review Board (CARB) meeting. The members were well prepared to discuss each of these agenda items. CARB members discussed each of these items in detail and reached an agreement as to what actions needed to be taken. The CARB appeared to be performing their function well.
- 4.7
- There are no DCPP mechanical systems in the Red (unsatisfactory health) category. DCPP has identified corrective actions necessary to return the mechanical systems rated Yellow (unsatisfactory) to White or Green, (both satisfactory) and have scheduled repair or replacement in a timely manner for most of the systems. It appears to the DCISC Fact-finding Team that some of the repairs are scheduled too far in the future. DCISC should continue to follow systems rated yellow and the completion dates scheduled for each.
- 4.8
- DCPP Nuclear Safety Oversight Committee (NSOC) meetings are satisfactory and continue to improve. All of the four external members were present. The Committee reviewed previous Action Items that were not completed and new meeting action item to assure clarity, understanding, schedule, and responsibility. Agenda items were pertinent to nuclear and personnel safety. There was good attendance and participation by both DCPP and external members was strong with effective questioning and three-way communication. The External Members were very effective in asking questions and making comments. The NSOC sub-committee system and reports works well, but the DCISC Fact-finding Team did not observe identification of action plans to address concerns or problems identified in the Subcommittee Reports. The DCISC should continue attending NSOC Meetings each year.
- 5.0 Recommendations:
- None
6.0 References
- 6.1
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, September 6 & 7, 2006 ”, Exhibit D.2, Section 3.9.
- 6.2
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, January 18 & 19, 2006 ”, Exhibit D.6, Section 3.6.
- 6.3
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, September 21 & 22, 2005 ”, Exhibit D.3, Section 3.7.
- 6.4
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, September 7 & 8, 2005 ”, Exhibit D.2, Section 3.1.
- 6.5
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, September 6 & 7, 2006 ”, Exhibit D.2, Section 3.10.
- 6.6
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, January 17 & 18, 2007 ”, Exhibit D.6, Section 3.11.
- 6.7
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, November 28 & 29, 2006 ”, Exhibit D.4, Section 3.2.
- 6.8
- “Diablo Canyon Independent Safety Committee Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, December 28 & 29, 2006 ”, Exhibit D.5, Section 3.7.