Report on Fact-finding Meeting by Diablo Canyon Independent Safety Committee (DCISC) at Diablo Canyon Power Plant (DCPP) on August 2 & 3, 2006 by A. David Rossin, Member and Jim E. Booker, Consultant [17th Annual Report, Exhibit D.1]
1.0 Summary
The results of the August 2 & 3, 2006, Fact-finding Trip to the Diablo Canyon Power Plant in Avila Beach, CA are presented. The subjects addressed and summarized in Section 3 include:
- Review 1st and 2nd Quarter 2006 QPAR
- Update on Outage Safety Plan
- ALARA Overview
- Review Status of Self-Assessment Program
- Engineering Workload Management Status for Project Engineering
- Review Employee Concerns & Differing Professional Opinion Programs
- Review System Health Reports that are in Red or Yellow with ICE System Engineering Supervisor
- Tour of Intake Structure Including Concrete Work Performed During 1R13 and 2R13
- Meeting with Plant Management
- NEXIS Project Overview
- Attend Plant Health Committee Meeting
2.0 Introduction
This Fact-finding Trip to the DCPP was made to evaluate specific safety matters for the DCISC. The objective of the evaluation was to determine if PG&E’s performance is appropriate and whether any areas revealed observations which are important enough to warrant further review, follow-up, or presentation at a public meeting. These safety matters include follow-up and/or continuing review efforts by the Committee, as well as those identified as a result of reviews of various safety-related documents.
Section 4 – Conclusions highlights the conclusions of the Fact-finding Team based on items reported in Section 3 – Discussion. These highlights also include the team’s suggested follow-up items for the DCISC, such as scheduling future Fact-finding meetings on the topic, presentations at future public meetings, and requests for future updates or information from DCPP on specific areas of interest, etc.
Section 5 – Recommendations lists specific recommendations to PG&E proposed by the Fact-finding Team. These recommendations will be considered by the DCISC. After review and approval by the DCISC, the Fact-finding Report, including its recommendations, is provided to PG&E. The Fact-finding Report will also appear in the DCISC Annual Report.
3.0 Discussion
3.1 Review 1st and 2nd Quarter 2006 QPAR
The DCISC Fact-finding Team met with David Taggart, Manager Nuclear Quality Verification (QV), and Bob Prigmore, General Supervisor- Plant Quality Assurance to review the 1st Quarter 2006 Quality Performance Assessment Report (QPAR) . The 2nd Quarter QPAR had not been issued yet. The DCISC Team also was briefed on other activities of the QV Department. The DCISC last reviewed this activity at the January 18 & 19, 2006 Fact-finding Meeting (Reference 6.1).
The QV Department Manager, Mr. Taggart, reports directly to Jack Keenan, Senior Vice President-Generation & Chief Nuclear Officer. The overall performance at DCPP as depicted by the QPAR performance ratings has leveled off and, in declined some areas, though the overall rating continued at a WHITE rating, a continuation of the 4th quarter of 2005. A number of areas had been designated as targets for improvement. Departments continue to undertake actions to close performance gaps; however some resolvable areas continued to linger during this period without reaching set goals. It is not apparent that all gaps to top performance have been identified or are clearly understood.
Performance during the two Emergency Diesel Generator (DG) (EDG) maintenance outage windows fell short of established practices. Behaviors were not consistent and lacked follow-through at times.
The need for improvement in Maintenance Services human performance was very evident during this period. Additionally, human performance for Engineering, Operations, and Site Services was also identified as “needing improvement” during Management Review Meetings. A number of supplier performance issues also revealed challenges to the plant during this period. Some of the actions taken to address previous INPO Areas for Improvement (AFIs) have not been fully effective and additional attention is required. An INPO assist visit also revealed identified the need for additional attention.
Additional effort to integrate the benchmarking and self-assessment processes is still needed. Mr. Taggart stated that the self-assessment schedule should be truly integrated and coordinated with station activities. This is a strategic function that requires senior plant management involvement. Assessment topics should result from a cognitive and deterministic process that chooses topics to get maximum benefit for outages, NRC inspections, QV audits and other key evaluations. A self-assessment of the new Plant Health Committee, Plant Health Subcommittee, and Project Review Committee should be scheduled with outside peers to ensure the process is reaping the benefits that were expected for such a process.
The Key Station Quality Performance Issues continue to be where DCPP must maintain focus. The Plant Health Committee, Plant Health Subcommittee and Project Review Committee provide for a structured framework for equipment health issues. The program provides for feedback and issue tracking. However, the process is complex and may become burdensome to key members who are required to implement the procedural requirements. The time investment to be able to implement the procedure will be large, and administration of the process is critical. Additions to the Key Station Quality Performance Issues include “ACE Quality,” “Human Performance,” and “Maintenance Work Package Quality/Documentation/Procedure Adherence.”
Challenges still exist in the area of outage planning and scope control. These are barriers to improving outage performance and meeting “industry best performance” levels. Though the indicator for all parts to be ordered and received was GREEN, many parts were subsequently identified that had to be reordered. These findings should have been identified prior to setting the milestone.
Strategic Projects activities progress was satisfactory this period with the exception of the NEXIS Project. After an excellent effort was established for various readiness-reviews, subsequent deferral to the project has impacted the department readiness plans. Project communication regarding the delay and the specific impacts to departments has been poor.
The 2nd Quarter QPAR had not been issued, but Mr. Taggart stated that it would include the problems that were found in the DCPP response to the NRC Generic Letter (GL) 2004-02 (Containment Sump Debris Blockage) in September 2005. QV has essentially completed a 100% verification of the analysis identifying 28 unique and significant analytical errors in the DCPP analysis for the their response to the GL. Resolution of these errors has resulted in considerable increases in the types and quantities of debris calculated to be generated and transported to the sump structures, as compared to that used in the prototypical sump screen head loss testing performed in the July 2005. These tests were reported to the NRC in DCPP’s September 2005 response to GL 2004-02. The sump team has reached consensus on the resolution path for the few remaining deficiencies and open items in the debris generation and transport analyses. Corrective actions are scheduled to support the modified screen testing scheduled for early August 2006. The DCISC plans to follow up on this issue at its October 18-19, 2006 Public Meeting.
The existing screens will either be modified by installing fiber eliminator inserts or they will be replaced with new screens, depending on the results the testing in August. Regardless of the screen modification ultimately implemented, appropriate debris source reduction efforts will be pivotal to DCPP’s successful resolution of the GL 2004-02 issue. DCPP also discussed the results of the Corrective Action (CA) Audit and the Maintenance Activities Assessment. The CA audit showed some improvement over last year, but they still have some problems. They had problems with the quality of ACEs because they did not follow the details of the procedure, and the CARB did not look at very many ACEs. The Corrective Action Group changed grading criteria and increased the number of ACEs going to the CARB for review. The Maintenance Group is still reporting problems related to following procedures.
Other major issues identified in the Maintenance Activities Assessment were work package quality and training of planners.
- Conclusion:
- It appears that Quality Verification (QV) continues to perform good assessments of performance of DCPP including the Quality Performance Assessment Report (QPAR). The audits and assessments identify issues that should have been identified and corrected during the line organization self-assessments. DCPP management should be asking the line organization why they are not finding these issues before QV identifies them during audits and assessments. The DCISC should continue to review the QPARs at future Fact-finding Meeting.
3.2 Update on Outage Safety Plan
The DCISC Fact-finding Team met with Brad Hinds, Outage Manager, to review the update on the Outage Safety Plan. The DCISC last reviewed the outage safety plan at the September 7-8, 2005 Fact-finding Meeting (Reference 6.2) and improvements in the safety plan for use in Outage 2R13 at the June 6-7, 2006 Public Meeting (Reference 6.3). Mr. Hinds reported that the next two refueling outages would be 1R14 – April 2007 and 2R14 February 2008.
Mr. Hines stated that due to disturbing trends in the number of loss–of-cooling events in the industry, DCPP completed a comprehensive overhaul of its outage safety plan in 2005. As a result, DCPP’s shutdown defense-in-depth (DID) program is more effective and practical. It ensures that forced core-cooling methods are maintained at all times. It also ensures that should the required forced cooling system or support systems fail, at least one passive core cooling method is available until forced cooling can be restored. It also identifies the necessary plant conditions that make passive cooling capability either delayed or unavailable and minimizes the time that the plant is in these “transition periods.”
The safety plan also provides operators with comprehensive guidance for taking appropriate mitigating actions and ensures that the plant condition and configurations are maintained to support the basis for that guidance. In addition, methods are established for evaluating emergent conditions that threaten or reduce DID. These factors reveal that DCPP has kept a robust focus on event mitigation and Containment closure capability in its program. While the DCPP DID program is effective and practical, it appears from internal and external operating experience that there is continued justification for strengthening DCPP’s means of event prevention.
INPO has requested all utilities to perform a detailed assessment of the shutdown DID. DCPP completed their assessment in March-June, 2006. As part of the INPO request, there was a set of 35 questions for each utility. The Shutdown DID self-assessment was performed in March-June, 2006 by DCPP employees and peers from Callaway Plant. This self-assessment identified 5 strengths, 3 areas for improvement, and 8 recommendations. The self-assessment also responded to the 35 questions on the INPO DID review areas.
The areas for improvement which are documented in Action Request (AR) A0663350 are:
- Improve methods of event prevention, including training, communications, and field posting of protected equipment.
- Clearly identify temperature limitations related to minimum Reactor Coolant System (RCS) and Spent Fuel Pool (SFP) temperatures in plant procedures, or ensure that there is analysis that support not having to include them. In addition, ensure that limitations related to fuel movements are adequately incorporated into procedures.
- Find ways to eliminate loss of vital power supply events NPO initiatives are not all on-track.
The recommendations which are documented in AR A0663350 are:
- Improve methods for event prevention.
- Incorporate “time to core uncover” into existing “time to boil” calculations and communications, as required.
- Implement a practice of routine switchyard walkdowns prior to and during single offsite power source operations.
- Clarify the terminology used in the outage safety plan and associated procedures, to distinguish between contingency planning based on Procedure AD7.DC10 “Contingency Planning” versus contingency strategies and compensatory measures employed in the outage safety plan.
- Ensure adequate analysis exist related to minimum temperature for criticality. Ensure that these analyses are adequately incorporated into plant procedures and instrumentation to provide guidance for minimum acceptable temperatures for: a) Spent Fuel Pool operations, b) RCS and reactor cavity operations, and c) fuel movement.
- Ensure mitigating actions credited for planning a reduction in DID are documented in the outage safety plan and comply with NRC regulations related to shutdown operations.
- Improve the outage safety plan by ensuring that all compensatory actions to mitigate planned reductions in DID are clearly documented and located in one place in the plan.
- Review the outage safety schedule for all Decay Heat Removal (DHR) transition periods, and ensure that they are documented in the outage safety plan. In addition, ensure steps are taken to minimize transition periods.
The final report should be out by the end of August 2006. The outage safety plan for 1R14 will incorporate the improvements and recommendations made in this assessment. The DCISC team expressed concern about possibly adding too many jobs in 2R14 because of the length of the outage. Mr. Hines stated he did not think so, and that the outage was scheduled for 69 days to replace the SG’s. The replacement of the SG’s has been contracted for. He reported there were three potential problems with schedule. These are containment sump screen work, the containment leak rate test and reactor head inspection.
- Conclusion:
- DCPP had made improvements in their outage safety plan in 2005 on their own initiative. The The Institute of Nuclear Power Operators (INPO) requested self-assessment with peer reviewers appears to have identified additional improvements and recommendations which should make for additional improvements in the outage safety plan. DCISC should review the final report and the improvements made in the outage safety plan prior to 1R14.
3.3 ALARA Overview
The DCISC Fact-finding Team met with Jeff Harker ALARA Manager, Dr. Linda Sewell, Dosimetry Supervisor and Curtis Hanson, ALARA Supervisor of the Radiation Protection Group. Bob Hite, Director, joined the discussion later. The DCISC has reviewed this program and its activities at almost every Public Meeting and during many Fact-finding Meetings.
The Team congratulated the group on its work during 2R13. The collective dose was a record low for a DCPP outage, and the Radiation Protection Group has been given accolades for its performance. The DCISC notes that DCPP has appropriately given much of the credit to its line organizations for their efforts in taking ownership of ALARA.
The Team commended Jeff Harker and Linda Hughes for their excellent presentation at the Public Meeting on June 21 on radiation control, planning and performance for 2R13, and lessons learned from experience during the outage. Their visuals were clear and informative, and the overall presentation was crisp.
The group discussed further the distinction between “project work” and the radiation exposures during work that is routinely scheduled during many outages. Tasks involved with the projects often require work in higher radiation fields. Planning for project work involves more detailed analysis of tasks, examining ways to reduce doses, training, mockups and practice. Efforts are made to include people who will participate in the work including the crafts in the planning process. This has resulted in new ideas, more in-depth understanding of the work and the radiation fields, and better cooperation.
In the 2R13 Quality Performance Assessment Report (QPAR) section on Radiation Protection Performance/ALARA, RP reported six areas in which significant dose reductions were achieved. One is “Change in Work Scope.” We were assured that this did not represent side-tracking any essential or safety-related work. Although most of the DCPP published information on 2R13 highlights the reduction in dose, Leadership continues to remind all that safety has the highest priority, and that work that has been scheduled after careful review of priorities will not be deferred to avoid radiation exposures, and certainly not in order to improve a performance indicator.
The Team asked about the effect of efforts to raise awareness and to motivate employees to suggest ideas. Radiation Protection feels that the “idea contest” and use of posters and TV reminders were valuable tools. Incentive awards and a “Daily Dose report” have been well-received by the plant.
Some of the new ideas came from benchmarking visits by members of the Radiation Protection group. Benchmarking visits were made to Seabrook and TMI. These plants are rated as “top performers” in Radiation Protection by INPO.
The collective dose for 2R13 shows a significant drop in exposure from the previous outages (74 vs 130 person-Rem typical). RP notes that collective doses were reduced by using fewer man-hours in containment and reducing the number of reactor containment area (RCA) entries. RP is continuing to review the initiatives they used to see which were most effective and where training will result in repeated performance.
Dr. Rossin commented that plants have been reducing collective doses since the industry made the commitment to do so several decades ago. While there are ways to reduce collective doses further, US plants have achieved large reductions, and shared experience effectively. Although opportunities for further reductions remain, numbers for routine outage work are approaching a level of diminishing return. Future outages will include more major project work in containment.
Dr. Rossin pointed out that future outages already call for more project work in containment, some of which will require additional exposures. Increases in collective dose need to be looked upon as understandable necessities. Plants will have to justify numbers that do not exhibit a further decline in collective dose. It is important for plant management to recognize these demands, and to reinforce the need to balance radiation exposures with plant safety and reliability goals. Plant managers are responsible for overall plant performance. INPO, in its quest for excellence, regards decreasing collective dose numbers as another opportunity to demonstrate their goal of “continuous improvement.” It will be important to keep goals in perspective.
The concept of ALARA is a commitment to reduce unnecessary radiation exposures and reducing exposures when procedures or technology can be applied to reduce exposures and still complete reactor work and improvements that are important to safety, reliability and performance.
A good ALARA program does the right things. It calls for planning, innovation and attention to training and detail. ALARA encourages good management. As Bob Hite said, “A well-managed plant that is a good performer has a good ALARA program and will have good numbers. Just having low collective dose numbers is no guarantee of good performance.”
The benchmark visit to the Seabrook plant provided insights into their program for dose reduction as well as best practices in a number of areas. Seabrook is rated as a top performer among U. S. nuclear power plants. Dr. Rossin noted that, “Seabrook is cited as a top performer because their collective dose numbers are low. Seabrook may well be a very good performer overall. However, if overemphasis on collective dose to achieve avoidance of miniscule radiation doses drives operating strategy, a plant can run the risk of delaying or making decisions not to do work or inspections because workers and middle-level managers may feel forced to minimize radiation dose.
Hite stated, “In a good ALARA program everyone is thinking about ways to reduce dose.” Jeff Harker said in a recent PG&E@WORK article, “By keeping that increased focus during the online maintenance period also, the approach eventually becomes more habitual. That’s the kind of culture we are striving for; to provide year-round repeatable successes.”
- Conclusion:
- The DCPP Radiation Protection Group is performing its work well, and applying its experience and initiatives to provide quality service to DCPP. This has resulted in improved line ownership of ALARA and a record low collective radiation dose in Outage 2R13
3.4 Review Status of Self-Assessment Program
The DCISC met with Gary Close, Performance Programs Supervisor, to review the status of the Self-Assessment (SA) Program. The DCISC last reviewed this program at the April, 2006 Fact-finding meeting (Reference 6.4). Mr. Close reported that they were now developing a schedule for self-assessments for the first six months of 2007 instead of waiting until later in the year as they did in the past. This will allow time for a team lead training course (required for self-assessments). They have trained about 40 people for this. He also stated that INPO has added DCPP as a plant for benchmarking in self-assessment programs.
DCPP is currently doing a self-assessment on rigging. They are also using NRC major inspections reports, Self-Assessment Review Board (SARB), and SA request forms for selecting where to perform SAs. A SA requires about 3 weeks: one week for preparation, one week for performing the SA itself, and one week to write up the SA. They have completed 9 formal SAs and 12 quick hits this year. Their goal is to complete 20-22 formal SAs this year. They have implemented line organization grading of the SAs. The SAs are reviewed at the Management Review Meetings (MRMs) that are held monthly and attended by Managers, Directors, and Officers. The SARB meeting is held 4 times per year to review the SAs and provide feedback on them.
DCPP will continue to do benchmarking with other plants on SA. They have just completed a SA of their own SA Program which should be out by the end of August 2006. Overall, it found that they have made improvements, but they need to improve on their benchmarking to the same level of improvement as they found in good benchmarked programs. A formal Self-Assessment summary is issued quarterly. The second quarter 2006 summary stated that in SA team training, leads are highly encouraged to prioritize and then recommend up to three or four recommendations. A greater emphasis will be needed to ensure they are driving toward root cause understanding and not making recommendations that solve only symptoms of larger issues.
Mr. Close then briefly reviewed recent SA of the Operations Department on Configuration Control and Mid-Cycle Evolutions. The purpose of the SA was to evaluate how effective their corrective actions have been addressing the Areas For Improvement (AFI) identified in the INPO evaluation during April 2005. Mr. Close then discussed the two AFI’s and what the SA identified. The SA also identified that internal operating experience is not being disseminated quickly enough throughout the organization and the overall use of operating experience, both internal and external, is inconsistent. OPS has developed recommendations for both of these AFIs.
A mid-cycle SA team performed a comprehensive review of DCPP operations during a two-week period starting on July 10, 2006. In all, eight industry experts were utilized, as well as DCPP personnel. The assessment team concluded that there has been significant improvement in plant performance since the last INPO evaluation and listed five significant accomplishments. In total there were eight Areas For Improvement (AFIs) and six Performance Deficiencies (PDs) as well as four previous AFIs that were listed as “open with concerns.” The following are considered to be the most significant:
- Some of the station’s improvement efforts are falling short of their targets
- Work order package instructions and overall quality continue to require improvements. There are still negative impacts to the daily work week plans for and refueling outages.
- Four 2005 INPO Evaluation AFIs remain at least partially unresolved:
On each of the AFIs, the Actual and Potential Consequences, Examples/Supporting Details, Causes/Contributors, Other Insights, and Recommendations were identified.
- Conclusion:
- It appears that DCPP is making improvements with their Self-Assessment Program. They still need to make improvements in their Benchmarking Program. The two Self-Assessments that were reviewed identified areas for improvement (AFIs), however these had been identified by INPO in 2005 and DCPP was not successful in taking corrective action to solve the problems. DCISC should review the actions taken in the AFIs at future Fact-finding meetings. The DCISC should attend the Management Review Meetings and Self-Assessment Review Board Meetings if they are taking place during future Fact-finding meetings.
3.5 Engineering Workload Management Status for Project Engineering
The DCISC Team met with Jack Shoulders, Manager, Project Engineering, to review the work load for Project Engineering. Engineering work load was last reviewed by DCISC at the February, 2004 Fact-finding Meeting (Reference 6.5). Mr. Shoulders first discussed the Engineering Services Organization which was reorganized one year ago. The purpose was to split Design Engineering into two groups to provide separate resources, one focused primarily on daily plant support and the other on long range support. The Design Engineering group consists of Design Drafting, Plant Support and EFIN (Engineering Fix It Now). The Project Engineering group consists of Mechanical/Piping, Electrical/I@C, Civil/Seismic, and Outsource.
The Engineering active workload is currently about 3,300 Action Requests (AR), Action Evaluations (AE), and AE returns. This workload is also tracked by section of Project Engineering. Mr. Shoulders discussed some problems they had in the past with outsourcing. These were where the work was performed offsite. PG&E had to do independent review. It did not work out well and costs were high.
They have established a goal of 30/80 limitation on the number of active design changes. 30 is the maximum number of design changes during each outage and no more than 80 non-outage design are to be in the works at any one time. They currently have 110 design changes, down from 200 a year ago. They are trying to reduce the number down to 80, but the trend for the last 3 months has been flat.
The management improvements in Engineering Workload are:
@#8226; Reorganization
@#8226; Better Management of due-date changes, overdue actions and ACEs
- Expectations established
- Results – Quality Problem due date extensions are less than last year, QP (a-type) none for all of DCPP, and N-type- only 1 is in Engineering
- ACE timeliness –4 of 16 Departments are over the goal of a 30 day average for ACEs
@#8226; Better Management of Projects
- EOC- Engineers are primarily onsite, experienced, and they have “bench strength,” their own quality program, currently have approximately 60 active projects
- Scheduling each project
- New process provides more rigor with formal studies, and scheduled scoping meetings
- Risk managing each project
- Oversight by PG&E – Owner review meetings
- A Project Manager for most projects in design phase. Following classical Project Management (PM) techniques. Provide formal training on PM.
@#8226; Daily Managers review of new ARs
- Proper priority
- Clear problem statement
- Proper assignment within Engineering
- Process improvements identified
- Conclusions:
- It appears that DCPP Project Engineering is making progress in managing their workload with their various trending programs. These trending programs include workload by department and section, due date extensions and overdue, and average age vs. goals. While they do not have a formal program to compare resources vs. work load, they do have a means to track what their work load is and how they are managing the work.
3.6 Review Employee Concerns & Differing Professional Opinion Programs
The DCISC met with Rick Burnside, Employee Concerns (EC) Program Supervisor to review status of the Employee Concerns and Differing Professional Opinion Programs. The DSISC last received an update on these programs at the June, 2005 Public Meeting (Reference 6.6). Mr. Burnside stated that the EC group consists of him and one lead investigator, which is enough for the number of concerns they receive.
The EC group has received 3 formal allegations from the NRC that were raised by the public. None of these concerns were substantiated by DCPP EC group. One internal concern was raised which was not substantiated. They have had 28 low-level contacts that were looked into. They normally have about 30 to 70 low-level contacts each year. Mr. Burnside distributed charts provided by the NRC on all allegations received by the NRC for all nuclear power plants.
Mr. Burnside stated that, even though DCPP does not receive very many concerns, he thinks there is no problem with employees raising concerns. They had a mini-employee survey in June 2006, and there were no problems raised about safety culture. He also thinks that employee moral is improving.
The last time the Differing Professional Opinion Program was used was in 2004. The program has never had much activity. Apparently differences in Professional Opinion are being resolved within the departments.
- Conclusion:
- It appears that the employees at DCPP feel free to raise concerns within their departments and accept the response. There have not been many concerns raised at DCPP in the last few years. The Employee Concerns Program is available if any employee needs to contact it.
3.7 Review System Health Reports that are Red or Yellow with I&CE System Engineering Supervisor
The DCISC Fact-finding Team met with Lou Fusco, Manager Instrumentation, Control and Electrical (I&CE) Systems to review System Health Reports that are Red or Yellow. The DSISC last reviewed System Health Reports like this at the April, 2006 Fact-finding meeting (Reference 6.7).
The System/Component Health is represented by a color:
- Green (satisfactory) indicates the structure, systems, and components (SSC) has no performance issues
- White (satisfactory) indicates the SSC has completed all actions to correct major performance/health issues and performance is trending toward the goal or target
- Yellow (unsatisfactory) indicates the SSC has major performance/health issues with actions scheduled for implementation
- Red (unsatisfactory) indicated the SSC has major performance/health issues and actions are being developed
In addition to the performance metrics, DCPP leadership has identified four key indicators for which SSC health should be reviewed and actions taken for improvement. They are: 1) SSC placed in Maintenance Rule (a)(1) status, 2) Prompt Operability Assessments (POAs), 3) Critical Equipment Event Clock Resets, and 4) Significant Adverse Trends in Performance.
Mr. Fusco reviewed each of the systems in I&CE that were in Red or Yellow rating, how long they have been in that rating, what needed to be done, and the schedule for returning them to White or Green rating. The systems in the Yellow rating are:
- Unit 1 Digital Feedwater Control
- Units 1 & 2 Anticipated Trip without Scram Mitigation System Actuation Circuitry (AMSAC)
- Units 1 & 2 Eagle 21, Solid State Protection System
- Units 1 & 2 Solid State Protection System (SSPS)
- Units 1 & 2 480V Vital & Non-Vital AC Electric Power Systems
- Unit 2 125V DC Vital and Non-Vital Electric Power (?) Systems
The system rated Red is: Unit 1 & 2 Plant Process Computer (PPC).
Unit 1 Digital Feedwater Control has been rated Yellow since June 2006 due to critical components approaching end of life and no accessible spares. Scheduled for replacement in 1R14 (May 2007).
Units 1 & 2 AMSAC has been rated Yellow since January 2006 due to backup power supply not being available. Scheduled for replacement in October 2006.
Units 1 & 2 Eagle 21 has been rated Yellow since June 2006 based on the past history of the system being chronically White for the past 3 quarters. The following actions are required to restore the system to Green status: 1) Westinghouse is to provide a software upgrade to eliminate parameter update lockups, 2) install software on the in-house warmup rack 13 for testing prior to installing changes in the plant, and 3) schedule and incorporate software changes to Eagle 21 system in the plant. This work is to be completed in June 2007.
Units 1 & 2 Solid State Protective System (SSPS) has been rated Yellow since July 2006 due to the systems being placed in Maintenance Rule a(1). There is a high potential for the test probe to inadvertently contact the relay casing during installation or removal of the probe during time-response testing. The corrective action is to install banana jacks onto terminal blocks on the contact side of the applicable slave relays to provide a safer test connection point. These will be installed in 1R14 and 1R14.
Units 1 & 2 480V Vital & Non-Vital Electric Power System was rated Yellow in June 2006 due to three adverse trends concerning 480 V components. Actions required to return the system to Green are 1) replace approximately 220 vital and non-vital buckets during the bucket replacement project. Estimated completion date is 2R17 (December 2012), and 2) replace 48 ERC Category 1A/1S thermal overload relays in local linestarters. Estimated completion date is 2R14 (next outage). The DSISC questioned why the corrective action for item 1 was so far out in the future (Dec. 2012) and Mr. Fusco agreed it was too far out, and he would look into why.
Unit 2 125 V DC Vital and Non-Vital Electric Power System was rated Yellow in June 2006 due to adverse trend battery 26 degrading capacity trend. Corrective action is to replace battery 25/26 in 2R14.
The Plant Process Computer (PPC) has been rated Red since April 2004 due to management direction for Unit 1 and Unit 2 due to its availability trending negatively. Unit 2 has had two failures in the last quarter. Aging issues are adversely affecting the system’s ability to maintain the needed design availability. The corrective action is replacement of the computer portion of the systems. Unit 1 replacement is scheduled to be complete in September 2007 and Unit 2 in November 2007.
Each System Engineer looks at his system monthly, and if anything changes the System Health report is revised. All System Health reports are reviewed by the Manager and Supervisor for quality and any recommendations are given to the System Engineer. The Plant Health Committee reviews system health at each of its weekly meetings and approves funding for needed improvements.
- Conclusion:
- DCPP has identified corrective actions necessary to return the Instrumentation, Control and Electrical (ICE) systems rated Yellow and Red (both unsatisfactory) to White or Green, (both satisfactory) and have scheduled repair or replacement in a timely manner (except for the Units 1 & 2 480V Vital & Non-Vital Electric Power System not being timely). DCPP will review the schedule for the 480V Vital & Non-Vital Electric System to determine if the schedule should be shortened. The DCISC should follow up on these actions.
3.8 Tour of Intake Structure Including Concrete Work Performed During Outages 1R13 and 2R13
The DCISC Consultant Booker met with Rob O’Sullivan, Senior Civil Engineer and Travis McRitchie, Civil Engineer, to tour the Intake Structure and review the inspections performed during 1R13 and 2R13. The DCISC last reviewed the intake structure inspections at the January, 2005 Fact-finding Meeting (Reference 6.8).
The group first toured the top of the intake structure where Mr. O’Sullivan pointed out areas of concrete that are degrading. Many of these areas were repaired in the 1994-1995 timeframe. The tour continued in the inside of the intake structure. Areas of new degradation were also pointed out. The overall materiel condition inside the intake structure appears to be good. Mr. O’Sullivan discussed the previous intake inspections of each outage and that they had not been able to get budget approval to start the necessary repairs. He reported that an AR had been issued which detailed the status of the degrading concrete of the intake structure and why repairs are needed in the near future.
The inspection results of 1R13 showed that the general condition of the concrete in Circulating Water Conducts (CWC) is structurally sound. There are several areas on the common wall in both CWCs that meet the DCPP requirements for concrete in need of repair. These areas have been previously evaluated and repairs have been deferred to a future outage. The overall condition of the concrete in Traveling Screen Forebays (TSFBs) 1-2 through 1-6 is fair to poor. These TSFBs were only visually inspected during this outage due to limited accessibility. The overall condition of the concrete at the Discharge Structure at elevation 85 feet is poor. Of the total 3,000 square feet of concrete inspected, 13.5 percent is delaminated. No concrete repairs were made during 1R13 outage.
The inspection results of 2R13 showed that the general condition of CWCs 2-1 and 2-2 is structurally sound. Also the general condition of the concrete in CWP Forebay 2-1 and 2-2 is good. However, the general condition of the concrete in TSFB 2-6 is very poor. No inspection was performed on TSFB 2-1 through 2-5, but there is no reason to believe that the conditions of these TSFBs are any different than that of TSFB 2-6 which is very poor. Many areas were not inspected due to limited accessibility. No repairs were made during 2R13.
The Civil Maintenance Rule Program stated that civil structures shall be considered for transfer to (a)(1) status if a structure is degrading such that if degradation were allowed to the next normally scheduled inspection, the structure may not meet its design basis (i.e. a clearly declining trend). The ACE performed for adverse trend concluded that “even though the structure can perform its intended function, it is trending to a point in which the marginal capacity of the building is being compromised.” As a result of this adverse trend, a goal-setting evaluation is required. AR A0654467 was issued for a goal-setting review for the intake structure.
The AR stated the cause was the ineffectiveness of the long-term plan process to successfully secure funding for the engineering of recommended repairs and preservation work in order to maintain and control concrete degradation at the intake structure. Due to the history of performance problems with the intake structure, the adverse trending in degradation and the lack of attention being afforded to the recommended repair and preservation, the AR stated that the intake structure should be conservatively placed in Maintenance Rule (a)(1) status. Even though the structure can perform its intended function, it is trending to a point in which the marginal capacity of the building is being compromised.
Corrective actions have now been identified and funded, and successfully completing the corrective actions will allow the intake structure to be removed from (a)(1) status. The corrective actions outlined in the AR, as well as the process to effectively monitor the plan’s success, is a long term commitment. Engineering estimates that by 12/31/2009 (2 outages per unit) the goals should successfully be met.
- Conclusion:
- DCPP Engineering has been performing good intake structure inspections at each Refueling Outage and recommending repairs but has had difficulty obtaining funding. Engineering took the unusual step of writing an Action Request on the funding process itself, and the proposed corrective action for repairs and preservation has finally been funded for completion by 12/31/2009. DCISC should follow the status of the corrective action at future Fact-finding Meetings.
3.9 Meet with Plant Management
Dr. Rossin, DCISC Member, met with Donna Jacobs, VP Nuclear Services, to discuss items reviewed in the Fact-finding meeting and other items of interest.
3.10 Nuclear Excellence Information System (NEXIS) Project Overview
The DCISC met with Jeanine Holtx, IT lead, and John Nystrom, Change Management Readiness Lead (Consultant) to review the status of the Nuclear Excellence Information System (NEXIS) Project schedule. The DCISC has reviewed NEXIS at the January, 2006 Fact-finding Meeting (Reference 6.9) and at the June 2006 Public Meeting. The purpose of this meeting was to get an update on the schedule and implementation of the Nuclear Excellence Information System (NEXIS) Project.
The new schedule for NEXIS is to go on line by June 30, 2007. They are now in the process of adjusting all procedures for the new date. Final testing is now projected for January 2007 with “string testing cycle” testing in August 2007. They still have a lot of work to be completed this year (2006) before final testing in January 2007 and before 1R14. NEXIS is run on the PG&E main frame computer, as are all PG&E corporate computer information. It is located in Fairfield, CA. The computer center has redundant emergency power and AC supply.
The training plan was to have all personnel signed up for training before the date of the changeover. The present schedule is to have those needing early training identified by Sept. 1. All others will plan their training sessions during the period starting October 1, and Dec. 1, 2006. Actual training sessions are to start by Jan. 15, 2007. There will be no training scheduled 7 weeks before 1R14 and no training during 1R14. After the outage there will be training for selected people through June 2007.
There are a total of 1557 employees needing training for NEXIS. There will be 9 classroom courses, 3 computer-based courses and 29 lesson sessions for very specific groups that will have limited use of SAP.
1R14 will be the first outage from which data will be fed to NEXIS. They will take ARs from 1R14 outage and use NEXIS SAP to see how it works.
PIMS will continue to run after NEXIS is operational for information but can no longer be used for active system work. All information will be transferred to the SAP System.
The new clearance handling program eSOMS was not delayed. It is a stand-alone program that interfaces with SAP. The new clearance program is now using PIMS and will transfer to SAP by June 2007.
PG&E has 66 projects involved with SAP and it was going to be difficult for DCPP and PG&E business projects to go on-line together on the old schedule. They currently have three concerns with getting NEXIS on line. These are: 1) Data conversion, 2) interface with all the systems, and 3) PG&E corporate business projects coordinating with DCPP. They stated that the support from SAP has been very good. Lots of corporations are going to SAP and 10 to 12 Nuclear Plants will be using SAP soon.
- Conclusion:
- It appears that the delay of startup of the Nuclear Excellence Information System (NEXIS) project was necessary and will allow more time for training (which will still require major resources). NEXIS should improve overall use of computer programs and gathering of information.
3.11 Attend Plant Health Committee Meeting
The DCISC Fact-finding Team was scheduled to attend the Plant Health Committee Meeting scheduled for 1:00 PM August 3, but this meeting had been rescheduled for early in the day and the DSISC was not informed until noon. The DCISC should attend these meeting in the future when they are being held during a Fact-finding meeting.
4.0 Conclusions
- 4.1
- It appears that Quality Verification (QV) continues to perform good assessments of performance of DCPP including the Quality Performance Assessment Report (QPAR). The audits and assessments identify issues that should have been identified and corrected during the line organization self-assessments. DCPP management should be asking the line organization why they are not finding these issues before QV identifies them during audits and assessments. The DCISC should continue to review the QPARs at future Fact-finding Meeting.
- 4.2
- DCPP had made improvements in their outage safety plan in 2005 on their own initiative. The The Institute of Nuclear Power Operators (INPO) requested self-assessment with peer reviewers appears to have identified additional improvements and recommendations which should make for additional improvements in the outage safety plan. DCISC should review the final report and the improvements made in the outage safety plan prior to 1R14.
- 4.3
- The DCPP Radiation Protection Group is performing its work well, and applying its experience and initiatives to provide quality service to DCPP. This has resulted in improved line ownership of ALARA and a record low collective radiation dose in Outage 2R13.
- 4.4
- It appears that DCPP is making improvements with their Self-Assessment Program. They still need to make improvements in their Benchmarking Program. The two Self-Assessments that were reviewed identified areas for improvement (AFIs), however these had been identified by INPO in 2005 and DCPP was not successful in taking corrective action to solve the problems. DCISC should review the actions taken in the AFIs at future Fact-finding meetings. The DCISC should attend the Management Review Meetings and Self-Assessment Review Board Meetings if they are taking place during future Fact-finding meetings.
- 4.5
- It appears that DCPP Project Engineering is making progress in managing their workload with their various trending programs. These trending programs include workload by department and section, due date extensions and overdue, and average age vs. goals. While they do not have a formal program to compare resources vs. work load, they do have a means to track what their work load is and how they are managing the work.
- 4.6
- It appears that the employees at DCPP feel free to raise concerns within their departments and accept the response. There have not been many concerns raised at DCPP in the last few years. The Employee Concerns Program is available if any employee needs to contact it.
- 4.7
- DCPP has identified corrective actions necessary to return the Instrumentation, Control and electrical (ICE) systems rated Yellow and Red (both unsatisfactory) to White or Green, (both satisfactory) and have scheduled repair or replacement in a timely manner (except for the Units 1 @ 2 480V Vital @ Non-Vital Electric Power System not being timely). DCPP will review the schedule for the 480V Vital @ Non-Vital Electric System to determine if the schedule should be shortened. The DCISC should follow up on these actions.
- 4.8
- DCPP Engineering has been performing good intake structure inspections at each Refueling Outage and recommending repairs but has had difficulty obtaining funding. Engineering took the unusually step of writing an Action Request on the funding process itself, and the proposed corrective action for repairs and preservation has finally been funded for completion by 12/31/2009. DCISC should follow the status of the corrective action at future Fact-finding Meetings.
- 4.9
- It appears that the delay of startup of the Nuclear Excellence Information System (NEXIS) project was necessary and will allow more time for training (which will still require major resources). NEXIS should improve overall use of computer programs and gathering of information.
- 5.0 Recommendations:
- None
6.0 References
- 6.1
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit D.6, Section 3.7.
- 6.2
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit D.2, Section 3.3.
- 6.3
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit B.9.
- 6.4
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit D.8, Section 3.5.
- 6.5
- “Diablo Canyon Independent Safety Committee Fourteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2003 – June 30, 2004”, Exhibit D.7, Section 3.4.
- 6.6
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, June 2005 – June 2006”, Exhibit B.9
- 6.7
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit D.6, Section 3.3.
- 6.8
- “Diablo Canyon Independent Safety Committee Fifteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2004 – June 30, 2005”, Exhibit D.6, Section 3.9.
- 6.9
- “Diablo Canyon Independent Safety Committee Sixteenth Annual Report on the Safety of Diablo Canyon Nuclear Power Plant Operations, July 1, 2005 – June 30, 2006”, Exhibit D.6, Section 3.10.