Edited Version of March 3, 1999
EIIP Classroom Online Presentation
"Year 2000 - A Risk Management Approach"
Robert Dykes
Director of Operations
EQE International
Newport Port Beach, CA
Thomas Roche
Project Manager, Y2K
EQE International
Newport Port Beach, CA
Neil Blais
Senior Project Engineer
EQE International
Newport Port Beach, CA
The original unedited transcript of the March 3, 1999 online Virtual Classroom presentation is available on the EIIP Virtual Forum (http://www.emforum.org). The following version of the transcript has been edited for easier reading and comprehension. Typos were corrected, date/time/names attributed by the software to each were deleted but content of discussions, questions, and responses are as stated by each participant. Answers from the presenter to questions by the audience are grouped beneath the appropriate question to facilitate meaning.
[Opening]
Amy Sebring: Welcome to the EIIP Virtual Classroom!
We are starting what we hope will be a series of sessions in the Classroom about different hazards, and we are starting off with one that is high on the attention list, Y2K. Next month we will be taking a look at drought and we also hope to do one on tsunamis.
One quick note about any URL's that may be used in the session; they are live links and you can click on them and view the referenced site in your browser window.
Subsequent "slides" may display behind your chat window, so you may need to bring the browser window forward. Background information for today's session may be found at <http://www.emforum.org/vclass/990303.htm>.
We will have a presentation for about thirty minutes, and then have audience Q&A for the last thirty minutes. I will review the instructions for Q&A as we are about to begin that portion.
Today we are revisiting the Y2K issue, this time from a risk management perspective.
[Introduction]
We are pleased to welcome EIIP partner EQE International today. With us are:
• Tom Roche, Project Manager for risk management and remediation programs associated with the Year 2000 issue for facilities in the water, wastewater, nuclear energy, and transportation industries
• Bob Dykes, Director of Operations and responsible for implementing the risk-based approach to the Year 2000 embedded systems analysis and Y2K auditor for the State of Washington
• Neil Blais, Senior Project Engineer at EQE International's Newport Beach, California office to whom we are grateful for arranging today's program.
Tom Roche: I'll wait a few seconds for Amy to return.
Avagene Moore: Just a moment, everyone. I can pick up where Amy left off if she doesn't return immediately. Just a moment. Amy called. She has problems. I will take over. Welcome gentlemen, and Tom, I understand you are going to begin the presentation. Tom, please go ahead now.
[Presentation]
Tom Roche: Please present SLIDE 1.
[SLIDE 1]
Tom Roche: EQE entered the Y2k embedded systems service area in 1997.
Our background in risk management from the systems and operations perspective and our analysis methodology of complex systems:
Bob has experience primarily in the petrochemical, shipping, and defense areas with a focus on internal events. I have worked primarily in the electric power and utility areas with a focus on external events.
We are now convinced that addressing the issue from the risk management and systems approach, rather than focussing on chips, clocks, and software, is an advantage. We will present 10 more slides covering the issue, a risk-based approach, supply chain and external interdependencies, and the end result.
One of the most common questions is how bad will it be? Our typical response is "we don't know". There is a small probability that this is all press and consultant hype, as well as a very small probability that this will lead to long-term, global, and irreversible consequences.
Amy Sebring: Tom, you still with us? I was afraid of that. They all lost their connection at the same time. While we are waiting for them to get back; I will put up the next slide to take a look at.
[SLIDE 2]
Tom Roche: Pardon the delay. I lost connection. Please present SLIDE 3.
[SLIDE 3]
Tom Roche: We defined disruption as recoverable and short-term, such as a severe storm, moderate earthquake, or localized civil unrest. The Catastrophe scenario is more widespread and longer duration; such as major a major flood, earthquake, or hurricane. Perhaps we should look at more serious events such as war to grasp the fact that this event could impact the entire world.
Most people in the risk-management field are familiar with the required phases of an effective Year 2000 program:
• Assess the issue and potential impact through risk assessment or triage
• Find the IT and embedded systems and assess compliance
• Remediate systems or devices that are not Year 2000 Ready.
Please present SLIDE 4.
[SLIDE 4]
Tom Roche: Documentation and Contingency or Continuity Planning are equally important elements, but most of us prefer to present in sets of three. We look at three basic issues when addressing virtually any risk issue: 1) What can go wrong? 2) What are the consequences? And 3) What is the likelihood? Please present SLIDE 5.
[SLIDE 5]
Tom Roche: With the likelihood of the Year 2000 event occurrence approaching 1.0 and limited empirical data on the likelihood of failure; it is difficult to prescribe an appropriate level of preparedness.
When a customer inquires about an appropriate supply of fuel or extent of testing for systems determined to be compliant by the manufacturer, we look at the consequences of failure. Please present SLIDE 6.
[SLIDE 6]
Tom Roche: SLIDE 6 was excerpted from a series of workshops for California water and wastewater utilities, thus, the earthquake analogy. Hurricanes, floods, or severe weather may be appropriate planning starting points for other areas.
Risk management and contingency or continuity planning for the Year 2000 event should follow a similar approach as other risk scenarios. The primary difference with this event is the convenience of having it scheduled well in advance.
We suggest a structured approach to evaluate potential consequences of failures. Questions presented on Slide 7 are examples of questions that should lead to a thorough understanding of the system and Year 2000 exposure.
A facilitator with key operations, engineering, and/or maintenance personnel can systematically review an operation or process. Documentation can be through a hazard analysis program or simple database or spreadsheets such as performed during a Hazard and Operability (HAZOP) study. Please present Slide 7.
[SLIDE 7]
Tom Roche: Past system or supply chain failures can be a key source of information to identify potential Year 2000 problems as well as mitigative actions. Please present SLIDE 8.
[SLIDE 8]
Tom Roche: Electrical power as well as communications, water, wastewater, and gas utilities are generally a key to most operations. Please present SLIDE 9.
[SLIDE 9]
Tom Roche: Interruptible power supplies (UPS) and emergency or standby generators are short-term solutions with less than perfect performance records based on past power outages. Past outages can provide some insight into the possible duration of power outages. Please present SLIDE 10.
[SLIDE 10]
Tom Roche: If grid failure occurs due to system phase, frequency, and voltage instability; the Northridge earthquake is a good data point, with the entire City of Los Angeles black for the first time in history. The Auckland New Zealand outage may be a better example if an outage results from chronic hardware failures. Please present SLIDE 11.
[SLIDE 11]
Tom Roche: In closing, this is a good opportunity to revisit emergency, contingency, continuity, or resumption plans from the system and operation perspective, rather an exercise to meet regulatory or other bureaucratic reporting requirements. This is the end of our slides. Thank you.
Amy Sebring: Tom, did Bob have a few words to add at this point?
Tom Roche: Bob is a little slower at typing. Old dog, new tricks.
Bob Dykes: No, Tom and I will work back and forth to answer the questions as they arise.
Amy Sebring: Ok, thanks Tom. We will now turn you over to our audience.
If you have a question or comment, please indicate by inputting a question mark (?) to the chat screen. Then compose your question but hold it until you are recognized. Please indicate to whom your question is addressed.
Audience Questions
Question:
Burt Wallrich: In talking to community-based organizations I take a four-pronged approach:
(1) technical - identifying critical systems that might be at risk and then checking them for Y2K compliance;
(2) household approach - encouraging staff and clientele to take prudent home disaster preparation steps;
(3) preparation for immediate disaster impacts (in my agency, e.g., that means being staffed on 12/31/99 at a higher level than usual for a holiday);
(4) preparation for longer-term recovery activities, e.g., if there is a significant loss of jobs leading to pressure on food and homeless services.
Bob Dykes: Although you are giving them good advice, it would appear that they still need to think about answering the basic questions mentioned in the presentation. How do I prepare?
Taking the risk-based approach allows them to think about what they really need and where to focus resources in the limited time remaining. This is what the risk-based approach is all about. Where to focus our efforts to minimize business or government interruptions.
Tom Roche: We do advise clients to focus on home and family so that staff outside distractions will be minimal.
Question:
Amy Sebring: Bob or Tom, do you find that the public and private sectors are working on this about the same, or is one in better shape than the other?
Bob Dykes: This depends on the location and leadership. The state of Washington has an active program that is directed by the governor; some industries are very aggressive on this. The approach is on how they want to reflect to their interested parties.
Neil Blais: Bob means, how is the company or agency seen by the stakeholders. This will often guide the approach.
Question:
Avagene Moore: One of your slides mentions a rule of thumb being based on preparation for consequences of an earthquake. Is that practical or a worst-case scenario?
Tom Roche: An earthquake may not be a worse case scenario, but may be appropriate, from the planning point of view. As is all contingency or emergency plans, it depends on the consequence of failures. Many of our clients look a few steps beyond their earthquake plan.
Question:
John DeAngelo: Do you think that media hype near years end will create undue nervousness with the general public? That could generate more inquiries to public safety agencies and they may not have all the answers that the public would expect from them.
Bob Dykes: In a recent visit to one of my agencies in Washington, my point of contact had just got off a phone call with a reporter from a local newspaper. They were looking for things wrong or potential problems to stir things up. So the answer is expect lots of hype.
Tom Roche: We do think the media has taken us to the disruption scenario even in the unlikely event that this is all hype.
Neil Blais: At the NEMA conference this year, there was discussion on the need to provide the proper public perception. This is often complicated by potential legal ramifications. FEMA and the State Directors all agreed that public perception is a critical issue which warrants watching.
Question:
Christopheer Effgen: Bob or Tom: What advice do you give them regarding home and family preparations?
Tom Roche: I'll take this one. I advise friends and family to prepare as we would for any disaster; keep the pantry stocked, propane bottles filled, and some water around. This is just good practice, especially in earthquake country.
Bob Dykes: But start preparations early, things may get interesting as the year progresses.
Question:
Amy Sebring: I would also like Burt Wallrich to tell us what he is advising in California also. Burt?
Burt Wallrich: Local governments in my area are trying to balance reassurance with reasonable preparedness information. I am afraid, though, that fear of civil disorder has become primary and they are downplaying the real potential problems in favor of reassurance.
Question:
Christopheer Effgen: The problem I see is that after an earthquake, every body goes home, what happens to the company?
Tom Roche: Our experience shows that most companies can resume business within a few days.
Bob Dykes: That is a good question. We addressed this to one of our clients that is in a remote area. They have a problem with getting people to the site, hours they can work and a multiple of supply problems. One of the things they are looking at is how they can reinforce the feeling among their employees that their families will be taken care of. This is a major part of their contingency plan.
Question:
Chip Hines: Are there any tools that you can recommend that would help with your approach to analysis and planning?
Tom Roche: We generally use Access database applications. The HAZOP risk analysis format has proven effective for process industries.
Bob Dykes: The tools can be dependent upon the size of the problem.
From simple Excel spreadsheets to complex databases. Think 125,000 items verses 2200 verses 200.
The legal requirements may become a large part of the documentation. We have a database that includes a picture (digital) of the piece of equipment and all the compliance information.
Question:
Amy Sebring: Can you elaborate some Tom regarding Access database? Is this for tracking the systems checked?
Tom Roche: We use access database applications to manage programs ranging from small facilities to nuclear power plants with several hundred thousand items. Generally to track the individual systems, devices, software, and interfaces.
Question:
Chip Hines: Have you applied any systems models such as iThink, or dynamo?
Tom Roche: We have not. Please send me info on these later.
Question:
Amy Sebring: Bob, how significant is the embedded aspects of this problem do you feel as a percent of the total?
Bob Dykes: The percent could be 30-50, depending on what the company produces. A production company will be much different from a financial institution.
Question:
Avagene Moore: Do you have any indication that more local government officials are taking heed and becoming aware of the Y2K problem? A few months ago, 50 percent weren't doing anything at all.
Tom Roche: Yes. A number of state and local organizations are becoming much more active. California is sponsoring workshops at the state level to promote the issue to local entities.
Bob Dykes: The State of Washington is working to get the cities and counties more involved. As one would expect, money is a major problem as the organizations get smaller.
Neil Blais: I believe you will also see a lot more State Agencies driving Y2K compliance or contingency planning down to the County and City level. This is due in part to the concern that the State and Federal Officials might be stretched if too many problems occur.
Final Question:
John DeAngelo: Without electricity, I can dream up lots of problems; do you have an opinion on how prepared the electrical grid will be?
Tom Roche: The electrical power grid and communications systems are the primary issues for virtually all of our clients. Electric utilities are generally working aggressively to address the issue.
At a National Electric Reliability Council (NERC) workshop, the industry discussed methods to increase grid stability through localized generation to isolate potential islands, excess capacity (more plants on-line), reduced output for individual plants to withstand transients, and close coordination among utilities.
We should not forget how fragile the grid can be. Damage to substations in Los Angeles resulted in power outages in seven Western States, Canada, and Mexico. A worse case scenario for the US would be to have all four coordinating councils go black (the four regions are separated by AC-DC-AC ties). I think local brownouts and blackouts are possible, if not likely. However, such events are recoverable, as demonstrated during past events.
Bob Dykes: One client is at the highest user for that electrical supplier, and is at the end of the line. They may have a power problem, but in their contingency plan they are working with the supplier to develop load shedding plans to keep operations going.
Amy Sebring: We will have to leave it at that for now. Thank you very much to EQE, Tom, Bob and Neil. Ava, coming events?
Avagene Moore: Thanks, Amy. Next Tuesday, March 9, 1:00 PM EST, John Laye, Contingency Management Consultants (EIIP Partner), will lead a Round Table discussion for us.
On Wednesday, March 10, 12:00 Noon EST, we will be in the Virtual Library for "The FireNet Story: Information Technology Use at the Aberdeen Proving Ground Fire Department" with Ann Willis, George Washington University, and Kevin Farrell, Aberdeen Proving Grounds. Make plans to join us for both sessions. Back to you, Amy.
Amy Sebring: Thank you audience. Our time is up, but we will be adjourning to the Virtual Forum room for a few minutes of open discussion, and you are invited to join us there.