Years ago, I was one of three people in a startup company providing design and development services for web hosting and online message boards. We started the company on a dining room table. As we expanded into the living room, we quickly realized that it was getting too cramped and we needed more space to let our creative juices flow, plus we needed to find a way to stop being at each other’s throats. We decided to pack up our laptops move into a co-working space in Venice, California. We were one of four other companies using the space and sharing the rent. It was quite a nice setup and we were enjoying the digs. We were eager to get to work in the morning and wouldn’t leave sometimes till very late in the evening.
One Thursday morning as we pulled up to the office to start the day, we noticed the door wide open. Someone had broken into the office in the middle of the night and stolen all of our equipment, laptops, computers etc… This was before the time of cloud computing, so data backup at that time was mainly burning CDs, which often times we would forget to do or just not do it because “we were just too busy”. After the theft, we figured we would purchase new laptops and recover from the latest available backups. As we tried to restore our data, none of the processes were going as planned. Either the data was corrupted, or the CD was completely blank or too old to be of any value. Within a couple of months, we bit the bullet and had no choice but to close up shop.
BY THE NUMBERS
Business interruptions come in all shapes and sizes. From natural disasters, cyber security incidents, system failures, human error, operational activities, theft, power outages…the list goes on and on. In today’s landscape, the lack of business continuity planning not only puts companies at a competitive disadvantage, but can spell doom for the company as a whole. Studies show that a single hour of downtime can cost a small business upwards of $8,000. For large enterprises, that number skyrockets to millions. That’s 6 zeros folks! Compound that by the fact that 50% of system outages can last 24 hours or longer, and we’re talking about scarily large figures.
The impact of not having a business continuity plan doesn’t stop there. As if those numbers weren’t staggering enough, a study done by the AXA insurance group showed 80% of businesses that suffered a major outage filed for bankruptcy within 18 months, with 40 percent of them out of business in the first year. Needless to say, business continuity planning (BCP) and disaster recovery (DR) are critical components and lack of planning in these areas can pose a serious risk to any modern organization.
We can talk numbers all day long about why BCP and DR are needed, but the bottom line is – THEY ARE NEEDED. Frameworks such as NIST Special Publication 800-53 Rev.4, 800-34 and ISO 22301 define an organization’s “capability to continue to deliver its products and services at acceptable predefined levels after disruptive incidents have occurred. They provide much needed guidance on the types of activities to consider when formulating a BCP. They can assist organizations in ensuring business continuity and disaster recovery systems will be there, available and uncompromised when required.
DISASTER RECOVERY – DON’T LOSE SIGHT OF SECURITY & RISK
Once established, business continuity and disaster recovery strategies carry their own layer of complexities that need to be properly addressed. A successful implementation of any disaster recovery plan is contingent upon the effectiveness of its design. The company needs access to the data and applications required to keep the company running, but unauthorized access must be prevented.
Security and privacy considerations must be included in any disaster recovery planning.
Security and risk are top priority at every organization, yet traditional disaster recovery procedures focus on recovery from an administrative perspective. What to do to ensure critical business systems and applications are kept online. This includes infrastructure, staff, connectivity, logistics and data restoration. Oftentimes, security is overlooked and infrastructure designated as disaster recovery are looked at and treated as secondary infrastructure, and as such, the need to properly secure (and budget) for them is also treated as secondary to the production systems. Companies invest heavily in resources, security hardware, software, tools and other solutions to protect their production systems. Typically, only a subset of those security solutions are deployed, if at all, to their disaster recovery systems.
The type of DR security that’s right for an organization is based on need and risk. Identifying and understanding what the real risks are can help focus efforts and close gaps. A lot of people simply look at the perimeter and the highly visible systems. Meanwhile, they’ve got other systems and back doors where they’re exposed, potentially leaking data and wide open for attack.
In a recent article, Barry Forbes, XYPRO’s VP of Sales and Marketing discusses how senior executives at a top five US Bank indicated that they would prefer experiencing downtime than dealing with a breach. (http://bit.ly/1mFNpRL) The last thing you want to deal with during disaster recovery is being hit with a double whammy of a security breach. Not having equivalent security solutions and active monitoring for disaster recovery systems puts your entire continuity plan and disaster recovery in jeopardy. This opens up a large, exploitable gap for a savvy attacker or malicious insider. Attackers know all the security eyes are focused on production systems and data, yet the DR systems whose purpose is to become production systems in case of disaster are taking a back seat and ripe for the picking.
Not surprisingly, the industry is seeing an increasing number of breaches on backup and disaster recovery systems. Compromising an unpatched or an improperly secured system is much easier through a DR site. Attackers know that part of any good business continuity plan is to execute the plan on a consistent basis. This typically includes restoring live data onto backup or DR systems and ensuring applications continue to run and the business continues to operate. But if the disaster recovery system was not monitored or secured similar to the live system using similar controls and security solutions, the integrity of the system the data was just restored to is in question. That data may have very well been restored to a compromised system that was lying in wait. No one wants to issue outage notifications couple with a breach notification.
The security considerations don’t end there. Once the DR test has checked out and the compliance box ticked for a working DR system and successfully executed plan, attackers and malicious insiders know that the data restored to a DR system can be much easier to gain access to and difficult to detect activity on. Therefore, identical security controls and inclusion of DR systems into active monitoring is not just a nice to have, but an absolutely necessity.
COMPLIANCE AND DISASTER RECOVERY
Organizations working in highly regulated industries need to be aware that security mandates aren’t waived in times of disaster. Compliance requirements are still very much applicable during an earthquake, hurricane or data loss.
In fact, the HIPAA Security Rule specifically calls out the need for maintaining security in an outage situation. Section 164.308(a)(7)(ii)(C) requires the implementation, as needed, of procedures to enable continuation of processes for “protection of the security of electronic protected health information while operating in emergency mode.”
The SOX Act is just as stringent, laying out a set of fines and other punishment for failure to comply with requirements, even at times of disaster. Section 404 of SOX discusses establishing and maintaining adequate internal control structures. Disaster recovery situations are not excluded.
It’s also difficult to imagine the PCI Data Security Standards Committee relaxing its requirements on cardholder data protection for the duration a card processing application is running on a disaster recovery system. It’s just not going to happen.
Neglecting to implement proper and thorough security into disaster recovery planning can make an already critical situation spiral out of control. Careful consideration of disaster recovery planning in the areas of host configuration, defense, authentication and proactive monitoring will ensure the integrity of your DR systems and effectively prepare for recovery operations while keeping security at the forefront and keep your business running. Most importantly, ensure your disaster recovery systems are secured at the same level and have the same solutions and controls as your production systems.
Steve Tcherchian, CISSP, PCI-ISA, PCIP is the Chief Product Officer and Chief Information Security Officer for XYPRO Technology. Steve is a Member of the Forbes Technology Council, on the NonStop Under 40 executive board and part of the ANSI X9 Security Standards Committee. With over 20 years in the cybersecurity field, Steve is responsible for strategy and innovation of XYPRO’s security product line as well as overseeing XYPRO’s risk, compliance and security to ensure the best experience to customers in the Mission-Critical computing marketplace.
Steve is an engaging and dynamic speaker who regularly presents on cybersecurity topics at conferences around the world.