ANSWER

Chaos Engineering is the practice of testing the resiliency of a system and its ability to withstand random and unforeseen disruptions by deliberately subjecting the system to a wide range of stress scenarios.  The goal of this discipline is to identify previously undetected points of system failure in order to correct them or to prevent them in the future.

How is Chaos Engineering Different from Testing?

Traditional testing tends to focus on the specific and required capabilities to be supported by the system.  It involves identifying a list of specific conditions and scenarios that must be validated to ensure the system works as desired.  In a nutshell, traditional testing and quality assurance focuses on testing known conditions.

On the other hand, the chaos engineering approach is much more exploratory in nature; it seeks to elicit unknown and unexpected behaviors from the system.  One way to look at it is to consider chaos engineering as an information gathering exercise which would lead to the creation of future scenarios to be considered in traditional testing.  In a nutshell, chaos engineering focuses on previously unknown outcomes.

If traditional testing is likened to verifying the skills of first responders to specific emergency scenarios, chaos engineering is likened to simultaneously subjecting the first responders to hurricanes, earthquakes, and wildfires.

What are some of the Chaos Engineering Approaches?

• Chaos Engineering at the Infrastructure Level - Evaluate the resiliency of the production platform by simulating failure of various system components, zones, and instances.

• Chaos Engineering at the Network Layer - Evaluate the system behavior in the face of the failure of various combinations of network components and connections.

• Chaos Testing at the Application Layer - Evaluate the application’s resiliency by simulating failures of various application components across architectural layers.

Chaos Engineering Best Practices

• Minimize the blast radius - While conducting experiments in production, it is vital to have safety measures in place to limit the impact (blast radius) of the chaos tests in order to avoid causing unnecessary impact to the users of the system, the customers..

• Create small chaos and then big chaos - Start by applying chaos engineering disruptions to a subset of the system in order to determine the effectiveness of the methods employed.  Once successful, expand the chaos to the whole system to determine its overall robustness.

• Simulate real world events - Define real-world scenarios which could be catastrophic to your system such as hardware failure, software failure, network failure, entire site outages, experts unavailable, etc.

• Automate chaos engineering experiments - Once organizations get good at the chaos engineering discipline, they should automate high-value experiments to occur at random (but regular) intervals.

What are some of the tools available to support Chaos Engineering?

Please be aware that this list of tools is rapidly changing as the chaos engineering discipline continues to evolve and mature.  Do your own research of the latest and most effective chaos engineering tools.

• Chaos Monkey
• Litmus
• Chaos Toolkit
• Gremlin
• ToxiProxy
• Swabbie
• Conformity Monkey
• Spinnaker
• Chaos Lambda
• Pumba
• Service Mesh
• Istio