Due to the storage replication and network routing requirements for keeping all of the containers interacting, an enormous amount of data must be available for diagnostics. Containers exponentially increase this complexity due to their ephemeral design, which does not include storage that automatically persists to keep log files, kernel panics, or other trace files that are often generated during an incident.Ī Kubernetes cluster running in a single region of a public cloud can be not only spread across several physical servers, but also across different data centers which are miles away from one another. In the cloud computing world, consumers do not even know the physical locations of virtual servers, making metadata invaluable for resolving incidents. We have now moved beyond virtual servers and entered the cloud era. Thus, successfully resolving incidents now requires more knowledge across more domains of information technology.

Since then, we have shrunk the size of servers and even removed much of the physical part altogether, since most servers are now virtual.

Dynamic and Ephemeral InfrastructureĬomputing took up entire buildings in the early days, and incident response took the form of actual bug hunting. This will allow for much more time to build automation for mitigating and remediating common errors until development teams can apply permanent fixes. The value of proper incident response lies not only in finding and fixing problems, but also in the whole process of documenting errors and resolutions, which in turn will provide operations teams with a knowledge base for future reference. This leads to significant technical debt as well as frustrated staff who feel that they never have time to do things properly in order to advance the organization. Over time, you will be unable to maintain the more problematic or complex parts of the infrastructure. This approach may work well for a while, but it is the technological equivalent of duct tape - and it never ends well. It’s human nature to see a problem and automatically start trying everything you can think of to get it fixed as quickly as possible and then hope that life returns to normal. The key to a successful organization is to have the tools and processes in place to handle these incidents and get systems restored in a repeatable and reliable way in as little time as possible.Įvery organization knows how to do firefighting. Inevitably, organizations that use technology (regardless of the extent) will have something, somewhere, go wrong.

Ibm resilient what is kubernetes