Datacenters are unpredictable

Here is why

  • Step 1. We have machines (computers and infrastructure etc) - reliability P0 and people reliabilty P1
  • Step 2. We increase P0 (we have to react on every emergency by fixing things so P0 improves)
  • Step 3. In time P0 grows to such a high level, that it becomes obvious, P1 now can be decreased safely
  • Step 4. So we decrease P1 (in simple terms we can hire idiots, because machines will work no matter who is around)
  • Step 5. We now can push P1 down infinitely (like on Step 2 we were pushing P0 up also infinitely)
  • Step 6. Is equlibrium. It's some combination of P0 and P1 that was impossible to predict. You might try.
  • Step 7. Now can fire all the management - 100% of them. And put new management : Step 1. It's a loop.
  • Not only datacenters of course. Any combination of humans and machines is like that.
    Or even any combination of reliable and unreliable stuff.
    Taleb describes those two matters in "Fooled By Randomness", but he does not elaborate on blends or equilibrium.

    Feb 25 2021