Comments on: Graceful Degradation is Not What We Want; Quick Failure is Better

By: Development and Integrity Management by Eli Lopian » Pushing and pulling

Development and Integrity Management by Eli Lopian » Pushing and pulling — Thu, 11 Jun 2009 18:36:15 +0000

[...] Now that the company is in integrity it is easier to fail fast and to fix our broken processes. As Johanna Rothman says Graceful Degradation is Not What We Want; Quick Failure is Better [...]

By: abby, the hacker chick blog

abby, the hacker chick blog — Tue, 02 Jun 2009 19:59:46 +0000

Couldn’t agree more! I wish that would happen with running shoes… they wear out so quickly but I don’t know how to tell when they start going. I just feel like I’m getting old or out of shape, and then eventually I get a new pair and woah! I can run again.

Never thought about it in these terms for software before… I can definitely see it for performance degradations. But maybe also for those things that don’t quite fit your way of thinking/doing business that you keep having to work around and work around and work around until finally you realize you’ve painted yourself into a corner? Hmm.

Charles: re: Unix philosophy, I like it :-)

By: Jim Ward

Jim Ward — Tue, 02 Jun 2009 19:36:11 +0000

Johanna,

I assume that you are familiar with the bathtub curve model of software system life. Failures are frequent at the beginning. Hopefully, we don’t implement until we have them eliminated, but this does not always happen. Then we have a relatively stable period of performance, hopefully also relatively long. Finally, through accumulated fixes and changing conditions, the software no longer adequately performs the functions it was developed for, and failures become frequent. Again, hopefully, we replace the software before the failures become too frequent. Quick or catastrophic failure is rare with software, it just keeps working less well and less dependably, requiring more maintenance which invariably further degrades it. The real trick is knowing when to pull the plug and replace it.

By: Kathleen

Kathleen — Tue, 02 Jun 2009 13:43:37 +0000

This kind of begs a discussion of knowing when to cut your losses. With imperceptible entropy, it’s hard to know. We went through a similar plumbing problem that we ignored for three years mostly because we thought it’d be expensive to repair. In the end, I said we had to do it because we wouldn’t be able to sell the house like that and in the end, it cost $100. I wish I had those three years back.

I read something the other day that was tangentially related. Contrary to what we all believe, research shows we learn more from success than failure…
http://bit.ly/d8VhV

By: Dwayne Phillips

Dwayne Phillips — Tue, 02 Jun 2009 11:22:48 +0000

Oh if the whole world were ones and zeros. Sadly (for some of us) it is mostly analog.

By: Other interesting blog posts (June 2/2009) « Analytical Mind

Other interesting blog posts (June 2/2009) « Analytical Mind — Tue, 02 Jun 2009 07:29:56 +0000

[...] A nice analogy between a hot water tank and software development by Johanna Rothman in her post called: Graceful Degradation is Not What We Want; Quick Failure is Better. [...]

By: janm

janm — Tue, 02 Jun 2009 04:06:28 +0000

Jim Gray called this “failfast” operation, and it is a useful technique for building fault-tolerant systems.

http://en.wikipedia.org/wiki/Fail-fast

By: Peter Edstrom

Peter Edstrom — Tue, 02 Jun 2009 01:03:02 +0000

I see this from the software bugs perspective: It is far easier to identify and fix syntax bugs because they break quickly and obviously. Logic bugs (“graceful” bugs) are much more insidious and hard to resolve because the cause and effect is not obvious.

By: Simon Morley

Simon Morley — Mon, 01 Jun 2009 23:48:03 +0000

Nice analogy!

What about performance monitoring? That should always help us to spot degradations in complex systems. Those regular/periodic reports fed back are then acted on to put fixes/improvements back into the pipeline.

This might be the equivalent of the gradual water temp dop-off, just that there was no action taken.

Yes, quick failures useful pointers – ideally under lab conditions and not out in the field.

By: Charles McKnight

Charles McKnight — Mon, 01 Jun 2009 23:00:25 +0000

Folks just need to follow the Unix philosophy on other items, to wit:

“Rule of Repair: When you must fail, fail noisily and as soon as possible.”

“Basics of the Unix Philosophy”, http://www.faqs.org/docs/artu/ch01s06.html