Is something metal burning?

On Friday night I experienced what is probably my most spectacular hardware failure yet. I was working on a project for our upcoming Interactive Show, a chandelier with 150 or so individually controllable 5 watt incandescent bulbs:

A small piece of the chandelier during testing.

Anyway, it was way too late and I was rushing to get the last controller board finished on the outer ring of the chandelier, which has 7 controllers and 52 lightbulbs. I plugged the very last controller in backwards and flipped the switch…

Fwoosh. The final controller went up in smoke. Because of the way the boards are set up, 24 volts from a beefy power supply shot backwards into the chip’s power rails. Of course, to get there, the voltage had to flow all the way around the ring, blowing a chunk out of every chip in line, killing the Teensy microcontroller I was using, then feeding into my USB port and frying my laptop, which started smoking as well. It was pretty spectacular – probably the most amount of money and effort I’ve burned in under a second.

Takeaways:

1) Don’t work tired. Nothing of quality ever comes from staying up late. Or rushing. Haste makes waste, as they say.

2) Think about how your power flows. More importantly, think about how your power might flow in different situations. If it’s possible for the power to be hooked up backwards, consider protection diodes, or isolating your power rails, or using a keyed connector.

3) Be thoughtful about your connectors. It’s easy and tempting to design simple things with 0.1″ headers, but this way lies danger. If your project can be damaged by reversing the connector, redesign the pinout, or use a keyed connector that can only be used in one orientation. This makes sure that your tired brain can’t do any serious damage.

4) Isolate your USB devices. A USB isolator might not be a bad investment if you work on a lot of USB devices, especially ones that also use higher voltages. They’re not terribly expensive, but unfortunately are not capable of handling High Speed (480 Mbps), although Full Speed (12 Mbps) is fine. But you can definitely isolate the power rails, which would have at least saved my laptop here.

5) If you find yourself in a similar what-the-heck-just-happened situation (if you don’t, you’re not pushing hard enough), use it as a learning opportunity. Stop and reflect on what happened and how you can improve next time. Ask questions! Just the act of explaining something to another person helps you understand it better. Then blog about it and press on.

Any other interesting #fails / learning experiences? Post ’em in the comments!