Test for your users. Not your testing devices.

Some of the most valuable signals look like noise. They come from unlikely sources, are not neatly packaged in datasets and lack explanations.

They get ignored.

Much of my insights, over 20 years as a marketer, came because i learnt to value such signals early in my career.

At a luggage company, we were testing a revolutionary new suitcase in the market. I spent a month visiting luggage shops across India. I would demonstrate the product and take feedback on the aesthetics, functionality, quality and expected pricing.

At one such shop, the owner looked at the suitcase, thumped his fists on it, lifted it to feel its weight, checked the locks repeatedly and then slammed it on shop counter.

“This won’t run. The side latch will break”, he said.

I told him that the latch was tested by opening and closing it 5000 times and it worked fine.

It will break, he repeated, with the conviction of a man grounded with strong roots.

Similar experience in a few other markets. No one could give me reasons for their views.

Back at office, I recommended delaying the launch to fix the side latch. I was hauled up in front of management. The Head of Design reminded me, “…5000 times”.

And then proceeded to open and shut the side latch repeatedly counting – 1,2,3…

I could sense my job on the line.

Just after 50, he stopped.

A small tear had formed at the edge.

The testing device opened and shut the latch in a vertical plane with no deviation. 5000 times.

But people handled the latch differently. At times they pulled to the right, at times left or front.

Test for your users. Not your testing devices.