In the late 1920s and early 1930s, at the Western Electric company's Hawthorne Works plant, a series of experiments was conducted that became among the most famous in the history of industrial psychology — and among the most misunderstood. Researchers altered the working conditions of relay assembly workers: they intensified the lighting, then dimmed it; they introduced additional breaks, then took them away again. Expectedly, productivity rose when conditions improved. Unexpectedly — it rose with almost any change, including worsening conditions and a return to the original ones. Attention to the people itself, rather than any specific change, turned out to be the main factor.
Later, the methodology of these experiments was repeatedly criticized, and the precise interpretation of the original data is still debated by researchers. However, the general term "Hawthorne effect" took root precisely in this sense: people change their behavior simply because they know they are being observed — and this effect is capable of completely masking the real outcome of an intervention.
For HR, this is a methodological trap into which almost all pilot programs fall. A new motivation system is launched, a new onboarding format, a new assessment tool — and the metrics improve. The temptation arises to conclude that it was the new tool that worked. But part of the effect — and sometimes the entire effect altogether — can be explained by the simple fact: employees know they are participating in a pilot, feel increased attention from management toward their group, and this is precisely what temporarily changes their behavior — regardless of the content of the innovation itself.
The problem is that this effect almost always fades away along with the end of the pilot group's "special" status. The company implements the tool across the entire organization, the effect of attention vanishes along with the novelty — and the results that seemed to be a sustainable improvement roll back to the baseline level.
What helps:
using a control group that also receives increased attention but does not receive the innovation itself — this separates the observation effect from the content effect;
measuring the result not immediately, but after a prolonged period once the novelty of the pilot has worn off;
treating the results of any HR pilot with an adjustment for the fact that part of the improvement is not the tool, but the mere fact that people were looked at more attentively than usual.