An analysis may be extremely precise about exactly the part of reality that happened to be easy to observe.
The drunkard searches for his key under the streetlight because there is light there, not because that is where he lost it. As a methodological metaphor, this describes how data availability and measurability determine what enters an evidence base at all.
An analysis may be extremely precise about exactly the part of reality that happened to be easy to observe.
In 1964, Abraham Kaplan used the ‘principle of the drunkard’s search’ as a warning: researchers may select problems according to where methods or data are available. The anecdote itself is older and has no reliably established single origin.
Adams and colleagues (2018) found geographical concentration in their sample of climate–conflict publications, which they linked to accessibility and data availability. This is domain-specific observational evidence, not a general law that readily available data always mislead. Their Scopus literature search mainly covered English-language journal articles.
Observability, measurement cost and data availability select evidence before inference begins. If the observable universe is unrepresentative of the target, precision may increase while coverage fails.
AI transfer to investigate: AI lowers the cost of analysing evidence that has already been digitised and may therefore concentrate attention even more strongly on what is machine-legible and retrievable.
Proposed test, not yet conducted: Compare human-only work, AI using systematically incomplete data, AI plus coverage metadata, and AI with access to one independent dark-source observation. Measure accuracy, confidence and revision.
Ask not only ‘is the evidence correct?’, but also ‘which relevant parts of reality could never have entered this evidence?’
The searchable corpus determines what this retrieval environment can show. Not found does not automatically mean absent; document the areas that were not searched.
Kaplan (1964), The Conduct of Inquiry
Kaplan, The Conduct of Inquiry, pp.11 and17 — Publisher-distributed later reprint preview
Adams et al.(2018), Sampling bias in climate–conflict research — published article
Kaplan’s metaphor and his qualification about a rational search order appear on pp. 11 and 17 of the later reprint used here. Adams and colleagues provide observational evidence about their specific sample of climate–conflict literature, not general or AI-specific validation.
Help improve this case
We review your contribution before updating the text or sources.