Community surveys

Why anybody bothered

Around 2010 the question of where a cache should be listed was genuinely open for the first time in years. Garmin had launched a free service, the Opencaching nodes were growing, and Navicache was about to change hands. Several people did the obvious thing and asked cachers what they wanted.

The surveys were small and self-selected. They circulated on forums and blogs, which means they captured the opinions of people engaged enough to answer a questionnaire about listing services — a group that skews experienced, opinionated and more likely to already use more than one service. Nothing below should be read as a measurement of the hobby. It is a record of an argument.

What came back

Three themes recurred, and the order mattered.

Quality over quantity. Asked to choose between a service with more caches and a service with better ones, respondents chose better ones by a wide margin. This was not what the competitive landscape at the time was optimising for; every new service led with how easy it was to list.

Better information before the walk. The second theme was that the existing signals about a cache were inadequate. An owner-declared difficulty rating and a terrain rating, set once and never revisited, tell you very little about whether a hide is worth an hour. Respondents wanted the opinion of people who had actually been.

Permanence. The third theme was the least discussed at the time and turned out to be the most important: cachers wanted to know a listing would still exist in five years. Nobody wants to build a series, accumulate logs and photographs, and then watch the service hosting it close.

Why the third answer turned out to matter most

At the time, permanence read as a soft concern: the kind of thing people say in a survey and ignore in practice. In hindsight it was the one that predicted outcomes.

The services that closed did not close because their software was worse. Garmin’s had the best rating system anyone had shipped. They closed because the organisations behind them decided to stop, and users had no say in that decision and no warning before it.

The services still running are the ones where the question “will this exist in five years” has an answer users can reason about: it exists because volunteers in their own community keep it existing, and those volunteers are reachable. That is a weaker guarantee than a company on paper and a stronger one in practice.

Reading them now

The surveys are worth knowing about for two reasons.

The first is that the priorities have not changed. Ask cachers the same three questions today and the answers come back in roughly the same order. The argument about find counts, the argument about cache quality and the argument about who owns the listing are the same argument in different clothes, and this site has a section on each.

The second is methodological. These were amateur surveys with obvious sampling problems, and the people running them mostly said so. That honesty is worth more than a larger, cleaner-looking dataset that hides where it came from. Where this reference cites a number, it tries to do the same.

Why nobody runs them now

Surveys of this kind largely stopped, and the reason is not apathy.

They were asking a question that was genuinely open in 2010 and is mostly closed today: where should caches be listed? In most countries the answer settled, and a survey asking people to choose between alternatives they do not use produces answers nobody can act on.

Where the question is still live (Germany and Poland in particular) the discussion continues, but inside national forums rather than as cross-community surveys. The subject moved rather than disappeared, which is worth knowing before concluding that the argument was won.

What a survey of this kind can and cannot show

These were self-selected samples, and it is worth being precise about what that does to the results rather than waving at it.

People who answer a survey about alternative listing services are people who already know those services exist and hold a view about them. That population is more engaged, more likely to have placed caches, and more likely to have a grievance than the general run of cachers. Every proportion in the results should be read as a proportion of that group, not of the hobby.

What self-selection does not damage is the open text. When several hundred engaged people are asked what would make them use a second service and their answers converge on the same handful of obstacles, that convergence is informative regardless of how the sample was drawn. It tells you what the objections are, even if it cannot tell you how widely they are held.

The distinction matters because the numbers from these surveys have been quoted since as though they described the whole community. They never did, and the people who ran them did not claim they did.

What would be worth asking now

If anybody were to run one today, the interesting questions have moved.

The original surveys asked about features, because in that era the alternatives were competing on what a website could do. That contest is over and the answer turned out not to be features.

Those are the questions the closures actually raise. The questions that would produce something now are about custody and continuity: whether cachers think a twenty-year record of logs should sit with one company, what they would want to happen to their own caches if a service closed, and whether they have ever checked that the data they have contributed can be exported at all. Those are the questions the closures in this section actually raise, and none of the surveys ever put them.