GWS 2023: On reliability of data

By Jon Freitag

The 2023 edition of Wargames, Soldiers, & Strategy magazine's Great Wargaming Survey is a wrap, and the results are in. This year, there were 9,282 completed surveys. That count is down from 2022, but 9,282 respondents still represent a solid sample size. Several new questions were added into the 2023 survey. There are still questions that have yet to see any analysis, some new twists to earlier analysis, and revisiting some old questions with a look using fresh data. Many topics to explore in the coming twelve months!

Let's start  the 2023 analysis cycle with a look at a topic first examined in May of 2023. That topic is On the Reliability of Data.  Why examine this topic again so soon after the last investigation? For one, the survey counts are down from 2022. Second, the age question was asked differently. Third, I wondered if discussing the conundrum of first-time respondents would move the needle at all.  Finally, we can assess the stability of the data with one more year added into the mix.

Prior Survey Response by Year

As identified in last year's analysis, the percentage share  of respondents having taken a prior survey remained in the low 50% range. After mentioning this result, I wondered if surfacing this tendency would affect results.  Well, it is impossible to say whether mentioning this attribute had any impact upon the results but the percentage of those who say they had taken the survey before jumped to 66%.

Years Spent Wargaming (Duration)

What about the number of years spent in the  hobby? Did that remain consistent to previous years? No! The group claiming to have 31 or more years in the hobby spiked to nearly 45%. Perhaps the hobby is graying?

Respondent Location by Year

Year after year, the location of respondents seems stable. We see roughly the same mix of locations each year.

Age Group by Year

Like last year's analysis, the tendency of age cohorts to increase over time continues.  In tandem with the Duration chart above, 2023 saw a jump in the number of respondents in these older age cohorts. Is this phenomenon due solely to an aging population or could something else be driving these results?

One of my frustrations from recent years is that compliance with answering the Age question has been slipping. In 2019, Age was changed from an age range to a discrete age to accommodate the psychometric study (one day, I will dig into those data!). Age was a required field. In 2020, Age became an optional field but remained discrete. 2021 and 2022 kept that same format, but non-compliance increased to the point that in 2022, nearly 1 in 5 respondents failed to answer the age question. Wanting to reverse this trend, I suggested returning to age cohorts rather than asking for discrete age. The result?  Non-compliance almost entirely vanished.  Could these non-compliant respondents have resided primarily in the older age cohorts?

Primary Interest by Year

Finally, we revisit primary interest in wargaming. The trend seen earlier of a shift from Fantasy/Sci-Fi to both Mixed and Historicals continues. If responses are drawn from a representative sample, then the notion holds that non-historical wargamers shift away from preferring purely non-historical wargaming as they age. Is the survey catering to and drawing from fewer non-historical wargamers over time, or is the trend illustrated here accurate? Other analyses may be able to confirm one way or the other.

Even with some shifts in the data, I still maintain that the hypothesis that these data are drawn from similar populations, across time, cannot be rejected outright.

Many questions. Some answers. Your comments are encouraged!