In our first installment of the Silph Research Group’s new Mythbusters series, we turn our attention to Pokémon movesets. During the early days of Pokémon GO, Silph researchers examined several factors that potentially influenced a Pokémon’s moveset upon evolution. The study did not reveal any correlation between those factors and resulting movesets.
With the addition of new mechanics such as weather, along with new methods to obtain Pokémon, there was renewed speculation about factors that could make certain moves more likely to occur. After examining over 142,000 Pokémon and their movesets, we’re ready to call this myth BUSTED.
- Weather has no effect on a caught Pokémon’s moveset.
- Encounter methods have no effect on a caught Pokémon’s moveset.
Weather increases the likelihood of encountering certain types of wild Pokémon, such as rain increasing occurrences of wild water-type Pokémon. Some trainers hypothesized that weather might also bias movesets toward move-types associated with the corresponding weather. In order to put this myth to the test, researchers were asked to record the weather at the time of catching a Pokémon along with their moveset.
When evaluating hypotheses in statistics, it is necessary to define a “null hypothesis“. This is the default statement that says variables are unrelated or groups do not differ. In this study, the null hypothesis is that the chance of receiving a specific Fast Move or Charged Move is independent of the weather. For example, Abra should have a 50% chance of knowing either of its two possible Fast Moves, Charge Beam and Zen Headbutt, regardless of the weather. If the null hypothesis is true, then these probabilities will not change, no matter what weather condition is active.
To test this hypothesis, we used a Pearson’s chi-squared test to evaluate whether a Pokémon’s Fast Move was independent of the weather condition in which it was caught¹. We filtered the full species list to only those that were caught more than 10 times in all weather conditions (140 species). The same procedure was then repeated for Charged Moves.
The result of performing a chi-squared test (or any hypothesis test) is a “p-value”. A p-value is the probability that you would get data as or more extreme if the null hypothesis were true. Hypothesis tests are constructed such that if the null hypothesis is true, p-values will be uniformly distributed between 0 and 1. Lower p-values provide more evidence to reject the null hypothesis. Typically researchers set a threshold below which they will reject the null hypothesis before conducting the test. Common cutoffs include 0.05 and 0.01.
However, because of the large number of hypothesis tests we performed, we expect that some of the species will have a p-value below a given threshold, even if the null hypothesis is true. In this case, we expect the fraction of tests below a given threshold to roughly match that threshold (for example, 5% of all tests should have a p-value less than 0.05).
For Fast Moves, we found that 4.8% of chi-squared tests gave a p-value below 0.05, and none fell below a p-value of 0.01. For Charged Moves, 6.4% of tests gave a p-value below 0.05, and 1.4% were below a 0.01 threshold. All of these results are within the expected variability and give no evidence to reject the null hypothesis, despite the huge amount of data collected. We, therefore, conclude that weather has no influence on a Pokémon’s moveset.
In addition to recording the weather at the time of catch, researchers were also asked to record how the Pokémon was encountered (wild catch, quest reward, evolution, or raid catch).
Using the identical analysis tools we used for weather, we examined a subset of 104 species for possible influence of encounter type on movesets. We selected species that had more than 10 encounters in at least two different types of encounters. As with weather, we found that the number of species-move type pairs that fall below a threshold of 0.05 and 0.01 to be within the expected variability (5.4% and 0.5% for each threshold, respectively). This strongly suggests that different encounter methods do not have any effect on movesets.
The question “Is there anything I can do to influence a Pokémon’s moveset?” was prevalent in the early months of the game because, at the time, a Pokémon’s moves could not be changed after it evolved. Travelers risked being left with moves that did not allow the Pokémon to reach its full potential in battles. Although that fear has been alleviated now with the introduction of TMs, a subtle boost dependent on weather or encounter type may still have been helpful to travelers in their hunt for the perfect moveset. Sadly, no such boost could be found.
We hope you enjoyed this article and help to spread the word to your local communities!
Until next time, see you on the road, travelers!
¹ We restricted this analysis to the five most common weather conditions – Partly Cloudy (29.15%), Clear & Sunny (27.19%), Cloudy (24.74%), Windy (8.30%), Rainy (7.76%). The weather conditions Foggy (1.06%) and Snowy (1.00%) were not included due to limited data.