Interested in joining the Silph Research group? Learn More »
Silph Study: #003
The Curveball Bonus in Pokemon GO
10 XP bonus for landing a
Curveball has led some travelers to adopt the throwing style. But the question has long remained: does throwing
Curveballs increase my odds of capturing a wild encounter?
Anecdotal reports have been conflicting, with many travelers swearing by the throw method and others claiming capture rates are only affected by other factors. Statistically meaningful results had not previously been gathered to confirm.
Silph researchers have gathered several thousand capture experiences in a controlled study. After statistical analysis, we are finally able to confirm:
There IS AN INCREASE in catch rate using
Both Nice and Great throws achieved statistical significance, while Excellent lacked sufficient samples (though the trend has been included for reference). Even though samples were not controlled for ring size, No Bonus curved throws still showed a significant capture rate increase.
Variables Controlled For
- Only samples collected before the recent
Type Catch Bonusmechanic launched were studied
- This study was only conducted on
- This study excluded Pokemon caught via
- Only Pokemon whose
base catch rateis .4 were included (e.g. Rattata, Pidgey, etc)
- We used
% of Max CPper species as a basic way to estimate Pokemon level (See Note 2)
The Null Hypothesis Vs. Results
The null hypothesis for this experiment is that the μ1 = μ2 for the sets Straight and Curve, that is, throwing a curve ball does not affect the catch rate.
The approach taken to determine significance was to use a Chi Squared Test for each throw bonus bin.
Chi Squared Results Table:
- p-value = A measure of how unlikely a measurement is. (If <.05, considered statistically significant)
- χ2 = (Chi Squared) A calculated value that is compared against a χ2 distribution (if the χ2 critical value > 3.84, considered statistically significant)
- Frequency = Number of samples in bin
From these results, we found statistically significant increases in the catch rate of
curveballs with a confidence level of 95% in the No Bonus*, Nice and Great categories. The sample size of the Excellent bin was too small to find a statistically significant effect. The data that was collected for Excellent has been calculated for completeness.
Based on our results, we can reject the null hypothesis in favor of the alternative. There is a statistically significant difference in catch rate using a
Throughout this analysis, you will see that we have continued to separate our data by "Throw Bonus" in order to ensure that the ring size variable is controlled. No Bonus bins have been included throughout the document for completeness, despite it being potentially biased due to the ring size variable being unconstrained.
A common question regarding this analysis is whether
Pokemon Level impacted the
throw bonus catch rate percentage.
Pokemon Level does impact catch rate. But fortunately, each throw bonus sample group had a roughly equal distribution of Pokemon levels (roughly approximated by % of Max CP).
The frequency distributions of each throw type bonus can be seen in the following bar charts. We had few samples of 62-72% and 72-80% Pokemon in this dataset due to only our higher-level Researchers having access to them, so their section of each bar is small. All of the other sections, however, are quite evenly distributed.
STRAIGHT THROWS BY % OF MAX CP
CURVED THROWS BY % OF MAX CP
After publication the Silph Research group discovered a small subset of data that may have been logged incorrectly. Fortunately, even with the affected subset removed, statistical significance is still achieved rejecting the null hypothesis.
This finding was shared on our subreddit on Oct. 18, 2016.
Contributions to this project were made by many Silph Researchers, but we want to highlight the Silph Scientists who helped design, manage, and analyze the projects that make these findings possible: