Reflecting on Pokémon GO Shiny Rates III: The Base Rate

In the first two parts of this series, we reviewed the Silph Research Group’s accumulated data and identified several key shiny rates. Below is a summary of those rates, along with their respective 95% confidence intervals (CIs):

  • Raid Day rate, used for Niantic’s three-hour special raid events, as well as some limited research events [CI of 1 in 10.9 to 1 in 10.2]
  • Legendary Raid rate [CI of 1 in 20.2 to 1 in 18.6]
  • Community Day rate [CI of 1 in 25.1 to 1 in 24.4]
  • “Permaboost” rate, used for at least a dozen Pokémon that can be found in the wild and whose shiny rates appear to be permanently much higher than the usual base rate [CI of 1 in 65 to 1 in 56]
  • “Medium Event” rate, often used for Pokémon with significantly boosted spawns during events [CI of 1 in 141 to 1 in 126]

Conspicuously absent from our previous overviews of event and other boosted rates was any discussion of the base rate. When the Silph Research Group estimated a base rate of 1 in 450 in 2018, we also speculated that this rate was biased compared to the true rate. We have never felt comfortable using our data to draw conclusions about the exact base rate for a simple reason: the base rate is small enough to be highly susceptible to errors in data.

In this article, we present a dataset that provides evidence for a base shiny rate near 1 in 500 or 1 in 512.


Systematic bias in shiny data collection

There are various reasons to expect systematic errors that increase the observed shiny rate above the true rate. The most pernicious form of error in a study like this one is reporting bias. Researchers are more attentive to reporting series of encounters in which a shiny was found than reporting data without a shiny. Although our Scientists issue frequent reminders about appropriate steps to take to minimize reporting bias, we cannot be sure that we have eliminated it. Another serious issue is that data from a boosted period might inadvertently be classified as data from a non-boosted period. For example, a Researcher might report data from an event after the event’s end and neglect to correctly backdate their submission.

To see why this sort of systematic error makes it especially hard to measure the base rate, let’s examine some hypothetical numbers. Suppose that our data contains one erroneous extra shiny for every 5,000 reported sightings. At a “true” rate of 1 in 500, a set of 5,000 sightings would contain on average 10 correct shiny reports and 1 extra shiny. This error causes almost a 10% increase in our estimated shiny rate! However, if the “true” rate were 1 in 100, this erroneous shiny would only cause an average error of 2%. Our best guess is that this is roughly the right order of magnitude for the error rate. The error rate is probably not as high as 1 per 1,000 sightings, but is likely greater than 1 per 10,000 sightings.

Minimizing Bias

To combat these potential sources of error, our project design team went back to the drawing board. In order to reduce reporting bias, the updated design requires researchers to declare which species they will track and for how long (up to a maximum of 7 days at a time). A reminder system also prevents researchers from neglecting to submit data in which they did not find a shiny. These steps make it significantly easier for researchers to accurately collect and report their data.

To illustrate, below is a comparison of data from the old and new project designs. Date-reporting errors have been reduced by considering only species that, as far as we are aware, have never had a boosted shiny rate in any event: Delibird, Ekans, Electrike, Koffing, Lunatone, Makuhita, both forms of Rattata, Sentret, Spoink, and Zubat.1

Category Seen Shiny Rate 95% CI low 95% CI high
Original Project Design 203,281 486 1/418 1/459 1/383
New Project Design 100,492 198 1/508 1/583 1/442

While the newer project is still collecting data, large differences between the two datasets have already started to emerge, providing further support for a base shiny rate near 1 in 500 or 1 in 512.

Despite the large uncertainty in the base shiny rate data, examining all of the shiny rates together can help form a clearer picture. In the final section of this article, we’ll speculate about exact values for the base rate as well as for the various other rate categories that we have identified.

Speculation on exact shiny rates

It is reasonable to hypothesize that the boosted shiny rates are, at least in some cases, exact multiples of the base rate. For example, the Permaboost rate appears to be roughly double the Medium Event rate, and the Medium Event rate is almost exactly four times the base rate. Multiplying the confidence interval bounds for these two shiny rates by their suspected multipliers suggests a base rate of at least 1 in 560 but no more than 1 in 450.

Two logical denominators within this range are 500 or 512. The numbers 500 and 512 are the only numbers between 450 and 560 whose reciprocal is a terminating (rather than repeating) decimal. A whole number has a reciprocal that is a terminating decimal if its only prime factors are 2 and 5. Many travelers on the Road have pointed out that a 1 in 512 rate is potentially more computationally efficient than a 1 in 500 rate, since a 1 in 512 chance can be evaluated by looking at the state of exactly 9 bits. However, some of our rate categories fit slightly better with a base of 1 in 500 than with a base of 1 in 512.

Under either the 1 in 512 or the 1 in 500 base rate hypothesis, with other rates determined by a multiplier, here is what the other rates appear likely to be:

Category Multiplier Rate with 1 in 512 base Rate with 1 in 500 base
Base Rate 1 1/512 1/500
Medium Event 4 4/512 = 1/128 4/500 = 1/125
Permaboost 8 8/512 = 1/64 8/500 = 1/62.5
Raid/Egg (if different from Permaboost)² 9 or 10 9/512 ≈ 1/56.9 or 10/512 = 1/51.2 9/500 ≈ 1/55.6 or 10/500 = 1/50
Community Day 20 20/512 = 1/25.6 20/500 = 1/25
Legendary Raid 25 25/512 = 1/20.48 25/500 = 1/20
Raid Event 50 50/512 = 1/10.24 50/500 = 1/10


Neither hypothesis is perfect. It seems that the 1 in 512 base rate hypothesis fits somewhat better for the Medium Event, Permaboost and Raid Event data, but the 1 in 500 base rate fits better for Legendary Raids. We also can’t rule out some combination of the two hypotheses: computational efficiency in determining shiny status seems to be more critical for wild encounters than for less-frequent events such as Raids and Eggs.

Parting Words

Shiny Pokémon continue to be a focal point for many collectors on the Road. Their value is inherent due to their rarity, but this rarity also makes the task of determining their rates difficult. We are optimistic, however, that this research continues to get us closer to understanding the intricacies of the shiny mechanics. We hope that these discoveries are useful to you while you are out hunting these elusive variants. That said, stay safe, and keep hunting those discolored monsters!


Acknowledgements

 – Lead Researcher Paleshadow was the primary author of all three articles in this series.
 – Scientists PiFlavour, Titleist, IvoryTinkler, Gluglumaster, and Lead Researcher Paleshadow worked on various pieces of the analysis.
 – Scientists JaxsFC, MothballMinter, PiFlavour, Titleist, and PhoenixCrystal worked on the design of the new shiny tracking project.
 – Scientists Stealerhat, PhoenixCrystal and JaxsFC led the new and original shiny tracking projects at various times.
 – Artwork created by Hectordraw
 – Article editors were Scientists Cham1nade, Skyeofthetyger, Titleist, and Lead Researcher Blondey
 – Thousands of researchers from all corners of the world have collected data on shiny Pokémon over the years. Want to become one? Join our Discord.


Footnotes

1 These data were collected before the April 2020 Incense Day in which Sentret’s shiny rate was boosted for the duration of the event.

² We should reiterate our doubts about a shiny rate for Raid/Egg-exclusive Pokémon different from the shiny rate for Permaboosted species. It is counterintuitive that two distinct rates were chosen while being so similar. Although it would be surprising to us if our Raid/Egg-exclusive data contained significant bias not present in our Permaboost data, we are suspicious of the possibility that the Raid/Egg-exclusive data contains a mixture of shiny rates — some at the 1 in 64 Permaboost rate, and some at a higher (i.e. more likely to be shiny) rate. One can imagine a variety of reasons this might be the case, including the possibility that the Raid/Egg-exclusive rate was changed at some point, but our data are too limited to make strong conclusions.