Interested in joining the Silph Research group? Learn More »

Silph Study: #051

Published:
08.20.2021

A Look Inside the Daily Free Box

Everyone loves a giveaway, and Pokémon GO players are no exception. When testing for quality-of-life features was announced in June 2020, players in locked-down countries and rural areas especially were excited by the promise of a daily free item box. One month later, this feature became available to travelers globally. Curious Silph Road Researchers noticed that, initially, the Daily Free Box contents would change every ten minutes if unclaimed. Then, in August 2020, the contents became fixed: the items would change at midnight, but only if the previous day’s box had been claimed. Thanks to our Researchers’ careful tracking, we were able to crack the secrets of this treasure box!

Key Findings

Each Daily Box contains four items which are determined randomly and independently.¹

  1. The number of each item received per box follows a binomial distribution.
  2. Item inventory likely has no influence on Daily Box contents.

Methodology

From July 2020 to June 2021, twenty Researchers collected data on 2,078 boxes. They recorded the items received in their Daily Free Box on each day where the previous box had been claimed, ensuring that the contents had been reset. This study consisted of active players over level 38.

Items Available in the Daily Box

Each Daily Box contains a random selection of four items, such as 3 Poké Balls and 1 Potion. The initial pool for Daily Boxes consisted of eight possible items: Poké Balls, Great Balls, Razz Berries, Revives, Potions, Super Potions, Hyper Potions, and Max Potions. January 2021 saw the permanent addition of Silver Pinap Berries to the Daily Box item pool. Although travelers on the Silph Road reported two periods where experimental groups had a different set of available items, no researchers involved in this study were part of these test groups.²

Item Frequency

Our Researchers observed a total of 8,310 items, with 40% recorded after the addition of Silver Pinaps to the item pool.³ The drop rates of each item following the introduction of Silver Pinaps are as follows:

Total % Total Items Item Type Total
Poké Ball 1,736 54.9% 75%
Great Ball 614 19.4%
Razz Berry 183 5.78% 5%
Pinap Berry 5 0.16%
Revive 13 0.41% 20%
Potion 595 18.8%
Super Potion 14 0.44%
Hyper Potion 1 0.03%
Max Potion 3 0.09%

Poké Balls, Great Balls, Razz Berries, and regular Potions make up the overwhelming majority of Daily Box items, with the other items combined comprising just 1% of item drops. Separating items into similar groups, we find that balls, healing items, and berries drop at a ratio of approximately 15:4:1. The balls category is further split into 75% Poké Balls and 25% Great Balls, whilst healing items consist of 95% Potions, 2% Super Potions, 2% Revives, and 1% Hyper/Max Potions.

Item Distribution within Boxes

This brings us to our next question: how are items distributed at the box level? More specifically, is each item selected independently (like what we see in item drops from Gyms), or do Daily Boxes contain balanced item contents (like what we see in the paid boxes)? If the contents of each box are chosen randomly, then we would expect the frequency of an item per box to match a binomial distribution. Conversely, a balanced box approach might try to guarantee at least one ball or potion per box and result in a skewed item distribution.

To test this, we broke down the frequency of items per box and compared them to the number we would expect under a binomial distribution. The table below shows this process for Poké Balls (using the probability of 54.9% that we observed above).

Poké Balls
per Daily Box
Number of
Daily Boxes
Expected Binomial
Distribution of Poké Balls
0 68 86
1 445 419
2 768 765
3 600 620
4 197 188
χ2 p-value 0.17

Using a Pearson’s Chi-squared (χ2) goodness-of-fit test, we found that the number of Poké Balls per Daily Box closely matched the expected frequencies under a binomial distribution of items (χ2(4) = 6.47, p-value = 0.17). We likewise evaluated the distributions of Potions, Great Balls and Razz Berries and found that they did not differ significantly from the expected binomial distribution.⁴ These calculations appear to show that items found in the daily box are independent (like PokéStop and Gym items) and do not fit in a predetermined ratio.

Can Daily Box items be manipulated?

In early November, Niantic hinted that trainers would begin seeing items that were most relevant to their Pokémon GO experience. Did that hint mean items received in Daily Free Boxes could be affected by the contents of a player’s inventory? Two Researchers cleared their item bags of any potions each day before midnight to test whether this would impact the number of potions in their boxes. One hundred and fifty-six of these “no potion” boxes were collected and compared to a control group: boxes without inventory management during the same period. We found that the proportion of potions received did not differ significantly between the two groups (χ2(1) = 1.82, p-value = 0.18). It seems unlikely that item inventory has any effect on the contents of each box, though it should be noted that only potions were looked at for this test, so we cannot generalize this to all items with certainty.

Test Group Total
Potions
Total
Items
Potion
Proportion
No Potions in Inventory 133 624 21.3%
Control Group 605 3,187 19.0%
χ2 p-value 0.18

Parting Words

The Daily Free Box offers players, particularly those in rural areas or unable to go outside, an opportunity to maintain their daily catch streak due to the high (though not 100%) chance of the box containing at least one type of Poké Ball. Future improvements to the Daily Box would be a welcome addition to a feature that offers a small respite to players lacking access to PokéStops.


Acknowledgements:
Authors: Scientist Pancake and Lead Researcher DeeDillyDawn
Analysis: Scientists CaroKann and Pancake
Editing: Scientist Cham1nade and Senior Researcher Jinian
Data collection: Twenty researchers collected data for this study. These 8 went above and beyond:

  • Brush are Best
  • DarkMighty
  • DeeDillyDawn
  • Gustavobc
  • Pancake
  • Sandercyanin
  • sleepbastian
  • TwitchToxin

Footnotes

¹ Two of the 2078 boxes contained only 3 items. It’s unclear whether this was a bug, or if there is a minuscule chance of rolling an empty slot. If the latter, the chance of an empty slot would be <0.05%.

² In December 2020, Remote Raid Passes began appearing in some Daily Boxes in what seemed to be a short-term test limited to certain trainers, with some trainers getting them frequently and many never receiving one. Niantic announced expanded testing for some trainers beginning on March 12, 2021, and reports of Incubators, Rare Candy, and Magnetic Lure Modules followed. These boxes did not have the usual four-item limit, and the test appeared to last for approximately one month before being removed with no further news being given to date.

There was early speculation that these test groups were linked to the then-active AR groups, as the bonus egg space testing had been. Each of our Researchers was asked to identify which AR group they were in, and the average numbers of the four most common items in each box by group are shown in the table below.

Average per Box Total average per Box
Group 1 Group 2 Group 3
Poké Balls 2.23 2.24 2.12 2.20
Great Balls 0.77 0.76 0.80 0.78
Razz Berries 0.24 0.23 0.25 0.24
Potions 0.71 0.73 0.80 0.75

There is no significant difference in drop rates between the three groups, and at least one researcher in each group received a Silver Pinap Berry. The average number of items received by each group was consistent before and after the dissolution of the AR groups, further indication that these groups were not used for testing purposes.

The total number of researchers and boxes collected within each group were as follows:

  • AR Group 1: Hyper Potions/Revives/Ultra Balls (5 researchers/550 boxes)
  • AR Group 2: Dratini/TMs/Silver Pinaps (6 researchers/812 boxes)
  • AR Group 3: Skarmory/Poffins/Poké Balls (7 researchers/693 boxes)
  • Unknown: 2 researchers/23 boxes

³ The low drop rate of Silver Pinaps means that the item distribution was largely unchanged from before their introduction (Jan 23). The table below breaks down the drop rates of each item before and after this date.

Before Jan 23 Percentage After Jan 23 Percentage
Poké Balls 2,833 55.05% 1,736 54.87%
Great Balls 997 19.37% 614 19.41%
Razz Berries 307 5.97% 183 5.78%
Silver Pinap Berries N/A N/A 5 0.16%
Revives 24 0.47% 13 0.41%
Potions 963 18.71% 595 18.81%
Super Potions 16 0.31% 14 0.44%
Hyper Potions 5 0.10% 1 0.03%
Max Potions 1 0.02% 3 0.09%

⁴ Using a simulated multinomial goodness-of-fit test, we compared the observed item distribution per box to the expected binomial distribution given each item’s overall drop rate. We found no significant deviation from the predicted binomial distribution of items (p-values: 0.50 (Great Balls), 0.84 (Potions), 0.97 (Razz Berries); number of simulations = 106).