It isn’t a priori obvious that animals (including us) would have a built-in, abstract sense of number. For example, the faculties for estimating amounts of food could easily be separate from those for estimating how many individuals are in a group of your species. If you aren’t planning how much food you’ll need at a party, how often would you need to be able to compare those? Alternatively, quantities detected by our different sense could be perceived differently. Most of the processing for (say) vision and hearing is completely different; it’s not obvious that we would notice a very high-level analogy between three apples and three beeps. In fact, for a long time the thoughts of Jean Piaget prevailed in psychology and education. Piaget held that there was no innate sense of number. Children are born as a blank slate, and they gradually come to understand the notion of distinct physical objects. From there they come to understand the idea of “how many” objects there are, and later, at the highest level of abstraction, they recognize the idea of abstract numbers that can equally well describe objects, sounds, or time periods.
Perhaps the most dramatic refutation of Piaget’s ideas came from the American psychologists Prentice Starkley, Elizabeth Spelke and Rochel Gelman. They placed a baby between two screens, one of which showed 2 objects and the other of which showed 3 objects. At the same time the baby is exposed to drum beats, in groups of either two beats or three. At first the baby will spend more time looking at the screen with three objects, as it is more complex, but after a while the baby will settle into a pattern: it will spend the majority of its time looking at the screen which matches the number of drum beats it is hearing! The experiment has been performed with babies down to a few months old, all showing the same result. The only really tenable explanation is that humans have some form of innate sense of quantity which we always use to understand the world. Curiously, the effect dies off for numbers greater than three; the innate number sense doesn’t discriminate precisely above that level.
For people interested in reading more about numerical cognition, and the research that's been done on it, I recommend the book "The Number Sense" by Stanislas Dehaene. I found his book when I was doing background research for this blog, and it's really fascinating. He hits on a lot of the same points I want to make, but our emphasis is extremely different; he's more interested in the neural underpinnings of our number sense, whereas I'm more interested in the many layers of abstractio