Comment by zdragnar
Tea also has noticeably less caffeine than coffee, yet it has been treated as an energizing drink since it was first discovered.
When you don't have any adrenal stimulants in your diet at all, even a small amount is noticable.
Caffeine also has a metabolic half life of roughly 5 hours in the body, if I remember correctly. A few berries might not do much, but surely a handful will be enough.
Tea bushes use an entirely different metabolic pathway to produce caffeine than coffee bushes.
Here is an article that further describes how we believe caffeine synthesis evolved in multiple land plant lineages.
https://www.pnas.org/content/113/38/10613
Caffeine synthesis involves several enzymes, but the enzyme family (called the SABATH family) involved in the final stages of its synthesis can trace its origins back to the first land plants. These first enzymes are thought to have been very promiscuous (capable of having activity with several molecules), partially contributing to how caffeine synthesis managed to evolve independently multiple times throughout the evolutionary history of land plants.