The problem is digesting that quantity of food, not the energy content. Elite athletes typically eat some potatoes but most of what they eat is more nutrient dense.
Seriously guys, get out your scale and weigh 13 pounds of potatoes. Could you really consume that much volume in a day without feeling sick? Let's do a reality check here.
I had a HS friend who was a serious swimmer (not quite Olympic level but he won state championships) and watching him eat was insane. He would eat about 3x of what we all ate. Like literally down 3 sandwiches while we had one. I think he was on a 6000 calorie a day diet. I believe the potato thing. It sure sits outside what I think I could eat, but having seen others do similar, it seems realistic.
I do weightlifting and have gone through times where I do more weightlifting and less. The body just absorbs food when you do more work. Can't tell you the mechanics but it's way easier for me to digest more food when I do more weightlifting.