Comment by cjs_ac
The main 'Bletchley Park' exhibition is good, but it focuses on the human experience of the code breakers. Head around the corner from the car park to the National Museum of Computing (also on the Bletchley Park site) to see more technical exhibitions: they give proper demonstrations of the machines invented at Bletchley, as well as the oldest working computer in the world (which was computing prime numbers when I visited).
Also, and not obvious, because these two entities are distinct despite occupying the same site: they're not always open at the same time. So if you want to see both, even if you plan to spend more time at one than the other, check they're both open.
Whether something is the first computer is - inevitably - a definitional argument, but TNMOC has several candidates (though not all of them) including (a modern reproduction of, the original was destroyed as a secret) Colossus which is famous because of its involvement in the war.
Bletchley Park is also still an actual stateley home, all the war stuff was built on somebody's grounds - there's a good chance you either don't care about stately homes or you're intending to visit a more interesting one (or indeed one of the Royal Palaces), in which case no need to care, but that's a third distinct thing on the same site.
[Edited to make clear there is no original Colossus, we destroyed it because it was a secret]