> Ie if you saw a video of undisturbed objects only affected by gravity, you couldn't tell if the video was reversed.
How does that work with things like black holes? If you saw an apple spiral out of a black hole, wouldn't you suspect that you were watching a reversed video? Even if you take account the gravitational waves?
Comets are in elliptical orbits around the sun so literally half the time they're traveling away from the sun.
Sometimes they don’t make it:
https://en.wikipedia.org/wiki/Sungrazing_comet
https://www.space.com/8478-comet-collision-sun-captured-3.ht...
That's the question of why time only goes forwards. It seems to be that the universe started in an extremely low-entropy state. It will go towards high entropy. In a high entropy state (e.g. heat death, or a static black hole), there's no meaningful difference between going forwards or backwards in time - if you reverse all the velocities of the particles, they still just whizz around randomly (in the heat death case) or the black hole stays a black hole.
Classical gravity doesn't work like that. An apple does not spiral into a black hole in space there. It's in an elliptical orbit. (A circle is a special ellipse.)