Comment by iNic

Comment by iNic 2 days ago

12 replies

View on Hacker News

The kind of intuition you gain for higher dimension tends not to be visual. It is more that you learn a bunch of tools and these in turn build intuition. For example high dimensional spheres are "pointy" and most of their volume are near their surface. These ideas can be defined rigorously and are important and useful. For medium dimension there are usually specific facts that you exploit. In my own work stuff like "How often do you expect random walks to intersect" is very important (and dependent on dimension).

david-gpu 2 days ago

> For example high dimensional spheres are "pointy" and most of their volume are near their surface

I had a visceral reaction to this. In what sense can a sphere be considered pointy? Almost by definition, it is the volume that minimizes surface area, in any number of dimensions.

I can see how in higher dimensions e.g. a hypersphere has much lower volume than a hypercube. But that's not because the hypersphere became pointy, it's because the corners of the hypercube are increasingly more voluminous relative to the volume of the hypersphere, right?

Reply View 7 replies
  • iNic 2 days ago

    There is a standard thought experiment where you start with a hypercube of side-length 2, centered at the origin. You then place a radius 1 sphere on each vertex of this hypercube. The question then becomes: what is the largest sphere you can place at the origin so that it is "contained" by the other spheres. As it turns out in like dimension 6 or so the radius of the center sphere exceeds 1. It will actually poke out arbitrarily far (while still being restricted by the corner spheres).

    Reply View 4 replies
    • carltg_ 2 days ago

      I hear this point parroted all of the time, but I think it is a misunderstanding and a poor visualization. Consider the same situation, but instead of focusing on the radius of the center sphere, focus on the distance between the spheres on the corners to the origin. For 1-dimension, these 'spheres' are unit intervals and so the distance is 1 (Central radius is 0). For 2-dimensions, these are circles at a distance of root(3) (Central radius is root(2)-1). 3-D: root(3) (Central radius is root(3)-1). Etc. So, it isn't the central circle getting more 'pointy' allowing the central radius to increase, but rather that the corner circles are getting further from the origin, allowing larger N-spheres (increasing proportional to the root of N). Thus, pointy is not the right way to conceptualize these spheres. For the more visual folk, I would recommend drawing this out and you can see this in action. More clearly, if a sphere became 'spikey' then the distance on the surface of the spike should be further than a neighboring point, which is NOT the case. Not trying to attack you, I just see this same point over and over and think that this warrants more thought

      Reply View 1 reply
      • iNic a day ago

        Yes that is true, but there are other ways to see spiky-like behavior.

        First, the volume of spheres (or balls rather) in higher dimensions goes to zero as the dimension grows. Said another way, to keep unit volume on a ball you need to grow the radius more and more (which I interpret as spiky).

        Second, the volume of spherical caps grows like ~exp(- d h^2 /2), in particular the caps lose volume fast in higher dimensions. To interpret this as "spikyness" I like to visualize it as two balls intersecting (which is just 2x the cap volume). If they are of the same radius, but their centers are just slightly off their intersection volume goes to zero quickly!

        Reply View 0 replies
    • Sharlin 2 days ago

      Yes, but that can be better understood as the hypercube becoming more pointy, not the sphere. And it's true; the cube's vertices get arbitrarily far from the origin, while the centers of its faces stay at ±1.

      There are other ways in which a hypersphere can be considered "pointy", though; for example, consider a point lying on the surface being moved some epsilon distance to a random direction. As the dimension increases, the probability that the point ends up inside the sphere approaches zero – the sphere spans a smaller and smaller fraction of the "sky".

      Reply View 1 reply
      • senderista 2 days ago

        Specifically, of course, d = sqrt(N), where N is dimension and d is distance of a vertex of the unit hypercube from the origin.

        Reply View 0 replies
  • [removed] 2 days ago
    [deleted]
    Reply View 0 replies
jochi427 2 days ago

I remember learning about the probability of returning to the origin in a 2D random walk versus a 3D random walk when I took stochastic processes. After we proved with probability 1 you return to the origin in a 2D walk (and with probability 0 you return in 3D) my professor said "that's why you hand a drunk man the keys to a car and not an airplane when he leaves the bar". After checking wikipedia it looks like he riffed off this quote from Shizuo Kakutani: "A drunk man will find his way home, but a drunk bird may get lost forever".

Reply View 3 replies
  • CamperBob2 2 days ago

    That's interesting, about the probability being zero in 3D. Is this on an integer lattice? The source that cannot be cited on HN without loss of karma says that the probability of returning to the origin in Z^3 is approximately 0.34.

    I don't see how it could possibly be zero, even for reals, unless you're relying on the idea that the probability of any given real emerging from a uniform RNG is zero. That would seem to apply in 2D as well.

    Reply View 2 replies
    • penteract 2 days ago

      Here's how to formulate the question in continuous space/time:

      Random walks can be defined on continuous space and time as a probability distribution on functions R -> R^n (Brownian motion in n dimensions).

      We can then ask whether Brownian motion beginning at the origin will ever revisit it i.e.

      Given 2D Brownian motion X such that X(0)=(0,0), the probability that there exists t>0 such that X(t)=(0,0) is 1.

      Given 3D Brownian motion X such that X(0)=(0,0,0), the probability that there exists t>0 such that X(t)=(0,0,0) is 0. (This is more clearly true when it doesn't begin at the origin, but it's almost certainly not at the origin at t=1, and you can divide the half open interval (0,1] into a countable number of intervals, each of which have 0 probability of passing through the origin.)

      Random walks in 2D are space filling curves; random walks in 3D are not.

      Reply View 0 replies
    • jochi427 2 days ago

      I'm sure I am just misremembering -- it was definitely on Z^3 so I guess its actually 34%. Thanks for letting me know

      Reply View 0 replies