Comment by ACCount37

Nah. That's a common "thought about it for 15 seconds but not 15 minutes" mistake.

Making a car that drives well on arbitrary roads is freakishly hard. Having to adapt every single road in the world before even a single self-driving car can use them? That's a task that makes the previous one look easy.

Learned sensor fusion policy that can compensate for partial sensor degradation, detect severe dropout, and handle both safely? Very hard. Getting the world that can't fix the low tech potholes on every other road to set up and maintain machine specific infrastructure everywhere? A nonstarter.

>Every time you pit the sheer violent force of end to end backpropagation against compartmentalization and lines drawn by humans, at a sufficient scale, backpropagation gets its win.

I fully agree, but your statement is quite ironic.

For driving, humans drive well because we operate more like Mu Zero model does - we can "visualize" the possibilities of the future states depending on what we do and pick the most optimal path. We don't need to know what the specific object is on the road, the fact that we can recognize that its there, in our path, and understand the physical interacting of a car hitting something that is taller than a bump means we can avoid it.

The way to implement self driving is exactly that - train your model to take sensor data and reconstruct a 3d space in latent dimensions, train another model to predict evolutions on the 3d space given time history with probabilistic output, and then your inference is a probabilistic guided search in that space with time constraints based on hardware. Mu Zero is nothing new, and already proved that you don't even need a hardcoded model of environment to operate in.

And you don't even need human driving data for this, as the model will be able to predict things collisions solely based on pure physics. And as a bonus, as you enhance it with things like physical models of the cars, where it can reconcile what it thinks the system is going to do versus what the physics calculations predict, you can even make it drive well in snow with low traction.

The irony of your statement is that everyone who is going end to end is manually hand coding all these hacks (like image warping in the case of Comma AI) to make the training work, all because the training data is just not sufficient, which is the exact same exercise as humans drawing lines.

And if you doubt that what Im saying is true, again, Mu Zero was proven to work. Driving is just another game where you can easily define a winning scenario, the board, and moves you can make, and apply the same concepts. The only technical part becomes accurately determining the board from sensor data.

Yes that’s why having both makes sense.

This is so dumb, I don't even know if you are serious. Nobody ever said it is lidar instead of cameras, but as additional sensor to cameras. And everybody seems to agree that that is valuable sensor-information (except Tesla).