Realistic 3D vision in our usual space-time

Question

Humans, animals and various devices only have 2D vision: that is, we see a 2D projection of light reflected from various 3D objects in the direction of our eyes (and processed by our brains).

I would like to describe a 3D vision, but without being fancy 5+ dimensional beings. Questions are both: how can it be realized and how would it feel to have such a vision?

I want to keep all the limitations of our usual vision: finite resolution (amount of pixels), finite field of vision (area and distance), finite time resolution, finite spectrum and spectral resolution, etc.

Of course I don't want to keep is to rely on electromagnetic spectrum alone. Better have a combination of scanning instruments, including various elementary particles.

1) One way is to have a million nanorobots (or organic analogue) with sensors floating in space and communicating with each other / some external processing device. If they are tiny enough they can even 'see' inside living tissue or some porous materials.

The processing needed to 'comprehend' such image in real time is enormous though. And also, it will not be a higher-dimensional analogue to our vision, because we won't be able to 'turn our head' and see another 3D projection of a 4D object, we'd be more like a right hand circle in this 2D space:

2) On the other hand a complete analogue the vision I want would be a usual 3D being seeing in 2D space:

In this image purple lines is what the blue ball sees: projection of a 2D space at different moments in time.

Since I don't want extra dimensional beings, let's say we are all 4D (including time) and go from there. Now it's much more complicated to realise, since 'turning our head' we'd be able to see a projection of 3D space existing at several moments of time simultaneously in the past and in the future.

I can't even begin to imagine how it would feel and how it could be processed.

---

In the end, I would like some help in fleshing out both options 1) and 2), possible ways to desribe them, and related problems and maybe some references to similar ideas?

There's a similar question here Would a 4-dimensional being be able to see inside other people? and one of the answers describes something similar to what I want https://worldbuilding.stackexchange.com/a/78789/32642, but all of it is not science based.

Answer 1

It makes no sense, allow me to deconstruct. Feel free to tell me if I've missed the point since I'm not sure if I really understood what you are asking.

All you need is 2 eyes closer to the object you are looking at than 2 other eyes. So your head would be something like a L shape. This would indeed enable you to see something unfold in time since you would get 2 images from two different moments in time. The problem here is that light is super fast and most things will not change that quickly. There is another option however:

Have 4 eyes again and receive information slower (at a rate you like) - that means with a delay - from one pair. The more pairs of eyes you have, the more moments you can capture and "see" at the same time. This indeed allows to see the state of a system change. An important question is: When would this be useful? For example when a bottle is about to fall from a table - in order to catch it you need two points in time in the case of linear movement - and you can catch it. Sounds like something you can already do? As I said, you would have to build in a delay + you would receive the same information as before again. So let's break it down even more:

This is already accomplished by humans because we story information. We remember things that just happened and are able to "see" movement, i.e., the change of a state of a object in time. All you need are two eyes and memory.

You are completely right, in a way we are 4-dimensional beings (let's not get picky here). And our species would've died out a long time ago if we wouldn't be able to see 4d. All of us would've fallen off a cliff because we couldn't have predicted what would've happened

Answer 2

First, humans do not see a 2D projection of space. Humans like many other animals have binocular vision, which means that the vision of our two eyes overlap and the brain uses the differences between the two images to get depth information on things close in-front of us.

This actually gets closer to what you probably want than it seems. In addition to piecing an image with depth information from two flat images the brain also "fills out" the image in several ways.

The trivial way is that the brain integrates previous visual data to current image. This is mostly used to remove noise and defects from the image, but it does allow the brain too detect finer detail than the eye can actually see.

More relevant is that the brain integrates what you see with existing models of objects. Again this is mostly used to reduce noise based on assumptions that sometimes turn out to be amusingly false, but it does allow you to know that a horse has four legs even when you only see three at the moment. You also have fairly good idea of what the other side of the horse looks like. I a very real sense humans already can see objects in 3D.

And yes, our brains, and brains of most animals with binocular vision, do build a model of objects over time as well. Binocular vision seems to be an adaptation to a need to make a moving part of your body accurately and reliably meet another moving object. So in a sense binocular vision counts as 4D even.

What is really falling short from what you seem to want is our ability to focus. Generally we only process the full "4D" information on the one object of interest and everything else is peripheral with some low-level automatic processing so you can react to attacks or walk on a surface without having to actually look.

Lots of text for "I don't really get your question", but it is actually kind of relevant. The way the visual system works brings up some good news and bad news.

The good news is that our visual system already can do what you want, kind of. There are lots of ways to improve it, but the basic processing ability is already there.

The bad news is that the main difference with what we already have and what you want is not in the senses or the visual system, it is in the way we think. Meaning that we can't really imagine how to build a human that could deal with the increased information flow and even if we could imagine it we would have no way to really understand how it would feel or actually work in practice. It is pretty obvious that such huge increase in information processing ability would have lots of side effects, after all.

I am not sure if this answers your question even partially, but I hope it is somehow useful.