Humans with 360 Degree Vision across IR, Visible, UV and XRay?

Question

Wearable and implantable technologies are increasingly blurring the line between natural and augmented reality. Currently, our commercially available computer-aided vision systems for the blind are barely capable of providing a blurry, black-and-white low res rendering at relatively high cost (~$250k). However, progress in the field is starting to look increasingly exponential, which brings to my mind questions about the potential use of vision systems for healthy humans.

Unaided human vision is limited to about 120 degrees per eye, most of it peripheral, with only a narrow 6 degrees of high-resolution vision (the macula). Perhaps we would be able to use an array of cameras (placed on a helmet or a band, say), connect them to a specially designed computer processor and then feed them into a brain implant that would provide a feed of the information to our visual centers.

Is it possible to create an artificial vision system with high-res depth-of-field-capabilities, 10-100x on-thought zoom (the machine will zoom as you think you want to zoom), blending several perceptual modes (passive visual, EM, IR, UV, and why not, active X-ray) and spanning the full 360 degrees something a man-machine system could send to the visual centers and have a user experience consciously?

Note that I'll judge answers on whether the feasibility of such a device is addressed, as well as the two criteria below:

Usability: Would a human be able to train herself to experience this, or would we have to strap them onto newborns or genetically engineer ourselves for it?

Usefulness: Would something like this be useful for anyone besides guards and ornithologists?

Answer 1

The simple fact of the matter is that the human brain is simply not evolved to handle that great an amount of data.

Without processing assistance, it would be necessary for the user to cycle through various input modes in order to gain the maximum benefit from the sensors. It would require quite a lot of training to gain the maximum benefit of such a system.

However, that is not the only way to implement this system. Instead of feeding the input to the visual cortex, the data could be processed in the hardware, and the neural implant could feed the results deeper into the brain, so that when connected, the brain would perceive this as data from a perfectly normal additional set of sensory organs, and would not require the brain itself to process or interpret the input, only to base decisions upon it.

It is difficult to imagine, but this style of input would overlay itself on our field of vision like extra eyes; we could close one (natural) eye or the other, or the artificial one. In addition, the field of vision from the artificial eye(s) would be 360° and vastly more detailed, and it wouldn't seem odd that it was so.

However, if the processing hardware was removed, it wouldn't be like simply losing an eye (in which case you'd just see nothing in that part of your field of view; i.e. blackness), but like losing even the part of the brain that processed that eye's data, so you'd remember having a larger field of view, but it simply wouldn't be there any more, not even blackness. Consider that the edges of a human's natural vision isn't black all the way around the back of the head to the other side, it just isn't there at all.

However a system such as this would be considerably more advanced than a simple feed to the visual cortex, and would probably take in excess of a hundred years additional R&D to perfect.

Answer 2

The limited spectrum of optical light that our minds and eyes have evolved to "see" and interpret already includes a lot of information that we ignore. For example, although we may have "120 degrees per eye", most of that is not in focus - i.e., our mind is already pruning information in order to avoid an information overload.

Currently, a full spectrum of electromagnetic radiation can enter the eye, but our mind only interprets the narrow optical spectrum in order to avoid an even more overwhelming information overload.

The scenario you describe would seem perfectly feasible if we are willing to sacrifice a lot of the new information (and some existing information) present in the light in order to avoid an information overload.

Another question to ask is how you would represent this new information. You cannot use our current colour sense, since that would mean "overwriting" existing optical information. This is a very problematic aspect of your scenario.

Answer 3

Additional or alternate senses have been researched for quite some time. The primary method of input is to "hijack" an existing sense to pass data from sensors to the brain. It takes training and exposure for the data to be processed usefully.

After wearing a belt with 13 phone vibrating units spaced around his waist, with the one nearest to north constantly vibrating, Udo Wachter states:

I suddenly realized that my perception had shifted. I had some kind of internal map of the city in my head. I could always find my way home. Eventually, I felt I couldn't get lost, even in a completely new place.

In the mid 20th century, Austrian researcher Ivo Kohler gave people goggles that flipped the visual image.

After several weeks, subjects adjusted - their vision was still tweaked, but their brains were processing the images so they'd appear normal. In fact, when people took the glasses off at the end of the trial, everything seemed to move and distort in the opposite way.

An electrode studded mouthpiece has been used to pass rudimentary visual data as well as accelerometer data to compensate for dizziness from an inner ear infection.

A SOES (Spatial Orientation Enhancement System) can be used by pilots to let them feel the orientation of the plane rather than relying on potentially poor visual cues.

A potential downside of this is that you are literally training (rewiring) your brain to accept these altered inputs. Udo Wachter said that he felt lost after he stopped wearing the belt. He bought a GPS unit and would obsessively glance at it.

The current incarnations of sensory prosthetics are bulky and low-resolution - largely impractical. What the researchers working on this technology are looking for is something transparent, something that users can (safely) forget they're wearing. But sensor technology isn't the main problem. The trick will be to finally understand more about how the brain processes the information, even while seeing the world with many different eyes.

Wired article : Mixed Feelings (April 2007)

As a separate example, I already do possess an extended viewing range while driving my car. My mirrors allow me to view my surroundings by minimally moving my head and gaze. I do not have constant focus on everything, but by carefully shifting my focus, I can keep a remarkably detailed picture of the traffic around my car without having to twist to look beside and behind me.

Answer 4

Usability

Edit: After some more reading on how the brain develops it's visual cortex during infancy and childhood, giving this ability to an adult just won't work. Too much about how humans see becomes hardwired at an early age. Perhaps this device could allow an adult to get an intuitive feel for IR, UV, etc, put you likely couldn't see in those bands.

Original: Given the ability of the human brain to learn and adapt at most any age, developing a device like this and linking it to a brain implant is a feasible strategy, though the younger, the better. We already have people learning to use complicated structures such as the human arm through physical therapy following a traumatic injury. People relearn to walk. The brain and nervous system can be trained for quite a few things. True, neuroplasticity changes with age and there's probably an age beyond which it's very difficult or impossible to learn to see in such broad reaching capability.

Usefulness

Being able to see beyond just visible in near IR through near UV would have huge implications to practically every profession. Geologists could more quickly differentiate rock types based on the reflective spectra. Mothers can tell if they've properly applied sunscreen to their children by looking at them in UV. Sports coaches can tell if their players are overheating by looking at them in IR.

More zoom is always good and the list of areas where higher zoom levels would benefit someone are beyond listing. Anyone who has ever squinted in order to see a little bit farther would appreciate this capability.

Feasibility

You probably don't want to use anything longer UV light as anything in the shorter UV range and beyond is relatively new and bad things happen when humans are exposed to that kind of radiation. Think, radiation burns, radiation poisoning, cancer, the list goes on and on.

Also, the aperture required to see in a different part of the EM spectrum varies directly to the size of the wave you want to look at. Note that radio telescopes are giant dishes, completely unlike visible telescopes. You need some very specialized equipment to see in the X-Ray bands. Restricting the apparatus to IR, visible and near UV should be plenty.

Zooming by a factor of 100x will require either a sacrifice in resolution (achieved by a digital zoom) or carrying a bulky optical setup on your forehead. Also, 100x zoom in a SLR is relatively new. 50x optical zoom cameras are available as of this writing. The laws of physics are brutal with regards to lense size and aperture size to achieve a given resolution/zoom level.

Seeing magnetic fields just isn't feasible as the only way to see a field is to put something into that field.

Answer 5

Is it possible to create an artificial vision system with high-res depth-of-field-capabilities, 10-100x on-thought zoom (the machine will zoom as you think you want to zoom), blending several perceptual modes (passive visual, EM, IR, UV, and why not, active X-ray) and spanning the full 360 degrees something a man-machine system could send to the visual centers and have a user experience consciously?

No

Wavelength

In order for photons to be detected, they must be absorbed. Furthermore, they must be capable of triggering a chemical reaction or electron cascade detectable by biological or semiconductor instrumentation.

Too long

One problem with long wavelength photons, is that they can only be absorbed by objects whose minimum dimensions approximate dimensions of that wavelength. Putting this another way, in order to detect $1 cm$ wavelength radio, you need an antenna approximately $1 cm$ in size.

This size restriction puts severe constraints on the resolution possible with many radio wavelengths. A human eye's dimensions might be capable of detecting 1 pixel of information for this wavelength and could not detect any radio waves longer than this.

Too Short

It would seem then that we should be able to get very high resolution images from very short wavelength photons and, in theory, this is true. However, very short wavelength photons possess very high energy levels. This makes them difficult to focus (by reflection or refraction) and becomes nearly impossible at very short wavelengths.

What is worse, photons must be completely absorbed or they can't be absorbed at all. What this means is that electrons in the atoms of our eyes, simply can't absorb this much energy and the photon passes right by them. These very short frequency ($\gamma$ rays) react mostly or only with the nucleus of the atom. When the do so, they tend to deliver enough energy to change the composition or structure of the nucleus and break bits off.

In layman's terms, they cause the nucleus to fission. If the nucleus is more massive than iron (atomic mass of 56) the fission event releases even more energy.

Regardless of whether the $\gamma$ ray absorption is endothermic or exothermic, it causes your material to transmute all nearby elements into other elements.

This could be a problem for you when it happens in your skull.

Other Issues

Some of what you propose is difficult or impossible to accomplish for other reasons.

These include:
1) The amount of information a given physical instrument can glean from observing this remotely is limited based, in part, upon the physical dimensions of the instrument. Electronic processing and other techniques frequently misused on TV (like CSI), can NOT add any information to what was actually collected. So a 100x zoom just would not be possible at some wavelengths.
2) When a biological organism evolves an ability, often it pays a penalty - even if that penalty is limited to adapting cells that used to do one job to do another. Human eyes are limited to the frequencies they can see because on Earth, these frequencies provide humans with the best sensory input. Biological organisms pay too high a penalty for the benefits gained when other frequencies are detected. It Costs Too Much

We can side-step the cost penalty to biological organisms by (as you say) developing implant technologies to make up the short comings of our current systems. I suppose this is true.

I also bet we wouldn't see much benefit by detecting other frequencies for the vast majority of people.