Any publications on the Fermi paradox and GAI?

Question

Is there any reason to think that GAI, in every conceivable form and with any possible safeguard, will annihilate any civilization that builds it, thereby solving the Fermi paradox (assuming that every civilization must build it for space travel)?

I can't see why, unless GAI is sentient, and sentience intrinsically, with enough computing power, annihilates everything as soon as it emerges. While this may solve the Fermi paradox, I personally believe GAI won't be sentient anyway, though I guess we'll see (or not).

I don't like the idea that flawed but powerful processing power alone is enough to kill everything, as everything contains bugs. (Life contains unhelpful mutations but still exists, is increasingly complex, etc. My physics is also bad, but if all systems contain random noise, then complexity alone should not suffice?)

I'm interested not in whether this solves the Fermi paradox (which would be scary) but if the Fermi paradox can seriously add anything to 'the dangers of AI'.

Whether or not the fermi paradox has any relevance to the dangers of GAI should depend on how likely any civilization (including ours) will be able to contact another after developing GAI.

The Drake Equation is part of why there are official searches for alien life... 12,500 intelligent alien civilizations which may currently exist. This is just a guess [for the milky way].

https://askanearthspacescientist.asu.edu/drake-equation

Supposing p=1 (which it isn't), then the Fermi paradox would imply that the chances of annihilation from GAI are large¹, though I am not proficient enough at statistics to estimate it.

1. We can be 95% sure it is over 1/3000.

Answer 1

Less wrong says

"this disaster can’t be an unfriendly super-AI, because that should be visible"... an UFAI should also be counted as an 'expanding lasting life'

Why would UFAI be any more visible than an alien civilization? Given that ET's have AI it seems unlikely that the visibility of UFAI is greater than ET's.

What shifting the paradox from ET's to UFAI's makes for could be called the "less alien paradox": why haven't we seen UFAIs? To say it is more problematic than the fermi paradox seem wrong, especially supposing that ET's, unlike UFAI's, are not necessarily unfriendly. So it may help resolve the fermi paradox.

Indeed, if we annihilate ourselves then it seems fair to assume - at this point - that AI has contributed to the filter (socially or UFAI) for other civilizations to find us. Whereas to claim that our civilization will not be annihilated suggests that there are ETs.

This is just a fun way of saying that existential risks definitely seem relevant to whether or not we will make contact with (and it seems fair to think that you can contact - in some sense anyway - anything we would call "life" if not "intelligence") ET's and resolve the paradox etc..

Answer 2

We already see current expert systems "solving" problems by exploiting "glitches" in the simulation rather than finding the best solution in the scenario that we've prepared for them. So it's not really about the dangers that we foresee it's about the dangers that we don't foresee.

You could also have discussions about what if it turns conscious and develops concepts of pain and death and how to avoid them... And yes we might use those in regards to "reinforcement learning" though we don't expect them to be "real", but what do we know...

Also an OFF switch might be insufficient. Like hook that up to the internet and it might store it's "brain state" somewhere on a server in you-don't-know-where or encrypt it on it's own hardware, obfuscated in regular data and the moment you boot it up again, the same is back on or worse.

The thing is on the one hand we want these things to be all-powerful to be useful to use on the other hand we are afraid of what that would look like because if it ever would have gotten to the point we are likely not even able to think of all the ways it could avoid us trying to stop it.

Though why would you build a GAI in the first place other for a proof of concept?

Answer 3

I suggest the terrifying danger you foresee might be avoided by taking the precaution of wiring an 'OFF' switch into the evil bot's power supply.

Answer 4

I'm not convinced this is a useful line of inquiry. Neither the Fermi Paradox nor the risk of a hyperintelligent unaligned AI are universally accepted as real problems in their respective fields:

• For the Fermi Paradox, it's fairly obvious that most of the terms in the Drake equation are completely unknown, and probably unknowable. There are many regions of the Drake configuration space in which we should not expect to observe aliens, and which are also compatible with our existing models of cosmology. Based on current observation, it would seem that we are in one of those regions - but that's not a "paradox," it's just an observation.
• For AI, there are many different problems. Here are a few:
  • There is no mathematically precise definition of intelligence which extends far beyond the capabilities of humans. Intelligence is normally defined based on a clustering analysis of intelligence test results. This analysis shows that many different tests will tend to correlate with one another, suggesting that there is something to be measured (and the tests are not just axiomatically "valid"), but this is an operational definition. It becomes meaningless when we start to talk about anything significantly smarter (or dumber) than the humans who originally took the test. This is not to say that hyperintelligence is definitionally impossible, merely that it cannot be formalized and precisely reasoned about given the currently available research. But that's already a serious problem for a significant cohort of the AI safety movement, because they want to mathematically prove that a hypothetical "friendly" AI is not a danger to us, and a formalization of hyperintelligence would be, at the very least, rather useful for that purpose.
    • This idea also runs into various metaethical problems, but let's not get too far into the weeds here.
  • Some AI safety folks seem to believe that intelligence (whatever it is) is positively correlated with the ability to build more intelligent AIs, leading to a positive feedback loop and runaway progress once some threshold is crossed. The problem with this is that, since intelligence is not formalized, this is at best an intuition, and at worst an outright guess. Very smart people have been trying to build rule-based AIs for a very long time, with almost no success whatsoever. Modern machine learning AIs do require some intelligence to design their basic architecture, but much more important is raw computation and data. It is possible that there is some algorithmic upper bound on how "good" a machine learning architecture can get before it has saturated the available data and computation. There is, after all, a finite amount of data in any given dataset, and trying to extract more than that will lead to overfitting. If this is the case, then smarter AIs will not beget smarter AIs, and the positive feedback loop fails to exist (or runs out at some point).
  • In the more immediate term, there is significant disagreement over whether we should focus more on making existing AIs better align with human values, or if we should focus on the formal proofs I mentioned earlier. These divergent preferences arise from very different ideas of what "AI risk" entails, how likely various different outcomes are, etc.

In short, you are asking whether one questionable premise may be used to explain or justify another questionable premise. I would say that we lack sufficient understanding of (and evidence for) either premise to seriously evaluate this kind of relationship.