Computational cost of RSA with keys of length $n$ bits is roughly $O(n^2)$ for public key operations (encryption, signature verification), and $O(n^3)$ for private key operations (decryption, signature generation). So RSA with a million-bit key will be roughly one billion times slower than RSA with 1024-bit keys (for the private key operations); the latter takes about 1ms on a common PC, so you're in for a fortnight of computation with your million-bit key.
Memory space is not a hard constraint, because RSA computations require only to keep a few values of the size of the key; a one-million-bit integer is 128 kB; you can have thousands of those in RAM. You will, however, exceed level 1 cache (that's 32 or 64 kB on a common PC) so you can expect some slowdown (with Montgomery's multiplication, data access is sequential, so this effect should be limited).
Of course, security is not only about resisting attacks; it is also about having confidence in being able to resist attacks. My confidence in a system which uses RSA with one-million-bit keys would be near zero... because it makes no sense. RSA is secure because big integers are hard to factor. The best known algorithms with the best available hardware fail at about 1024 bits; 2048 bits are more than enough. Going beyond is making a wild and totally unsubstantiated guess about what not-yet-discovered algorithms may look like, which is speculation on rumours of legends about mythical glimpses of the future. When someone talks to me about having an oversized RSA key, like a 8192-bit key "just to be safe", I see it as if he was talking about buying a SUV to demonstrate his manhood (in your case, you are advocating buying an aircraft carrier).