Using (pseudo-)random numbers and the Law of Large Numbers to simulate the random behavior of a real system.
Monte Carlo methods are used to mimic the random behavior of real random phenomena by drawing sequences of random or pseudo-random numbers, usually via a computer code. These numbers are incorporated as inputs to the model describing the phenomena of interest, as proxy to a true random sample. They require a (pseudo-)random uniform generator; the most common implementations provide a number from 0 to 1 that mimics the random draw of a $U(0,1)$ variate, let us call this (pseudo-)random variate $U$.
For instance, to simulate a fair coin toss, draw a random number $U$ and associate values in $[0,0.5)$ with heads, and values in $[0.5,1)$, with tails. More complex models require a more involved use of the random uniform generators, as well as smart ways of transforming the basic $U[0,1]$ variates into random variables with the targeted distribution, e.g., using acceptance-rejection methods or Metropolis algorithm. In general, a random generator will be a function $\Psi$ of a random number of uniform variates, $X=\Psi(U_1,U_2,\ldots)$.
In Monte Carlo studies where time and budget are an issue, an efficient use of variance reduction techniques is often warranted. These include antithetic samples, importance sampling, control variates, etc.
Besides the direct use of Monte Carlo methods to model random phenomena, they are also used in mathematical physics and statistics to evaluate multidimensional integrals. Such integrals can be represented as expectations of an integrand against a wide range of (importance) distributions. A point is drawn randomly from such a distribution (often with support the support of the integrand); the value of the integrand is added to the running sum; the process is repeated until the desired accuracy is achieved. The approximation is converging by virtue of the Law of Large Numbers. The advantage of the method is that the error decreases at the rate $O( N^{-1/2})$, independent of the dimension of the space. Low discrepancy, or quasi-Monte Carlo, sequences are arguably better suited for this purpose.
References:
Lemieux, C. (2009) Monte Carlo and Quasi-Monte Carlo Sampling. Springer. Amazon link.
Liu, J. S. (2001) Monte Carlo Strategies in Scientific Computing. Springer. Amazon link.
Robert, C.P. and Casella, G. (2004) Monte Carlo Statistical Methods. Springer. Amazon link
See also: tags sampling, simulation.
