Why does my BODY border ignore my DIV blocks when I make them Float:Left? When there is no float, the BODY border is on the outside of the DIVS?

Question

body {
    margin: 20px;
    border: 2px solid rgb(0, 0, 0);
    background-color: pink;
}

p {
    background-color: blue;
    border: 1px solid green;
    text-align: center;
    margin: 5px;
    padding: 5px;
}

div {
    background-color: white;
    width: 200px;
    height: 200px;
    border: 1px solid green;
    background-color: cyan;
    text-align: center;
    padding: 10px;
    margin: 5px;
    float: left;
}

If Float is added the Border jumps over the DIVS, if I delete Float, the border goes around all Elements.