First, you should consider if Wilcoxon is here the best choice. Check How to choose between t-test or non-parametric test e.g. Wilcoxon in small samples thread for great answer by Glen_b where he compares pros and cons of different hypothesis tests for small samples. It appears that in very small samples $t$-test may be the most robust choice.
Second, removing the missing values in most cases is not the best option. What do you know about missings? How did it happen that you lack information about those cases? If they are missing totally at random then one thing that you could do is to impute missing values with some values (e.g. mean, median, mode), use multiple imputation, or a number of different techniques of dealing with missing values. If there is some pattern in when values are missing then maybe you should consider the fact that some cases are missing in your statistical model as well?
For general theoretical introduction you could try the Rubin (1976) paper "Inference and Missing Data", classic book by Little and Rubin "Statistical analysis with missing data" or a number of different sources that can be easily found on Google Scholar.