男 and 女 are neutral with politeness in formal contexts. In conversation or colloquial style, they may be (but not necessarily) used to imply rudeness, and 男の人, 女の人, 男性, 女性 are more polite.
When you listen to Japanese news, you will hear both 男 and 女, and 男の人, 女の人, 男性, and 女性. That is a very shameful aspect of Japanese culture, and it reveals that Japanese society is still immature. In these contexts, the announcers are expected/pressured (by the society/broadcasting company) to express personal feelings against criminal suspects by the use of language. 男 and 女are used for offenders (or suspects as well in earlier days) to express that the announcer is siding with the victim and is hence showing a personal dislike to the offender/suspect. This kind of language use is generally subsumed under the notion of 呼び捨て. Other examples include: avoiding the polite affix さん when referring to criminals. This departs from the nature of journalism being a neutral and factual information source, and its idea is to mete out non-official punishment to criminals/suspects in addition to the legal process.