According to Bertrand Russell, colonialism is justifiable if the result is the spread of civilization and the survival of the fittest. The dispossessed is either raised to a higher plane of civilization or is taken out of misery. In order to justify colonialism, the colonizers must also be able to enjoy his fruit of labour and be able to flourish in the climate of the conquered land.
By a "war of colonization" I mean a war whose purpose is to drive out the whole population of some territory and replace it by an invading population of a different race. Ancient wars were very largely of this kind, of which we have a good example in the Book of Joshua. In modern times the conflicts of Europeans with American-Indians, Maories, and other aborigines in temperate regions, have been of this kind. Such wars are totally devoid of technical justification, and are apt to be mor ruthless than any other war. Nevertheless, if we are to judge by results, we cannot regret that such wars have taken place. They have the merit, often quite fallaciously claimed for all wars, of leading in the main to the survival of the fittest, and it is chiefly through such wars that the civilized portion of the world has been extended from the neighborhood of the Mediterranean to the greater part of the earth’s surface. The eighteenth century, which liked to praise the virtues of the savage and contrast them with the gilded corruption of courts, nevertheless had no scruple in thrusting the noble savage out from his North American hunting grounds. And we cannot at this date bring ourselves to condemn the process by which the American continent has been acquired for European civilization. In order that such wars may be justified, it is necessary that there should be a very great and undeniable difference between the civilization of the colonizers and that of the dispossessed natives. It is necessary also that the climate should be one in which the invading race can flourish. When these conditions are satisfied the conquest becomes justified, though the actual fighting against the dispossessed inhabitants ought, of course, to be avoided as far as is compatible with colonizing. Many humane people will object in theory to the justification of this form of robbery, but I do not think that any practical or effective objection is likely to be made.
Russell, Bertrand. The Ethics of War, 1915
Being colonized is equivalent to emigrating to another country. In the former case, the border crosses the person, in the latter case, the person crosses the border. In the former case, a more desirable way of life is brought to a person; in the latter case, that person seeks out a more desirable way of life. A lot of people who hated colonialism would happily emigrate to a more developed country while being totally oblivious of the apparent contradiction in their opinions: what is the difference between emigrating to Austria and cloning a piece of Austria in your neighbourhood? If the cloning is done by the colonizer, you don't even have to worry about copyright infringement.
In plain English, natives never lived lives thug-free; civilized foreign thugs were a lot better than indigenous thugs. Towards the end of the 19th century, colonialism had become a vanity fair of the civilized, and there was a healthy rivalry between world powers. Civilized countries built universities, hospitals, rail roads and even legal infrastructure in other parts of the world which were mostly medieval and barbaric. This type of colonialism was entirely beneficial to the then-medieval Chinese. WWI put an end to it.
Even during WWI, Bertrand Russell recognized Germany as a civilized nation and called for swift British compromise in order to bring the war to an end.