Colonists are the living inhabitants of each RimWorld colony. Each colonist has specific skills, positive and negative traits, and a character backstory. In general, every human is a colonist except for raiders/travelers/prisoners. The colonists come in various outfits based on their background, which can be seen by looking at the 'Character' tab in game.
Prisoners are people, usually raiders, hostile tribals, or your own colonists undergoing a mental break, who have been taken captive. They are unable to leave their prison barracks and can only use facilities (such as a food dispenser or shelf) on their own if they are placed inside the prison barracks.
You should notice that your colonists won't haul food away from cells, even when directed to do so. Animals may still enter the cell on their own and eat that food however.
Raiders, are enemies who attack in groups at random times, sometimes one after the other and others from time to time. Their group size, health, skill, and weaponry will depend on both the set game difficulty and the time already elapsed in the game. The first raid will only have one wounded raider. Subsequent groups have three to five raiders, and eventually groups can mob more than 20 with the lowest health being above 80.
Visitors are friendly faction members that come and mill about the player's colony. Sometimes one of the them has items to trade, indicated by a question mark. Their arrival could prove advantageous if the colony is assaulted by a mutual enemy while they are present. When leaving, visitors will often rescue their own downed pawns.