Even though the colony is shown as a utopia, the Puritans still recognize that there will be crime and death, making both necessary (original sin). This also sets up a dark tone & gloomy setting for the novel. The prison is symbolic of condemned and disgraced people.
