If you want a good read on this I recommend "The Fall of the House of Dixie: The Civil War and the Social Revolution That Transformed the South". The author was an undergraduate professor of mine.

One of the interesting things examined in the book is how the minority slave owning class got the other 75% of the population in the South to go along with the "peculiar institution of slavery".