At least 56,000 people of all 150 Civil War prisons died. Camp Sumter was a Confederate military prison, located in Andersonville, Georgia. This prison is known as the worst in all Civil War prisons. It existed for 14 months where about 13,000 died, both inmates and guards.