So sorry a little sleepy I guess, you said Union prisoners and not Confederate. The majority of Union prisoners were taken to Richmond, Virginia and when Andersonville, Georgia opened its gates in February of 1864 some of them were transferred there.