Early in the morning of February 20, 1864, General Seymour's army left Barbers' Station and moved westward towards Lake City. Because of the necessity of posting garrisons at Jacksonville and elsewhere, the Union force consisted of between 5,000 and 5,500 men. The small army was divided into three brigades of infantry, one brigade of mounted troops, and supporting artillery.
The Federals advanced in three columns along the Lake City and Jacksonville Road, which ran roughly parallel to the Florida Atlantic and Gulf-Central Railroad. The Federal cavalry was in the vanguard, followed by the slower-moving infantry. By mid-day the Federals had reached Sanderson, where they briefly stopped for lunch. While it Sanderson, Seymour and his staff were warned by a defiant southern woman: "'You will come back faster than you go."' The Union officers were amused at her boldness.
In the early afternoon of February 20, a few miles west of Sanderson, the advance elements of the Union cavalry began skirmishing with a few southern horsemen that appeared to their front. This skirmishing was maintained for several miles, with the Federals driving the Confederates westward towards the railroad station at Olustee, about ten miles east of Lake City. Southern resistance intensified as the Federals neared Olustee.
In the days since the February 11 skirmish at Lake City, General Finegan had moved his force to Olustee Station, located about ten miles east of Lake City There the Confederates found one of the few defensible locations in the area where the railroad passed through a narrow corridor for dry ground that was bordered by impassable swamps and bays to the south and a large body of water known as Ocean Pond to the north. The Southerners built strong earthworks and awaited the Federal advance. When Finegan learned of the enemy's approach on February 20, he ordered his cavalry forward to skirmish with the Federals and to drive them towards his main line. Unfortunately for Finegan, the fighting east of his main line intensified, forcing him to send out additional troops to help those already deployed. A major engagement soon developed about two miles in front of the Confederate line.
As the skirmishing intensified, both Finegan and Seymour fed additional troops into the battle. Finegan advanced first the 64th Georgia and part of the 32nd Georgia, followed by the 6th, 19th and 28th Georgia Regiments, and Gamble's Florida Artillery. General Colquitt commanded the detached units, while Finegan remained behind with the main body. General Seymour brought forward the 7th Connecticut, followed by the remainder of Hawley's Brigade, the 7th New Hampshire and the 8th United States. By midafternoon the skirmishing has escalated into a major battle.
The battle threatened to turn rapidly into a rout for the Federals. While Colonel Hawley was positioning the 7th New Hampshire, a wrong command was given and the unit fell into confusion. The 7th soon collapsed, with some men running to the rear and others milling about in a disorganized mob.
The collapse of the 7th New Hampshire directed southern attention towards the 8th United States Colored Troops, which occupied the left of the Union line. The 8th was an untried unit, having been organized only several months before. Prior to Olustee the regiment had seen no combat, and in fact the men were not even completely trained. Colonel Charles Fribley tried to steady his men, but he soon fell mortally wounded. The raw troops of the 8th held their ground for a time, suffering more than 300 casualties. Finally, however, they retreated in some confusion, leaving the Confederates in virtual command of the battlefield.
With the dissolution of Hawley's Brigade, General Colquitt ordered the Confederate forces to advance. Since the beginning of the engagement Finegan had sent additional units (the 6th Florida Battalion; the 1st, 23rd, 27th, and the remainder of the 32nd Georgia Regiments, and the Chatham Artillery) to Colquitt's support, so by now the Confederate lines stretched for about one mile, north to south. Colonel Harrison commanded the Confederate left, and Colquitt the right, although the units of their brigades were somewhat intermingled.
To stop the southern advance, General Seymour hastily ordered forward Colonel William Barton's Brigade of the 47th, 48th and 115th New York. The New Yorkers stopped the Confederate advance, and the battle lines stabilized for a time . The Union commander would later be criticized for reacting slowly to an increasingly dangerous situation, and fordeploying his forces piecemeal into the battle. In fairness to Seymour, the battlefield's terrain somewhat limited his options. The Federals lines were bordered by swamps on both flanks so there was little room to maneuver, and the field itself was an open pine barren with little cover.
The fighting during this middle period of the battle was particularly severe, with each side suffering heavy casualties. During this seesaw combat, the Confederates captured several Union artillery pieces and threatened to overwhelm the Federal infantry. Although the Yankees were under intense pressure, at a critical moment the surging Confederates began running low on ammunition. Men searched the pockets and cartridge boxes of their wounded and dead comrades to obtain additional rounds, but still the southern fire slackened. Several regiments held their place in line despite being completely out of ammunition. After what seemed to be an interminable delay, ammunition was brought forward from Olustee, along with the remaining reserves: the 1st Florida Battalion and Bonaud's Battalion. General Finegan also reached the battlefield at about this time.
With the arrival of these reinforcements, the Confederates again began advancing . By late afternoon, General Seymour had realized the battle was lost. To prevent a rout and to cover his retreat, he sent forward his last reserves, Colonel James Montgomery's Brigade, which consisted of the 35th United States Colored Troops and the famous 54th Massachusetts Volunteer Infantry. Montgomery's Brigade stopped the Confederates for a brief time, enabling Seymour to begin withdrawing his other forces. One white veteran of the battle states: " The colored troops went in grandly, and they fought like devils"
By dusk, the Union forces had begun their long retreat back to Jacksonville. The 54th Massachusetts, the Federal cavalry, and part of the 7th Connecticut covered the withdrawal. Many wounded and a large amount of equipment had to be abandoned in the hasty retreat. Fortunately for the Federals, the Confederate pursuit was poorly conducted, enabling most of the Yankees to escape. The southern cavalry, led by Colonel Caraway Smith, was particularly criticized for its lackluster performance. That night the Federals retreated all the way back to Barbers, where they had begun the day. By February 22, Seymour's battered army was back in Jacksonville.
The casualties at Olustee were staggering compared to the numbers that fought there. Each side had about 5,000 men present. Union casualties were 203 killed, 1,152 wounded, and 506 missing, a total of 1,861. Confederate losses were 93 killed, 847 wounded, and 6 missing, a total of 946. This works out to about 40% for the Federals and 20% for the Confederates. The 47th New York had 313 casualties and the 8th U.S.C.T. had 310. Among the Confederate units, the 32nd Georgia lost 164 men and Bonaud's Battalion 107. For the North, the casualty percentage was among the highest of the war, and Olustee ranks as the third bloodiest for the Union when comparing the casualties to the number on men engaged. Letters and diaries from the men involved indicate that the battle was the equal of, if not worse than, the savage fighting a number of the veteran regiments had experienced in the campaigns in Virginia or the Western theater.
A regrettable episode in the aftermath of the battle was the apparent mistreatment of Union black soldiers by the Confederates. Contemporary sources, many from the Confederate side, indicate that a number of black soldiers were killed on the battlefield by roaming bands of southern troops following the close of the fighting.
The Olustee defeat ended Union efforts to organize a loyal Florida government in time for the 1864 election. The Federals were somewhat more successful in meeting the expedition's military objectives. Jacksonville remained in Union hands until the end of the war, open for trade with the north; the operation had undoubtedly disrupted the supply of Florida cattle and other foodstuffs to the rest of the Confederacy; and the increased area of Federal control made it easier for Florida blacks to reach Union lines and for recruits to fill the ranks of northern military units. Of course, all of these objectives could have been met simply by the occupation of Jacksonville and without the nearly 2,000 casualties suffered at Olustee.
Military operations continued in Northeast Florida throughout the remainder of the war. Union troops frequently raided out from Jacksonville to harass Confederate supply operations. Captain J.J. d*ckison of the 2nd Florida Cavalry earned fame during the period as the "Swamp Fox" of Florida. In a series of minor yet spectacular victories, and with only a small force under his command, d*ckison was able to thwart many of the Union drives into the interior of the state, although the Federals handed him a stinging defeat at the first "battle" of Gainesville. Despite d*ckison's successes, by early 1865 it was obvious that the defeat of the Confederacy was near. In early March, 1865, the last military operation of any significance in Florida took place. At the Battle of Natural Bridge, a scratch force of Confederate militia and home guard units, along with a few regular troops, stopped a Union raid against St. Marks, on the coast south of Tallahassee. When Tallahassee was finally occupied by Northern troops in early May, it was the last Confederate capital east of the Mississippi to fall into Union hands. The Civil War was over. Thank God.