Both Lincoln and Johnson had foreseen that the Congress would have the right to deny Southern legislators seats in the U.S. Senate or House of Representatives, under the clause of the Constitution that says, "Each house shall be the judge of the ... qualifications of its own members." This came to pass when, under the leadership of Thaddeus Stevens, those congressmen called "Radical Republicans," who were wary of a quick and easy "reconstruction," refused to seat newly elected Southern senators and representatives.  Within the next few months, Congress proceeded to work out a plan for the reconstruction of the South quite different from the one Lincoln had started and Johnson had continued.

Wide public support gradually developed for those members of Congress who believed that African Americans should be given full citizenship. By July 1866, Congress had passed a civil rights bill and set up a new Freedmen's Bureau – both designed to prevent racial discrimination by Southern legislatures. Following this, the Congress passed a 14th Amendment to the Constitution, stating that "all persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside."  This repudiated the Dred Scott ruling, which had denied slaves their right of citizenship.

All the Southern state legislatures, with the exception of Tennessee, refused to ratify the amendment, some voting against it unanimously. In addition, Southern state legislatures passed "codes" to regulate the African-American freedmen. The codes differed from state to state, but some provisions were common. African Americans were required to enter into annual labor contracts, with penalties imposed in case of violation; dependent children were subject to compulsory apprenticeship and corporal punishments by masters; vagrants could be sold into private service if they could not pay severe fines.

Many Northerners interpreted the Southern response as an attempt to reestablish slavery and repudiate the hard-won Union victory in the Civil War.  It did not help that Johnson, although a Unionist, was a Southern Democrat with an addiction to intemperate rhetoric and an aversion to political compromise.  Republicans swept the congressional elections of 1866.  Firmly in power, the Radicals imposed their own vision of Reconstruction.

In the Reconstruction Act of March 1867, Congress, ignoring the governments that had been established in the Southern states, divided the South into five military districts, each administered by a Union general.  Escape from permanent military government was open to those states that established civil governments, ratified the 14th Amendment, and adopted African-American suffrage.  Supporters of the Confederacy who had not taken oaths of loyalty to the United States generally could not vote.  The 14^th Amendment was ratified in 1868. The 15th Amendment, passed by Congress the following year and ratified in 1870 by state legislatures, provided that "The right of citizens of the United States to vote shall not be denied or abridged by the United States or any state on account of race, color, or previous condition of servitude."

The Radical Republicans in Congress were infuriated by President Johnson's vetoes (even though they were overridden) of legislation protecting newly freed African Americans and punishing former Confederate leaders by depriving them of the right to hold office. Congressional antipathy to Johnson was so great that, for the first time in American history, impeachment proceedings were instituted to remove the president from office.

Johnson's main offense was his opposition to punitive congressional policies and the violent language he used in criticizing them. The most serious legal charge his enemies could level against him was that, despite the Tenure of Office Act (which required Senate approval for the removal of any officeholder the Senate had previously confirmed), he had removed from his Cabinet the secretary of war, a staunch supporter of the Congress. When the impeachment trial was held in the Senate, it was proved that Johnson was technically within his rights in removing the Cabinet member. Even more important, it was pointed out that a dangerous precedent would be set if the Congress were to remove a president because he disagreed with the majority of its members. The final vote was one short of the two-thirds required for conviction.

Johnson continued in office until his term expired in 1869, but Congress had established an ascendancy that would endure for the rest of the century.  The Republican victor in the presidential election of 1868, former Union general Ulysses S. Grant, would enforce the reconstruction policies the Radicals had initiated.

By June 1868, Congress had readmitted the majority of the former Confederate states back into the Union. In many of these reconstructed states, the majority of the governors, representatives, and senators were Northern men – so-called carpetbaggers – who had gone South after the war to make their political fortunes, often in alliance with newly freed African Americans. In the legislatures of Louisiana and South Carolina, African Americans actually gained a majority of the seats.

Many Southern whites, their political and social dominance threatened, turned to illegal means to prevent African Americans from gaining equality. Violence against African Americans by such extra-legal organizations as the Ku Klux Klan became more and more frequent. Increasing disorder led to the passage of Enforcement Acts in 1870 and 1871, severely punishing those who attempted to deprive the African-American freedmen of their civil rights.