Allen was born in Edenton, North Carolina. His sister, Mary Granberry Allen, married Pleasant Thurman, and their son, Allen G. Thurman, followed in his uncle's footsteps, becoming a lawyer and politician. Allen moved to Chillicothe, Ohio in 1819 and he and his sister lived there together.
He served as a Representative from Ohio from 1832 to 1834, when he lost a bid for re-election, and Senator from Ohio from 1837 to 1849, losing a bid for re-election in 1848. Allen then retired to his farm, "Fruit Hill," near Chillicothe, Ohio, and did not return to public service for nearly a quarter century. He served as Governor of Ohio from 1874 to 1876. He unsuccessfully sought a second two-year term in an 1875 election.