The old Falls Plantation was inherited by William Byrd I (ca. 1652-1704) from his maternal uncle Thomas Stegge. The Byrd family lived there from the 1670s until 1693 when they moved downriver to Westover in 1693. The land remained in the Byrd family and was later called Belvidere Plantation. William Byrd III (1728-1777) lived at Belvidere for a time with his first wife.