Built of brick laid in Flemish bond, the Harvey Mansion rises to a height of three stories over a full basement delineated by a molded water table on the front (north) and rear facades. The ground slopes greatly to the south, which causes the basement to be exposed as a full story across the rear facade. Of the four original interior end chimney shafts, only two remain, the other pair having been destroyed by hurricanes. Both facades terminate in brick modillion cornices above which is a fractable gable roof.

Many changes made on the house have greatly altered its plan and fenestration. Seemingly the original house was five bays long with a pair of entrances on the front. These entrances later were changed to windows and the central bay opened for a door. In the early 1940s the central entrance was eliminated, and pairs of small windows replaced the former six-over-six sash in the central bay at all three levels. An entrance was cut in each side and covered by a small porch whose elaborate cornice was taken from the second-floor ballroom, which was shortened considerably during the renovation of the two window openings above each porch, only those on the east are original.

Harvey seems to have been a merchant of considerable wealth. His personal estate inventory, returned by his wife, Mary Harvey, in 1828 list the schooner Tillman, a large amount of tools and naval supplies, slaves, stock in the Bank of New Bern, a long list of notes due the estate, and a considerable stock of goods-including 870 bushels of alum salt and 103 bags of coffee. Neither the real estate owned outside the city in Craven Count nor the property in New Bern is listed in the above inventory. In a division of the real property in 1830 the "wharf, buildings and appurtenances" on South Front street went to Mary Harvey, so that she retained possession of the mansion and its supportive buildings - NRHP, 21 September 1971