From The Sag Harbor Historical Society

The Port of Sag Harbor was founded in 1707, and is located on the south fork of Long Island, New York about 100 miles east of Manhattan. By the time the Revolutionary War began, Sag Harbor had built up a thriving coastal and foreign trade. In 1789, when George Washington approved the creation of Sag Harbor as a Port of Entry for the newly formed United States, the village had more square rigged vessels engaged in commerce than the Port of New York. 

A unique and interesting village, Sag Harbor has the distinction of having had the first custom house in the U.S., the first volunteer fire company in the state of New York, and the first newspaper printed on Long Island.

The lower part of the business district, thrice destroyed by fire, rose from the ashes each time, grew and prospered through a spectacular whaling industry, a period of industrial growth, and finally as a place where the population swells each summer with an ever increasing influx of visitors and a bay filled with pleasure craft.

23 Howard Street was originally built around  1820 for Erastus Bassett and the house appears on the earliest maps of Sag Harbor. 

Through all of its 300 years, Sag Harbor has retained its special history, beauty, and individuality.