Formally organized on July 7, 1874, St. George's-by-the-River began as a summer church for families that resided in what was then known as Rumson Hills. In 1908, the widow of one of the church's founders offered to build a new church in his memory, in one of the more populated areas of town. Its current location is near the Shrewsbury River near the Sea Bright Bridge. Some of the original building was incorporated into the new church, such as the cornerstone, Celtic cross, lectern and a memorial window. After World War II, when Rumson and the surrounding towns became year-round communities, St. George's became a year-round church.
With many groups and programs in which to participate, St. George's is an active, involved community. From Bible studies to community service programs to mission trips, parishioners have many opportunities to worship with each other and serve the greater community.
Services are held regularly on Sunday mornings at 8:00 a.m. and 10:00 a.m.. The early service follows Rite I and is without music; the latter service follows Rite II from the Book of Common Prayer and includes either the adult or children's choir. Occasionally there is a combined service of Morning Prayer and Eucharist. Two Eucharists are celebrated on every Wednesday: one at noon with a Service of Healing and short family-oriented Village Eucharist service at 6:00 p.m.