The parish of Calthorpe lies in northeastern Norfolk roughly 4 miles north of the market town of Aylsham. Calthorpe sits, in lanes, about 2 miles west of the A140 road which connects the city of Norwich with the coastal resort of Cromer. There is very little to Calthorpe, a straggle of properties running westwards, the lane being named "The Street", from the Scarrow Beck which divides the parish from Erpingham, Calthorpe being, in reality, little more than a westward extention of its larger neighbour. Like most Norfolk parishes Calthorpe would have been a farming community, arable in the main although there would have been pastures alongside the Scarrow Beck. This area of the county is awash with tiny parishes and Calthorpe, covering only a little more than 1,000 acres was one of many within a few miles.

St Margaret's church sits on the southern side of The Street just east of the main crossroads of lanes. For once Pevsner is stumped as to the dating of this church, apparently the whole church was re-faced in the 19th century and much dating detail remains hidden behind that surface. The chancel, which shows a rendered surface, is dateable and comes from the late 13th century at the cusp of Early English Gothic transforming into the Decorated style. Elsewhere the church shows Perpendicular styled windows which presumably are original and indicate a build covering much of the late 13th and 14th centuries, a date period typical for a Norfolk church. The western tower, however, is an enigma, today it appears a typical Norfolk flint tower but drawings of 1823 show no flint present, the lancet at its southern base could be indicative of Early English Gothic and mid 13th century. A rough verge outside the church allows parking for a single car, the churchyard is above road height with a sandy pathway rising to the gates. Whilst the churchyard is compact and surrounded with trees there is just sufficient space to get back far enough to adequately view the building.