The parish of Kirby Cane lies in southeastern Norfolk close to the border with neighbouring Suffolk. The parish lies on slightly higher ground to the north of the valley of the River Waveney about 4 miles northwest of the Suffolk market town of Beccles. There is no significant compact village to Kirby Cane, it is merely a scatter of farms and cottages grouped along a lane which runs north of the A143 road which runs from Beccles to Diss. The area rises strongly from the valley of the Waveney, Pewter Hill climbing up a notable slope (for Norfolk) where the course of the river over history has created a prominent former river cliff line. Kirby Cane, nowadays almost runs into Ellingham to the south but to the north is a relatively empty agricultural plain stretching northwards towards Loddon.

All Saints' church is sited in the grounds of Kirby Cane's eponymous Hall, to the west of the parish's main lane. The site is rather picturesque with some rather attractive twisted pines growing on the sandy soil. As the round tower indicates All Saints has a pre-Norman origin. The church is a rarity in being pretty much un-restored, some interiors were changed but much of the architecture remains from the building periods. The tower is Saxon, the south doorway is Norman and most of the remainder can be dated to the early 14th century.