If you take the great swathe of Norfolk that runs from Fakenham to the east coast, taking the gap between Cromer and Aylsham, you pass through no towns. There are no medieval churches that Simon Jenkins saw fit to put in his premiership in England's Thousand Best Churches; and yet you will find as many medieval churches in this narrow band as most English counties have in total. Given that Norfolk has so many churches, certainly more per square mile than anywhere else in western Europe, it might be thought that to visit even all those in this swathe would be a hard slog. But this is not the case. Most of these churches are kept open, or are at least accessible; many of them are lovely, and some have wonderful things to see. And just a few are of great interest, but little known, perhaps because there are so many churches here.

St Giles, Colby, is a great case in point. Set awkwardly north of its village along a lane going nowhere in particular, an indeterminate number of miles south of Sheringham, it is interesting to look at, it is beautiful inside, and it has several extremely interesting features. I doubt that it will be found in many people's top fifty Norfolk churches - it is certainly in mine.

Externally, it is rather odd. The tower is pencil thin, which in East Anglia always starts you humming and hawing about the Normans, but that is not the case here, I think, for it looks all of its late 13th century origins. Then, there is a massive late 15th century south porch, with image niches and wonderfully carved spandrels.

That on the east side is clearly St Michael dispatching a dragon. Another dragon is being dispatched on the west side, and this is claimed by the guide books as St George. I wonder. He carries no shield - St George usually carries a shield. He is on foot - St George is usually shown on horse back. There is something very similar on the porch at St Michael at Plea in the centre of Norwich. Perhaps he is a wild man, albeit a civilised one. Perhaps, more likely, he is the donor of the porch, emulating his Saintly hero in the other spandrel by dispatching evil.

Unusually, the porch lets straight into the nave - there is no aisle, no clerestory. You wonder if there is one around the north side. You set out around the church, and are met with a bit of a surprise, for the whole northern side of the church has been rebuilt in 18th century brick. Perhaps they did remove an aisle, but as you head east you see that they also rebuilt the chancel in its entirety.

This, then, is the skin. Perhaps it suggests what we might find inside. 13th century origins that have not been overwhelmed. A late medieval donor, pious and rich enough to provide the porch, but not to add clerestories or aisles - or, at least, not aisles that would last. An 18th century enthusiasm that was strong enough to rebuild in confidence. Perhaps, also, there is a further impression, that this is a parish that has had to maintain and beautify its church by the graft of its own hands - the key, for instance, the idiosyncratic work of some early 19th century blacksmith.

Your first step inside will reveal all of these at once. Light streams into the church through a simple Y-tracery west window. The font is late 15th century, and topped by the Georgians in a rural manner. The 18th century provided the view to the east, of which more in a moment.

In a county of interesting fonts, Colby's is one of the most interesting. It is set curiously; it stands so that no panel faces east, south, west or north; rather, being off-centre, the two easterly panels face east-south-east and east-north-east. It may have been moved; but the pedestal it stands on does the same thing, and appears to be original.

The most significant panel, and most important, is that to the ENE. It shows the Madonna and Child set in what I believe is known as the Seat of Wisdom. Now, this is an extraordinarily rare image to find on a font. It is undamaged by Anglican or Puritan iconoclasts.

The panels either side are also extremely unusual. That two the south shows two kneeling figures in 15th century dress. They are paying homage, and are almost certainly the two donors of the font. To the north, another figure, a huntsman, kneels with an axe. In front of him is what appears to be a small dog. This is more problematical. It is claimed as the figure of St Giles, who saved the life of a hind that turned out to be Christ. The creature, it is suggested, is the hind. The problem is, it is unusual to find St Giles carrying a weapon - and the creature does look like a dog. However, we know that this church owned a relic of St Giles before the Reformation - his finger, in a silver reliquary. And somebody kindly sent me a photograph of a statue of a huntsman carrying an axe with a deer at his feet - it stands high above the streets of Liverpool, on top of the turret of a 19th century pub. The name of the pub? The St Giles Tavern.

All the panels are in super condition, and must have been cemented over during the Anglican reformation of the 1540s. The other five panels are interesting for what isn't there. The symbols of the four evangelists are paired either side of the 8th panel. And this, incredibly, is still cemented over. What on earth could be beneath it? A crucifixion? A Holy Trinity? I couldn't help thinking that it looked like 19th century cement, in which case it was probably exposed by the Victorians at the same time as the other seven panels, and then cemented over again for being badly damaged.

Around the base of the font are reset brass inscriptions that echo the font's evidence of 15th century important people. And then, the view east is simple and delicious. Not too much money was spent by the Victorians on the nave - the walls are still panelled, the benches seemly, the tiles restrained. There is no coloured glass.

And then, there is that gorgeous chancel - a small east window above an 18th century sanctuary which is a rural vision of what a City of London church might be like. Even Moses and Aaron, flanking the now-removed decalogue boards, are locally painted. It is claimed that this reredos came from the church at Gunton on the occasion of its rebuilding by Robert Adam. This may be so, but I find it difficult to make the dates add up, and it does seem to fit perfectly here. To top it all off there are some fine 15th century glass figures reset in the otherwise clear east window: Christ in Majesty and a Mary of the Annunciation above the altar, two angels on their wheels either side at the top, and other figures including St James with his pilgrim staff and bag, St John with his poisoned chalice and St Peter with his keys.

There is more of interest in this immaculately well-kept church, including the inscription to the 17th century Richard Snelling; he left one pound to be distributed on every Christmas Day among the aged poor of Colby, and ten shillings on every Easter Day to cloath four poor children of the said parish. The altar cloth in exquisite needlework set in a frame on the east wall has an inscription recording that it was worked by Miss Charlotte Coleby (1797-1869), elder daughter of the Reverend George Coleby... by some means the frontal passed into the custody of the Royal School of Needlework, and it lay more or less forgotten until the School moved to new premises in 1962, when it was returned to this church. It is a rare and poignant reminder of the Oxford Movement-inspired revival in the Church of England as it was actually carried out by people on the ground - it was, perhaps, the first altar frontal that the parishioners of Colby had seen for several centuries, and it is still here, in the place where it was made and fondly used.

Simon Knott, August 2005