It was eight o'clock in the morning, at the start of one of those wonderful days that seemed to fill July 2006. The haze over the park of Hoveton Hall was thinning, and the sun was getting higher above the trees. Even the Roys of Wroxham empire, actually on the Hoveton side of the river, looked romantic down in the valley. It was going to be a scorcher. The shops and leisure developments on the Hoveton side sometimes give Wroxham a bad name, I'm afraid, but you don't have to go far out into the lanes to bring yourself back into proper rural Norfolk.

In the narrow lanes to the north sits one of Norfolk's prettiest small churches; the red brick, thatched St Peter, with its stepped gables and bellcote. It was a perfect sight early on a summer's day, and more so for being so tiny, so precious, like a cornelian jewel box in a bed of green velvet. It is all of a piece.

St Peter was built in one go in 1624. As Pevsner observes, this is an unusual date for a church to be built in England. I assume it had something to do with the proximity of the Hall. Hoveton's medieval parish church, St John, is a good mile and a half a way. The family who lived at the Hall were the Blofelds, and although the current Hall wasn't built until the end of the 17th century, I assume that there was an earlier building on the same site. There are two bays of windows in the nave, a west window and an east window, and a little porch in perfect proportion. It made me think of a toy church, perhaps even one of those china ones you see on some people's mantlepieces.

Churches in this area are enthusiastic openers, and despite the early hour the kind churchwarden had already been to unlock. We stepped into a tiny space, one of Norfolk's dozen smallest parish churches.

When I first read about this church, my heart had sunk on discovering that the restoration in the late 19th century was by Herbert Green. Green was the mediocre Diocesan architect at the turn of the 20th century, and I feared a dark, gloomy place. It is true that entering the building now is to enter a 19th century building, to all intents and purposes, but it is a restrained one, full of light, and still very much suits it.

Herbert Green's great-grand-nephew David had recently written to me, agreeing with some of what I have said about him, but pointing out that he was also the architect of that masterpiece, the public library in Kings Lynn, which is surely among East Anglia's ten best pre-WWI buildings of the century. So he is forgiven a lot. And, indeed, for here, because Green was responsible for the Y-tracery windows, which really suit the church.

There is a delightful font from when the church was new, as Arthur Mee would say, a gorgeous moulded Stuart royal arms set on a painted back cloth. This church was built the year before James I died. There are hatchments and other memorials to the Blofelds, and an interesting memorial to Mary Aufrere from the middle years of the 18th century. But most of all, this is a simple, pretty, prayerful space, welcoming and inclusive. On this summer day it was full of light and the sound of birdsong, and I'll always remember that.