Houghton St Giles is an utterly lovely, tiny village on the outskirts of Little Walsingham. When people talk about 'Walsingham', they usually mean this place as well. Here, Norfolk is hilly; the vigorous River Stiffkey cuts through the vale, and the houses and parish church of Houghton St Giles, or 'Houghton-in-the-Dale' as it is sometimes known, flank the top road. This parish has thousands of visitors every weekend in summer, pilgrims drawn here by one of the touchstones of English Christianity. However, it is not to the Anglican parish church that they come, but to the Catholic National Shrine of Our Lady, just across the river from St Giles. The great 1982 Church of the Reconciliation sits barn-like in the vale, the 14th century slipper chapel and the other shrine buildings beside it.

Inevitably, some pilgrims cross the forded river to visit St Giles, a witness to the 'glory that was once theirs'. Imagine what it must be like for some of these pilgrims when they visit this beautiful village for the first time. Arriving after a long coach journey along motorways from bleak post-industrial inner city parishes in Middlesbrough, Glasgow or Liverpool, they step out into a gently rolling landscape of high-hedged fields and lanes, of bubbling brooks and ancient buildings, of birdsong and clear sunlight. Perhaps it is like a vision of heaven. They must often think about it afterwards, daydreaming of this remote place and imagining their return.

St Giles sits on the road from Fakenham to Walsingham. Half a mile or so earlier, travellers will have passed the lovely ramshackle church of All Saints East Barsham, perhaps not even noticing it, its low tower surrounded by trees. St Giles is quite different. This is a neat, shipshape building, thoroughly done up in the 19th century by enthusiastic Victorians. It sits at a right-angle to the road, its rather stark lines ameliorated by a raised bed of 18th and early 19th century gravestones to the south-east.

You enter into what is essentially a 19th century interior, retaining some of the survivals of its medieval life. The best and most interesting of these is the 15th century roodscreen. Norfolk has no shortage of roodscreens, but no other is quite like Houghton's. On the north side is a sequence of Holy Kinship - the only other in Norfolk is on one of the side screens at Ranworth. Each panel depicts one of the women associated with Christ and the Disciples, although the first panel has never been satisfactorily identified. The inscription says Scta Anna - but this is not St Anne. We'll come back to it in a moment. Meanwhile, the others show, from left to right, St Mary Salome with the infant St James and St John, the Blessed Virgin and Christ child, St Mary Cleophas with four infants who appear intended as St Simon, St Jude, St Philip and St James the Less, St Elizabeth with the infant St John the Baptist, and finally St Anne teaching the Blessed Virgin to read. So, who does the fourth panel depict? MR James suggests St Emeria, the sister of St Anne and mother of St Elizabeth. However, as the child appears to be a boy he may be St Servatius, held to be her ancestor. No other explanation seems more likely, but still...

on the south side, the figures are more familiar. The four Latin Doctors, Saints Gregory, Jerome, Augustine and Ambrose, are commonly found in Norfolk. The last two are St Silvester and St Clement. Silvester appears to have the donor of the screen kneeling at his feet.

Unsurprisingly, given our location, there is much evidence here of 20th century Anglo-catholic enthusiasm, including a fine mosaic icon and a truly grim statue, both of St Giles. However, the most poignant survival is from the 19th century, and sits in a side window of the chancel. It depicts Christ as the Good Shepherd, carrying a lamb in a familiar, formulaic pose - there is an identical image across the county at Alburgh. However, it is the inscription which intrigues. In Memory of Dec 30 1876, it reads. He shall gather the lambs in his arm. And that is all - no names, no other details. The obvious deduction is that the window commemorates the death or loss of a child, or children. The incident was obviously well-known enough to require no further explanation, but here we are 130 years on, and it is lost to us. What can have happened on that midwinter night, out here in this wild and remote vale? Was the snow deep on the ground, and the temperature descending towards freezing? Was the River Stiffkey in full spate? I have pondered and investigated this window for more than ten years, but I have never been able to find a satisfactory answer.

The east window of the chancel is a curiosity for a different reason. It depicts the Crucifixion, with Mary and John flanking the cross. And yet it is in a style unlike any I have seen elsewhere, and Mary in particular seems to have been rendered deliberately ugly, as if this was some old Dutch Master at work rather than the late 19th century glass which it appears to be.

There is much to wonder at here, from the screen, to the Crucifixion, to that extraordinary memorial window. And something else to think about: when Tom and I came this way in August 2007, we found the church locked. This was something I had not encountered in perhaps a dozen previous visits, and a locked church is almost unheard of in this part of Norfolk. The reason, it turned out, was that every year in mid-August, Travellers and Gypsies descend on Walsingham to celebrate the Feast of the Assumption. They are Catholics, of course, and will be processing to attend Mass at the modern church on the other side of the river. But I couldn't help thinking what this told us about the local parish's attitude to outsiders, if it locks its church rather than allowing them access to a sense of the ancient, numinous and divine, to the Glory that once was theirs as well.