The setting of this pretty little church is just about perfect, on the edge of the Raynham estate and set away from the nearest road, with the old Rectory next door and copses and fields for company. At the entrance to the graveyard is a wrought iron kissing gate, of a type which must once have been common, put together by some long-dead village blacksmith. As at the other Raynhams, the graveyard is still full of fascinating old headstones, with none of the modern fashion for tidying up in the name of motor mowers. The windows of the church are a delight, like a pattern book of the later medieval period. The simplicity of the tower belies the fact that this whole structure is probably a rebuilding of the 14th century, embellished in the late 15th.

St Martin has the feel of a proper rural church. Here, there are no outstanding treasures, but there is much of interest, and an atmosphere which speaks of the long generations who have seen it as the heart of their community. It is a touchstone.

Perhaps the most outstanding feature is the surviving mensa, or altar stone, now in place back on the altar. For many years, it served as a step into the chancel. The pattern on the edge is older than the building in which you see it. Pevsner thought it might be 12th century, but it may be older, and is certainly at the end of the Norman period, if not earlier. This makes it the oldest surviving mensa in East Anglia, and one of the oldest in England.

This is a rustic estate church, without the memorials to famous names which you find across the fields at East Raynham. Here, it is easy to imagine the blacksmith and the plowboy sitting uncomfortably for the afternoon sermon, and even further back in Catholic days the village women at their private devotions. I stood in the silence, the birdsong from outside a counterpoint to the sunshine falling through the windows. I thought of Eliot: If you came this way, taking any route, starting from anywhere, at any time or at any season, it would always be the same: you would have to put off sense and notion. You are not here to verify, instruct yourself, or inform curiosity or carry report. You are here to kneel where prayer has been valid.

One of the delights of this part of Norfolk is that, apart from the jarring exception of the two aptly-named Weasenham churches, virtually all the churches around here are open and welcoming. It feels like a privilege to come into one and sit in its silence for a while, a reminder that the old buildings are the greatest act of witness which the Church of England has got. I wonder how many people who slake their thirst for a sense of the numinous, and fill the God-shaped hole within themselves, do so by starting with a visit to a place like this?

It has been eight years since I began the websites for the Churches of East Anglia, and throughout that time I have promoted the cause of open churches, seeing it as one of the primary reasons for the existence of these buildings. I am proud that many more churches are open in Suffolk than were a decade ago, and although I think there are several reasons for this, I am pleased when people tell me that my websites may be one of them.

However, in the last year or so that I have begun to have some small doubts - not about the need for open churches, of course, but about what I have begun to see as a rather horrible militant anti-religious feeling in this country, fed by the cynicism of celebrity culture, a poorly educated understanding of Islam and Catholicism in particular, and the often articulate champions of the neo-fascist television philosopher Richard Dawkins. What forms might this anti-Christian sentiment begin to take?

Several times this year in Norfolk I have encountered acts of vandalism which were clearly not the work of kids, or football hooligans, or organised burglars, but simply attacks on churches because they were places of worship. Increasingly, BBC Radio and broadsheet newspapers have featured writers with extreme views on the subject of belief, including a promotion of Dawkins' idea that religion is a mental illness, and that bringing up a child in the Christian Faith is a form of child abuse. I fear that the attacks I have already witnessed will not be the last. Churches like St Martin are a part of our cultural and social inheritance. I can only hope that they remain open to us for as long as we need them, and certainly for longer than we deserve.