A bid by the exclusive Ham Polo Club to build a new all-weather arena to allow the game to be played during the winter has been rejected by planners following a backlash from locals.

The application drew more than 200 objections from residents who warned it would have a disastrous impact on protected landscape on the banks of the Thames within sight of the Grade 1 listed Ham House and its fabulous gardens.

Central to the plan was the building of a 1.5m high close board fence around the new all-weather arena – which would measure 90m by 40m – so interrupting views.

The fence is required for a winter form of polo, which is likened to indoor football, where players bounce a leather ball off the boundary as part of the game.

There has been long-running antagonism between the operators of the club and residents in Ham and Petersham, who accuse it of effectively shutting out locals while causing nuisance with, for example, arrivals by helicopter of its wealthy visitors.

In a letter of objection to planners, one resident suggested the new fence was designed 'so that the toffs don't need to be seen by the hoi polloi when playing in the sand with the horses'.

He added: "This is just another attempt to spoil the amenity of the area. You don't need this stuff, we don't need to see your ugly fencing."

In reality, the residents of Ham and Petersham include a collection of academics, musicians, authors, captains of industry and many others, who are quick to act on any threat to the local environment.

Not only does the Polo Club sit on protected Metropolitan Open Land (MOL) next to Ham House and gardens, but the area was also featured by Charles Dickens in Nicholas Nickleby.

Objections to the schemes were heard by the planning committee this week and they included a powerful contribution from Adam Gatehouse, the celebrated conductor and more recently Artistic Director of the Leeds International Piano Competition.

He told councillors: "The proposal, particularly the fencing, will bring harm to the MOL and harm to the openness of the MOL, harm to the heritage assets and their setting, including Grade 1 Ham House and conservation area."

He said there would also be harm to the views, particularly from the neighbouring Melancholy Walk, which is a public footpath used by generations.

In its application, the polo club argued that the new arena would allow it to operate all year round, rather than just in the warmer months. The club and its supporters said this would allow it to invite more people from the local community, charities, schools, universities and the underprivileged to take part in the sport.

Mr Gatehouse said there was no evidence for this, adding: "On the contrary, horse riding and polo are minority sports practiced by a small financially advantaged section of the public."

He pointed out that the club charges adults £185 per hour with £85 per hour for children.

He added: "If this proposal is accepted, the planning department would be sending a message that this Council supports the interests of a very small and wealthy minority against those of a vast community of equals who have long benefited from free access to an area of outstanding natural beauty offering iconic views unimpeded by encroaching man-made constructions."

Deputy leader of the Green party on Richmond Council, Cllr Andree Frieze, criticised the plan saying the sports arena 'for the few' would cause substantial harm to the Ham conservation area.

She said: "What this application would be allowing is a 3,700 square metre arena for high intensity games played by shouting riders and cantering horses, thwacking balls that hit off the new wooden fence creating noise disturbance as well as visual intrusion in a tranquil beautiful location."

Local Lib-Dem councillor, Cllr Penny Frost, joined the criticism saying the club had failed to show there are 'very special circumstances' that would justify overriding the normal planning rules covering protected landscapes.

She said: "Supporters of this scheme describe it as a great asset for the community … but only the community that play polo not local people, who could not afford the astronomical cost of polo lessons.

"It is many years since local people were even welcomed onto the site to picnic and watch the matches in the summer."

Cllr Frost said: "This is an area in the curtilage of 17th century Ham House, greatly loved and valued for its tranquillity and beautiful views, enjoyed for many centuries by locals and visitors alike. This will be lost once the fencing goes up."

She said there was nothing in the scheme for the local community, but the only beneficiaries would be wealthy visitors to the club.

One of the club's directors, Benji Davis, said the new arena would allow the club to operate through the winter. "This will be a huge benefit to the club and the local community," he said.

"There seems to be a common misconception that the club is primarily concerned about making money to line the pockets of its members, this could not be further from the truth. The club's enshrined constitution is to promote polo and riding in the local area.

"All of the revenue generated by the club, whatever it is from, is reinvested in the club to improve its facilities and, in turn, enhance the range of activities offered to the local and wider community."

He said the organisation has a waiting list for polo and riding lessons, while it has formed links with local charities and Roehampton University with a view to allowing them to use its facilities.

"The club was established in 1926 and is at the heart of Richmond borough's heritage and history. The arena extension will enhance equestrian activities within the borough," he said.

Objections to the scheme came from local residents' associations and the National Trust, which argued the fencing would harm views of historic Ham House.

Despite the many objections, council planning officers recommended approval for the scheme, subject to conditions. However, councillors on the planning committee overwhelmingly rejected the application, arguing the harm outweighed any benefits.