Eamonn Hayes always knew he was going to be a fisherman. Just like his fathers and brothers. It was what you did in Kilmore Quay, a tiny rural coastal community on the south-east coast – there wasn't much else. But six years ago, Hayes saw the tough times coming and traded his commercial hat for a charter cap. Now he is the skipper of an angling boat, the Autumn Dream.

Because tourism, he thought, was about to take off in this quaint old village, with its thatched cottages and whitewashed walls running down to the sea.

As with commercial fishing, Kilmore Quay has long had a reputation as an angling site for bass, cod, shark, mackerel, bream, and pollock, among others.

Hayes takes groups wreck, reef and deep- sea fishing. He'll do long day trips, short evening excursions. Tours around the Saltee Islands too – he is not fussy. If you want to get out on the water he will tailor a package to suit your needs.

And the area has a lot more than just fishing to offer – a vast wealth of unharnessed beauty, with miles of rugged coastline, sandy beaches and the Burrow – famous for its dunes 15 metres high – wildlife and walks.

The Saltee Islands, which lie in the bay directly in front of the town , are one of Europe's foremost bird sanctuaries, with puffins, soars, oyster catchers, shags and cormorants to name but a few. Day-trips to the islands are becoming more popular – you can even camp there overnight and be completely cut-off from civilisation.

Remoteness, believe it or not, is still possible to find. Just two hours from Dublin and 20 miles from Wexford, the feeling of tranquillity and isolation is remarkable here.

The tranquillity, of course, is all part of the charm. But it's a little too quiet here right now, at the height of summer. Where are the tourists? Why are golden miles of beach so empty? The charter boats are lying idle in the harbour, tied up alongside the fishing trawlers.

Hayes is not happy, and his story is a typical one here. "I started fishing on my school holidays when I was young lad. I left school early and went onto the family boat and fished with my father and three brothers for about eight or nine years."

In the 1980s, when fishing was lucrative, Hayes and one of his brothers bought their own trawler. But after 35 years, he decided to get out – his business is solely a tourist one now.

"The charter business can be good, but it is totally weather-dependent, and only for three of four months of the year," he says. "Relying on tourists is not feasible, and for me it's not a stand-alone business. If I don't pick up other work in the winter I am in serious trouble."

This season has already started badly for Hayes and the many like him in Kilmore Quay. "Last year the weather got us, but the books were full. The tourists were here and you didn't mind too much if you had a bad week. This year the weather is bad again – but the difference is the books aren't full – no one is booking ahead."

Hayes believes the marina, built in the 1990s and part-funded by the community, has great potential but needs expansion and development. "These seas are great for angling and we get groups from Dublin, Wales, all over. We are only a half hour from the UK here. We really need to tap into that market."

But while Hayes remains hopeful about the future of his business, scallop fisherman Noel Carroll is not.

Last year, he invested €500,000 in a boat. He hasn't been fishing in weeks and when he does it's barely worth it, he says. "As scallop boats we don't even have quotas to deal with. France is our main market but it is now buying cheaper farmed scallops from South America. I understand we need to import and export within the EU, but I think member states should have some protection from imports from outside the EU. What's the point otherwise? I don't know what the future holds, I really don't."

It's safe to say that Kilmore Quay is feeling the recession. It's the weather of course. The 'Irish summer' is wreaking havoc on this small town, which relies heavily on both tourism and fishing –neither of which is a lucrative or stable industry right now.

In May, the Wooden House Lodge Hotel, a focal point and watering hole for both tourists and locals, went into liquidation, its empty shell and dishevelled garden a sign of the times. Refurbished in 2005 by owner Cormac MacCartaigh, the property is for sale by receivers with a €3.5m price tag, down from €5.5 last year.

"It was a complete refurb and a lot of money was invested," said one local businessman. "It went from a small local pub to super pub with nine bedrooms in the main hotel building, a restaurant and function room."

But the newer bigger version just couldn't cope. "People want to see the local guy behind the bar, and if he's not sometimes it doesn't work, especially without a leisure centre or more facilities, it just couldn't attract the kind of clientele it needed."

But Hayes and MacCartaigh were not the only ones who thought Kilmore Quay was about to take off. Everyone did, says Nicky Cullen, the town's best-known and best-loved character. Owner of Cullen's grocery-cum-deli, he is a typical small-town character – a 'lifer' – nothing happens in this village without him knowing, and he is all too ready to give his considered opinion. People come to Cullen's not just for their breakfast roll but for a chat, a laugh, the latest gossip, or maybe just a friendly ear.

Cullen, or the Silver Fox as his friends call him, after the restaurant of the same name which he set up and ran for 16 years, worked as a chef with the navy for many years before returning to Kilmore Quay to open its very first seafood restaurant. It became a hugely successful family business, and was given a glowing review by restaurant critic Tom Doorley. Cullen maintains it was the best review Doorley ever gave.

"We were at the top of the trees with it, butI got out while it was still good. My mother used to bake the bread for us. The year she died, 2003, I gave it up."

Now in his 60s, Cullen retired for a year and a half, but then got bored. His latest empire surveys the harbour proudly. He sells all the household staples, but his hot food deli is his pride and joy – he does a hot roast dinner for €7.49. Rolls, sandwiches, salads, sausages, beef, pork – everything.

But not fish. Someone else over the road does that, and it's important everyone sticks to their own, he says.

Cullen works seven days a week, 7am till 10pm, goes to the local pub a few nights a week, likes a flutter on the hounds and, according to one local, loves to sing.

He is worried about the future of his beloved village and is hoping its unrealised potential will be developed in careful way.

The Celtic Tiger, he says, ran right past this tiny corner of the country. There was no great investment to speak of because the town's infrastructure simply could not handle any major builds – a long-promised sewage treatment plant never materialised.

The countryside is not dotted with new estates or the telltale tax-relief hotel, thrown up during the boom, and there is nothing to really keep people in town for more than a day.

There are some tourists about of course, when the rain lifts, but not enough of them to make an impact.

In 10 years' time, the spectre of Kilmore Quay as a ghost town – perhaps preserved as a monument to an Irish fishing village of old, is a very real concern for some local people. "When the fishermen are not working it's very noticeable. There is no activity in the harbour," says Cullen.

And just as the sun breaks through the clouds on this decidedly inclement July day, Colin Parle speaks of his plans to breathe new life into the old dog. Born and bred in the area, he, like everyone else, has a vision for the place – and it's nothing new for Kilmore Quay. Just something that needs to be gently exploited – diving.

"Diving in Ireland is really taking off – and this is, and always has been, a great spot for it," maintains Parle, who works in construction. "I dive regularly with four friends and we are constantly getting enquires from clubs and groups about coming here to dive, but there are no facilities for them."

More than 1,000 wrecks lie off the Wexford coast, from Hook Head to Arklow. "It's called 'graveyard of a thousand wrecks'," says Parle. "We have seen 30 or 40 wrecks but we haven't even tipped on it. We do it for fun, but realised what an attraction this could be for the town."

Parle and his friends are hoping to open a dive centre in the Guillemot – an old lightship cemented into the harbour and maintains they are simply waiting for a grant to come through.

"We will start it as a voluntary group, and maybe eventually will be able to run it as a business, but that is not the first priority.

"There is a lot of doom and gloom around at the moment but if we get the proper support we can really make a go of this and could have up to 12 dive boats going out a day."

Chris Lawlor, owner of Kehoe's pub, the town's heartbeat, wants to let people know that rural Ireland is open for business.

He, his wife Ann, and three children, 14, 11 and eight, left Dublin three years ago "to get away from the M50." Originally from Dundrum, Lawlor ran various pubs in Dublin, but wanted a better life for his family outside the city.

"It wasn't too difficult to move. Kids are very good adaptors and they didn't mind leaving Dublin. There's a great sense of community here and the children have freedom and a healthy life. We are all in and out of each other's houses. It's a different way of life, a lot more laidback – maybe the way Dublin was years ago."

Now the town's only traditional pub, Kehoe's is a wonderful melting pot, buzzing with that friendly hum you only find in small towns and villages. It's not something you can fabricate, or replicate in the city, and if you never left the pub at all you'd learn all you need to know about Kilmore Quay and its inhabitants.

James Kehoe, the pub's namesake, spent almost 30 years diving and retrieving artefacts from wrecks in the nearby waters. The pub is adorned with maritime memorabilia. Lawlor didn't just buy a pub when he moved here, he bought into a lifestyle – the local pub, a thatched cottage, four horses – and he loves it. It's the Florida of Ireland, he says, and talks about the great weather, the wildlife and amenities with the passion and knowledge of a real local.

"This part of the country is really unique. It's been proven to be the warmest part, and there is flora and fauna growing here that doesn't anywhere else. I even have a customer who drives down from Dublin every weekend because he loves the food and the pace down here. My repeat business is a testament to the drawing power of the area."

Something else is needed though, to draw people to Kilmore Quay. There is no escaping it, every conversation comes back to two things– the future of fishing and a development plan.

"Look around," says Rob Roche, owner of the Saltee Hotel. "We are dying on our feet here. Roche has built residential developments in Rosslare, Wexford and Bunclody, but more importantly owns a prime site in Kilmore Quay that he wants to see utilised, sooner rather than later.

Actively involved in the town's development association, he wants to help attract an internationally branded hotel with a leisure centre and indoor facilities to make the town a year-round destination.

There is only one stumbling block, a hurdle that has prevented any kind of investment for many years now – infrastructure.

"The site is zoned for a hotel but we can't get the planning until the town's sewage treatment plant goes ahead. It all hinges on that. I have the land, and want to get the thing going. The only reason our applications for development have been turned down in the past is because of the town's sewage.

"Developers take one look, see the problem and move onto the next site. The funding has been earmarked. We are hoping it comes sooner rather than later.

"The only way forward is tourism, and to exploit our natural amenities. What else will sustain us?"

Ever the optimist, the town's self-appointed spokesman Nicky Cullen, with a glint in his eye, gets the last word in.

"I am thinking positively. Look at all we have to offer, someone will come up with a plan to save us."

Everyone here is definitely trying.