Accessing RAFCommands

[Lyffe]

I've been having considerable difficulty in accessing the forum recently and wonder if anyone else is having problems? For instance I lost it just before midnight on the 22nd and was still unable to get in around midday on 23rd (admittedly I was in bed between midnight and 8 am). Today, and especially this afternoon it's been almost impossible to use - I tried to post this around 5 pm but gave up as the system seemed to get itself in a loop when I tried to preview the post..

Brian

[Oldduffer]

No specific problems.

I live in the midlands of England and have appallingly awful internet speed (half a MIP) but that is nothing to do with the site. I usually find an acceptable time to connect.

Have you tried a broadband speed check to see if the problems are speed related?

Colin Cummings

[Ross_McNeill]

There is a bit of a feeding panic from the web crawlers at the moment.

Out of 51 users 37 are web bots yahoo, msn, google etc to this is added quite a few crawlers from China ISPs.

All these use up the limited access ports to the server,

I have been closing the board peridically to try to bump them off the connectiions but they are a necessary evil to index the board for search engines.

A few weeks ago 112 users were bots with only 15 registered/visitor users.

Regards
Ross

[Lyffe]

Thanks for responding Colin, but there are no problems there. I have no difficulty with any other forums/websites - in fact I watched an interview with Andy Murray immediately after my last hiccup. I know Peter (Resmorah) has been having similar problems - I'm in Wiltshire, he's in Berkshire.

Brian

[Lyffe]

Much obliged Ross - sorry these idiots are giving you so much unnecessary hassle.

Hope you didn't mind my asking.

Brian

[Ken MacLean]

I'm in Ottawa, Ontario. Past day and a half the site was very sloooooow. Now, at 5:58 p.m. ET on Thursday it is responding at warp-speed, which Captain Kirk assures me is very fast indeed.

[HughAHalliday]

I do not pretend to understand the explanation tendered by Ross (techno-idiot that I am) but his description of his work as a "necessary evil to index the board for search engines" makes sense.

I have found that people unfamiliar with RAF Commands will nevertheless find it through posts which are narrowly titled. By this I mean that a Thread Title such as "Need Help" will not likely be read by anyone unregistered with the site. On the other hand, a thread title such as "Orkney Air Crashes" or "Jane Cartoons" may be picked up within a week or two by search engines and read by the most unlikely people. I have been approached by people who, in doing genealogical research, have blundered upon a two-year old thread that mentioned the family name in its title.

For a demonstration, log off this site, then google "Short Bursts Newsletter".

[Resmoroh]

Lyffe/Hugh/Ross,
Assumed the problem was, if not, DoS attacks then getting perilous close to it!
I also find that a number of other, allied, historical aviation sites do, from time to time, exhibit the notice "Sorry, we are doing Essential Maintenance tasks". This would seem to be the same sort of thing?

Ross,
Do you have any advice as to how "Thread Titles" might/should be written (re Hugh's post) in order to reduce this unwanted nuisance? (I appreciate you can't eradicate it entirely).

HTH
Peter Davies

[HughAHalliday]

Perhaps I am reading Peter's last post incorrectly, but the ability of search engines to pick up posts on this forum (via thread titles) is hardly a "nuisance"; rather, it makes our little club more relevant to a wider audience. The family of Paul Gingras may never have heard of "RAF Commands", but if they were to google his name today they would find a relevant posting regarding his career - because his name appears in a thread title.

I believe Ross would agree that the processes and tasks he performs, which occasionally slow down the board for regular members like myself, is nevertheless useful to draw others to its contents. Am I wrong, respected moderator, web-master and arbiter of all things connected thereto ?

[Ross_McNeill]

As to thread/post titles:

The better search engines will index and search on any word or symbol in the body of the post but when the result is presented it has a short header of the first few words in the page/thread/post title.

Hence short titles such as "Beaufort xxxx" or "F/L Prune No.633 Sqn" are presented without shortening but "does anyone know a chap who was in the RAF in 1939 called I think Prune" will still be found on a search for prune but the title will be something like "does any one know a chap"

The casual button clicker will think that this result is not a good fit for their search.

The web spider/bot problem is with the more recent ones from India/China.

MSN/Yahoo/Google etc will only search a few levels from the original page so the number of access requests via the ports is self limiting.

The newer, less friendly, bots will try to click all the links on a page then will create new bots to follow the links on the linked pages etc. Soon numerous bots from the same source are all clicking links on the board as fast as their little electric tentacles will allow.

They also click the post reply buttons then index and link the "No permission" error pages . Does not take much to put them into a circular loop.

All the time this is going on the valid user/guest is not able to get access to a port so is presented with either the page not found internet page or a timeout error page.

It is impossible to ban all the bad search engine pages without hourly updates or reducing the access of the good guys hence my comment of necessary evil.

To this bot activity is added a background of spam bots doing the same thing with the registration pages trying to find valid ISP and user email combinations.

It is a trade off more ports will give better access but at a cost and that will effectively reduce the storage allocation reducing the space for the post archive.

At the moment I'm not overly concerned on the access speed as it will probably be cyclical.

Regards
Ross