26 Jan 2012; Review of 2011 Exam

[PJW]

Good point.

I have previously prodded the examiners regarding a similar situation which some Indian candidates face that basically utilise a colour light form of mechnacical signalling feeaturing "islands" around stations but relatively long block sections in between. Basic dilemma when asked to "signal to standrrds of your railway" and "provide 3 minute headway" for a railway that never has such a requirement!

There is a current initiative in the UK to develop a modern, low cost basic colour light system "modular signalling" for secondary routes to replace the remaining mechnical signalboxes controlling tiny areas and long axle counter blocks between sites. So it is quite possible to conceive that a sub-set of UK signal engineers will specialise in such; initially judged suitable for up to 5 trains per hour.


Probably outside the exam review that I think will focus on module 1 and module 3 because of poor results; I know that module 2 had a very respectable pass rate.
However your thought reinforces that for module 2 and 3 layouts, the IRSE ought to be at very least advising candidates how to approach if their railway "doesn't fit" the question paper. Metro candidates such as from London Underground have the same type of problem for doing Control Tables. So yes I'll certainly "add it to the list". I think I'll find out early in the New Year whether my offer of a paper for Aspect 2012 has been accepted; if so it'll be in, if not I shall find some other way to highlight.

Hi,

Would it be possible during or outside of the exam review to have a look at what would have been expected for Module 2 this year if speed signalling was taken as the perferred signalling type? Living in Victoria, Australia and the 160km/h line speeds with 2 minute headway was not possible with speed signalling. I think it could be an interesting discussion as to what measures should be taken, particularily in an exam scenario for these figues and it does make you think about the operations options available to you as a signalling engineer.

Would be great to see other thoughts on it.
Thanks

[Peter]

So yes I'll certainly "add it to the list". I think I'll find out early in the New Year whether my offer of a paper for Aspect 2012 has been accepted; if so it'll be in, if not I shall find some other way to highlight.

Peter

We offer to raise items at the exam review on behalf of people who cannot be there if people raise it here. Is there any merit in giving the exam committee sight of the issues so that they stand a better chance of encouraging suitable examiners to be there.

[PJW]

So yes I'll certainly "add it to the list". I think I'll find out early in the New Year whether my offer of a paper for Aspect 2012 has been accepted; if so it'll be in, if not I shall find some other way to highlight.

Peter

We offer to raise items at the exam review on behalf of people who cannot be there if people raise it here. Is there any merit in giving the exam committee sight of the issues so that they stand a better chance of encouraging suitable examiners to be there.

Definitely agree that there needs to be a means by which those who cannot get to London have a means of getting their question asked / point made. We offered to be the conduit and of course we'll honour that; I hope I didn't imply otherwise- perhaps it read too dissmissively unintentionally.

It is just that I am a little dubious of whether that is the best forum for this particular issue for the following reasons:
a) mod 1 results (I have heard consistent overall story from various different sources) imply attention will be correctly focussed there. I have been in contact with one of the relevant examiners and she has confirmed that they are planning some significant feedback session,
b) similarly if the rumours re mod 3 are true then that may also come in for more detailed feedback
c) mod 2 conversely seemed to have been done by most relatively easily.
d) the particular issue raised, which is very relevant to candidates from non-UK backgrounds, isn't one that affects those likely to be attending the event in London,
e) there is not nomally any mod 4 or 6 coverage; I think the numbers sitting could be a bit higher this year so may warrant.

Yes it probably is worth compiling a list of items that the students / candidates would like on the Agenda. Perhaps we do that early in the New Year by which time we will hopefully know some results; for the remainder of this year people will either be winding down for a long Christmas break or building up for a busy commissioning period and are unlikely to receptive either way.

[PJW]

Peter,
I hear that Tony Kornas as chairman of the Exam Committee will certainly be leading the attendance and has "adequate representation" from the committee.
Have suggested to Martin F that the YM may care to encourage those there at the earlier seminar to write things they want raised but would be uncomfortable in speaking out themselves, so that they can be asked anonymously.
I will be busy in India until just prior; can you take the initiative re your suggestion of giving forewarning of the issues that we know of

[PJW]

Since the candidates were sent a list of results tabulated per candidate number, this year it has been possible to get an overview of the results in advance of the meeting. If my adding up is correct, the mod 1 pass rate is actually a tad higher than that which was previously suggested and the results graphically are per the attached, errors and omissions excepted as they say.
No guarantee therefore that the numbers are 100% accurate, but as an overview I am sure they are good enough to show the picture from several perspectives.
Obviously the information missing is how the various exam centres compare with each other. My knowledge is limited to knowing that:
a) the number of candidates from the whole of Australia was similar to Bristol & Birmingham combined,
b) around half of the total modules sat were taken in India. Note that 2011 was only the second year in which there have been exam centres in India, so this growth has been very rapid. I hadn't realised quite what a significant percentage this now constituted; it may be one factor that has caused the recent change in the mod 1 results, but obviously not the only one since there was a distinct falling trend prior to this.

[PJW]

The IRSE have now supplied me with some data relating to the performance of candidates in particular modules sat at specific exam centres. It makes interesting comparisons and confirms my assertion that the Indian module 2 results are broadly on a par with the UK whereas most other modules are significantly worse.

I suppose that I am not so surprised about this in the context of module 1 & 7, but the module 3 results do seem most peculiar.
See attached presentation that illustrates how exam performance varies between UK and India (which have broadly comparable number of entries) and also the remainder of the world with rather less entries and thus more prone to a single persons result making a large statistical effect.

Note however that by treating countries as a whole does mean that some significant differences are not visible. The results of the admittedly small number of modules sat in Birmingham were almost always worse than the equivalent ones sat in India!

I compiled these graphs from various sources of information that do not seem to be 100 percent consistent. I do know that the original pdf listing sent to candidates did contain a few bogus allocations due to the legibility of certain candidates' handwriting which were discovered when the candidate numbers were matched with names and the modules for which they had applied which may explain. Certainly the overall totals of modules sat does not precisely accord with that given by the examiners at the exam review.

When I get a chance I will try to cross check my various sources, but whereas there are some discrepancies to resolve, I am sure that the overall picture presented is good, although where numbers are small a mis-recorded result can obviously make a significant difference on a percentage basis

[Jerry1237]

The results seem to indicate:
1) the exam isn't easy;
2) the failure rate decreases for the more specialised modules;
3) it would appear people are applying before they are ready (but no indiciation if that is coporate pressure or misunderstanding of the level required) but the number of near misses is interesting;
4) there needs to be an audit on one of the older M1 papers versus the last two years for complexity.

It would probably be useful to discuss with the exam panel whether the level of rigour has changed with respect to marking the papers. However, that the standard of paper appears not to have dropped and the Institution is not trying to massage figures gives me great encouragement. We [as an industry] need to address how to train, mentor and support people into improving their level of skill which will allow them to pass the exam.

Jerry

[PJW]

The results seem to indicate:
1) the exam isn't easy;
2) the failure rate decreases for the more specialised modules;
3) it would appear people are applying before they are ready (but no indiciation if that is coporate pressure or misunderstanding of the level required) but the number of near misses is interesting;
4) there needs to be an audit on one of the older M1 papers versus the last two years for complexity.

It would probably be useful to discuss with the exam panel whether the level of rigour has changed with respect to marking the papers. However, that the standard of paper appears not to have dropped and the Institution is not trying to massage figures gives me great encouragement. We [as an industry] need to address how to train, mentor and support people into improving their level of skill which will allow them to pass the exam.

Jerry

I now know that in more than one company in India there is corporate pressure on those who joined the industry only 6 months ago to attempt all 4 modules in 2012.

Thinking back to BR days when a graduate (in 1980s terms rather a rarer commodity than in 2011 terms) joined the industry they were spoon fed a training scheme deliberately designed to give a wide experience for 2 years, during which time expected to do nothing really productive. Then a year in their first appointment, including attending a pair of 2 week courses specifically geared to the examination, so they would sit the equivalent of 4 modules after 3 years. It is just not reasonable to expect those in what is probably a very production focussed, insular design office environment to achieve the same within the same period- most certainly not within just one year of employment in the industry. No wonder the failure rate is so appalling.

[PJW]

I took quite a lot of notes last night; the IRSE was not recording the Exam Review by video or audio, but Christine was there also taking notes so I guess the official minutes will appear eventually. I aim to get my unofficial notes posted here within the week.

However the immediate headlines:

1. Module 1 exam for 2012 will follow the format of 2011; there now needs to be a period of stability following the recent changes.

2. Module 2 exam for 2012 will follow the format of 2011 and indeed has been unchanged for many years. However the examiners did undertake to consider the situation for candidates from railways which do not have lines with the level of capacity that is usually requested . This may be advice regarding what they should do re the braking and headway portion of the question, or potentially may be by provision of another blank layout (in addition to the current "Mainline" and "Metro") more appropriate to that kind of railway.

3. Module 3 exam for 2012 will also follow the format of 2012; in particular the same IRSE blank format control tables will be available in 2012- these were used by some 50 percent of candidates, with the remainder bringing in their own which will also continue to be permitted, however with the additional stipulation that SUCH BLANKS MUST NOT INDICATE ANY COMPANY AFFILIATION which could inadvertantly give clues to the identity of the candidate.

However the examiners did indicate that they would be prepared to discuss the possibility of making the layout provided for the Control Tables less UK mainline specific, or possibly providing an alternative one to suit candidates from elsewhere should there be a significant number of applicants from a specific railway administration.

4. Modules 4,6,7 will similarly remain of unchanged format.

5. Modules 5 will remain of broadly unchanged format BUT
CANDIDATES ARE ADVISED TO COVER THE ENTIRE SYLLABUS AND NOT RELY THAT PARTICULAR QUESTIONS THAT HAVE TRADITIONALLY BEEN SET WILL APPEAR EVERY YEAR.
In particular the numbers of answers to the TC calculation and level crossing questions in 2011 were very high, suggesting that candidates have focussed their studies mainly on these limited areas rather than being capable of addressing questions over a significant range of the syllabus. It therefore seems likely that the examiners will omit at least one of these questions in the 2012 exam; the advice was very clear- make sure that your preparation for the exam is sufficiently wide that you still have a decent choice of questions whould neither of these "regular bankers" feature.
YOU HAVE BEEN WARNED IN PLENTY OF TIME!

[PJW]

Attached are my notes from the Exam Review Meeting.

I took quite a lot of notes last night......I'll aim to get my unofficial notes posted here within the week.

However the most important items:

Module 3- Candidate's blank Control Tables MUST NOT INDICATE ANY COMPANY AFFILIATION.

Modules 5- CANDIDATES ARE ADVISED TO COVER THE ENTIRE SYLLABUS AND NOT RELY ON PARTICULAR REGULAR QUESTIONS (e.g. Track calculations , Level Crossing operation).

2012 Results will not be published until mid-late JANUARY 2013.