2015 Results

[StrongLifts5x5]

Eagerly waiting for Results so that I can plan for 2016 examination....

I know that they went before Council last week but also that there may be a bit more of a delay before being sent out because of personal illness - do not forget that the IRSE is very dependent upon volunteers to undertake many activities.
Overall I hear that the results across many if not all modules are disappointing with lower pass rates than 2014 so statistically the chances aren't good; however as ever some have done well so bucking the trend.

I don't think people should delay in studying waiting for results; some I know have determined to start studying for another module in 2016 taking the view that if they didn't pass those they took in 2015 then they can either decide to resist that alongside the 2016 ones or revisit in 2017- seems a sensible approach to me.

Peter I know its staffed by volunteers but its still very disappointing that it takes so long- approaching the end of the 5th month- people have lives i.e kids, holidays, careers etc that they have live, and having to think what dates can I arrange study time and exams is hard enough without fully knowing what or even if I will be studying.

The examiners are a fantastic bunch and the blame does not lie with them its with the IRSE- the IRSE has to either reduce the amount of people doing the exam by say having a pre-check as to whether a candidate is of a high enough standard to do the exam which will increase overall pass marks, or the IRSE has to increase the resources if provides for marking exams and getting feedback students!!

Sorry for the rant!!

[dorothy.pipet]

Will the delay owing to illness mean that the exam review date will move?

[PJW]

Will the delay owing to illness mean that the exam review date will move?

Don't think so. The date was agreed at the time when it wasn't actually certain that the results would even go before Council in February; we got the examiners to agree that broad feedback could be given even if the individual results were yet to be "rubber stamped".  
The overall figures re the pass percentages for each module are known- it is just the availability of the formally checked master record that can be used to inform individuals which is the current issue.

It is all about risk, late changes and version control.  If the figures for a module given at the session are wrong by a percent or two, it is barely significant to anyone as the overall picture really does not change.  If however one candidate is misinformed, then to them the error is huge.

[PJW]

Eagerly waiting for Results so that I can plan for 2016 examination....

I know that they went before Council last week but also that there may be a bit more of a delay before being sent out because of personal illness - do not forget that the IRSE is very dependent upon volunteers to undertake many activities.
Overall I hear that the results across many if not all modules are disappointing with lower pass rates than 2014 so statistically the chances aren't good; however as ever some have done well so bucking the trend.

I don't think people should delay in studying waiting for results; some I know have determined to start studying for another module in 2016 taking the view that if they didn't pass those they took in 2015 then they can either decide to resist that alongside the 2016 ones or revisit in 2017- seems a sensible approach to me.

Peter I know its staffed by volunteers but its still very disappointing that it takes so long- approaching the end of the 5th month- people have lives i.e kids, holidays, careers etc that they have live, and having to think what dates can I arrange study time and exams is hard enough without fully knowing what or even if I will be studying.

The examiners are a fantastic bunch and the blame does not lie with them its with the IRSE- the IRSE has to either reduce the amount of people doing the exam by say having a pre-check as to whether a candidate is of a high enough standard to do the exam which will increase overall pass marks, or the IRSE has to increase the resources if provides for marking exams and getting feedback students!!

Sorry for the rant!!

Yes it is and I agree that it really isn't acceptable and I have indeed voiced that opinion.

The IRSE has actually significantly increased the number of examiners.  However it is partly the law of diminishing returns and this then increases the amount of co-ordinated effort between them to ensure insistence and fairness and the logistics increasing in complexity exponentially making dates even harder to agree and thus pushing out timescale for other reasons.

The answer is certainly the former; the problem is not primarily the number of modules to mark- it is their quality.  I have spoken to three different examiners over the last couple of weeks and getting the same story from all.  It takes far longer to deal with a poor quality paper than a good one.  The disparity in marks given by examiners proportionally differs far more and therefore triggers extra effort to adjudicate between the scores.  The personal motivation for people who are in the final analysis volunteers in their spare time to wade through a large pile of generally poor offerings is lowered to the extent that several are asking themselves why they are doing it.   Frankly continuing how things currently are is not sustainable.  I had a similar conversation over a pint or two yesterday evening with the Chairman of the Exam committee.

On the other hand I have helped some people over the years who have repeatedly failed one or more modules of the exam and I do have quite a deal of sympathy with their plight. In general, and there are some notable exceptions, there is not a lot of active support from employers in the workplace.  The nature of the industry in the UK (and let's admit it a very high percentage of people from places such as India who sit the exam are fundamentally in the sub-contract chain from the UK industry) is very "project delivery, achieve next deadline, minimise costs in the short term" focussed; it is not like the more long term, stable, "worth investing in people for the future, doing things that are the right thing to do even if no immediate payback" railway that I joined 35 years ago.  Not the same loyalty from employer to employee and vice-versa as there was.  I know several intelligent people who seem capable in their day job who have yet to get through the exam and in some cases have evidently given up. In many cases I put this down to lack of adequate effort and exam focussed preparation; some of this might flow from changes of expectation and the general education system. When running Study Groups I find that some 50% drop out when they realise the level of work needed as  I pile on the pressure; one could argue that this is a good thing if they subsequently don't sit the exam only to fail it. However not  everyone has someone that  keeps their nose  on the grindstone when perhaps that is what they need! Others are employed in one small niche of the industry without much opportunity to experience a range of roles, range of technology, different stages in the lifecycle; it is definitely very difficult for them. The sad thing is that they are really not in a position to be able to succeed in the exam; it is not their fault, there is little they can do about it but given their situation the hard truth is that the IRSE exam is not appropriate to their situation.  The exam is doing exactly what it is supposed to, sorting the sheep from the goats.  The same person in a different scenario could very likely find the exam much easier so it isn't so much about the innate potential as the accumulated experience and unless the person has been able to acquire that experience by having the right exposure, it is hardly surprising that when being asked to demonstrate it that they cannot.


It is certainly a complex problem and the current status is very unsatisfactory.  The IRSE most certainly recognises this; there is a small working group (including the IRSE president so very much a current focus of attention) currently trying to propose a workable way of weeding out those who are really not ready to sit the exam.  It is actually quite a tricky problem as we don't want to block anyone unnecessarily nor discourage interest.  It is also not going to be easy to construct an initial precursor paper for the following reasons:
a) there is no common body of knowledge that can readily be tested which is appropriate to people from a variety of types of railways and countries
b) it is often not the lack of basic knowledge which is actually leading to exam failure.  Far more about not being able to interpret correctly what the question requires, not being able to connect isolated facts to construct a full understanding, not being able to present an answer clearly and effectively within the time constraint.

In fact I will be attending the second such session on Friday afternoon; the recent Survey Monkey exercise which those who applied for the 2015 exam was as a result of our first meeting.  On Friday We will be analysing the outputs of this in order to see what evidence it gives to support / refute our various working assumptions.  We do have a couple of proposals, none of which is a panacea and none without disadvantages but perhaps some or all will emerge as "the least bad".

However if you have any specific suggestion to add into the brainstorm session, then please feel free and share it.

[StrongLifts5x5]

Eagerly waiting for Results so that I can plan for 2016 examination....

I know that they went before Council last week but also that there may be a bit more of a delay before being sent out because of personal illness - do not forget that the IRSE is very dependent upon volunteers to undertake many activities.
Overall I hear that the results across many if not all modules are disappointing with lower pass rates than 2014 so statistically the chances aren't good; however as ever some have done well so bucking the trend.

I don't think people should delay in studying waiting for results; some I know have determined to start studying for another module in 2016 taking the view that if they didn't pass those they took in 2015 then they can either decide to resist that alongside the 2016 ones or revisit in 2017- seems a sensible approach to me.

Peter I know its staffed by volunteers but its still very disappointing that it takes so long- approaching the end of the 5th month- people have lives i.e kids, holidays, careers etc that they have live, and having to think what dates can I arrange study time and exams is hard enough without fully knowing what or even if I will be studying.

The examiners are a fantastic bunch and the blame does not lie with them its with the IRSE- the IRSE has to either reduce the amount of people doing the exam by say having a pre-check as to whether a candidate is of a high enough standard to do the exam which will increase overall pass marks, or the IRSE has to increase the resources if provides for marking exams and getting feedback students!!

Sorry for the rant!!

Yes it is and I agree that it really isn't acceptable and I have indeed voiced that opinion.

The IRSE has actually significantly increased the number of examiners.  However it is partly the law of diminishing returns and this then increases the amount of co-ordinated effort between them to ensure insistence and fairness and the logistics increasing in complexity exponentially making dates even harder to agree and thus pushing out timescale for other reasons.

The answer is certainly the former; the problem is not primarily the number of modules to mark- it is their quality.  I have spoken to three different examiners over the last couple of weeks and getting the same story from all.  It takes far longer to deal with a poor quality paper than a good one.  The disparity in marks given by examiners proportionally differs far more and therefore triggers extra effort to adjudicate between the scores.  The personal motivation for people who are in the final analysis volunteers in their spare time to wade through a large pile of generally poor offerings is lowered to the extent that several are asking themselves why they are doing it.   Frankly continuing how things currently are is not sustainable.  I had a similar conversation over a pint or two yesterday evening with the Chairman of the Exam committee.

On the other hand I have helped some people over the years who have repeatedly failed one or more modules of the exam and I do have quite a deal of sympathy with their plight. In general, and there are some notable exceptions, there is not a lot of active support from employers in the workplace.  The nature of the industry in the UK (and let's admit it a very high percentage of people from places such as India who sit the exam are fundamentally in the sub-contract chain from the UK industry) is very "project delivery, achieve next deadline, minimise costs in the short term" focussed; it is not like the more long term, stable, "worth investing in people for the future, doing things that are the right thing to do even if no immediate payback" railway that I joined 35 years ago.  Not the same loyalty from employer to employee and vice-versa as there was.  I know several intelligent people who seem capable in their day job who have yet to get through the exam and in some cases have evidently given up. In many cases I put this down to lack of adequate effort and exam focussed preparation; some of this might flow from changes of expectation and the general education system. When running Study Groups I find that some 50% drop out when they realise the level of work needed as  I pile on the pressure; one could argue that this is a good thing if they subsequently don't sit the exam only to fail it. However not  everyone has someone that  keeps their nose  on the grindstone when perhaps that is what they need! Others are employed in one small niche of the industry without much opportunity to experience a range of roles, range of technology, different stages in the lifecycle; it is definitely very difficult for them. The sad thing is that they are really not in a position to be able to succeed in the exam; it is not their fault, there is little they can do about it but given their situation the hard truth is that the IRSE exam is not appropriate to their situation.  The exam is doing exactly what it is supposed to, sorting the sheep from the goats.  The same person in a different scenario could very likely find the exam much easier so it isn't so much about the innate potential as the accumulated experience and unless the person has been able to acquire that experience by having the right exposure, it is hardly surprising that when being asked to demonstrate it that they cannot.


It is certainly a complex problem and the current status is very unsatisfactory.  The IRSE most certainly recognises this; there is a small working group (including the IRSE president so very much a current focus of attention) currently trying to propose a workable way of weeding out those who are really not ready to sit the exam.  It is actually quite a tricky problem as we don't want to block anyone unnecessarily nor discourage interest.  It is also not going to be easy to construct an initial precursor paper for the following reasons:
a) there is no common body of knowledge that can readily be tested which is appropriate to people from a variety of types of railways and countries
b) it is often not the lack of basic knowledge which is actually leading to exam failure.  Far more about not being able to interpret correctly what the question requires, not being able to connect isolated facts to construct a full understanding, not being able to present an answer clearly and effectively within the time constraint.

In fact I will be attending the second such session on Friday afternoon; the recent Survey Monkey exercise which those who applied for the 2015 exam was as a result of our first meeting.  On Friday We will be analysing the outputs of this in order to see what evidence it gives to support / refute our various working assumptions.  We do have a couple of proposals, none of which is a panacea and none without disadvantages but perhaps some or all will emerge as "the least bad".

However if you have any specific suggestion to add into the brainstorm session, then please feel free and share it.

Peter

Thanks for your very detailed and informative reply, it has certainly made me think about the range of knowledge and experience of candidates, I don't want to count my chickens and jinx myself as all I need is Mod 1 to go for full membership I do have some thoughts and suggestions but I'm sure finer minds than mine  will be working on solutions.

I like your goats from sheep comments and from my own experience I had 20 years signalling experience 10 years at a managerial grade before I started the exam, maybe thought should be given to ensure candidates have a sound grounding and are not fresh out of university and looking to gain membership to further their career before they have a sound signalling knowledge, call me old fashioned. I also appreciate a candidate could be say a project manager and therefore have little signalling principles and studying for the exam will greatly tie their knowledge together as it has for an old 40 something like me!

Also someone with several years service is obviously as we used to call a railwayman and will wait until the IRSE decides they are ready to start, improving the candidates will increase marking rates and stop the examiners from getting frustrated. Just a thought as I feel too many unsuitable candidates are taking the exam, I so hope I haven't jinxed myself!

Hopefully it's resolved as its obvious the numbers seeking to start the exam will keep growing, anyway Peter thanks.

[mohanakrishnan]

Hi Peter,

I have a suggestion which you may like to discuss during the exam review session.

The exams can be conducted twice a year. Once in April and Once in October. We can have three modules in April - say Module 2, 4 and 7 and four modules in October Module 1, 3, 5 and 6. Overall in a year the same seven module question papers have to set and corrected but conducting two times a year will reduce the resource requirement and will help the candidate to focus on couple of modules in April and couple of module in October rather than all in one go. The results can also be announced much quicker - may be 2-3 months rather than 5 months. Knowing that April exams are for different modules candidates will not wait for Oct results and vice versa and hence preparation can start immediately after the previous exams. Thorrowgood Scholarship award can be given to the candidate whose overall score in the year i.e.April + October is higher.
I am sure the logistics needed for conducting twice a year is more but probably worth. This might also increase the awareness of IRSE exams as it happens more often and people know about it more.


Mohan

[dorothy.pipet]

Mohan,
I'm not sure I see how that would help the examiners. [PJW do you know how many, if any, examiners work on more than one paper?]. I think it may help some candidates, others will try and prepare in too short a time and we'll get more fails. I already can only manage one paper a year to do well. Worth discussing but may not help the underlying issue.
I hear that Thorrowgood Scholarships already take into account those who have achieved outstanding results but not all in the same year.

So in my view the things we need
* more examiners, without burying the benefit in co-ordinating and invigilating between them;
* Candidates who take the exam when they are ready i.e. better preparation of those who take the exam and few trying to take it to early in their railway careers.

And as I see it we need those who have passed well in recent years to both become examiners and help the candidates.

[PJW]

Mohan,
I'm not sure I see how that would help the examiners. [PJW do you know how many, if any, examiners work on more than one paper?]. I think it may help some candidates, others will try and prepare in too short a time and we'll get more fails. I already can only manage one paper a year to do well. Worth discussing but may not help the underlying issue.
I hear that Thorrowgood Scholarships already take into account those who have achieved outstanding results but not all in the same year.

So in my view the things we need
* more examiners, without burying the benefit in co-ordinating and invigilating between them;
* Candidates who take the exam when they are ready i.e. better preparation of those who take the exam and few trying to take it to early in their railway careers.

And as I see it we need those who have passed well in recent years to both become examiners and help the candidates.

No examiners mark more than one paper.
The prime difficulty seems to be that people underestimate the preparation they need to do prior to the exam- we suspected this before but this is also what the survey results told us.  
Twice a year exams would increase costs for everyone, require more volunteers to give up time for invigulating etc.; given that the reason why we seem to be in the current situation is that inadequate studying and practice gets done, I don't see the solution is to change the exam.  The one thing we might do is to keep it to be annual but only certain modules certain years........ 

If in a year where there is no one worthy of a Thorrowgood, then the examiners do look to see if there is anyone who has achieved a suitable level of performance spread over that year and the previous one, but the emphasis is on those doing all 4 modules at once.  I advise people who are perhaps thinking of it to study for modules, do not sit, then study for more modules and when they feel they are ready then sit the lot altogether.

As for your * points:
*1.  Does happen and probably at the level for most modules that caouldn't readily be extended
*2.  That is where our focus now is.  Watch out for forthcoming IRSE News.

Either help, or be examiners- not both!

Since both the existing mod 2 examiners are into their retirement, I am sure that fresh blood is needed- be careful what you wish for!

[dorothy.pipet]

I agree people underestimate the exam prep required, plus I think most of them don't take it in when we try and tell them!

I also agree that the focus is more helpfully on the candidates (and I did mean Either help OR examine, poorly phrased, sorry).

I wondered if a set of preparatory tests, for candidates to use to test their readiness would help? Something they can use as their own tool to say "If I can't do well enough at this then I'm not ready yet". Not really thought through what form it would take or how often it would need to be updated though.

[PJW]

I agree people underestimate the exam prep required, plus I think most of them don't take it in when we try and tell them!

I also agree that the focus is more helpfully on the candidates (and I did mean Either help OR examine, poorly phrased, sorry).

I wondered if a set of preparatory tests, for candidates to use to test their readiness would help? Something they can use as their own tool to say "If I can't do well enough at this then I'm not ready yet".  Not really thought through what form it would take or how often it would need to be updated though.

A series of preparatory tests have been discussed many times I think.  In principle it is a good idea, but as ever the "devil is in the detail" and can't devise something that would be practicable.

We are all used to online tests which are ok for testing something very specific such as PTS knowledge, but making it accessible to all from different countries, different roles, different type of railway isn't going to be easy.  The keeping it fresh and updated makes it even more difficult if not impossible.  Only real option would be to make people study " a set book" such as an IRSE textbook and ask questions on that.

However I do like your twist on the idea of giving a tool to the student to use rather than treating it as an initial part of the exam itself.  This certainly does overcome some, but not all, of the difficulties.  It may have some mileage but I do feel that those who should use it would be the least likely to do so.After all we have never had huge quantities of attempts posted on this Forum or indeed had much response when asking for attempted answers prior to a YM IRSE event in  the UK.

The real killer for any automated test or self-check template is that I can't see it could do more than assess level of knowledge, so even if someone passes this and does really have adequate knowledge of facts, it does not mean that they would be able to pass the exam.  As I understand it whereas some candidates grasp of facts is poor, those aren't the really time consuming ones to mark; it is those who write lots of stuff that is reasonably accurate but barely relevant to the question which poses the real problem to the examiner.  The problem seems to be a lack of deeper understanding, inability to see what the question is really requiring and poor technique in answering.  

It's hard to see there is any pre-qualification test of these, other than something akin to an IRSE exam itself; which brings us back to how many actually practice past papers and show their attempts to anyone else?  Hence this is where we feel the thrust must be.

Reminds me- I am still carrying around one of your attempts- really must find the time to respond; now that I've done what I needed t for that recent meeting, it can hopefully get higher up my action pile!