2016 Exam Review- March 2017

[PJW]

Attached are the beginnings of my notes; certainly not finished but since it'll probably be a week or two before I can complete I decided that it was best to make available what I have got as soon as I reasonably could.

There is a lot of similarity with 2015 Exam Review which is why I took that report as the basis and started deleting / editing / retaining that content as applicable.  Then I re-started from the beginning and have covered all the general discussion, before the sections on the individual modules that I'll leave for a later update. 
Certainly there is significant content to add for both mod 1 and mod 3; probably not that much for mod 2. 
There was little if any coverage of mods 4/6/7 anyway. 
Need to check my notes for the end portion also, but I don't think there was much other than I have already included just from memory.

[Jerry1237]

Thank you Peter for you continual efforts. Much appreciated by many but far too few thanks offered.

It seems there is an opportunity to benefit the Institution in confirming whether those whose first language is not English didn't succeed due to language limitations or ability on the day. Should it be asked why the pass figures have not altered significantly with the new initiatives?

[PJW]

Thank you Peter for you continual efforts. Much appreciated by many but far too few thanks offered.

It seems there is an opportunity to benefit the Institution in confirming whether those whose first language is not English didn't succeed due to language limitations or ability on the day. Should it be asked why the pass figures have not altered significantly with the new initiatives?

I have the opportunity this year of having two Indians at Chippenham who unsuccessfully attempted a module each in 2016.   My perception at present is that the primary problem is narrowness of experience of railways and limited scope of the range of the designs they have worked on.

Actually the pass figures have noticeably increased; after two very poor years the pass rate is back to the 10 year average.  I agree it could have been even more dramatic.

[Jerry1237]

So, could part of the feedback be careful consideration of who is considered suitable for mentorship then? Agree with the committee a 100% pass rate is unacceptable and a to return to a norm is good but not what was aimed for and the non-pass rates seem to high still!

Should we encourage the language thing to be put to bed or at least to be formally reviewed/ratified? Based on the review comments I see, those who struggle do so at the most basic levels; lack of domain knowledge, reading the questions, sufficient preparation etc. A certain SW UK based group is a, and not the only, good example of what is needed to pass and what structured application does to candidates.

[PJW]

So, could part of the feedback be careful consideration of who is considered suitable for mentorship then? Agree with the committee a 100% pass rate is unacceptable and a to return to a norm is good but not what was aimed for and the non-pass rates seem to high still!

Should we encourage the language thing to be put to bed or at least to be formally reviewed/ratified? Based on the review comments I see, those who struggle do so at the most basic levels; lack of domain knowledge, reading the questions, sufficient preparation etc. A certain SW UK based group is a, and not the only, good example of what is needed to pass and what structured application does to candidates.

I think you are right, but in reality any of us who are approached have a difficult balancing act to perform.  One of the candidates I sponsored last year failed one of the two modules; the one that he came to me quite last minute to support instead of one we had been covering in the Study Group.  He had a rational argument that it was far better matched to his experience and indeed I am sure that starting in June that he COULD have got to the state that he would have achieved a good pass.  However I was not "on his case" since he is relatively senior and had decided to plough his own furrow separately to the group I was leading.  Probably distracted by work, probably over confident; certainly had the knowledge and experience but had not got himself exam ready.  Not really my fault, but still feel disappointed particularly because I had felt I had had "a gun at the head" needing to sign the form more or less there and then.  I had got him to attempt 3 questions over night and they were rather marginal, but having gone over them with him I felt that he'd got the message.  Obviously I was wrong.

A lot of those who are approached to sponsor will really not know what is expected and so are less likely to decline.  Even I am likely to be in an awkward position this year as a number of my candidates are a bit "left field"- experienced yes, but not in mainstream signal engineering.  It's good they want to do the exam, it would be good in a "broad church" to get them into the fold, but asking me in APRIL to make a judgement whether, having started to study in January that they are likely to make the grade in October just isn't reasonable.  In August I'd probably have a far better idea of their rate of progress.
Similarly the two Indians.  It makes total sense for them to try the exam again this year; whilst they are in the UK they have a better chance- more support, less travel time commuting, more spare time and less family responsibilities, so very logical to attempt in 2017 before they return home.  However I have little time to judge whether they'll be able to pass this year; the "safe" decision for me would be decline to do so as they are "high risk", but is that the right decision in the circumstances?

For some, language is most certainly part of the issue.  It cannot help when trying to get the nuances of the question.  It is not always the technical words that cause people to flounder.  It also limits how readily they can assimilate information from reference material.  Having to make an effort to interpret many words, construe a sentence means that it is very difficult to maintain an overview of the question as a whole. They see the individual trees, but not the wood.

[Jerry1237]

Totally agree that it is difficult to judge but the "gun held to head" is not a positive for the current "system". The results obviously depend on the day's performance and dedication to the task prior (too many underestimate the effort required to pass, let alone do well, in the exam). It would be interesting to see if any of the mentors needed a little support to understand the requirements placed on them and the expectations [of the Institution] on what level the candidates need to be at to sit it.

Would suggest the "lefties" often do well due to the additional effort they put in as signalling/telecoms/systems design is not the "day job". However, that should apply to many due to the diversity of the modules anyhow, not that all appreciate that fact.

Language is a funny thing. It would be good to talk about the issues of language in the exam over a cordial or two. Seems many native speakers struggle with some of the exam question's intentions so those who don't would also struggle. Certainly, I struggle at times with the technical version of my second language. It would also be interesting to hear how the examiners deal with a response that clearly has the jist but where the language is lacking.

The exam is certainly important and deservedly not simple to pass. Certainly would be interesting for an open discussion on ways the playing field could be levelled and ensure the candidates taking it are appropriate to the task.