As a result of the SQA’s failure to answer our direct questions, we sent a second letter pointing out the inadequacy and inappropriateness of the SQA’s first response and reiterating that we required specific answers to these questions.
The SQA’s reply stated that the RMPS Assessment Panel at their Autumn meeting had “considered our “feedback” in full with respect to this year’s examination ( i.e.2010) and confirmed that the questions were valid.” In fact this was not just “feedback” that I gave to the SQA but was a professional criticism of the validity and reliability of many of these assessments and marking instructions. I would therefore have expected a detailed and professional response to those substantiated criticisms. However, the SQA did not provide this and I subsequently also learned that our 70 page submission had not even been circulated or discussed at this meeting and that only our two page cover letter had been presented. How then could the SQA claim that they had “discussed the submission in full?
It was clear from this that the SQA representatives had withheld our full submission from Panel members. This has serious implications for the manner in which the SQA has dealt with these issues as it was now clear that “openness and transparency” from the SQA apparently does not apply to answering direct questions about the validity of their assessments.
Over the following 12 months (from October 2010 until October 2011) the school pursued these issues with the SQA, including a detailed letter to the Chief Executive herself explaining the SQA’s failure to answer our questions and requesting her to investigate the circumstances of the SQA’s failure to do so. However, even the Chief Executive refused to answer our questions and simply reiterated the SQA default position that these assessments were valid and reliable thus continuing to provide unsubstantiated claims concerning these issues.
Given this impasse, the Rector continued to insist that the SQA should give us a written answer to our many complaints and questions and we eventually received what the SQA termed “a full response” from the Director of Qualifications Development in early October 2011 (15 months after our initial correspondence with them). In this response none of our 196 questions was answered. However, the SQA stated that the issues we had raised “demonstrated a misunderstanding of the SQA approach to assessment” but did not explain why; it was simply another unsubstantiated statement and avoidance of the issues.
Following this totally inadequate and mostly irrelevant response, I personally replied to the SQA Director of Qualifications Development asking a number of further questions particularly about why she had not answered these specific complaints and questions. I also requested her to clarify several points she had made which were contrary to information given in the SQA Arrangements for the Higher RMPS Course. She failed to respond to these questions and stated that the SQA considered the matter closed and that, should I wish to pursue this, I should do so via the Ombudsman.
It is pertinent to state that, during the correspondence process with the SQA, it was apparent to us that the SQA were giving us misleading, irrelevant, contradictory and erroneous information; this was in addition to their significant procrastination in responding to us on a number of occasions and also making unsubstantiated statements regarding what they termed our “misunderstanding of the SQA examination processes.” We have documentary evidence to support these claims and, in addition, when I contacted the SQA requesting more specific information via the Freedom of Information Act, they continued to fail to provide answers to a number of these further questions in relation to the validity of questions and marking instructions.
The Scottish Public Services Ombudsman (SPSO’s) “investigation” of my complaints against the SQA
The investigation of my complaints carried out by the SPSO failed to resolve the issues. This was mainly due to the fact that the SQA submitted a legal objection to the SPSO’s jurisdiction to investigate them on the basis that, as I was not a candidate who sat these examinations, then I could not directly claim injustice or hardship in relation to the actual questions in these assessments. This objection was upheld by the Ombudsman. Consequently, the SPSO did not investigate my claims of maladministration and service failure by the SQA in this Course.
It would appear from this Ombudsman ruling that teachers or schools have no recourse to seek independent scrutiny of the quality and accuracy of the SQA examinations via the SPSO. As teachers and schools can also be directly affected by SQA assessment maladministration, this seems an unfair and significant injustice within the current SPSO 2002 Act. I therefore request that this anomaly be addressed by the Scottish Parliament.
However, the Ombudsman did uphold other aspects of my complaints, namely that the SQA had not responded reasonably to these complaints in relation to the validity and reliability of the assessment questions and marking instructions. But, as the role of the SPSO is limited to recommending that decisions (by the SQA) be re-taken they accepted that, given the specific nature of these issues and the time lapse since they originally emerged (it was now almost two years since we had first raised them and this was solely due to the unnecessarily extended correspondence period of 17 months primarily due to the SQA’s continued refusal to answer our questions) that this was virtually impossible, impractical and ineffective. It was far too late to make any effective difference to the actual examination or the results that candidates received. So, despite the fact that the SPSO had upheld this part of my complaint, they could not recommend that the SQA answer these direct questions we had asked. The Ombudsman also stated that the SQA were not required to answer the specific complaints we had made about the examination questions nor to explain why they had produced large numbers of marking instructions which, in our view, were wrong, inappropriate or irrelevant to the questions asked. However, the SPSO did not explain why this was the case. This is clearly an indictment on the effectiveness of the Ombudsman’s office that it cannot require the SQA (a public service organisation and sole examinations provider for Scotland) to be accountable for the services they provide. Thus, the original complaints remain unresolved. This is a further reason I am now requesting that the Scottish Parliament takes steps to set up an independent regulatory body to investigate such issues concerning SQA assessments and marking instructions which fail to comply with the SQA’s own Arrangements for the Higher RMPS (or, indeed, any other) Course.
Correspondence with the Cabinet Secretary for Education and Lifelong Learning and the Minister for Learning, Science and Scotland’s Languages
As a result of the failure of the SQA to answer these complaints, and the SPSO’s inability to investigate the issues of alleged maladministration and service failure by the SQA, I contacted my local MSP in August 2012. Consequently, my MSP wrote to the Cabinet Secretary for Education and Lifelong Learning to explain my concerns accompanied by a detailed account outlining the history of my communications with the SQA and explaining the lack of resolution of the questions and issues I had raised via the Rector of the school. The response we received from the Minister of Learning, Science and Scotland’s Languages, who has responsibility for the SQA, did not take us one step nearer a resolution of these issues. This was principally due to the fact that the Minister did not request the evidence of my detailed analysis of these assessments despite my offer to provide this for investigation. Had such an investigation been carried out, I am certain that it would have provided clear and unequivocal evidence of a wide range of errors in these assessments and marking instructions to justify my claims of maladministration. Instead, he relied totally on an SQA response which was virtually identical to one that the SQA had previously sent to us, which also failed to address and answer these questions and issues. Consequently, I did not regard this as an appropriate response from a Government Minister with responsibility for the SQA.
It should also be noted that, while the details I have presented initially relate to the Higher RMPS assessments during academic session 2009/10, the majority of the complaints and issues also relate to subsequent academic sessions. This is because the SQA have used the same internal assessments (NABs) from session 2010/11 onwards and these continue to exhibit the same errors as those in session 2009/10. Additionally, the 2011 and 2012 final external examination papers have many similar errors as the SQA have continued to produce questions which do not reflect the mandatory content or assessment criteria of the Course.
The processes we have been through have clearly confirmed that currently, there are no structures or avenues available to teachers and schools to which they can refer in order to request an independent investigation of such complaints. The SQA are both judge and jury of their own assessments and marking instructions but they refuse to explain or justify them when requested to do so. Additionally, the SPSO cannot investigate such matters because of the SQA’s legal objection to do so and now even the Government Minister responsible for the SQA is not prepared to initiate such an investigation of alleged maladministration and service failure in their provision of the national examinations service.
This is indicative of a totally unjust and unacceptable situation currently operating in Scotland which is protecting the SQA from transparency and accountability for their practices and procedures and also from independent public scrutiny of the examinations and assessments they produce.