Welcome to the General Aviation Advocacy Group of New Zealand

Archives for March 2017

Client Satisfaction Surveys in a post-factual world…

The CAA is either living in a fools’ paradise and thinks all its customers are happy, or it is deeply afraid to ask them. We can’t think of any other reason why the CAA refuses to hire a neutral organisation and tell it go to the customers and ask some honest questions. We also have convincing proof that no company or organisation can rely on anything other than an independent monitoring of its customers.

Here’s why…

In January this year, we wrote to the Chairman of the CAA Board pointing out that the Authority engaged Colmar Brunton to undertake a client satisfaction survey in 1995 and 1998; five years later in 2003, the same research company carried out a similar survey. Since then, there have been no further CAA Client Satisfaction Surveys specifically targeting all CAA document holders. The 2003 survey’s results did not heap praise upon the CAA.

In his reply, the Chairman said:

“Your interest in surveys is topical as the CAA will shortly be repeating its flagship ‘Feel Safe’ survey conducted by Colmar Brunton. Management are currently planning on supplementing the survey, or even splitting its content, so that while retaining its prior focus on the CAA’s ultimate customer (the public) it also provides more information from the aviation sector”.

OK, it’s costing you $4.50 a minute. But do have a problem with me?

The Colmar Brunton “Feel Safe” surveys have typically only contained results from 300 to 400 respondents. We have written to the Chairman and asked that all CAA document holders be made aware of his planned consumer survey. But we reminded Mr Gould that this is just a consumer survey, and what is actually required is a CAA document-holder survey related to specific client issues, and promoted via the CAA’s email user database and Vector magazine.

Post-audit or inspection feedback to the CAA

In our letter to the Chairman, we pointed out that MoBIE has for many years operated a system in which, after every investigation, an inspector is required to send a client satisfaction survey to all parties involved. Something similar would be a valuable tool for the CAA Director for feedback about, to quote just one example, the knowledge and performance of auditors and the way they interact with aircraft operators and aircraft maintenance facilities. A client feedback system also provides useful information to senior managers of departments to gauge the performance of individual staff members as part of their yearly performance reviews.

The Chairman said:

“… at the end of each audit or inspection, the CAA already surveys the parties involved in order to gather information on their views on the effectiveness and efficiency of the regulatory intervention they have just been involved in. While that provides limited information about the performance of one regulatory function, we are interested in gathering a broader range of views to inform risk management and decision-making. Both the management and Board of the Authority recognize the importance of such information from the sector to inform our planning and improvement efforts. In part, this is one of the reasons we place so much importance on the relationship with representative bodies like the Aviation Community Advisory Group.”

We then sought to verify what the Chairman had claimed was correct – “the CAA already surveys the parties involved in order to gather information on their views on the effectiveness and efficiency of the regulatory intervention they have just been involved in”.

We asked a number of operators if what the Chairman claimed (or had been told by his officials) actually happened in practice.

  1. Did the Chairman’s statement have validity?
  1. Did operators feel comfortable providing feedback to the CAA when their AOC is at stake?

We believe the replies below speak for themselves, and we’d welcome further comments.

  • No, I’m not aware of any statement like that. Imagine if an operator criticised them at that point, they could be deemed not fit or proper and their AOC withdrawn or delayed when the auditors got back to Wellington. Yes there is a discussion, but who would dare to complain? It proves how out of touch the Chairman is.
  • I never had the auditors ask me how the audit went or what I thought of CAA. If you wanted a good audit report you would never consider expressing your thoughts on that issue. Ask the Chairman for some of the recorded comments.
  • No, I haven’t been asked for feedback (that’s not to say I haven’t given verbal feedback to auditors at a subsequent audit however!).
  • I am not aware of any formal feedback system. At one point during an audit, I found I was being asked the same questions at each audit such as “Is your fax number still xxxxxxxxx. Is your mailing address still xxxxxxxxx. Is your phone number still xxxxxxxx. When I’m paying over $4.50 PER MINUTE for CAA to ask me the same questions at each audit to which there was no change, I wasn’t happy. I suggested they send me an A4 sheet with those questions on it prior to the audit and I could go through and tick the boxes, thus saving valuable minutes. They thought that was a great idea but nothing was done.
  • I have never been asked at all, fair to say I have offered my opinion on a number of occasions regarding their efficiency levels, no one has ever asked me for it tho and I have undergone approx 25 audits over the years.
  • The last CAA audit that I went through was about 2 years ago for our former Part 145 maintenance organisation. There was certainly a discussion on how we viewed the conduct of the audit but our responses were very much along the lines of what you would expect of “the regulated” talking to “the regulator”. No matter how the CAA Chairman might like to dress it up, it is extremely difficult to get anyone whose livelihood relies on holding an aviation document to be critical (no matter how well intentioned or constructive) of the organisation that issues said document.
  • I am in full agreement for the auditing process to be outsourced, what a great idea, at least if this happened there would be no favoritisms by the auditors. It’s imperative that all operators record* audits.  In that way it keeps the auditors honest and semi under control.
  • Yes this is an interesting subject and I totally agree with the other comments.  After last year’s audit we received a survey, which I filled out with my own feelings regarding CAA.  But I then thought it wasn’t a good idea to return it as it may have been held against me.  So I never posted it. I am certain that industry is running scared of CAA and feel that there are “bully” tactics going on. CAA is determined to change the aviation industry to make it highly regulated to cover their own arses.  Hence industry is finding this hard to come to terms with.
  • A further point about commenting on audit skills etc is as long as they are sitting at your table you are paying for that time.  You want them in and out of your place as soon as possible so the audit doesn’t cost you an absolute fortune.  So no, we wouldn’t think there would be anyone out there brave enough to criticise the CAA openly and in front of the auditors, as it would make their lives a misery.

The Chairman also said:

“On the information theme, please note that management is also introducing a ‘balanced scorecard’ to improve the information available on organisational performance. While not yet complete, over the next 12 months you can expect to see information being sought from the sector and others to inform the assessment of matters of regulatory effectiveness, efficiency and regulatory burden”.

  • Record, in this context, means to “tape record”. The GAA recommends that all important dialogue you have with CAA personnel –  on your premises, over the telephone or at the CAA’s office – should be recorded on audio and/or video. Before making any recording, you should advise your intentions to the CAA personnel, ensure they agree to an uncensored recording and offer to provide them with a copy of it.

CAA Notices – The tentacles of regulation just keep growing longer

In mid-February, the CAA announced its intention to publish CAA Notices as a means of ensuring that “the regulatory framework remains responsive and adaptable to change.”

On the surface, these objectives almost appear noble. But when we peel back the layers, what emerges is a worrying erosion of the checks and balances that have underpinned aviation rule-making since the early 1990s.

A CAA Notice will be a mandatory stipulation of requirements and must be complied with. It will be reinforced by a rule, but the notice itself will be issued by the CAA rather than going through the rule-making process.

Parliament did not vote for this

When concerns were raised about the advent of CAA Notices without any consultation, the response from CAA management was that there is no proposal to consult on the concept of Notices because these are currently permitted under Section 28(5) of the Act.

Section 28(5) was inserted into the Civil Aviation Act in 2010 and does confer certain powers to impose requirements or conditions; however, some seven years later, the CAA is being somewhat cute in its interpretation of how far the law-makers (Parliament) intended these powers to extend.

One of the positives of our system of government is that it keeps records, and these very clearly show that:

  • In recommending amendment of the Civil Aviation Act to Cabinet, the Minister sought to grant the Director power to determine technical matters such as testing equipment, syllabi and examinations
  • Cabinet agreed to amend the Civil Aviation Act to provide for rules that delegate to the Director the power to determine technical matters, such as testing equipment, syllabi, and examinations
  • When introducing the amendments to Parliament for debate, the Associate Minister highlighted them as being about the power of the Director to determine technical matters under the Civil Aviation Rules.

Parliament intended the granting of powers to determine technical matters in a narrow range of areas. It never intended the establishment of a process where the CAA becomes the risk-definer, the solution-chooser, the rule-maker and the law-enforcer.

This entire CAA Notices concept is a flossed-up attempt by the regulator to twist the intent of the law-makers in order to extend its tentacles and shield itself from the scrutiny of its masters. Aviators will not have recourse to the MoT or Minister regarding the content of CAA Notices.

Rather than looking for mechanisms to increase its powers and subvert the checks and balances that exist, the CAA should be focusing on getting rules and rule amendments right the first time so that they progress through the existing process in a timely manner.