Opinion poll findings - October 26, 2007

The editorial comment, included with my contribution on the above topic, further complicates the matter and makes me feel more doubtful about the validity and reliability of the study conducted by Allied Consultants for Allied Newspapers.

The editor refers to the "total sample frame of 3,000 randomly selected households". By "total sample frame" one normally refers to the universe, that is, the population the sample represents. But in par 1.3.3 the universe is declared to be all voters in Malta and Gozo (N=317,925)! In reality, it would appear that in this study 3,000 households were selected, and of these only 21.3 per cent response was achieved. I repeat that this is very low, and the resulting sampling error is so high that it makes the results very shady.

A number of other statements in the editorial comment are facetious, and difficult to uphold scientifically. For example, how can an outside observer scientifically establish whether "there was" or "there was not" a "specific pattern of non-responses" or that "the respondents are a reflection of the general population"? The indication within the report points in the opposite direction: the published data was weighted in an attempt to make the final sample of a sample appear to reflect the demographic profile. Weighting is indeed acceptable in some cases, but not in this kind of exercise!

In person-to-person interviews replacements are normally minimal, and those randomly chosen need to be matched by a number of factors to ensure the same sample profile. The sample size does not change according to how many interviewees accept to be interviewed: the size is a function of the sampling error accepted prior to the start of the project and should not change in the process of interviewing! To start with 3,000 and end up with about one fifth of them is indeed very, very strange.

Other mistakes call for explanations: The stated base of voters for the individual districts is factually wrong in a number of cases! This study appears to be a very carelessly done exercise indeed, and this is further compounded by the absolute sample size for each of the electoral districts, an aspect which was commented upon in my last contribution.

Indeed I did examine the whole report before I wrote, and as I said there are other reasons why I doubt the reliability of the study. As I said in my earlier contribution my only professional interest is that whoever produces these studies must ensure their reliability and validity. Otherwise it is just gigo - garbage in and garbage out!

The readers of The Times, and every Maltese for that matter, deserve better.

In the circumstances it is indeed somewhat surprising that the editor assumed responsibility for technical matters as proxy for the authors of the report: It is my view that it would have been preferable if the compilers of the study were allowed to assume full responsibility.