Public Opinion Polls And The Hong Kong Legislative Council Elections

The Hong Kong Legislative Election will be held on September 7, 2008.  There is a great deal of interest about the developments.  Who is leading?  Who is lagging?  Who is gaining momentum?  Who is losing momentum?

Those answers have to come through public opinion polls.

Some candidates and political parties may be conducting their own public opinion polls, but they don't usually share the results with the public.  Even if they publish some proprietary polling results, the public ought to be skeptical about selective leaking to satisfy certain agenda.

Therefore, the public has to rely on the public opinion polls that are conducted by independent third parties (such as university research institutions) and released to the general public. 

The three major public opinion polls on the Legco elections are the following:

• Hong Kong University Public Opinion Programme.  Unfortunately, you won't find any real data because of this situation:

The above data links contain findings from POP's Legislative Council Election rolling and exit polls. They are provided to sponsors for their first use. The links will be open for public consumption after the election.

You will have to read the sponsoring media (e.g. Apple Daily, Ming Pao, South China Morning Post, etc) to find the actual poll results.  But please be careful because some of the sponsoring media have been selective in publishing partial data to satisfy certain agenda.  That is, they do not release all of the poll results; they only publish selective slices every day in conjunction with editorial statements (e.g. vote for democrat XXX in district YYY because there is an emergency situation!).

• Hong Kong Institute of Asia-Pacific Studies, The Chinese University of Hong Kong (in Chinese only).
  　
• Hong Kong Research Association (in Chinese only).

In the following, I will run an analysis of these public opinion polls.  I am particularly interested in certain aspects of the polling process that may impact the end result.

The first question is: what is the universe of persons that is getting measured?  Many public opinion polls gauge attitudes on public issues (such as the domestic helper tax, inflation, satisfaction with the job performance of the HK SAR Chief Executive, fire safety, etc) and it is reasonable to use the universe of all adults (age 18 or over) in Hong Kong.  But since this is about the Legislative Council elections, it seems reasonable to restrict the universe to all registered voters in Hong Kong.  But if the purpose is to gauge the possible election outcome, it seems reasonable to further restrict the universe to all registered voters who are likely to vote.

For the HKIAPS-CUHK polls, the voter preference results exclude those who indicate that they do not intend to vote.  That leaves those who "will definitely vote," "may vote" or "are undecided/unsure."  During August 18-24, 9.1% said that they won't vote, 22.5% said that they may vote, 66.4% said that they will definitely vote and 1.9% said that they are undecided/unsure.

For the Hong Kong Research Association, the voter preference results are restricted only to those who say that they will vote.  That excludes those who "won't vote," "don't know" and "have no opinion."  During August 1-8, 62% of registered voters said that they will vote, 12% said that they won't vote, 15% are unsure and 11% have no opinion.

The HKU POP site does not publish any information on the universe because the data is reserved for first use by the sponsors.

Immediately, you can see that there are different choices for the universe.

The next question is about the polling methodology.  Due to cost considerations, the only cost-feasible choice that delivers data of decent quality is the telephone methodology.  Face-to-face, door-to-door personal interviews are cost-prohibitive, while mail/internet surveys deliver poor quality data.

This leads to questions about the coverage of the telephone sampling frame.  The starting point may be the telephone directory.  There are known deficiencies about the coverage based solely upon the telephone directory.  The most important factor is that telephone penetration is not universal.  At one point back in time, those without telephones are most likely poor people who cannot afford to have a telephone.  There is also a small number of people (such as celebrities) who opt out of the telephone directory listing.  More recently, there are young, affluent people who use mobile phones exclusively (note: the telephone directory only covers landline phones and not mobile phones).

Another important question is how the telephone interviewing is conducted.  The preferred method is interviewing conducted by a real person.  HKU POP explicitly adds this footnote (see, for example, this poll):

The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

This dig is aimed at the Hong Kong Research Association, which uses an interactive voice system.  This is somewhat abstract, but I can make it more concrete with my personal experience when I was called up by HKRA for one of their interviews:

The telephone rings.  I pick up the telephone.  I am asked: "Are you age 20 or older? If yes, press 1; if no press 2."  I press 1 and the interview continues.

Thus, they are using the first adult that picks up the telephone.

In my household, there are three adult residents.  When the telephone rings, I always pick up because there is a strong likelihood that it is for me (e.g. business associates, friends, service calls, etc).  The other two adult residents are both over 80 years old, barely receive any calls and would not know how to respond to my calls.  Therefore, they will always let me pick up the phone if they know I am home.

Therefore, this first-adult methodology gives me a 100% chance of selection and 0% of them.  The resulting polling results will be biased.

By the way, Apple Daily and Oriental Daily/The Sun also use interactive voice systems to conduct quick-and-dirty polls.

There are many issues that may bias poll results, including: sampling frame coverage; response rate; respondent selection within household; weighting; question phrasing; etc.

Given that these three polls differ in many aspect, there is no reason to expect that they should deliver in the same poll results.

How different are they?  Here are the most recent poll results that I can find: