Public Opinion Polls And The Hong Kong Legislative Council Elections

The Hong Kong Legislative Election will be held on September 7, 2008.  There is a great deal of interest about the developments.  Who is leading?  Who is lagging?  Who is gaining momentum?  Who is losing momentum?

Those answers have to come through public opinion polls.

Some candidates and political parties may be conducting their own public opinion polls, but they don't usually share the results with the public.  Even if they publish some proprietary polling results, the public ought to be skeptical about selective leaking to satisfy certain agenda.

Therefore, the public has to rely on the public opinion polls that are conducted by independent third parties (such as university research institutions) and released to the general public. 

The three major public opinion polls on the Legco elections are the following:

The above data links contain findings from POP's Legislative Council Election rolling and exit polls. They are provided to sponsors for their first use. The links will be open for public consumption after the election.

You will have to read the sponsoring media (e.g. Apple Daily, Ming Pao, South China Morning Post, etc) to find the actual poll results.  But please be careful because some of the sponsoring media have been selective in publishing partial data to satisfy certain agenda.  That is, they do not release all of the poll results; they only publish selective slices every day in conjunction with editorial statements (e.g. vote for democrat XXX in district YYY because there is an emergency situation!).

In the following, I will run an analysis of these public opinion polls.  I am particularly interested in certain aspects of the polling process that may impact the end result.

The first question is: what is the universe of persons that is getting measured?  Many public opinion polls gauge attitudes on public issues (such as the domestic helper tax, inflation, satisfaction with the job performance of the HK SAR Chief Executive, fire safety, etc) and it is reasonable to use the universe of all adults (age 18 or over) in Hong Kong.  But since this is about the Legislative Council elections, it seems reasonable to restrict the universe to all registered voters in Hong Kong.  But if the purpose is to gauge the possible election outcome, it seems reasonable to further restrict the universe to all registered voters who are likely to vote.

For the HKIAPS-CUHK polls, the voter preference results exclude those who indicate that they do not intend to vote.  That leaves those who "will definitely vote," "may vote" or "are undecided/unsure."  During August 18-24, 9.1% said that they won't vote, 22.5% said that they may vote, 66.4% said that they will definitely vote and 1.9% said that they are undecided/unsure.

For the Hong Kong Research Association, the voter preference results are restricted only to those who say that they will vote.  That excludes those who "won't vote," "don't know" and "have no opinion."  During August 1-8, 62% of registered voters said that they will vote, 12% said that they won't vote, 15% are unsure and 11% have no opinion.

The HKU POP site does not publish any information on the universe because the data is reserved for first use by the sponsors.

Immediately, you can see that there are different choices for the universe.

The next question is about the polling methodology.  Due to cost considerations, the only cost-feasible choice that delivers data of decent quality is the telephone methodology.  Face-to-face, door-to-door personal interviews are cost-prohibitive, while mail/internet surveys deliver poor quality data.

This leads to questions about the coverage of the telephone sampling frame.  The starting point may be the telephone directory.  There are known deficiencies about the coverage based solely upon the telephone directory.  The most important factor is that telephone penetration is not universal.  At one point back in time, those without telephones are most likely poor people who cannot afford to have a telephone.  There is also a small number of people (such as celebrities) who opt out of the telephone directory listing.  More recently, there are young, affluent people who use mobile phones exclusively (note: the telephone directory only covers landline phones and not mobile phones).

Another important question is how the telephone interviewing is conducted.  The preferred method is interviewing conducted by a real person.  HKU POP explicitly adds this footnote (see, for example, this poll):

The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

This dig is aimed at the Hong Kong Research Association, which uses an interactive voice system.  This is somewhat abstract, but I can make it more concrete with my personal experience when I was called up by HKRA for one of their interviews:

The telephone rings.  I pick up the telephone.  I am asked: "Are you age 20 or older? If yes, press 1; if no press 2."  I press 1 and the interview continues.

Thus, they are using the first adult that picks up the telephone.

In my household, there are three adult residents.  When the telephone rings, I always pick up because there is a strong likelihood that it is for me (e.g. business associates, friends, service calls, etc).  The other two adult residents are both over 80 years old, barely receive any calls and would not know how to respond to my calls.  Therefore, they will always let me pick up the phone if they know I am home.

Therefore, this first-adult methodology gives me a 100% chance of selection and 0% of them.  The resulting polling results will be biased.

By the way, Apple Daily and Oriental Daily/The Sun also use interactive voice systems to conduct quick-and-dirty polls.

There are many issues that may bias poll results, including: sampling frame coverage; response rate; respondent selection within household; weighting; question phrasing; etc.

Given that these three polls differ in many aspect, there is no reason to expect that they should deliver in the same poll results.

How different are they?  Here are the most recent poll results that I can find:

Geographical Constituency List Party HIAPS-CUHK
Aug 18-24
HKRA
August 1-8
HKU POP
Aug 10-18
Hong Kong Island 1 Lam Chui Lin Liberal Party 0.4 1.2 @
@ 2 Ho Sau Lan Cyd Civic Act-up 6.7 4.4 @
@ 3 Kam Nai Wai, Yeung Sum Democratic Party 10.5 16.0 11.3
@ 4 Tsang Kin Shing League of Socialist Democrats 3.4 4.4 @
@ 5 Tsang Yok Sing Jasper, Choy So Yuk DAB 14.6 17.6 21.0
@ 6 Siu Man Wa Myra @ 0.4 0.4 @
@ 7 Lo Wing Lok @ 8.8 4.4 @
@ 8 Chan Tanya, Eu Yuet Mee Audrey Civic Party 30.9 22.8 31.3
@ 9 Ip Lau Suk Yee Regina, Shih Tai Cho Louis @ 23.8 23.6 23.5
@ 10 Lai Chi Keong Joseph @ 0.6 5.2 @
@ @ @ @ @ @ @
Kowloon East 1 Wu Chi Wai Democratic Party 6.6 5.0 @
@ 2 To Kwon Hang Andrew League of Socialist Democrats 6.1 5.0 @
@ 3 Chan Kam Lam DAB 16.5 20.8 19.3
@ 4 Leong Kah Kit Alan Civic Party 31.8 35,2 27.0
@ 5 Li Wai Ming Democratic Party 13.3 15.7 20.6
@ 6 Wong Kwok Kin Hong Kong Federation of Trade Unions 25.7 18.3 23.5
@ @ @ @ @ @ @
Kowloon West 1 Chong Wing Charn Francis @ 0.0 0.0 @
@ 2 Lee Wai King Starry DAB 8.8 18.9 14.7
@ 3 Lung Wai Man James @ 0.6 0.6 @
@ 4 To Kun Sun James Democratic Party 18.9 19.5 19.3
@ 5 Mo Man Ching Claudia Civic Party 10.8 8.2 14.8
@ 6 Lam Yi Lai @ 0.6 1.3 @
@ 7 Leung Mei Fun @ 8.8 10.1 @
@ 8 Wong Yuk Man League of Socialist Democrats 14.5 5.0 @
@ 9 Tien Michael Puk Sun Liberal Party 10.0 11.3 9.1
@ 10 Fung Kin Kee Frederick ADPL 18.2 13.2 18.0
@ 11 Lau Chin Shek @ 7.6 9.4 9.1
@ 12 Lau Yuk Shing @ 0.0 0.6 @
@ 13 Tam Hoi Pong @ 1.3 0.6 @
@ @ @ @ @ @ @
New Territories East 1 Tien Pei Chun James Liberal Party 11.8 14.5 11.8
@ 2 Siu See Kong Party for Civic Rights & Livelihood of the People of Hong Kong Limited 0.4 1.2 @
@ 3 Lau Wai Hing Emily The Frontier 6.5 12.1 8.8
@ 4 Wong Shing Chi Democratic Party 8.9 4.8 @
@ 5 Leung Kwok Hung League of Socialist Democrats 5.3 7.2 7.2
@ 6 Lee Chi Wing Alvin @ 1.0 0.6 @
@ 7 Cheng Kar Foo Andrew Democratic Party 17.4 13.3 17.3
@ 8 Tong Ka Wah Ronny Civic Party 17.4 16.9 18.4
@ 9 Pong Scarlett Oi Lan @ 5.9 2.4 @
@ 10 Lau Kong Wah, Chan Hak Kan DAB 25.3 27.1 27.8
@ @ @ @ @ @ @
New Territories West 1 Cheung Chiu Hung Civic Party 7.4 4.9 @
@ 2 Lee Cheuk Yan Hong Kong Confederation of Trade Unions 11.5 11.4 8.8
@ 3 Tam Yiu Chung, Cheung Hok Ming DAB 20.1 26.0 23.2
@ 4 Ho Chun Yan Democratic Party 14.4 15.5 15.4
@ 5 Tandon Lal Chaing ADPL 1.6 1.2 @
@ 6 Yuen Wai Chung @ 0.9 0.4 @
@ 7 Chan Wai Yip Albert League of Socialist Democrats 4.0 2.0 8.5
@ 8 Chow Ping Tim @ 0.9 0.4 @
@ 9 Wong Kwok Hing @ 5.7 6.1 @
@ 10 Cheung Yin Tung @ 2.0 0.8 @
@ 11 Leung Suet Fong Blue Intelligent Union 0.0 0.4 @
@ 12 Chow Liang Shuk Yee Selina Liberal Party 8.1 10.6 11.1
@ 13 Leung Yiu Chung Neighbourhood and Worker's Service Centre    15.8 8.1 10.9
@ 14 Lee Wing Tat Democratic Party 7.4 12.2 10.6

The polls do not yield the same results (in the sense that they do not yield the same list of people being elected to the 30 Legco seats).  But then again they are not expected to due to time period differences, survey biases (including those listed above) and sampling error.

How do we know which poll is correct (or more correct)?  We don't.  And we cannot know because survey bias is defined as the expected value of the survey estimate minus the true number.  If we know the true number, we would not have to conducting any polling.  This is tautologically unknowable.

So what good is any of this?  Well, each and every one of the polls may be flawed in many ways.  But sometimes it is possible to be right for all the wrong reasons when errors cancel themselves out.  How do we know that?  We can check with the past performances.  When a poll did not work out last time, can we determine why and make an appropriate adjustment?

Let us review what happened in the 2004 Hong Kong Legislative Council election results.

The final voter turnout was 1,784,140 out of 3,207,227 at a rate of 55.6%.  This was better than the two previous Legco elections in 2000 and 1998, but it is less than the 60% forecast (and never mind the 70% wished for by the Democratic Party).  The results of the directly elected geographical constituencies are shown in the table (with the winners in bold letters), and you can find the results of the functional constituences in this other post.

Geographical District/candidate %HKU 9/11 poll # of actual votes % of actual votes
Hong Kong (5 seats)
1. Ma Lik/Choy So-yuk (DAB)
2. Rita Fan (independent)
3. Tsang Kin-shing
4. Yeung Sum/Martin Lee (DP)
5. Kevin Wong
6. Audrey Eu (Article 45 Concern)

15.5 %
23.6 %
  1.3 %
27.4 %
  0.7 %
31.6 %

  74,662
  65,660
    5,312
131,785
    2,829
  73,841
 
21.1 %
18.5 %
  1.5 %
37.2 %
  0.8 %
20.9 %
Kowloon East (5 seats)
1. Fred Li (DP)
2. Chan Kam-lam (DAB)
3. Albert Cheng (independent)
4. Alan Leong (independent)
5. Chan Yuen-han (HK FTU)

16.9 %
13.3 %
21.5 %
23.1 %
25.2 %

56,409
55,188
73,424
56,161
52,520

19.2 %
18.8 %
25.0 %
19.1 %
17.9 %
Kowloon West (4 seats)
1. Lau Yu-shing
2. Frederick Fung (ADPL)
3. Lau Chin-shek
4. Tsang Yok-sing (DAB)
5. James To (DP)
6. Bruce Liu (ADPL)
 
  0.4 %
23.3 %
23.7 %
19.5 %
28.0 %
  5.1 %
 
  1,824
46,649
43,460
61,770
60,539
13,452

  0.8 %
20.5 %
20.4 %
27.1 %
26.6 %
  5.9 %
New Territories East (7 seats)
1. Leung Kwok-hung (April 5 Action)
2. Tso Wung-wai (HK Progressive Alliance)
3. Andrew Cheng, Emily Lau, Ronny Tong
4. James Tien (Liberal)
5. Andrew Wong
6. Lau Kong-wah/Li Kwok-ying (DAB)
 
  7.9 %
  3.5 %
43.6 %
20.2 %
  7.5 %
17.3 %
 
  60,925
  14,174
168,833
  68,560
  23,081
  95,434

14.1 %
  3.3 %
39.2 %
15.9 %
  5.4 %
22.1 %
New Territories West (8 seats)
1. Albert Chan
2. Lee Wing-tat (DP)
3. Albert Ho (DP)
4. Leung Yiu-chung (Neighborhood & Workers)
5. Chow Ping-tim
6. Stephen Char
7. Tam Yiu-chung/Cheung Hok-ming (DAB)
8. Ng Tak-leung
9. Selina Chow (Liberal)
10. Lui Sau-tuen (New Century Forum)
11. Lee Cheuk-yan (HK CTU)
12. Yim Tin-sing (ADPL)
 
  5.4 %
15.2 %
12.2 %
12.6 %
  0.2 %
  2.9 %
18.7 %
  0.8 %
14.6 %
  1.0 %
12.7 %
  3.6 %

  36,278
  62,500
  62,342
  59,033
    1,725
    9,116
115,256
    1,920
  50,437
    4,511
  45,725
  14,570

  7.8 %
13.5 %
13.5 %
12.7 %
  0.4%
  2.0 %
24.9 %
  0.4 %
10.9 %
  1.0 %
  9.9 %
  3.1 %

I have included the HKU final public opinion poll numbers to illustrate a longstanding complaint from the DAB.  I have extracted these numbers:

Why are the HKU POP poll numbers so wrong for the DAB?  This is a matter of conjecture.  One explanation that is often given is that the DAB supporters are not included in sampling frame (i.e. they do not have telephones) or else they hide their opinions when interviewed.  My preferred explanation is that the DAB has a superior GOTV (Get Out The Vote) effort.  According to the public opinion polls, about 90% of the people said that they will vote.  In reality, just over 55% voted.  The DAB is simply better able to get their supporters to vote because they have more manpower and resources (as in reminder phone calls, personal visits, etc).  Therefore, the DAB end up with a higher share of the actual votes than the polls would indicate.

Here is an illustration with a hypothetical example:

Constant Model (60% voter turn out for all candidates)

Candidate Projected Votes According to Poll Actual Votes % Votes
1 20,000 12,000 20%
2 20,000 12,000 20%
3 20,000 12,000 20%
4 20,000 12,000 20%
5 20,000 12,000 20%
Total 100,000 60,000 @

Differential Voter Turnout Model (80% for candidate 1 and 55% for candidates 2-5)

Candidate Projected Votes According to Poll Actual Votes % Votes
1 20,000 16,000 27%
2 20,000 11,000 18%
3 20,000 11,000 18%
4 20,000 11,000 18%
5 20,000 11,000 18%
Total 100,000 60,000 @

This means that the poll results that you see in media reports are suspect.  All this is just of academic interest to those people who are interested in the horse race results. 

The more important issue is what should you do as a voter?  How should you maximize the impact of your vote in light of the current trends as indicated by poll results?

I suggest that the answer is very simple.  As a voter, you should just ignore the poll results.  Instead, you should look at the credentials, records, policy platforms, debate performances and other direct aspects of the candidates, and you make your choice.  As for the self-selected spins on the poll results, pleas to distribute the votes within your households, last-minute emergency appeals to save 'democracy' in freely distributed newspapers ... well, they are just statistics, lies and damned lies.  If you cannot even vote according to your own conscience and judgment, what is the point of even casting a vote?  You might as well as sign a proxy card and let someone decide for you.