Hong Kong Legislative Council Election Post-Mortem Analysis

For the Hong Kong Legislative Council election of 2008, there had been a series of public opinion polls conducted by the Hong Kong University Public Opinion Programme, the Institute of Asia-Pacific Studies at the Chinese University of Hong Kong, the Lingnan University Public Governance Programme and the Hong Kong Research Association in the weeks leading up the vote itself.  On election data, a number of organizations conducted exit polls.  The question here is whether the public opinion polls and exit polls predictive of the actual vote.

In Hong Kong, this is not just a simple research question or some esoteric academic debate over survey methodology and executive.  Instead, this is a politically controversial subject.  Are the public opinion polls (or, at least, the selective release of certain parts of the data) serving some special political agenda (such as persuading a certain candidate to withdraw from an apparently hopeless situation)?  Are the exit polls used to guide strategic voting blocs on election day?

Here is the table of the actual voting data along with the four pre-election public opinion poll average and the Hong Kong Research Association exit poll results. 

Geographical Constituency List List Party 4-poll Average % HKRA Exit Poll % Actual % # Seats
Hong Kong Island 1 Lam Chui Lin Liberal Party 1.2 1.2 0.7 @
@ 2 Ho Sau Lan Cyd Civic Act-up 10.0 9.0 9.9 X
@ 3 Kam Nai Wai, Yeung Sum Democratic Party 12.8 9.9 12.7 X
@ 4 Tsang Kin Shing League of Social Democrats 2.2 4.1 3.3 @
@ 5 Tsang Yok Sing Jasper, Choy So Yuk DAB 18.2 26.3 19.3 X
@ 6 Siu Man Wa Myra @ 0.3 1.5 0.6 @
@ 7 Lo Wing Lok @ 7.0 6.8 6.5 @
@ 8 Chan Tanya, Eu Yuet Mee Audrey Civic Party 24.7 18.3 26.4 XX
@ 9 Ip Lau Suk Yee Regina, Shih Tai Cho Louis @ 22.4 20.9 19.5 X
@ 10 Lai Chi Keong Joseph @ 1.4 2.0 1.3 @
@ @ @ @ @ @ @ @
Kowloon East 1 Wu Chi Wai Democratic Party 7.1 8.1 6.9 @
@ 2 To Kwon Hang Andrew League of Social Democrats 10.3 9.7 12.1 @
@ 3 Chan Kam Lam DAB 19.2 29.0 22.6 X
@ 4 Leong Kah Kit Alan Civic Party 23.9 14.2 16.6 X
@ 5 Li Wai Ming Democratic Party 19.9 17.4 20.4 X
@ 6 Wong Kwok Kin Hong Kong Federation of Trade Unions 18.6 21.6 21.3 X
@ @ @ @ @ @ @ @
Kowloon West 1 Chong Wing Charn Francis @ 1.0 1.0 0.5 @
@ 2 Lee Wai King Starry DAB 16.7 25.4 18.9 X
@ 3 Lung Wai Man James @ 0.1 0.5 0.3 @
@ 4 To Kun Sun James Democratic Party 12.8 11.4 14.4 X
@ 5 Mo Man Ching Claudia Civic Party 12.0 6.6 8.4 @
@ 6 Lam Yi Lai @ 0.7 0.5 0.3 @
@ 7 Leung Mei Fun @ 8.3 11.9 9.6 X
@ 8 Wong Yuk Man League of Social Democrats 14.7 14.1 18.2 X
@ 9 Tien Michael Puk Sun Liberal Party 7.9 5.8 6.3 @
@ 10 Fung Kin Kee Frederick ADPL 15.9 16.1 17.2 X
@ 11 Lau Chin Shek @ 8.9 5.3 5.1 @
@ 12 Lau Yuk Shing @ 0.5 0.4 0.1 @
@ 13 Tam Hoi Pong @ 0.6 1.0 0.8 @
@ @ @ @ @ @ @ @
New Territories East 1 Tien Pei Chun James Liberal Party 11.4 8.5 8.0 @
@ 2 Siu See Kong Party for Civic Rights & Livelihood of the People of Hong Kong Limited 0.8 0.8 0.3 @
@ 3 Lau Wai Hing Emily The Frontier 8.6 7.7 9.2 X
@ 4 Wong Shing Chi Democratic Party 7.9 9.5 12.2 X
@ 5 Leung Kwok Hung League of Social Democrats 10.2 9.9 12.4 X
@ 6 Lee Chi Wing Alvin @ 2.7 1.8 1.1 @
@ 7 Cheng Kar Foo Andrew Democratic Party 14.2 10.3 11.6 X
@ 8 Tong Ka Wah Ronny Civic Party 14.7 9.4 11.1 X
@ 9 Pong Scarlett Oi Lan @ 6.5 6.0 5.7 @
@ 10 Lau Kong Wah, Chan Hak Kan DAB 24.5 36.1 28.4 XX
@ @ @ @ @ @ @ @
New Territories West 1 Cheung Chiu Hung Civic Party 6.7 5.8 7.0 @
@ 2 Lee Cheuk Yan Hong Kong Confederation of Trade Unions 11.7 10.2 10.6 X
@ 3 Tam Yiu Chung, Cheung Hok Ming DAB 24.3 27.3 23.1 XX
@ 4 Ho Chun Yan Democratic Party 13.9 8.7 9.2 X
@ 5 Tandon Lal Chaing ADPL 1.1 2.4 1.7 @
@ 6 Yuen Wai Chung @ 0.1 1.1 0.3 @
@ 7 Chan Wai Yip Albert League of Social Democrats 4.9 6.6 8.1 X
@ 8 Chow Ping Tim @ 0.1 1.0 0.4 @
@ 9 Wong Kwok Hing @ 6.8 9.0 9.0 X
@ 10 Cheung Yin Tung @ 2.5 2.4 2.5 @
@ 11 Leung Suet Fong Blue Intelligent Union 0.5 0.6 0.3 @
@ 12 Chow Liang Shuk Yee Selina Liberal Party 7.5 5.6 5.4 @
@ 13 Leung Yiu Chung Neighbourhood and Worker's Service Centre    11.6 9.7 10.7 X
@ 14 Lee Wing Tat Democratic Party 8.9 9.6 11.5 X

Public Opinion Polls

For the 2004 election, it was noted that the HKU POP numbers were systematically understating the DAB votes everywhere.  I have extracted these numbers (see Table):

For the 2008 election, the corresponding numbers are:

For this election, the pre-election four-poll averages compared against the actual election outcomes resulted in:

Out of thirty contested seats in the geographical constituency, the pre-election polls got 27 out of 30 winners (90% success rate).

Exit Polls

Exit polls had been controversial already (Banning Exit Polls in Hong Kong) and it became even more controversial earlier in the week.

(The Standard)  It's all a matter of opinion.  Mary Ma.  September 4, 2008.

A boat with a hole bored in its hull is in imminent danger of sinking.

Pollster Robert Chung Ting-yiu attempted yesterday to repair the hole he has drilled - unintentionally - in the University of Hong Kong's public opinion program with his contentious plan to release exit poll data to a number of electronic media outlets while voting is still ongoing during Sunday's Legislative Council election.  Only on Tuesday, Chung vigorously defended the decision to start releasing updated information to the media 41/2 hours after the first vote is cast. But he backtracked yesterday, saying updated exit poll data will only be given to sponsors from 8pm.

Although whether Chung's center will provide refunds to the sponsors is more an issue between him and the stations, the storm he has whipped up could prove to be a curse for the opinion program he so painstakingly set up in 1991. Prior to the pan-democrats' unanimous appeal for voters not to answer any exit poll questions, online discussion forums are already filled with calls urging electors to give the wrong answer to confuse the findings. Both scenarios are surely the last thing a pollster wants, since they would inevitably undermine the accuracy any trustworthy survey strives to achieve.

Should accuracy be in doubt, all polls will be stripped of any reference value and become worthless. This is the danger now facing Chung and his colleagues at the university's research center. Chung is fully aware of this, which helps explain why he published a statement after the pan-democrats' press conference. Warning that a boycott or giving the wrong answer would endanger the long- term development of a civic society and the professional development of polling and research, he called on the politicians to reconsider their opposition.

However, with the public outcry having reached such a fever pitch, it is doubtful whether the pollster's concession to delay the release of data to the evening will be good enough to repair the damage, since it will be very difficult for one to change his or her mind once a view is formed. Have the people formed their views after so many days of hues and cries? If anyone is to be blamed, it will be Chung himself who has helped to push his own boat to the edge of the waterfall.

If there is anything else to be learned from this saga, it is that it exposes the leniency of the election law in dealing with anyone breaching the guidelines that prohibit pollsters and concerned organizations to publicize the results of exit polls before voting is completed, as this could unduly influence those who have not yet cast their ballots. According to the guidelines, the maximum penalty for violating the rules is either a reprimand or a censure in the form of a public statement. Not much of a deterrent, I'd say.

The exit poll data were not released to the public while the voting was still going on.  At 11:00pm, the Hong Kong Research Association published its data on its website (HKRA Exit Poll ).  The numbers are included in the table above on this page.

The glaring discrepancy is in the exit poll data for the DAB.  In each case, the HKRA exit poll numbers for the DAB party are higher than the public opinion poll average, and the actual voting is closer to the public opinion polls than the exit polls.

This will be known hereafter as the "Just say DAB effect."  Right before the election, there was a controversy whether the exit polls were being used exclusively by the DAB and/or other pro-establishment forces to allocate campaign resources.  Since the government refused to take action, some citizens started a "Just say DAB" campaign (see @Facebook Tell the pollster: "I voted for DAB!") -- when asked by a pollster after voting, the voter is supposed to say: "I voted for the DAB!"  This  tactic is supposed to distort the poll results and ruin its purported utility.  Indeed the poll results have been distorted systematically.  But there is no evidence that the exit poll data figured in any final-two-hour surge of 'iron' votes from the leftists towards designated candidates.

In comparing its predicted winners against the actual results, the HKRA exit poll performed as follows:

Out of the thirty contested seats in the geographical constituency, the HKRA exit poll got 28 out of 30 winners (93% success rate). 

In spite of the "Just say DAB effect," the only major mishap to the DAB might be with its number two candidate Choy So-yuk.  Did the DAB think that they were already secure with two seats from their overwhelming lead and therefore mistakenly allocated resources to Regina Ip's list?  Here are some known facts.  At 2pm, Choy So-yuk issued a personal "urgent rescue" appeal.  At 8pm, the DAB list leader Tsang Yok-sing proclaimed that the sixth and last seat was a contest between Choy So-yuk and Yeung Sum (Democratic Party).  This was regarded as a tactic to draw anti-DAB voters towards the apparently hopeless Yeung instead of the better placed Audrey Eu and Cyd Ho.  These actions do not seem to indicate that the DAB was actively transferring votes to Regina Ip.

Why did the HKRA exit poll still work well in spite of the clear "Just say DAB effect"?  Because the DAB effect resulted only in transferring votes among the top vote-getters only and not outside the circle almost all of the time.  For example, in Kowloon East, there were six candidates contesting for four seats.  Four candidates (including the DAB one) had a big lead over the other two candidates.  The "Just say DAB effect" made the DAB number larger, but it does not change the fact that those four would win.  Similarly in Kowloon West, New Territories East and New Territories West, the DAB lists were the top vote-getters already and the "Just say DAB effect" was not big enough to suggest that they could get an extra seat.  The only exception was in Hong Kong Island, where the top vote-getter was the Civic Party list, and the "Just say DAB effect" was enough to push the DAB past them in the exit poll data.

The Hong Kong University Public Opinion Programme released its interim data to its electronic media sponsors at 8pm.  As soon as the polls closed at 10:30pm, the electronic media began to broadcast the exit poll results.  There are no numbers, just qualitative assessments of the chances ("extremely high chances," "high chances," "equal chances," ...).  On the news, it was reported that the response rate to the HKU POP exit poll was around 50%, compared to 67% in the 2004 Legislative Council election.  (HKRA reported a 50% response rate for its exit poll.)  The almost 20% drop in response rate is due to the publicity that Robert Chung (of HKU POP) caused earlier in the week when he said that the exit poll data would be released to the electronic media at noon and 5pm, and then he relented after the pan-democrats called for a boycott of all exit polls.  Here were the HKU POP exit poll results.

Hong Kong Island:
Extremely high chances: Kam Nai-wai (DP), Jasper Tsang (DAB), Tanya Chan (CP), Regina Ip (IND), Cyd Ho (CA)
Equal chances (for the last seat): Choy So-yuk (DAB), Lo Wing-lok (IND), Audrey Eu (CP), Louis Shih (IND)

Kowloon East:
Extremely high chances: Chan Kam Lam (DAB), Wong Kwok Kin (FTU)
High chances: Alan Leong (CP)
Equal chances (for the last seat) Andrew To (LSD), Fred Li (DP)

Kowloon West:
Extremely high chances: Raymond Wong (LSD), James To (DP), Frederick Fung (ADPL), Starry Lee (DAB)
Equal chances (for the last seat): Claudia Mo (CP), Leung Mei Fun (IND), Michael Tien (LP)

New Territories East:
Extremely high chances: Wong Shing Chi (DP), Leung Kwok Hung (LSD), Andrew Cheng Kar-foo (DP), Lau Kwong-wah (DAB), Chan Hak-kan (DAB)
Equal chances (for the last 2 seats): James Tien (LP), Emily Lau (The Frontier), Ronny Tong (CP), Scarlett Pong (IND), Mok Kam Kwai (DAB)

New Territories West:
Extremely high chances: Lee Cheuk-yan (CTU), Tam Yiu-chung (DAB), Cheung Hok-ming (DAB), Wong Kwok-hing (FTU), Lee Wing-tat (DP)
Equal chances (for the last 3 seats): Leung Yiu-chung (Neighbourhood and Worker's Service Centre), Cheung Chiu-hung (CP), Albert Ho (DP), Chan Wai-yip (LSD), Selina Chow (LP), Leung Che Cheung (DAB)

This has the same characteristic as the HKRA exit poll data in terms of overstating the likelihood for the DAB (such as the third DAB candidates being competitive in New Territories East and West).  In Kowloon East, Fred Li ended up with 20.4% of the votes compared to 12.4% for Andrew To, and this should not have been close.  In all remaining districts, the candidates with 'extremely high chances' won and the remaining seats come from candidates with 'equal chances.'

The Hong Kong Research Association also provide 'qualitative' data (such as "extremely high chances," "relatively high chances," "relatively lower chances", "extremely little chances").

Hong Kong Island:
Extremely high chances: Cyd Ho (CA), Kam Nai-wai (DP), Jasper Tsang (DAB), Choy So-yuk (DAB), Tanya Chan (CP), Regina Ip (IND)
Relatively high chances: Lo Wing-lok (IND)
Relatively low chances: Tsang Kin-sing (LSD), Louis Shih (IND)
Relatively little chance: Audrey Eu (CP)

Kowloon East:
Extremely high chances: Chan Kam Lam (DAB), Wong Kwok Kin (FTU), Alan Leong (CP), Li Wah-ming (DP)
Relatively low chances: Wu Chi-ming (DP), Andrew To (LSD)

Kowloon West:
Extremely high chances: Raymond Wong (LSD), James To (DP), Frederick Fung (ADPL), Starry Lee (DAB), Leung Mei-fun (IND)
All others have 'relatively small or little chances.'

New Territories East:
Extremely high chances: Wong Shing Chi (DP), Leung Kwok Hung (LSD), Andrew Cheng Kar-foo (DP), Lau Kwong-wah (DAB), Chan Hak-kan (DAB)
Relatively high chances: James Tien (LP), Emily Lau (The Frontier), Ronny Tong (CP
Relatively low chances: Scarlett Pong (IND)

New Territories West:
Extremely high chances: Lee Cheuk-yan (CTU), Tam Yiu-chung (DAB), Cheung Hok-ming (DAB), Albert Ho (DP), Wong Kwok-hing (FTU), Lee Wing-tat (DP), Leung Yiu-chung (Neighbourhood and Worker's Service Centre)
Relatively high chances: Cheung Chiu-hung (CP), Chan Wai-yip (LSD), Selina Chow (LP)

These numbers are not that much different from the HKU POP data (except for the obvious case of Audrey Eu (CP) on Hong Kong Island).

Conclusions

Question 1: Were the pre-election public opinion polls useful?

It depends on how you want to use them.  Without the public opinion polls, you would have no idea who was leading and who was lagging.  You would have no idea about how things changed as a result of a turning point.  An example was the case of Raymond Wong (League of Social Democrats).  He began the campaign indifferently in the middle of the pack.  Then came the televised debates from which his segments went viral on the Internet.

Raymond Wong's numbers in the IAPS-CUHK tracking polls over time were:

    August 6-12: 6.6% (6th place)
    August 14-20: 13.2% (4th place)
    August 21-27: 12.9% (3rd place)
    August 28-September 3: 16.1% (2nd place)

And he finished second place in the actual election.

Were the public opinion polls useful in predicting the election outcomes? 

The above data showed that the pre-election four-poll average predicted 27 out of 30 winners (90% success rate).  There are some obvious explanations about the failures.  The known facts are (1) 90% of survey respondents said that they will definitely vote or are likely to vote and less than 10% said that they definitely won't vote; (2) the actual voter turnout was only 45.2%; (3) the no-show's are likely less committed to specific candidates.  The public opinion polls did not catch on that the Liberal Party candidates had soft support.  Their supporters may be willing to state their preferences over the telephone but they did not actually show up to vote.  Thus James Tien and Selina Chow flamed out.  The third case of the 'independent' Leung Mei-fun is less obvious.  We might guess that her voters were disproportionately "undecided/don't know/refuse to say" in the public opinion polls, but they showed up to choose a high-profile non-party-affiliated candidate.

Q2: Was the HKRA exit poll useful?

Here, I don't want to engage in any speculation about its possible role in the strategic decisions of insiders who had access to the data during the day.  I don't know.

Speaking as a citizen, there was only one time window (11pm when the data was first published and before the election results were announced at some time after 4am).  As it turns out, the HKRA exit poll predicted 28 out of 30 winners (93% success rate).  That is a respectable record.  Compared to the pre-election public opinion polls, the HKRA exit poll imparted the new information that Leung Mei-fun (IND) was a winner and Selina Chow was ousted.

The Apple Daily story on the "Just say DAB effect" used the headline: "Voters devastated exit polls."  If you need a precise estimate of the number of votes that a candidate is getting, you should wait for the actual election results.  If you want an overview on who the likely winners are, you will find the exit polls useful and interesting (with new information not present in the pre-election public opinions polls).

But you say: The exit polls are statistically biased due to respondent non-cooperation!  Well, you run a telephone survey and 40% of the contacted persons refuse to participate.  There is a potential survey bias because the non-responders may have different behaviors and attitudes than the responders.  But that does not mean that the survey is useless.  With experience in data usage over time (for example, comparing enough public opinion polls versus actual election results), you will know how to interpret the data correctly.