The Hong Kong Youth Association Survey On Queen's Pier
The following information was released to the press this past weekend. Excerpts were reported in various newspapers. You may have seen some of this, but here is the whole thing.
Hong Kong Youth Association Survey Questionnaire.
Gender
1. Male
2. FemaleAge
1. 18-24
2. 25-30
3. 31-35Highest Educational Level
1. Third-year middle-school or less
2. Fifth-year middle-school or university preparatory
3. University or higherOccupation
1. Student
2. Professional/Manager
3. Clerical, technical, service
4. Administrator
5. Housewife
6. Unemployed
7. Self-employed/otherQ1. Have you been paying attention to the news reports and information about the demolition/relocation of Queen's Pier?
1. Yes
2. No
3. Not aware of this matter [skip to Q7]Q2. In order to better traffic conditions around Central as well as carry out future development plans, the government wants to demolish/displace Queen's Pier. With respect to this policy, do you ...
1. Agree
2. Disagree
3. Don't know/Not SureQ3. The government spent seven years of consultation before legislating the plan for developing Central. Do you think that the process was ...
1. Appropriate
2. Inappropriate
3. Don't know/Not SureQ4. With respect to Secretary of Development Carrie Lam Yuet-Ngor's dialogue with the opponents of the demolition/relocation on July 29, you ...
1. Strongly approve
2. Approve
3. Neither approve nor disapprove
4. Disapprove
5. Strongly disapproveQ5. Certain preservationists oppose the demolition/relocation of Queen's Pier in the name of collective memory. Do you ...
1. Agree
2. Disagree
3. Don't Know/Not SureQ6. With respect to the opponents of demolition/relocation staying at Queen's Pier and conducting a hunger strike, you ...
1. Disapprove a lot
2. Disapprove
3. Neither approve nor disapprove
4. Approve
5. Approve a lotQ7. Personally speaking, do you consider the Queen's Pier to be a cultural heritage site?
1. Yes
2. No
3. Don't Know/Not SureQ8. The government wants to follow the Murray House case to disassemble Queen's Pier and re-assemble it at another location. Do you agree?
1. Agree a lot
2. Agree
3. Disagree
4. Disagree a lotQ9. Personally speaking, do you think whether the preservation of Queen's Pier would affect the development plan for Central?
1. Agree
2. Disagree
3. Don't Know/Not SureQ10. Personally speaking, when cultural heritage is incompatible with development, which would you choose?
1. Development first
2. Preservation first
3. Don't Know/Not SureQ11. Generally speaking, do you think that the government has done enough in its publicity about the balance between preservation and urban development?
1. Enough
2. So so
3. No enough
4. Don't Know/Not Sure
Date: August 1 to August 9, 2007
Locations: Eighteen districts in Hong Kong, Kowloon and New Territories
Methodology: The questionnaires were distributed and the respondents filled out the questionnaires themselves. The researchers were responsible only for distributing and collecting the questionnaires.
Survey Universe: Chinese citizens between the ages of 18 to 45 living in Hong Kong [note: yes, the description is 18 to 45 when the questionnaire only allows for 18 to 35 on the age question].
Sample Size: 570 questionnaires were distributed [note: there is no explanation as to whom] and 521 were collected (at 91.4% return rate). 21 of the questionnaires were excluded either because there was no response to one or more question, or more than one answer was checked for some question. The number of valid intab questionnaires is 500.
(聞.見.思.錄) Dreadful Polling, Dreadful Reporting. August 13, 2007.
[in translation]
If you pay regular attention to the news, you must have noted that an obligatory section on Sunday television news reporting is the survey results from various organizations. These surveys and their conclusions -- regardless of the scientific validity of their methods and their conclusions -- will also occupy prominent space in the newspapers on the next day. This showed up yesterday on television and today in the newspapers.
I am talking about the public opinion poll on "Preservation and Development" conducted by the Hong Kong Youth Association. Last night, I was watching the 6:30pm TVB news program and I did not pay much attention at first. I only heard that "64% of the people opposed the preservation of Queen's Pier" and some such. I read the various reports in the newspapers today. Except for <Apple Daily>, every other newspaper reported on the survey results, such as "more than half of the interviewees oppose the preservation of Queen's Pier" (see <Hong Kong Economic Daily> page A23), "59% of young people supported the demolition/relocation of Queen's Pier and more than half does not agree with preservation for the sake of collective memory" (see <Hong Kong Daily News> page A7) and "More than half of the interviewees do not support the preservation of Queen's Pier for the sake of collective memory" (see <Oriental Daily> page A27). At first sight, the "more than half of young people" and "more than half of the young people" became a tidal wave of voices against those who opposed the demolition/relocation of Queen's Pier.
But I was curious enough to start studying the questionnaire and methodology, and I can only use the adjective "dreadful" to describe the situation. Last night, I went back to review how <Cable TV News> reported on the survey and I found out a very important fact -- this survey was not based upon a "random sample" to select respondents. Instead, it was based upon using "association members to distribute questionnaires" (see <Hong Kong Economic Daily>). This is not the usual way to do public opinion polls. Compared to telephone surveys, there are literacy issues (but this problem is less likely in Hong Kong). The Hong Kong Youth Association website claims that the survey questionnaires were distributed in eighteen districts of Hong Kong, but there is no information about how many questionnaires were distributed per district. How many of those surveys were distributed by the association members on their own initiative? How many of those respondents were friends and relatives recruited by the association members to fill up the quota? How many of these questionnaires were distributed on relatively scientific basis? There exists some element of doubt here.
As for the survey questions, I found many problems. For example, in the second question:
Q2. In order to better traffic conditions around Central as well as carry out future development plans, the government wants to demolish/relocate Queen's Pier. With respect to this policy, do you ...
1. Agree
2. Disagree
3. Don't know/Not Sureand the third question:
Q3. The government spent seven years of consultation before legislating the plan for developing Central. Do you think that the process was ...
1. Appropriate
2. Inappropriate
3. Don't know/Not Surethere is a strong element of leading the respondents. The result was that 59% of the people agreed with demolition/relocation in Q2 and 55% thought that it was appropriate in Q3. In Q10, the respondents were forced to choose between one (and only one) choice between preservation and development. The previous questions were leading the respondents in one direction. By the time that they reached Q10, it was unsurprising that the 64% of the respondents "disagreed with affecting the development plan for Central in order to preserve Queen's Pier." This question was naturally the focus of the newspaper reports. Besides, this answer reflects the current division into the pro-government and anti-government sides. You are forced to choose between standing on either one side or the other ...
In the report of Hong Kong Youth Association, the survey results were said to be "clearly mainstream opinion." But the survey report did not disclose the results of each and every outcome in each question ... the survey questions and the sampling methodology were also flawed. So I very much question how "mainstream" this survey can be. I do not wish to speculate about the position of the Hong Kong Youth Association, whether it is left, centrist or right, or whether it is pro- or anti- demolition/relocation of Queen's Pier. I simply have too many doubts. I felt this survey has presuppositions and the purpose was to make way for the government action to demolish/relocate Queen's Pier by offering apparent public opinion support.
In my view, the Hong Kong Youth Association survey was unprofessional. Most media reports consist mostly of copying the press release verbatim and thus reproducing the so-called "analyses" in the "summary of survey results." They did not examine the methodology of the public opinion poll. Worse yet, the survey universe of this poll was "young people" but that did not appear in the various newspaper headlines. <Headline News>, <The Sun>, <Sing Pao> and <Wen Wen Po> referred to "citizens" while Hong Kong Daily News referred to "the people." From a survey based vaguely on "young people", this was turned into 64% of the citizens not accepting the preservation of Queen's Pier at the cost of affecting the development plan for Central. That may yet be the case, but this inference was just dreadful from this particular public opinion poll.
As I said before, I was unable to find the corresponding report in <Apple Daily>. I wonder why they chose not to report this time: Was it because the survey methodology was too "feeble" or was it because the conclusion was "too disagreeable"? Furthermore, in scanning through all the news reports, only <Sing Pao> interviewed Chow See-chung of Local Action. It seems that all the other media forgot about the journalism school's ABC of getting the reaction of the other side. Chow See-chung's interview pointed out that the Hong Kong Youth Association "only distributed the survey questionnaire for its members to fill out" and this allegation was serious enough that it should have destroyed any remaining credibility for this survey. But is that true? No matter what, I think that this survey and the corresponding news reports can only be characterized by one word: "Dreadful 糟糕"!