Metro wants your opinion except when they don't.
Carla Axtman
This story is a teensy bit older than I might generally like, but I'm behind.
Last year, Metro spent $76,000 to implement an Opt-In survey program,. The goal of the survey, as I understand it, is to engage citizens and get their feedback on issues and policies within Metro's purview.
It would seem that Metro isn't loving the public feedback so far.
Nick Christensen, Metro news staff writer:
The panel is still far from perfect. Much of the discussion was centered around the underrepresentation of conservatives and suburban residents on the panel. It also under represents blacks and Latinos, and is dramatically short on participants who never went to college.
That doesn't demean the project's purpose, said Rebecca Ball, an associate at DHM Research, which manages Opt In. She pointed out that of the 4,000 participants of a recent survey, 90 were Latino, 79 were Native American, 100 were Asian and 36 were African American.
"That's by far more than we would have gotten had we done a scientific survey," she said. "We have enough comments from these groups to look at them in a way that is meaningful."
Now this part has a ring of legitimacy to me. If the demographics are narrow that's a reason to figure out how to get it further into the community. But it seems like they have enough data from the demographic groups in order to make an informed decision.
Here's where it starts to get silly:
Metro Council President Tom Hughes expressed concerns about the possibility of the tool undermining the elected officials' power to use judgment in representing their constituents.
"If we doubled the size of it now, it would still be a small slice of the population," Hughes said. "There is a tendency to say 'If 85 percent of people who responded to a survey said you ought to do X, and you did Y, what's the value of the survey? Or how come you don't do what the people told you they wanted?'"
Seems like maybe this argument should have been hashed out before Metro dropped 5 figures on the survey in the first place. If you're not interested in the input because you think it undermines your judgement, then why did you do it?
Frankly, this kind of attitude is hugely problematic in my view. Just because you're elected doesn't mean that you still don't need to hear from and listen to constituents in an ongoing manner. This strikes me as arrogant and quite tone deaf on the part of Hughes.
More from Christensen:
On a similar track, two Metro councilors disagreed about the importance of offering breakdowns of survey responses by Metro Council districts.
Councilor Rex Burkholder said he'd like to see the district-based data, instead of the county-by-county breakdown now offered in survey results.
"The county boundaries aren't very good boundaries in terms of analyzing this data for us," Burkholder said.
But Councilor Kathryn Harrington said the councilors' charge isn't to vote for their districts.
"We are making regionally-based decisions," she said. "There are some folks who say 'You represent your district,' and yes – but I'm supposed to look at the regional context when I make those decisions."
Indeed Metro Councilors are making regionally based decisions. But you need to dance with them that brung ya, Kathryn. And that's your district. Seems odd that she'd want to dismiss that data so readily.
 |
More Recent Posts | |
Albert Kaufman |
|
Guest Column |
|
Kari Chisholm |
|
Kari Chisholm |
Final pre-census estimate: Oregon's getting a sixth congressional seat |
Albert Kaufman |
Polluted by Money - How corporate cash corrupted one of the greenest states in America |
Guest Column |
|
Albert Kaufman |
Our Democrat Representatives in Action - What's on your wish list? |
Kari Chisholm |
|
Guest Column |
|
Kari Chisholm |
|
connect with blueoregon
10:12 a.m.
Feb 28, '12
re: the final point. people have been arguing about what representation is for centuries. both Burkholder & Harrington are right. in terms of good policy, the Council needs both the broad data & the specific.
and Hughes misses the point very badly. the goal is to hear from as many people as possible. to be as inclusive as possible. cast many nets, and cast them widely. if he doesn't think Opt In brings in enough voices, he needs to help make sure it does. Metro has a ton of tools at its disposal and it needs to use them all to engage as many citizens as possible. otherwise, the Tea Partiers & govt-haters end up being justified.
11:52 a.m.
Feb 28, '12
Excellent write up, Carla. I believe the opt-in debacle reveals quite a lot about Metro's often clumsy management of communications and outreach.
There should be outrage that non-Portlanders and non-whites are poorly represented on this panel. It is Metro's responsibility to ensure this panel more closely resembles its constituency. Without proper representation from all Metro communities, this survey tool is DOA.
1:48 p.m.
Feb 28, '12
Steve: To be fair, I think Metro is working to expand the demographic. But even with the less represented constituencies, they're still getting a large enough number to get a reasonable sampling of opinion from those groups.
What's weird to me are the backhanded compliments for the survey program which are, in essence, "Yes we think this is a great tool. No we're not really interested in using it to fundamentally shape how we do stuff."
Or at least that's how it's looking.
3:55 p.m.
Feb 28, '12
Their numbers are also skewed, as they have me in the system multiple times and no matter of work on my part has been able to get me down to just one. I wonder how many others are in there multiple times at different email addresses?
That also seems like a lot of money for polls like the ones they're doing. Would be one thing if these were done live by people, but it's all online. You could have it on your own site for a few thousand and then all you have to do is load the questions/choices each time.
7:19 p.m.
Feb 28, '12
Carla, you missed what I think is the most important part of the Christensen story (emphasis mine):
"Metro spent $76,000 on Opt In in 2011, generating more than 20,000 responses – about $4.50 per completed survey.
By comparison, said a staff report for Tuesday's work session, Metro spent about $400 per open house attendee during the 2010 roll-out of then-Metro chief operating officer Michael Jordan's growth and policy recommendations; those numbers soar to $2,800 per completed survey at each of those open houses. The agency also spent $35 per attendee at the dozens of stakeholder meetings Jordan attended."
Opt-In isn't the equivalent of voting, the survey doesn't sit on the council and cast votes (if you want it to do that you can try to get it elected). Opt-In is just another way for people to connect with Metro. Compared to traditional ways of doing community outreach it is cheap and reaches many more people. Why is that a bad thing?
7:56 a.m.
Feb 29, '12
It's not a bad thing. Unless you dump a bunch of money for the survey program and then decide to disregard it because it doesn't jive with you've already decided to do.
Then, not so great.
9:02 a.m.
Feb 29, '12
Hmmm. I don't think we should expect elected officials to simply put their finger in the wind and do whatever the voters want - whether in the region or the district.
Even if the system was a perfectly calibrated measurement device, that's not a definition of leadership that I recognize. We elect leaders to use their best judgment. And effective leaders work to shape public opinion as much as they reflect it.
That said, constituent input and feedback should be a critical part of the decision-making process. It can help leaders to understand the tradeoffs, discover previously-unrecognized impacted communities, and help shape the strategies they employ to engage in leadership.
If they're completing ignoring the feedback they're getting, then they should stop using it. But it's probably better to fine-tune the tool and figure out how to use it to get meaningful and actionable data.
11:27 a.m.
Feb 29, '12
Kari, you are way too generous. The statements from all parties involved are so misguided that I don't quite know where to start.
First, an "opt in" sample is almost always biased toward better educated, older, higher income respondents. They can weight the results if they want, but the response demographics are exactly what they should have expected if they'd even given this a moment of thought.
I hope Tom Hughes doesn't really mean what he said in the quote, because ANY survey is a "small slice" of the population. He's run for office enough times that I'm sure he knows this, which is why I think it's unfortunate that he's reinforcing a commonly held misperception about sample surveys.
Comparing per-respondent costs at a focus group to an opt-in survey is not helpful; these are completely different research devices intended to provide different kinds of information. They are not apples to apples comparisons.
Rebecca Ball's statement is similarly not helpful. Garbage responses are STILL garbage no matter how many you have. You can generate margins of error out the whazoo but they are meaningless because THEY ARE BASED ON THE ASSUMPTION OF RANDOM SAMPLING!
You can made far more valid inferences from 100 carefully sampled responses than from 305 opt-in responses. Metro could have easily oversampled among targeted groups if they were worried about the ability to draw inferences about those groups.
I'm sure that I'm applying scientific criteria to quotes from a new story, but some of these statements are misleading to the public.
The size of sample is what determines its precision, not the size relative to the population.
There are criteria and formulas for weighting Opt-In panels to make them more reflective of the population at large. I truly hope that is going on in anything reported to the Metro Counselors.
The website gives very little information methodology, so that's all I can say from this perch. I have no idea whether this is good informative work or not.
11:34 a.m.
Feb 29, '12
Ok, I need to retract a few paragraphs in that response. There are methodological statements at the website.
DHM makes it very clear how and why this tool is useful (see any of the published reports on the left hand side of the screen).
I don't understand Metro's response. They knew what this tool was and how it would be used. I can't believe DHM did not inform them ahead of time.
Carla is right, to now criticize a tool for providing precisely what it was supposed to provide is a bit disingenuous.
5:59 a.m.
Mar 1, '12
Opt-In isn't comparable to a focus group, but I think it is very comparable to an open house.
8:15 a.m.
Mar 1, '12
Quite a few people have expressed concern about the design of the actual surveys. I haven't got the details in front of me, but several folks have pointed out that some of the choices on a transpo survey sounded more like the political surveys that are designed to shape opinion rather than discover it.
10:07 a.m.
Mar 3, '12
The surveys are often pretty cringe-worthy, apparently designed to sample extremes of conviction about choices that are not in fact mutually exclusive. They seldom present options for compromises or win-win solutions that are under discussion and/or should be on the table. Often, the only way to register an informed opinion is in the comment box, outside the data stream.
I'm not sure who's responsible for survey content and/or form, but too often the polling lends undue credibiliity to simplistic talking points. Like those that heavy-hitting development interests have successfully sold to to many Washington County/Metro officials (e.g. if we want jobs, we need more flat farmland to offer big employers, plus more and bigger roads, bridges, dams, etc. to serve all the businesses and people who'll be pouring in here soon as the economy gets back to normal)
For a more-realistic view of our constraints and options, see Bill McKibben's "eaarth". (No, that's not a typo; he's making a point.)
11:44 a.m.
Mar 3, '12
Carla has a very accurate perception here as it completely fits thetop-down hierarchy that metro continues to exhibit as I've witnessed for 3 years now. Tom Highes needs to listen to his constituents vs. spend tons of monies for input that is completely ignored. The top-down hierarchy does not work for elected officials unless they carefully attend to their constituents. Tom Hughes drastically affects METRO and in very negative ways as our citizens in Washington County can attest to.