meaning: The Macer View/5

	The Macer View
	Comment and analysis from Tim Macer Number 5, August 2003

Resources

Software reviews

Software database

Reports & Thinkpieces

Conference papers

Press articles

The Macer View

Defining best practice in the use of research technology

As a member of the judging panel for the MRS/ASC Joint Award for Technology Effectiveness, I am working through a reassuringly healthy crop of entries, which are a cause for celebration and concern.

Celebration, because the range of entries shows that, despite the economic doldrums of IT, imagination and pragmatism are still alive among our specialist technology developers and in-house teams.

The concern is not due to the entries or the entrants, but to the dilemma of how to judge excellence when we, as an industry, have never set out what we mean by good practice in the technology arena.
The MRS and ESOMAR are pretty much silent on this whole area. Take weighting for example. There is a Code of Conduct obligation to distinguish between weighted and unweighted data and not to pass off one as the other. But the profession offers no guidance on where weighting becomes unreliable, or how to detect unreliability.

Weighting, in the hands of the unscrupulous, can transform dodgy data into sampling frame nirvana. Done wittingly, or unwittingly, the result is equally unacceptable from a good practice perspective.
Take editing. How much editing of data is acceptable? What obligations are there to keep audit trails and ensure processes are both understandable and reversible?

We are moving into an era of mixed modes of interviewing because technology now allows it. These layers add more complexity, which can influence results or mean systematic errors go undetected. At present, many of these errors and influences are barely understood.

The award is certainly stimulating some debate among the judging panel on what is good practice in the application of technology, but it is a debate that the industry as a whole must engage in. We need to do this, not just through prescriptive codes of conduct, but also through professional training and qualifications. And we need to ensure that software suppliers cannot forever dodge behind the caveat that it is a matter of ‘how you use it’ and therefore not their responsibility to make it easier to do the right thing than the wrong thing.

What we really need is an industry-wide better practice framework for computer assisted interviewing and analysis that both technology users and technology creators can follow. But first, we need to start discussing what the ‘right thing’ is.