Usability interviews

The third experiment evaluated Prompt and the alternative user-interface CogZ. The experiment focused on evaluating the cognitive support provided by the tools in terms of their effectiveness, efficiency, and satisfaction [Falconer 2009], Researchers assigned eighteen matching and comprehension tasks to participants that they had to perform using each tool (nine per tool). The evaluators then measured the time that it took a participant to complete the task and accuracy with which they performed the task. They measured the participant satisfaction via exit interviews and the System Usability Scale [Brooke 1996],... 

When viewing the interviews, make notes on the transcripts to help you remember what someone s energy level is like, if there are problems such as flies or a microphone in the frame, or whether someone sneezes. Some portions of the interview may be usable but only as voice-over others may be useful as information only. Better to write it down once than to go back to the same bite three times in the course of the editing session because you forgot that there was a reason you didn t use it in the first place. (You re also writing down time code that corresponds to the transcript, so you can find material quickly.)... 

Both interview and questionnaire techniques were used in this survey. Fifty, semi-structured interviews were conducted on-site with a randomly selected, representative, stratified sample of employees. (See Appendix B for an example of some of the prompt questions used in the interview note that this can only be developed after some open interviews/discussion sessions are completed.) A questiormaire was generated from the interview data and distributed to 520 staff to be completed anonymously. This produced a 45 per cent usable response rate for analysis. The objectives were to gain an understanding of the perceptions of quality within the company and readiness for change to provide baseline data for evaluation purposes to identify quality improvement opportunities and potential barriers to change and to help management develop a sense of awareness about quality and quality improvement needs in the plant. The questionnaire included measures of job satisfaction (Warr et aL, 1979), organizational commitment (Porter et al., 1974), perceptions of cooperation and morale in the plant and measures of certain job characteristics associated with quality work performance, such as skill variety, autonomy and feedback (Hackman and Oldham, 1975). 

The usability test consisted of five phases Introduction, training session, scenarios, questionnaire and interview. In the introduction the participants were provided a small introduction of the evaluation and the prototype, and were then asked to sign a form of consent. In order to prepare the users for the prototype, there was a small training session in which the users were trained in the functionality of the prototype. A technical assistant was the tutor for this session, and helped the participants navigate and understand the functionality of... 

After the performance of the tasks, an interview was carried out. During that interview the participants were asked to fill in two questionnaires. One questionnaire addressed system s usability (SUS). The other questionnaire was used to indicate how active or passive the system was during usage. This active/passive questionnaire was adopted from the AttrakDiff [14]. 

The company is expecting some concrete advantages (from the improvement effort) to appear. (...) The product should be more usable, and there should appear clear savings in money and increase in sales. (Interview, usability speciaUst)... 

Eric (a development manager) told me the basic reasons why we would participate. We need to be more convincing in the eyes of the customer. That way we could dictate some things, for example UI issues. The project would offer facts which could enable us to do that. (Interview, usability specialist)... 

When we were making the budget for this year, the question was why spend resources on this (usability) It costs money when people participate in this they spend time on that. What can you get out of this (...) This type of questioning exists and it is good, because it all comes down to money and resources. We have limited resources and must have clear arguments. (Interview, development manager)... 

Until now, we have observed one company in Denmark, one in India and five companies in China. In each company, we did field observation with video cameras of thinking aloud usability test sessions and afterwards interviewed the evaluators, test users and the usability department manager to get more deep information. We used three observer/interviewers a Dane, an Indian and a Chinese. I will analyze the data in my PhD thesis. 

Vatrapu, R., Perez-Quinones, M.A. Culture and Usability Evaluation The Effects of Culture in Structured Interviews. Journal of Usability Studies 1(4), 156-170 (2006)... 

---