The focus of this test plan is on study areas and navigation, as that is the information we most need to move our project on and to determine our priorities (eg. if people are consistently choosing search options, then we should prioritise SEO work)
Based on meeting of Friday, 26 June, with Andrew Bredenkamp, Joe Lobotka (Unlicensed), Paul Seiler (Unlicensed) and Sam Motion (Unlicensed).
We want to know/outcomes
- How students approach finding information about what they want to study (e.g. browse, search (from google or internal)
- How decisions are influenced by whether the student has some idea of what they want to do at university or not.
- How the terms in the current IA compare to the proposed new ones.
- Which terms students recognise and which don't make sense to them .
- How well our subject groupings work. We hope to see evidence that our approach would work better
Reaching students
We decided that gathering rich information through talking to students in person was our priority.
It can be adapted for remote testing or larger group testing if necessary.
School visits (preferred option)
Spend a day in each of three schools. Select schools with different demographic makeups.
We'd need to have staff identify students we can meet with beforehand. The only requirement would be that they be interested in going to University and have a reasonable shot at getting in. We could ask for one Māori or Pasifika student per school.
In the morning, meet with 2-3 students one-on-one. Meet for roughly an hour.
Over lunch, get together a group of around 8 students for a focus group.
In the afternoon, meet with 2-3 more students one-on-one.
Students visit us
This was not discussed in the meeting but as schools are going on holiday for two weeks next week it's an option we need to consider. This would involve reaching out to students either through schools, marketing channels or personal networks and asking them to come to us.
We could ask for a room at Te Aro or Pipitea so students don't have to come up to Kelburn.
Remote testing
Easy to set up and cheap to run, but the quality of responses will be lower than we'd like at this stage of the project. Something we can run alongside our other testing, or after to finesse our results and get numbers behind them. Could also be something we can get started if there are delays in reaching students.
Questionnaire
The below plan is written for a one-on-one or very small group testing situation. It could also be adapted for larger groups or remote testing. It comprises a set of core questions, which are worded to avoid leading the student to one particular answer through language.
In one-on-one situations, the interviewer should:
- encourage the student to talk about the decisions they are making
- ask questions about what words mean to the student and why they chose certain options.
1. Do you have some idea of what you’d like to do at university, or careers that interest you?
Purpose: Set up for test outcome 2. May also give us some terminology insight around specific subjects/programmes and careers. If hey have no idea, follow up by asking what subjects they like at school and choosing one of those areas to follow up on.
(Remote testing - no change)
2. [With blank browser tab open]
Let's say you wanted to see what Victoria has to offer in that area. Show me what you'd do.
Purpose: Test outcome 1. Note if the student starts at a search engine, what they search (Victoria? Programme? Career? Subject? Course?) Try not to guide them too much in the way that they find Victoria. Stop them before they get too far into the site, as we don’t want the current options to bias answers about future options.
(Remote testing - delete question. We could ask people to tell us about whether they prefer searching to browsing, but I wouldn't trust the answers)
Try not to guide them too much in the way that they find Victoria. Stop them before they get too far into the site, as we don’t want the current options to bias answers about future options.
(Remote testing - delete question)
4. High level navigation options.
It would be good to have these questions asked before the user has had too much time to look around the site, to avoid prejudicing their answers towards the current terms.
| Study at Victoria | Future students | Learning & Teaching | Faculties & schools |
| Current students | Life at Victoria | About us | Use search |
Note: these are a mix of current wording options and proposed future options. Options that don’t directly relate to this audience and these questions have been removed. Any that might cause confusion have been retained.
4a. If you were trying to decide what to do at University, where do you think you’d find information about what Victoria offers? Why? If they pick search, ask them to pick a second option too.
4b. Which would you pick if you wanted to enrol? Why?
Purpose: Test outcomes 1, 3, 4.
(Remote testing - add comment box after. remove option to select a second answer if they pick search)
5. Read the following options. Which one would you select if you were exploring what to do at University?
| Programmes | Degrees | Subjects | Topics |
| Study areas | Courses | Qualifications | Careers |
Purpose: Test outcome 4
(Remote testing - add comment box after)
6. The next exercise is about seeing how easy it is for you to find information about a particular subject we offer at Victoria. We have an online tool for this.
Bring up the treejack test. (URL will be: https://victoria.optimalworkshop.com/treejack/july-subjects we should have this open in a different window or tag).
The participant just needs to put in their name (so we can match their results to other notes later).
They will see some visual instructions on how to do the task.
The questions have been loaded into the tool, so the interviewer can read them as they come up.
6a. Earlier we talked about things you might be interested in studying. Where would you find information about that?
They see the list of 15 top-level areas and work though the navigation to find the thing they want. When they find it, they can hit a button called "I'd find it here".
6b. Let's try another subject you're interested in. Where would you find information about that?
Same as above. I have made this task 'skipable' so if time is short you can move past it and close the tool.