The focus of this test plan is on study areas and navigation, as that is the information we most need to move our project on and to determine our priorities (eg. if people are consistently choosing search options, then we should prioritise SEO work)
Based on meeting of Friday, 26 June, with Andrew Bredenkamp, Joe Lobotka (Unlicensed), Paul Seiler (Unlicensed) and Sam Motion (Unlicensed).
We want to know/outcomes
- How students approach finding information about what they want to study (e.g. browse, search (from google or internal)
- How decisions are influenced by whether the student has some idea of what they want to do at university or not.
- How the terms in the current IA compare to the proposed new ones.
- Which terms students recognise and which don't make sense to them .
- How well our subject groupings work. We hope to see evidence that our approach would work better
Reaching students
We decided that gathering rich information through talking to students in person was our priority.
It can be adapted for remote testing or larger group testing if necessary.
School visits (preferred option)
Spend a day in each of three schools. Select schools with different demographic makeups.
We'd need to have staff identify students we can meet with beforehand. The only requirement would be that they be interested in going to University and have a reasonable shot at getting in. We could ask for one Māori or Pasifika student per school.
In the morning, meet with 2-3 students one-on-one. Meet for roughly an hour.
Over lunch, get together a group of around 8 students for a focus group.
In the afternoon, meet with 2-3 more students one-on-one.
Students visit us
This was not discussed in the meeting but as schools are going on holiday for two weeks next week it's an option we need to consider. This would involve reaching out to students either through schools, marketing channels or personal networks and asking them to come to us.
We could ask for a room at Te Aro or Pipitea so students don't have to come up to Kelburn.
Remote testing
Easy to set up and cheap to run, but the quality of responses will be lower than we'd like at this stage of the project. Something we can run alongside our other testing, or after to finesse our results and get numbers behind them. Could also be something we can get started if there are delays in reaching students.
Questionnaire
The below plan is written for a one-on-one or very small group testing situation. It could also be adapted for larger groups or remote testing. It comprises a set of core questions, which are worded to avoid leading the student to one particular answer through language.
In one-on-one situations, the interviewer should:
- encourage the student to talk about the decisions they are making
- ask questions about what words mean to the student and why they chose certain options.
1. Do you have some idea of what you’d like to do at university, or careers that interest you? Y/N
1a. If yes, what are you considering?
Purpose: Set up for test outcome 2. May also give us some terminology insight around specific subjects/programmes and careers.
(Remote testing - no change)
2. [With blank browser tab open] Show us how you’d start exploring your university options.
Purpose: Test outcome 1. Note if the student starts at a search engine, what they search (Programme? Career? Subject? Course?), or if they go straight to a particular website.
(Remote testing - delete question. We could ask people to tell us about whether they prefer searching to browsing, but I wouldn't trust the answers)
3. [Still in browser] Let’s look at your options at Victoria.
Purpose: Test outcome 1. Make notes about how they navigate to Victoria’s website – do they search their subject and Victoria? Do they find Victoria’s site then search/browse?
Notes: Try not to guide them too much in the way that they find Victoria. Stop them before they get too far into the site, as we don’t want the current options to bias answers about future options.
(Remote testing - delete question)
4. High level navigation options.
| Study at Victoria | Future students | Learning & Teaching | Faculties & schools |
| Current students | Life at Victoria | About us | Use search |
Note: these are a mix of current wording options and proposed future options. Options that don’t directly relate to this audience and these questions have been removed. Any that might cause confusion have been retained.
4a. If you were trying to decide what to do at University, where do you think you’d find information about what Victoria offers? Why? If they pick search, ask them to pick a second option too.
4b. Which would you pick if you wanted to enrol? Why?
Purpose: Test outcomes 1, 3, 4.
(Remote testing - add comment box after. remove option to select a second answer if they pick search)
5. Read the following options. Which one would you select if you were exploring what to do at University?
| Programmes | Degrees | Subjects | Topics |
| Study areas | Courses | Qualifications | Careers |
Purpose: Test outcome 4
(Remote testing - add comment box after)
6. The next exercise is about seeing how easy it is for you to find information about a particular subject we teach at Victoria. We'll show you some navigation options, and you follow the path that makes sense to you.
6a. To start, find something you're interested in studying.
7. Tree-jack questions (Paul Seiler (Unlicensed) add them here if you like)