Page Comparison

...

...

...

...

...

...

...

...

...

Based on meeting of Friday, 26 June, with Andrew Bredenkamp, Joe Lobotka (Unlicensed), Paul Seiler (Unlicensed) and Sam Motion (Unlicensed). 

We want to know/outcomes

1. How students approach finding information about what they want to study (e.g. browse, search (from google or internal)
2. How decisions are influenced by whether the student has some idea of what they want to do at university or not.
3. How the terms in the current IA compare to the proposed new ones.
4. Which terms students recognise and which don't make sense to them .
5. How well our subject groupings work. We hope to see evidence that our approach would work better 

Reaching students

We decided that gathering rich information through talking to students in person was our priority. 

It can be adapted for remote testing or larger group testing if necessary. 

School visits (preferred option)

Spend a day in each of three schools. Select schools with different demographic makeups. 

We'd need to have staff identify students we can meet with beforehand. The only requirement would be that they be interested in going to University and have a reasonable shot at getting in. We could ask for one Māori or Pasifika student per school. 

In the morning, meet with 2-3 students one-on-one. Meet for roughly an hour. 

Over lunch, get together a group of around 8 students for a focus group. 

In the afternoon, meet with 2-3 more students one-on-one.

Students visit us 

This was not discussed in the meeting but as schools are going on holiday for two weeks next week it's an option we need to consider. This would involve reaching out to students either through schools, marketing channels or personal networks and asking them to come to us.

We could ask for a room at Te Aro or Pipitea so students don't have to come up to Kelburn.

Remote testing

Easy to set up and cheap to run, but the quality of responses will be lower than we'd like at this stage of the project. Something we can run alongside our other testing, or after to finesse our results and get numbers behind them. Could also be something we can get started if there are delays in reaching students. 

Questionnaire

The below plan is written for a one-on-one or very small group testing situation. It could also be adapted for larger groups or remote testing. It comprises a set of core questions, which are worded to avoid leading the student to one particular answer through language. 

In one-on-one situations, the interviewer should:

• encourage the student to talk about the decisions they are making
• ask questions about what words mean to the student and why they chose certain options.

1. Do you have some idea of what you’d like to do at university, or careers that interest you?

Purpose: Set up for test outcome 2. May also give us some terminology insight around specific subjects/programmes and careers. If hey have no idea, follow up by asking what subjects they like at school and choosing one of those areas to follow up on. 

(Remote testing - no change)

2. [With blank browser tab open] 

Let's say you wanted to see what Victoria has to offer in that area. Show me what you'd do. 

Purpose: Test outcome 1. Note if the student starts at a search engine, what they search (Victoria? Programme? Career? Subject? Course?) Try not to guide them too much in the way that they find Victoria. Stop them before they get too far into the site, as we don’t want the current options to bias answers about future options.

(Remote testing - delete question. We could ask people to tell us about whether they prefer searching to browsing, but I wouldn't trust the answers)

Try not to guide them too much in the way that they find Victoria. Stop them before they get too far into the site, as we don’t want the current options to bias answers about future options.

(Remote testing - delete question)

4. High level navigation options.

...

Completed - page retained here for reference

Testing outcomes

1. Understand what different navigation labels 'say' to school leavers.
2. Understand what information school leavers need and what they need to see first. 
3. Understand the relevance of faculties and schools to school leavers.
4. Learn how school leavers deal with the terminology we use: which terms are understood by them and which are unclear.
5. Test how well our subject groupings work.

And while we are seeking these outcomes I hope that we can identify the main gaols for our personas

Notes for facilitators

• Is more 'research' than 'testing'.
• Conversational - discussion evolves naturally. Not an interview.
• But make sure key areas are covered.
• Recommend that one person facilitates the discussion and the other makes observations.
• Note taking should be unobtrusive. Leave time for discussing and recording findings after the session.

Participants

• Five or six interviews and then determine if we need more
• $30 Prezzie card per participant
• Will meet at Law or VBS to make transport easier for them

Equipment and set up

• Victoria laptop with the following:
  • Vic site open in Chrome
  • Vic site open in IE
  • Screen recording software
  • Treejack test in a new window
• Printed copies of ethics form: Participant Information Sheet.
• Printed copies of IA tasks (tasks 2 and 3)
• Printed copy of test script.

Introduction

The introduction is very important.  

1. Put the student at ease and explain the purpose

Key messages:

• We're interested in learning about your information needs when you're thinking about tertiary study.
• There will be a bit a chatting and we'll also get you to do a few things online.
• We're not testing you or your website skills! Every answer you give us is helpful and RIGHT!
• Your participation is highly valued - you're helping make a difference - thank you.

2. Explain privacy - but in a low-key non-scary way

1. The testing is confidential

"You won't be identified in any notes - at the most we'll use your first name."

2. We'll use screen recording software. 

"We'll use software that records the screen in case we need to go back over anything after the session. You won't notice that it's running. It will record your voice, but not a photo of you."

3. Please sign the ethics form.

Conversation guide

1. Interests/scene setting

Talk about them as a person - their interests, leading to their study interests now. Leading to tertiary study - goals, hidden aims - worries, challenges.

Find out their level of understanding of how university works - stage of preparedness.

2. Exploring and talking

Let's have a look at the information that's on the Victoria website now to do with (student interest area). Have a go.

Victoria site is open.

Student will either browse or use the site search (or a mix).

Talk to the student about what they are looking for - get them to talk out loud.

What's important here is what they are interested in - what they need to know (and in which order) rather than how well the current site works for them.

Explore:

• What are their personal priorities? (eg, career info, specific field, degree info)
• What's useful, and what's less useful?
• Did they find out what they were looking for? Was it enough? What's missing?
• What do they need to find or know for the topic they're interested in?

Is there anything else you need to know when thinking about enrolling at a university?

(Student may mention costs, student life, who are the lecturers, faculties and schools)

3. High-level navigation options

(a) Where would you look on the Victoria website to find information to help you make decisions about university study? Why?

(If they pick search, ask them to pick a second option too.)

Note: these are a mix of current wording options and proposed future options. Options that don’t directly relate to this audience and these questions have been removed. Any that might cause confusion have been retained. 

4a. If you were trying to decide what to do at University, where do you think you’d find information about what Victoria offers? Why? If they pick search, ask them to pick a second option too.

4b. Which would you pick if you wanted to enrol? Why?

Purpose: Test outcomes 1, 3, 4. 

(Remote testing - add comment box after. remove option to select a second answer if they pick search)

5. Read the following options. Which one would you select if you were exploring what to do at University?

Purpose: Test outcome 4

(Remote testing - add comment box after)

...

(b) Which area would you select next? Why?

(c) Going back to the first list, which would you pick if you wanted to enrol? Why?

4. Treejack

We're looking at new options for how we provide information about areas of study. We'd like you to use an online tool to help with this.

Bring up the treejack test. (URL will be: https://victoria.optimalworkshop.com/treejack/july-subjectswe  We should have this open opened in a different window or tag). 

The participant just needs to put in their name (so we can match their results to other notes later). 

They will see some visual instructions on how to do the task. 

The questions have been loaded into the tool, so the interviewer can read them as they come up. 

6a. Earlier we talked about things you might be interested in studying. Where would you find information about that?

• Task 1: They see the list of 15 top-level areas and work though the navigation to find the thing they want. When they find it, they

...

• hit a button called "I'd find it here".

6b. Let's try another subject you're interested in. Where would you find information about that?

Same as above. I have made this task 'skipable' so if time is short you can move past it and close the tool. 

...

• Task 2: Same action and visuals as above. Task can be skipped if you're short of time.

5. Faculty and school

Would you be looking for info from a faculty or school?

What's your understanding of the role of a faculty or school at a university? (Compare with school departments)

Where on the Victoria Uni website would you look to find a faculty or school page?

Close out

• Pressie card
• Copy of GUS 
•  Tell them how to find out about open days