Under development

The plan for and results from user testing of UG areas of study and degree pages. By users we mean students who are either considering university study or have recently started, as well as staff who use the site to support student decisions.

Testing outcomes

1. Understand if the topic page leads to the outcomes/call to action in the content strategy
   1. Decides to learn more about the degree in which he/she can study a subject in this topic (at Victoria), evidenced by moving over to relevant degree page
   2. Decides this topic is not a good fit and move to another topic
   3. Contact us using the details provided.
2. Understand if the programme page leads to the outcomes/call to action in the content strategy
   1. Applies to enrol.
   2. Contacts us for help with course planning
   3. Registers for an open day/Student Recruitment event
   4. (Additional to the actions) Understand the degree requirements, or more specifically the courses he/she should enrol in next.
3. Identify where users have issues with the build
4. Identify any unmet user needs

Prerequisites

1. All study area pages are now available on our production servers. This means:
   1. SME have approved the content,
   2. Highest and high priority bugs have been fixed, and
   3. All high priority Confluence table corrections have been made.
2. Area of study navigation page includes links to all live areas of study page (not to the subject pages).
3. Old subject and degree pages have been deleted, or at least access has been deactivated.

Notes for facilitators

• We are doing "user testing", meaning that (after the greeting and introduction) they drive the computer and we guide, prompt and most importantly, observe.
• As natural as possible, given the circumstances and tasks
• Go with the flow, redirecting at the end if key areas (tabs, blocks, functions, etc) were not covered.
• Paul to facilitate and (ideally) somebody else to observe. Can always swap roles.
• Will request permission to record screen and voice.

Participants

• Five to ten, with some being from our previous senior secondary school students and others being new 1st year students.
• Do they have to be studying something in our first areas of study pages?
• $30 Prezzie card per participant
• Will meet in a testing or meeting room.

Equipment and set up

• Paul's own laptop or a standard deskop:
  • Internet acces
  • Screen recording software
  • Voice recording software
• Printed copies of ethics form: Participant Information Sheet.
• Printed copy of testing script.

Introduction

The introduction is very important.  

1. Put the student at ease and explain the purpose

Key messages:

• This part of our website aims to help new students decide what to study and to do so at Victoria University. Therefore, we need to ask you how well it meets your needs.
• We're interested in observing you play with our new web pages and listening to your feedback.
• We're not testing you or your computer or website skills! Every answer you give us is helpful and right! 
• Please don't hold back, as we need to hear all your thoughts, no matter how critical.
• Your participation is highly valued - you're helping make a difference - thank you.

2. Explain privacy - but in a low-key non-scary way

1. The testing is confidential

"You won't be identified in any notes - at the most we'll use your first name."

2. We'll use screen recording software. 

"We'll use software that records the screen in case we need to go back over anything after the session. You won't notice that it's running. It will record your voice, but not a photo of you."

3. Please sign the ethics form.

Conversation guide

1. Welcome, build rapport and set the scene 

Engage them in conversation about themselves - their interests and activities, leading to their study interests now. This might be what they are doing this year, how it's going, what they plan to do next year, etc.

Explain that user testing is used before, during and after development. That we have a continuous improvement process that allows us to consider user feedback.

Need to articulate your thoughts and impressions aloud, so that we can hear/capture them.

We have only written up one three or four areas of study and the relevant degrees so far. While you can browse the other areas of study we offer, for the purposes of this exercise we need you to pretend that you are considering one of the three or four.

2. Topics

Let's start with what Architecture looks like at Victoria.

About tab

Was the information clear? Easily understood? Do you know what UG and PG mean?

Do they use tool tips? Do they try clicking on different things?

Watch for how early they jump to subjects (and is it from tab control, tab advance control or links)

And even to degrees (in which case bring them back to topics once they finish their roaming).

"What content seems the most interesting, confusing, boring, etc?" Careers? Stories?

What did you learn on that page?" "How do you feel about that page?"

Do they get to the contact details block (here or on any tab)?

Subjects tab

Way and order content is consumed?

When choosing UG study is it of interest to know about PG options?

Do they try to use anchor links (not working)?

Is the "What you might study" section useful to you? Why? What else would you want to know?

Careers and People & Stories

Less important, so don't guide them to these if they don't naturally go there.

If they did ask why and whether page met their needs?

Close out

• Pressie card
• Copy of GUS 
•  Tell them how to find out about open days

After testing

Sit down, go through notes from testing session, and:

• Decide what problems are worth solving.
• Consider whether issues (i.e. interactivity limitations, unrelated content etc.) with the prototype may have negatively influenced any of the problems. 
• Suggest action steps to solve outlined problems. If resolving a problem needs team discussion, raise to the team meeting.
• Prioritise action steps.