This wiki page is provided for recording small group discussions about issues relating to the assessment of student contributions to virtual world activities.
Participants
Denise Chalmers
Glenn Finger
Scott Grant
Catherine McLoughlin
Scribe: Ardis Cheng
Discussion question:
When university students are asked to demonstrate their learning by participating in a virtual world environment, what academic standards, and assessment and reporting practices are essential or desirable?
Authoring? Creation? Interaction?
Second Life - given island and students given some authoring (create own avatar) but others can be more restricted
Lecture theatre - avatars come into tutorial group virtually
Traditional course design - how to design assessment using virtual worlds? Its a space.
Choce of name is unfortunate
Social purposes - term is true to name but educationally, not the same - it is an interaction space
Immersion - medical space - students engage in life in this space - it is a separate space
Platform for enabking communication
For different geographic spaces
Ability to be in the same virtual space in real time is most important
Peer interaction
Various spaces for students to go into - social space, cafe space
Immediacy to connect to other students or other people
Not significant teacher involvement - more "play"
Experiential
(Scott Grant examples)
Beginner chinese students - task-based, encourage team work, peer support, very focused
Media students - interaction of other people, not just students - both in and out of university
Task based activities - beginner chinese students - also very cognitive (ordering food, shopping), getting them to think of cultural skills etc.
Similar to role-playing
Experessive dimension, student can create something new, being creative, not just acting out a pre-scripted role
The task not totally controlled, no script - there are some conditions (like which type to buy) - why do it in virtual environment? Because of the numbers - can't just send 100 students to a restaurant
Learning about the processes about doing things in China - what is the process
VW - second life, virtual classroom/space with simulated activities/tasks
Network learning environments
What kind of assessment in these spaces?
Risk management - doing things in an interface istead of messing up in a lecture, etc
Traditional forms of assessment? Co-created assignments? Other web 2.0 technologies?
Social experiential vs cognitive processes
A play and social area - immediacy, "play"
Beginner students - going in and practice their skills
MEdia students - 3 former assessments - classroom based assignments evaluating articles/chinese media, interviewees to speak with, working people active in secodn life - students shown around a location, the interviewee talks about it, students ask questions, then the interviewee comes back and they do a chat show (this all done in second life) - setting/editing etc done by group in States who are studying the fiming/editing process - assessment conditions set prior to the chat show, and then create a written news report (separate to second life) that was assessed more traditionally and then a formal news report (reading) - clear rubric defined as well for this last stage
Problem - detailed rubric meant students went by it word for word
Blended learning strategies - using different technologies for different activities - perhaps not so much for assessment but for the interaction
Media studies assessed as groups and as individuals
All tasks may have had all elements but some may have been more heavily weighted as individuals or as a group
Learning environment and learning is central, assessment is the formal stuff (validity, reliability)
Students engage in the assessment - this is a problem
If students don't see how it works with the assessment, less driven to do it
Integration of developmental and summative - getting feedback from peers and from instructors - easier to plan the VW because that's what they're engaging in
Factors such as safety, geography, creating a certain mood are conducive to using a VW but VW is not 100% necessary
Incidental information and targeted information getting picked up by the students in VW - but how do you assess that?
Environment - presenting parts of China - learning should be student focused and he presents a window and let them choose their perspective
Lesson design and task
What they do hopefully translates into the real world - but how to assess that?
How much do instructors contribute? Set-up?
Students not a lot of input - the end goal is set but the way to get there, there are many options. Once task starts, instructors step back and let the students go - ex. looking at window for advertisements or go to someone to ask
Social construcivism - instructors there in a supportive role, peer interaction
Want to measure peer-to-peer learning
Non-character players are flexible (not word for word), gives instructors to be able to walk around and address individual students - gives instructors time - they can go around and help individuals oppose to a normal tutorial where you're engaged with everyone unlike a non-character player for other students to interact with
Assessment principles?
Integration of development and parts that gets assessed - key tasks to build
Issues with team based and individual based
Pre-testing - give students info about a process, let them do the task, then follow up with a post-teat (task-based learning - very sim to traditional methods)
We haven't crossed the bridge yet - using new technologies but using traditional assessment methods
Real-time engagement across locations in similar tasks
Students taken to a place with an interviewee in VW - its very open ended for students
Media studies - the environment tells them little about the place but its about the interaction and engagement
Beginner studies - more focused on cultural, being able to pull info out
James Paul Gee(?) "If they've completed the task then they've achieved" (very narrow) - but how does this fit in a uni environment?
All students work in the VW all in the same lab (to ensure that the students are the avatars) at the same time - phsyically present so can be sure that they are the avatar - but this doesn't work when working with people in different geographic locations
Rubric guiding students? Details on how to get to the next level. Sometimes, the examination committee scrutinize marks (uneven distributions) which makes it hard to justify - it is good for the learning environment but for admin, it can be difficult
Need to make sure why tasks aren't completed - was it the technology?
Shouldn't have a normal curve with marks
If can demonstrate why all the students might be at one end of the curve, then that should be good
No moderation, no checking - problems of people just spreading things over a normal curve
Academic standards (it is important to follow these) - validity, reliability
Instructor creates the environment (no student authoring) vs students do author (this engagement can be broken down into more generalizable skills - the process and problem-solving, not so much the end product)
As long as its clear what you're asking of the students
Creation of development and summative end-product
What is good assessment? Students doing work, authentic, related to context of what they're working, clear to students
Driven by principles of good assessment - when design a course, are driven by learning objectives, blend in assessment (doesn't have to be VW or product)
Working in space, sharing with peers - do they acknowledge who contributed what? This is important
In VW, students can create their own stuff but how do they convince instructor that they are learning? Critical reflection - deeper learning
The final news report task - students should be able to look at other Chineses reports and think about what is involved (move beyond the superficial)
Assessment prinicples still apply
Use of VW is driven by learning requirements (the learning required is separate from the platform - important that you are not using the platform purely for using the platform and just for the sake of assessment)
Need for clear objectives - process, developmental, experiential, social, participation, production
Students are working in a new environment - need to make distinction of the student's role
This generation of students is not that engaged with this technology (the emerging technologies - more familiar with submersive technologies)
Avg age 30-35 in Second Life
This generation more practival minded - don't explore or play, need some form of assessment - what is the value to the student?
Need a rubric
Foster self-direction - student/teacher created (negotiation)
Distinguish between group and individual creation (averaging of marks - high students brought down, lowers students brought up)
Group WITH individual assessments - in the end, the individual gets the grade
In a VW, the group is important because of the interaction
Capacity to moderate and evaluate to assign grades
External accreditors should be able to a look at the work
Pros of using VW are: text-based work can be logged and reviewed later, can record voice (audio more important than video), (with media students), and provided with rubric, a person could take a look at all of this and make an assessment
In a normal classroom, can't capture a lot but you can in a VW
Combos (ex. Noodle and Second Life) can be good to use
Other pros of VW: digital timeline, logged dates, collaborative space for the assessers, persistnent online environment (people can come in after the fact and review) - can get different people to come and take a look, becomes an artefact of their learning for students, instructor and also review and compare it back to their rubric, potential for student to log in reflections (students did go in and listen in to their recordings)