Hans Hummel
UBC 24 maart 2010
Derek Kuipers & Wim Westera
Hans Hummel, Walter Geerts, Aad Slootmaker,
dilemmas in teacher education
Optimizing online group work on classroom
Structuring Collaboration Scripts
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Use serious games to enable collaboration on
professional problems
• Explain the role of scripts in collaboration games
• Define and optimize the structure of collaboration scripts
• Introduce the mastership game
• Pose hypotheses
Introduction
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Professionals are lifelong learners that continuously
face problem situations in their workplace that change
dynamically and rapidly
• Organisations’ tacit knowledge can only be expressed
and accessed in direct collaboration on professional
tasks (e.g., Nonaka & Takeuchi, 1995)
• Games may take workplace contexts as starting point
to stimulate learners acquire new knowledge by
sharing and co-creating (e.g., Bell et al., 2008)
Introduction: workplace learning
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Collaboration scripts (Kobbe et al., 2007) are an
instructional method that structures the collaboration by
guiding the interacting partners online through a
sequence of interaction phases with designated activities
and roles
• Collaboration scripts have hardly been tested in more
open and flexible learning environments like serious
games (Dillenbourg & Hong, 2008)
• No research has focused on optimizing essential
structure elements, nor has measured the learning
effects of collaboration scripts in serious game play
Introduction: collaboration scripts
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• An optimal level of structure appears to be a key
success factor for effective learner support
• Risks of over-scripting: ‘mathematanic’ versus
‘mathemagenic’ (Rothkopf, 1996)
• Segmentation and inter-dependency within the task
constitute the main structure elements (e.g.,
Dillenbourg, 2002)
Introduction: structure
Hans Hummel
CSEDU2014 / 1-3 abril 2014
Selecting a classroom dilemma (phase 1)
Hans Hummel
CSEDU2014 / 1-3 abril 2014
Discussing assigned themes (phase 3)
Hans Hummel
CSEDU2014 / 1-3 abril 2014
Negotiating declined themes (phase 4)
Hans Hummel
CSEDU2014 / 1-3 abril 2014
Peer assessment (phase 6)
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Will less structure lead to more ‘natural’ and effective
collaborative learning? (We hypothesize that the
individual reports of those that played the low-structured
game will be objectively graded higher by their teachers
and peers)
• Will less structure in the collaboration be appreciated
more by students? (We hypothesize that students will
subjectively appreciate the low-structured game higher
on a number of aspects)
Introduction: hypotheses
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• 29 teachers-in-training
• Experiment with 3 conditions: high- and low structured
game, (non-playing) control group ( definition,
elements of structure)
• Learning effects measures to proof first hypothesis
( correction model)
• Student satisfaction measures to proof second
hypothesis (
questionnaire)
Method
true
false
true/false
true/false
true
true/false
true/false
true
true
true
-
1_2
2_1
2_2
2_3
3_1
3_2
4_1
4_2
4_3
5_1
5_2
6_1
6_2
Hans Hummel
-
-
-
-
-
-
-
-
3/2/1
-
-
-
1
3/2/1
(cards to draw)
(has to finish scene)
true
a2
a1
1_1
Scene
-
-
-
true/false
true
true
false
false
false
true/false
false
false
false
false
(feedback required)
b2
CSEDU2014 / 1-3 abril 2014
-
-
-
true/false
true
true
true/false
true/false
true/false
true/false
true/false
true/false
true/false
true/false
(wait for others)
b1
-
-
-
false
false
false
false
false
true/false
false
false
false
false
false
(others draw cards)
b3
Method: structure elements
The elaboration is
not correct
E. Richness / correctness
Hans Hummel
The answer does
not contain nuance
D. Nuance / complexity
Total score
The problem has
not been framed /
focused
No reflection
B. Reflection
C. Focus
Refers to others:
“They will solve the
problem”
(0 points)
Insufficient
A. Ownership
Subscales
Good
CSEDU2014 / 1-3 abril 2014
The
elaboration is
partly rich and
correct
The elaboration is
rich and correct
0-10
0-2
0-2
The answer is
correctly linked to (a
network of) more
design patterns
The answer is
correctly linked to
one design
pattern
0-2
The problem is
rich and has been
correctly focused
The problem
has partly been
focused
0-2
0-2
Score
Rich reflection
The answer
shows real
commitment.
(2 points)
Some
reflection, partly
rich
“I will take
action”
(1 point)
Sufficient
Method: correction model
Aspect
U
Q
Q
U
S
T
S
U
S
T
U
I
Q
Q
Item
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Feedback (assigning cards, peer assessment, etc.) from co-players was useful (in
further elaborating my assignment).
Mutual interaction and collaboration proceeded well and were useful.
The way to collaborate during each phase was too complex.
The time allowed for each phase was too high.
The amount of structure in each phase is too high.
The time allowed for each phase was too low.
The amount of game structure is too low.
The time allowed to play was too low.
Group play was possible without teacher intervention, the collaboration process has
been determined well in advance.
The user-interface of the game is clear and user-friendly.
The composition of the group was good (regarding interest and level of expertise).
The elaborations (of practical assignments) by co-players were of sufficient quality
The way to play the game is clear, playing rules are clear.
Statement
quality.
Hans
1-3 abril 2014
The elaborationsCSEDU2014
of the exploratory /assignments
by co-players were of sufficient
+ Hummel
+
+
-
-
-
+
-
-
+
+
+
+
+
+/-
Method: satisfaction questionnaire
7.93
Peer rating
.66
1.59
Hans Hummel
6.44
Teacher grade
SD
1.04
1.03
SD
7.68
6.05
M
0.89
0.93
SD
(n = 10)
Control
CSEDU2014 / 1-3 abril 2014
7.52
7.35
M
(n = 10)
(n = 9)
M
structure
structure
Assessment
Low
High
Results: teacher grades
7.70
6.62
M
0.87
1.29
SD
(N = 29)
All
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Most individual reports (76%) were graded as sufficient
(M = 6.62, SD = 1.29)
• Condition effect is ‘marginally’ significant
(F (2, 26) = 3.072, MSE = 4.428, p = 0.063, ηp2 = 0.18)
• Low-structure outperforms control significantly
(t (18) = 2.97, p < 0.01), an effect not significant for highstructure (t (17) = 0.67, p = 0.51)
• Low-structure outperforms high-structure significantly
(t (17) = 4,86, p = 0.042)
• Weighted Kappa was acceptable (Kw = .47, δ = .08), and
without sub-scale B even good (Kw = .68, δ = .09)
Results: teacher grades
1.41
0.76
1.31
1.25
1.40
0.89
1.25
1.49
0.87
0.83
0.53
1.51
1.36
0.99
0.99
1.31
1.04
1.07
1.19
Hans Hummel
3.00
4.00
3.50
2.88
2.38
2.25
3.13
3.25
2.75
2.13
4.00
3.38
2.88
3.13
2.88
2.50
4.25
3.00
3.00
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
SD
2.70
4.00
4.20
3.50
2.67
1.40
3.40
4.20
2.30
2.20
2.90
2.80
3.00
3.33
2.33
2.56
3.56
3.67
3.22
M
2.85
4.00
3.89
3.22
2.53
1.78
3.28
3.78
2.50
2.17
3.39
3.06
2.91
3.21
2.73
2.53
3.88
3.18
3.12
M
1.30
0.76
1.13
1.17
1.42
0.81
1.13
1.31
0.92
1.15
1.19
1.35
1.14
0.89
1.01
1.23
1.16
1.08
1.36
SD
(N = 19)
All
CSEDU2014 / 1-3 abril 2014
1.25
0.82
0.92
1.08
1.50
0.52
1.07
1.03
0.95
1.40
1.37
1.23
0.90
0.82
1.16
1.24
1.24
1.55
1.56
SD
(n = 10)
(n = 9)
M
structure
structure
Item
Low
High
.64
1.00
.20
.27
.67
.02
.62
.13
.32
.89
.04
.38
.88
.68
.46
.93
.23
.39
.75
p∆
∆
Results: student satisfaction
“The way to
collaborate
during each
phase was too
complex”
“Time allowed
to play was
too low”
2.25
3.13
3.25
2.75
2.13
4.00
3.38
2.88
3.13
2.88
2.50
4.25
3.00
3.00
6
7
8
9
10
11
12
13
14
15
16
17
18
19
1.51
1.36
0.99
0.99
1.31
1.04
1.07
1.19
0.53
1.25
1.49
0.87
0.83
0.89
1.41
0.76
1.31
1.25
1.40
Hans Hummel
3.00
4.00
3.50
2.88
2.38
1
2
3
4
5
SD
2.80
3.00
3.33
2.33
2.56
3.56
3.67
3.22
2.90
3.40
4.20
2.30
2.20
1.40
2.70
4.00
4.20
3.50
2.67
M
3.06
2.91
3.21
2.73
2.53
3.88
3.18
3.12
3.39
3.28
3.78
2.50
2.17
1.78
2.85
4.00
3.89
3.22
2.53
M
1.35
1.14
0.89
1.01
1.23
1.16
1.08
1.36
1.19
1.13
1.31
0.92
1.15
0.81
1.30
0.76
1.13
1.17
1.42
SD
(N = 19)
All
CSEDU2014 / 1-3 abril 2014
1.23
0.90
0.82
1.16
1.24
1.24
1.55
1.56
1.37
1.07
1.03
0.95
1.40
0.52
1.25
0.82
0.92
1.08
1.50
SD
(n = 10)
(n = 9)
M
structure
structure
Item
Low
High
.38
.88
.68
.46
.93
.23
.39
.75
.04
.62
.13
.32
.89
.02
.64
1.00
.20
.27
.67
p∆
∆
Results: student satisfaction
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• No significant differences for peer ratings
• No significant differences on student
satisfaction, with just two exceptions
Results: student satisfaction
Hans Hummel
CSEDU2014 / 1-3 abril 2014
• Collaboration can be enabled by serious games
• Over-scripting has disruptive effects (e.g., order
and turns)
• Collaboration scripting holds promise for future
research but requires a. theory of guidance,
b. dedicated authoring, and c. studies into
various collaboration patterns
Conclusions
Hans Hummel
CSEDU2014 / 1-3 abril 2014
Questions / Discussion

Download Report