EOY MOSL Assessment Administration and Scoring - N211

EOY MOSL Assessment Administration and Scoring
April 9th and 11th 2014
Network 211
Jean McKeon, Network Leader
Network 211 MoSL Points: Josh Metz, Sharon Mahabir
Data/Application Support: Iris Battino Bernstein
Cluster 2 MoSL Specialists: Nadya Awadallah, Anna Kavrakis
1
EOY MOSL ASSESSMENT
ADMINISTRATION
2
3
Scheduling Suggestions…
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Submit
Scores
Archive
Assessment
Materials
Norming and Scoring may occur during the following
times:
•
Faculty Meetings and grade-level conferences
•
Per session scoring time (afterschool, etc.) – SAM memo
•
Extended day time (37.5 minute periods)
•
Anniversary Day, 6/5 (formerly Brooklyn Queens Day)
•
June Clerical Half Day, 6/3 (elementary or middle schools)
•
Teacher release time / coverages
•
Early Release Days
*Submit a calendar change request (if eligible) for a shortened
day or full PD day to Norm and Score NYC Performance
Assessments Parents notified at least 4 weeks in advance.
4
Step 1: Plan for EOY Administration and Scoring
Printing
Administration
Norming &
Scoring
Grades K-2
April 2- May 19
May 19- June 6
Grades 3-8
April 2- May 2
May 5- May 16
Through June 6
High School
April 1
April 2- May 16
Through June 16
5
Step 2: Prepare Assessments
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
• Select and download certain online
assessment(s).
• Print assessment materials
• Receive or select texts
• Print ATS score sheets (when necessary)
• Ensure teachers have the relevant materials
necessary to administer the assessment
(assessment tasks, texts, etc.)
Submit
Scores
Archive
Assessment
Materials
6
NYC Performance Assessment ZIP FILE CONTENTS
Printing Guidance
• Who should “get” each document?
• How many copies should I make?
Performance Assessment
• Coversheet – Administration guidance, G6-12 student honor code
• Assessment – EOY performance assessment
• Translations – Available in social studies, math and science
Rubric
• Coversheet – Guidance for use, reference to handbook
• Rubric – Teacher-facing performance level indicators for each trait
Teacher Scoring Guide
• Coversheet – Guidance for use, reference to handbook
• Scoring Guidance – Guidance for scoring
• Annotated Student Work – 6 samples annotated for rubric trait performance,
each following by scoring with commentary
7
Step 3: Administer Assessments
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
• Ensure the security of testing materials
• Follow administration guidelines in the
Handbook
• Collect assessment materials.
Note: Even if multiple teachers in your school are evaluated on the same
assessment, students should only take each assessment once.
Submit
Scores
Archive
Assessment
Materials
8
Steps 4 and 5: Prepare for Scoring and Score
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Scoring Overview:.
• Scoring tasks may include the following:
• Norm scoring practices
• Review scoring policies
• Finalize scoring schedule
• Coordinate “school-based distributed
scoring” for all assessments
Submit
Scores
Archive
Assessment
Materials
9
Steps 4 and 5: Prepare for Scoring and Score
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Distributed Scoring:
Submit
Scores
State teacher evaluation law stipulates that
teachers with a “vested interest” cannot
score the assessments of students they
directly teach. Whenever “practicable”
schools must establish processes whereby
other teachers would score a teacher’s
MOSL assessments.
Archive
Assessment
Materials
10
Steps 4 and 5: Prepare for Scoring and Score
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Submit
Scores
Archive
Assessment
Materials
Distributed Scoring- Who is Eligible to
Score?
• It is preferred that assessments be scored by teachers
who hold a license in the subject area and/ or are
currently teaching the subject area. Principals, APS,
Achievement coaches or other network staff may
participate in scoring at the discretion of the network
and principal.
• Teachers may score assessments from grades that
they are not currently teaching. For example, a 9th
grade ELA teacher may score a 10th grade ELA
assessment.
11
Steps 4 and 5: Prepare for Scoring and Score
1
Scoring Guidelines:
2
In accordance with the state regulations, the DOE
recommends that schools choose from one of the
following scoring models:
Plan for
Administration
and Scoring
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Submit
Scores
Archive
Assessment
Materials
• Option 1: Teachers in the same grade and subject
swap to score end-of-year assessments.
• Option 2: Teachers of the same subject in different
grade levels swap to score end-of-year assessments.
• Option 3: At the principal’s discretion, teachers who
are properly normed in an end-of-year assessment
may score.
Clusters and networks will establish structures to support
schools where the options listed above are not
feasible (i.e. schools where there is only one content
area teacher for the whole school).
12
Distributed Scoring Considerations / Suggestions
Small Schools:
• Pair up different grade-level teachers within the same
content area
• Foster an environment of collaboration
• Consider administering assessments at beginning of
window
• Consider hiring subs, or pairing up with another small
school
• Literacy coach and other teachers may administer the
assessments
13
Steps 4 and 5: Prepare for Scoring and Score
Group Norming Sessions for NYC Performance Assessments*:
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Submit
Scores
Archive
Assessment
Materials
Schools should conduct norming sessions after administering but
before scoring the assessment in order to ensure that all teachers
have a common understanding of the rubric and increase fairness of
the scoring process.
Group norming sessions should follow this process (approx. 2-3 hours):
Step 1: Review the rubric and discuss expected performance at each
performance level for each rubric trait.
Step 2: Review annotated sample student work and discuss the
evidence for each score.
Step 3: Individually score one EOY assessment, then discuss as a
group the evidence for each score.
Step 4: Collectively score several students’ NYC Performance
Assessments and come to consensus on a score for each trait. The
scorer ID for these collectively scored papers can be 999999.
14
Deep Dive: NYC Performance Assessments
Activity #1 – Task Design Investigation
Objective
Participants will be able to interpret a sample NYC Performance
Assessment by analyzing the requirements for students and predicting
students’ strengths and struggles.
Activity
Preview the sample NYC Performance Assessment and consider the four
key questions on the handout. Use the chart to take notes while you’re
reviewing the task and texts. Share your reflections with your table.
15
Deep Dive: NYC Performance Assessments
Activity #1 – Task Design
16
Deep Dive: NYC Performance Assessments
Activity #2 – Norming & Scoring
Objective
Participants will be able to apply the rubric as an evaluative tool for scoring
student writing and establish a shared understanding of the expectations
of the NYC Performance Assessments.
Activity
Individually read and score Sample 1 according to the rubric. Be sure to
provide a score for each individual trait, and then discuss your findings
with your table. Refer to evidence in the sample of student work and the
rubric to justify your choices and seek consensus.
17
Deep Dive: NYC Performance Assessments
Activity #2 – Norming & Scoring
Norming and Scoring – Grade 1
Instructions: Individually read and score Sample 1 according to the rubric. Be sure to provide a score for each trait
individually. Then discuss your findings with your small group. Refer to evidence in the sample of student work
and the rubric to justify your choices and seek consensus.
1st Grade ELA Sample __
Rubric Dimensions
My Score
Normed
Score
Evidence from the Text
Trait 1:
Focus: Information
Trait 2; Structure
Trait 3: Development
Trait 4:
Directionality/Sense
of Sentence
Trait 5: Word/Spelling
Knowledge
Trait 6: Punctuation/
Capitalization
Trait 7: Explanation of
Reading
18
NYC Performance Assessments
Planning for Instruction
19
Performance Assessment Menu (TEVL)
20
NYCPA – Printing Score Sheets
NYCPA scores are printed using the ATS function RTEV.
After choosing the correct EOY assessment ID, schools may
print score sheets by:
• Stars Course Code / Section
• Official Class
• Grade Level
NOTE: RTEV does not cross check assessment with course
code, official class or grade level. Schools should be very
careful when selecting students and ensure the correct score
sheet is generated for the correct students.
21
RTEV – Initial Printing
Note: Use the Printing Tracker to record the combination you use
to initially print the Score Sheet. In order to re-print the batches
of score sheets, the same combination must be used.
22
NYCPA – Printing Score Sheets
If a NYCPA score sheet is lost or damaged it can be re-printed
using the ATS function RTER.
NOTE: Score sheets may only be re-printed using the same
student group (course code / section, official class, grade
level) or by score sheet page number (recommended).
Use Performance Task Print Tracker to record how documents
were printed, should you need to re-print any scan sheets.
23
RTER – Re-printing Score Sheets
1. Use previous criteria to print a batch
2. Print by individual student ID
3. Print a single page or page-range
24
Best Practices for Printing
• Whenever possible, print answer documents using the ATS
printer/scanner you intend to use for scanning. If this is not
possible, print answer docs only on supported printers
(Lexmark T640 or later), not older or incompatible printers.
(Printers should not be labeled Post Script (PS) in DFS –
https://ats.nycboe.net/atsprint)
• After printing batches of answer documents, a “test page”
will print. Scan the test page to confirm the ink quality for a
batch. Trailing test pages are specific to each exam to
better diagnose scanning issues ahead of testing period.
Make sure to scan the test pages and with enough time to
reprint the answer documents as necessary.
• Answer documents must be printed on white paper only.
• Ensure only one answer document is printed per student
per exam to avoid possible data errors.
25
RTES – Teacher Scorer ID
*CHANGE*
Scorer IDs are
automatically
assigned
26
ATS Score Sheet
*CHANGE*
Scorer IDs are
now 6 digits
long
*CHANGE*
Fewer opportunities for
invalid scores
*CHANGE*
Single or multiple days?
27
NYCPA – Score Sheet Update
If changes to score sheet results are needed only the Principal
may enter them manually into ATS using the function
RTEU.
28
NYC Performance Assessment (NYCPA)
NYCPA scores are recorded on ATS NYCPA score sheets by
scorers and, once completely scored, scanned into ATS
using attendance scanners.
ATS automatically assigns Scorer IDs to every teacher. (RTES)
Administration
• Students
take
NYCPA
EOY exams
Score Sheet
Printing
• Schools
print
NYCPA
EOY score
sheets
Scoring
Scanning
• Scorers
record
results on
NYCPA
score
sheets
• Completely
scored
exams are
scanned
into ATS.
•Bubble sheet now has a place for a scorer to indicate the following:
•Test Date – the date the assessment was given.
•Days for Assessment – single or multiple choices.
•Bubbles added for the following cases:
First year ELL; NYSAA; and Invalid Score
29
Step 6: Submit Scores
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
Submit
Scores
Archive
Assessment
Materials
• Scan ATS Score Sheets
• Use the attendance scanner.
• Documents with errors in reading bar codes will
be rejected by the scanner with an error
message. Documents with bubble read errors
will be indicated on the confirmation sheet(s) and
may be investigated in ATS. These pages will
most likely need to be re-printed, re-bubbled,
and re-scanned after attending to the scanner
(cleaning it, checking toner, etc.)
• Generate reports and fix errors.
• Review RTEA reports for “warnings” noted to
inform of scan read errors, including multiple or
omitted responses, or stray marks.
• Review the automatically generated SCNS
report, which provides information on scanning
status and whether ATS successfully received
scanned data.
30
RTEA – Checking for Warnings/Errors
T=Status
I=Item
S=Summary
31
Step 7: Archive Assessment Materials
1
Plan for
Administration
and Scoring
2
3
4
Prepare
Assessments
Administer
Assessments
Prepare
for Scoring
5
Score
6
7
• Archive Assessment Materials
It is the responsibility of the school to retain and store all
Advance assessment-related materials. Assessments,
student work, data collection tools, scanning confirmation
pages and Assessment Results Approval Forms should
be organized by course and section and stored in an
accessible, secured location by the principal, assessment
coordinator or designee for a minimum of three years.
Submit
Scores
Archive
Assessment
Materials
32
Next Steps
3. Administer, Norm, Score & Submit Scores
NYC Performance Assessments
Administration
• Students
take
NYCPA
EOY exams
Score Sheet
Printing
• Schools
print
NYCPA
EOY score
sheets
Scoring
• Scorers
record
results on
NYCPA
score
sheets
Scanning
• Completely
scored
exams are
scanned
into ATS.
33
Thank you!!!!
Questions or Concerns?
Contact:
ATS and Scanning:
Iris Battino-Bernstein
[email protected]
MoSL:
Josh Metz
[email protected]
Sharon Mahabir
[email protected]
34

Download Report