ASAP

Guidelines for ASAP Phase One Vendor Demonstration

The results of the ASAP Phase One Vendor Demonstration Data have been published (http://www.scoreright.org/NCME_2012_Paper3_29_12.pdf) and includes aggregate data for the vendors. We are now making the individual vendor data and results available to qualified researchers so they may examine the results more fully.

1 Procedures for Requesting ASAP Data

1.1 The following categories of users are generally considered qualified to request ASAP data:

Researchers/Policymakers: Includes researchers or policymakers who are affiliated with either an educational institution (e.g., college/university, public/private primary or secondary school system), a governmental research agency or legislative commission, or a private research or policy institution.

State Education Departments: Includes state education agencies, state departments of education and central agencies of state systems of higher education.

Institutions/Districts: Includes persons with authority to represent public and private primary and secondary schools, local school districts, and institutes of higher education, such as colleges and universities.

Other Interested Parties: Includes recognized educational organizations (e.g., state education associations, the National Council of Teachers of Mathematics, etc.).

1.2 All requests for ASAP data from a qualified requester must be submitted via this website and include the following information: (1) the rationale for the data request; (2) any and all purpose(s) for which the data will be used; (3) name(s) and title(s) of all parties having access to the data; and (4) the name of the agency funding the research, if applicable. In addition, the qualified requester must agree to a licensing agreement. Those requesting ASAP data should allow at least two to three weeks for TCP to review and respond to the request. If the request is approved, TCP will provide the requester a link where the data can be downloaded.

1.3 Data that includes student essays may require sign-off from the state department of education, depending on the purpose of the research. In this case, Data will not be released without the appropriate sign-off.

1.4 TCP reserves the right to refuse any data request involving ASAP data; however, nothing herein shall prohibit participating vendors or state departments of education from releasing data pertaining to their own organization.

1.5 The requesting party is NOT permitted to retain data after the completion of their work. The data must be permanently destroyed and written certification of destruction must be sent to TCP.

1.6 The requesting party is NOT permitted to share or otherwise distribute data to third parties.

1.7 At all times the requesting party is ultimately responsible for ensuring that complies with the terms of the license agreement executed with TCP.

2 Guidelines for Posting Data on the Internet

All recipients of data are NOT permitted to post ASAP data on their websites, or republish or post data that appear on scoreright.org without the express written consent of TCP.

Recipients of the Data may publish the results of their studies provided they comply with all aspects of the licensing agreement. Dissemination of data or study results in any format that could compromise institutions’, vendors’, and individuals’ rights to confidentiality is strictly prohibited.

Requesters of data must be familiar with relevant and applicable state and federal statutes governing such issues and professional practices in protecting testing data (see The Standards for Educational and Psychological Testing, 1999, AERA, APA and NCME). A violation of any such law or standard may, at the sole discretion of TCP, result in the immediate revocation of the license for the provided data and the refusal to license and/or provide data in the future. 

About the Automated Student Assessment Prize

The Automated Student Assessment Prize (ASAP) invited data scientists worldwide to take on the challenge of creating new approaches to high-stakes testing for state departments of education. Sponsored by the William and Flora Hewlett Foundation, the first phase of ASAP was designed to demonstrate the ability of existing summative writing assessment products and to accelerate innovation in machine scoring. Phase one focused on the ability of technology to assess long-form constructed responses (essays). ASAP was designed to answer a basic question: Can a computer grade a student-written response on a state-administered test as well as or better than a human grader?

The results of these two studies present unique opportunities to:

  • Establish standards for state departments of education to utilize assessment technologies.
  • Advance the field of machine scoring in the application of student assessment.
  • Introduce new players with different and disruptive approaches to the field.

ASAP remains committed to its role as an open, fair and impartial arbiter of machine scoring and writing assessment capabilities through a series of scientifically rigorous studies and field trials. ASAP does not endorse or promote any specific technology or provider. Instead, ASAP seeks to deliver critical information to administrators, educators, families and students during a time when student assessment is undergoing a critical shift in schools across America.

Recent Articles

Additional Links

show nav