The Maryland State Department of Education would like to address a news piece that aired on Fox 45 on April 17, 2023, concerning the disclosure of MCAP data on the Maryland Report Card website. The news piece omitted key pieces of information about federally required data disclosure avoidance methods and included information that was incorrect and mischaracterized the MSDE and the agency’s intentions. Through this statement, MSDE is providing clarification on this news piece, the U.S. Department of Education’s requirements on data disclosure avoidance methods, and the steps MSDE took to ensure the agency complied with those reporting requirements.
The news piece in question was put together by Fox 45 reporter Chris Papst and aired in the 10:00 p.m. newscast on Fox 45. In the piece, Mr. Papst omitted information that was provided to him by the MSDE on April 10 and April 14 which explains the federally mandated disclosure avoidance methods for reporting student data, the enhancements applied to the post-pandemic release of data, and why certain pieces of data needed to be suppressed. By omitting this information, Mr. Papst manufactured a news piece that implied MSDE was “hiding information from the public” and “staging a cover up.” Not only are these statements inaccurate, but they are entirely baseless as MSDE has been cooperative and transparent in providing Mr. Papst with all the information he asked for.
MSDE staff presented the 2022 MCAP assessment results at the January State Board of Education meeting. For the release of the 2022 MCAP assessment data and for all data released on the Maryland Report Card website, MSDE procured the services of a vendor, The Canton Group. MSDE prepares data for release on-site and provides suppressed data in tables that support dynamic data views and data files available for the public to download. Prior to any data published on the MDreportcard.org website, there is a shared quality assurance process with the vendor and MSDE staff before final sign off from MSDE. In January 2023, an embargoed data download file with MCAP data was provided to the vendor. The file was incorrectly moved to production prior to the Maryland State Board of Education meeting during an overall website enhancement update to the website. The file given to the vendor was embargoed because it used MSDE’s standard data suppression rules in place since 2010. However, to comply with federally required standards, MSDE needed to update these suppression rules and apply additional data disclosure avoidance methods to the data prior to publishing the data.
Mr. Papst gained access to the embargoed data that was prematurely posted by the MSDE web vendor on the Maryland Report Card website. He then published the embargoed data and has been using it to run with a misleading and inaccurate narrative that MSDE is hiding information from the public.
MSDE was alerted to the reporting of the embargoed MCAP data when an LEA informed MSDE that Fox 45 had published the embargoed data that was incorrectly moved to production. The data download file was subsequently removed.
Enhanced Data Disclosure Avoidance
Because this was the first reporting of MCAP data after the COVID-19 school disruptions, MSDE staff as part of the review process identified the need to enhance the disclosure avoidance methods for reporting student data to ensure a “reasonable person in the school community” could not identify individual students.[1] MSDE added two additional suppressions that were put in place as part of the release of the 2022 MCAP data and when the percentage of students proficient was ≤ 5% of students.
· All performance level data was suppressed. The tested count and the percent proficient reporting of ≤ 5% of students continued to be reported.
· The N-size (tested count) was increased from 10 to 30. When fewer than 30 students were tested, the tested count and the reporting of the percent proficient was suppressed in addition to the performance level data. An example demonstrating the need for this additional suppression is a school having 20 students; 5% proficient would be one student and therefore there is a risk for reidentification of that student.
MSDE is currently retroactively updating data going back five years to be consistent with these enhancements. The 2022 MCAP data are now published as both data downloads and as dynamic pages of the Report Card with these federally required enhancements to disclosure avoidance. All data suppressed due to the risk of individual student identification is noted with an asterisk. Throughout the Maryland Report Card website, the following footnote is provided:
'*' indicates no students or fewer than 10 students in category, or '*' indicates the percentage for the category is either ≤5 or ≥95 and the corresponding counts have been suppressed.
Summary of Correspondence with Fox 45
On April 10, 2023, Mr. Papst, on behalf of Fox 45, contacted MSDE with questions about the 2022 MCAP data reported on the MSDE website. He specifically referred to the differences in data reported in January 2023 compared the data reported in March 2023 and included data spreadsheets posted on those respective dates. MSDE promptly responded to the inquiry that same day explaining why there were differences in the reported data. The response explained that the data released in January 2023 was embargoed and prematurely posted to the website by MSDE’s web vendor prior to the data going through the federally required data disclosure avoidance methods. MSDE removed the spreadsheet with embargoed data from the website and, in March, posted a spreadsheet with the same set of data, but with the required disclosure avoidance methods applied. The response to Mr. Papst’s inquiry also included a detailed explanation of what disclosure avoidance methods were applied to the data and why, as well as a link to a document explaining the U.S. Department of Education’s federally required reporting methods.
On April 12, 2023, Mr. Papst contacted MSDE again with follow up questions to his original inquiry. He specifically asked for MSDE to further explain the disclosure avoidance methods applied to the data. MSDE responded to the follow up questions on April 14, 2023. The response reiterated the federally required disclosure avoidance methods applied to the data and provided additional information about why data suppression is a federally mandated step when reporting student performance data to protect the privacy of individual student records under the Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. § 1232g; 34 CFR Part 99). The response also explained that MSDE needed to apply enhancements to the disclosure avoidance methods post-pandemic to further protect the personal information and identification of individual students.
On April 17, 2023, at 4:50 p.m., Mr. Papst reached out to MSDE again with more follow up questions asking about several topics including who is MSDE’s web vendor and why disclosure avoidance methods used for 2022 data was retroactively applied to data from previous years. Before MSDE could provide Mr. Papst with an answer to his follow up questions, Fox 45 aired a news piece in the 10:00 p.m. newscast on April 17, 2023. The news piece omitted key information provided to Mr. Papst making the piece inaccurate and mischaracterizing.
Next Steps for MSDE’s Data Reporting
As a direct result of the mistakes made by the vendor and project management oversight, MSDE is implementing the following corrective actions:
· MSDE will require additional layers of sign-off including directly from the Division of Assessment, Accountability and Performance Reporting Assistant State Superintendent. This level of sign-off was previously not required and will be required going forward.
· MSDE will put in place additional checks of the vendor deliverables and website by agency staff during web testing and prior to publication of data on MDreportcard.org.
· MSDE will publish a fact sheet on the Maryland Report Card website highlighting the disclosure avoidance methods, including data suppression methodologies, that are currently in place. This fact sheet will be updated as future enhancements and improvements are implemented.
In conclusion, MSDE values the press and transparency, as well as facts and accuracy. Therefore, MSDE must do its due diligence to ensure the false information is corrected and that the public has all the facts. Data released in the 2022 Maryland Report Card covers many facets of student and school performance and with that comes a nuanced approach to reporting that data to protect the privacy of our students and remain compliant with federal law. Context is a critical component to telling this story for the sake of accuracy and ethical journalism. MSDE will continue to be responsive and transparent on important questions and issues as the agency sincerely values trust and understanding from the public. MSDE maintains its stance on ensuring the public has all the facts and context on the stories about our students and educators.
