“You have this effort to add these elements without mechanism with which to examine the new variables, as well as a system to ensure their good implementation,” said Cook. “You could almost think that the one who implemented it did not know what they were doing.”
Cook has helped advise the education department on IPEDS data collection for 20 years and served in technical examination panels, which are normally summoned first to recommend changes to data collection. These panels were dissolved earlier this year, and there is no configuration for the new Trump’s admission data proposals.
Cook and other data experts cannot understand how a decimated education statistics agency could access this task. The six employees of the NCES who participated in the collection of IPEDS data were dismissed in March, and there are only three employees left outside 100 in Nces, who are managed by an interim commissioner who also has several other jobs.
An official of the Department of Education, who did not want to be appointed, denied that no one who left inside the Department of Education has Died experience. The manager said that the staff of the data director, who is separate from the Statistics Agency, has a “deep familiarity with IPEDS data, its collection and its use”. The former employees of the Department of Education told me that some of these employees had experience in data analysis, but not in collection.
In the past, there were up to a dozen employees who have worked closely with RTI International, a scientific research institute, which manages most IPEDS data collection work.
Technical review eliminated
According to the government’s efficiency department, also known as DOGE, also known as DOGE, was particularly concerned about the fact that the RTI $ 10 million annual contract was worrying to carry out data collection, also known as DOGE. These severe budget cuts eliminated the technical revision panels that proposed the changes to the IPED and ended the training so that colleges and universities correctly submit the data, which has helped the quality of the data. RTI did not answer my request for confirmation of the cuts or to answer questions about the challenges he will face by expanding his work on a reduced budget and staff.
The Department of Education did not deny that the IPED budget had been reduced by half. “The RTI contract focuses on the most critical activities of the mission,” said the head of the education department. “The contract continues to include at least one task in which a technical examination committee can be summoned.”
Additional elements of IPEDS data collection have also been reduced, including a contract to verify the quality of the data.
Last week, the scope of the new task became more apparent. On August 13, the administration published more details on the new admission data it wishes, describing how the education department attempts to add a whole new survey to DIs, called admissions and consumer transparency supplements (acts), which will dismiss all admission data and most data on results and financial assistance of students by race and gender. The college will have to report both in the first cycle admission and higher education. The public has 60 days to comment, and the administration hopes that colleges are starting to report this data this fall.
Complex collection
Christine Keller, Executive Director of the Association for Institutional Research, a commercial group of higher education leaders who collect and analyze the data, called the new survey “one of the most complex collections ever attempted”.
Traditionally, it took years to make much smaller changes to the IPED, and universities are given for one year to start collecting new data before they are required to submit it. (Around 6,000 colleges, universities and professional schools are required to submit data to the IPED as a condition for their students to contract federal student loans or receive PELL federal subsidies. Founding of fines and the threat of losing access to federal aid to students.)
Normally, the education department would reveal screenshots of data fields, showing what colleges would need to enter the IPEDS computer system. But the ministry did not do this, and several of the data descriptions are ambiguous. For example, colleges will have to point out the results of the tests and the GPA by the quintile, broken down by race and ethnicity and sex. An interpretation is that a college should say how many black candidates, for example, have obtained a score above the 80th centile in SAT or the law. Another interpretation is that the colleges should report the SAT or ACT score of the 20% most important of black candidates.
The association for Institutional Research used to train colleges administrators on how to collect and submit data correctly and sort the confusing details – until Doge eliminates this training. “The absence of complete and financed training by the federal government will only increase the institutional and risks for data quality,” said Keller. Keller’s organization now plunges into its own budget to offer a small amount of free IPEDS training to universities.
The Department of Education also obliges colleges to report five years of data on historical admissions, broken down into numerous sub-categories. The institutions have never been invited to keep data on candidates who have not registered.
“It is incredible that they are asking for five years of previous data,” said Jordan Matsudaira, an economist of the American university who worked on educational policy in the administrations of Biden and Obama. “It will be square in the pandemic years when no one has declared test results.”
“Deputy results”
Matsudaira explained that the IPED had planned to ask colleges more academic data by race and ethnic in the past and that the Department of Education has finally rejected the proposal. One of the concerns is that the slice and maze of data in increasingly small buckets would mean that there would be too few students and that the data should be deleted to protect students’ privacy. For example, if there were two Amerindian men in 20% of SAT scores in a college, many people could guess who they were. And a large amount of deleted data would make the entire collection less useful.
In addition, small numbers can lead to eccentric results. For example, a small college could only have two Hispanic male candidates with very high SAT scores. If the two were accepted, it is a 100%admission rate. If only 200 white women out of 400 with the same test results were accepted, it would only be a 50%admission rate. On the surface, this may resemble racial and sexist discrimination. But that could have been a stroke of luck. Perhaps these two Hispanic men were athletes and musicians. The following year, the school could reject two different Hispanic male candidates with high test scores but without such impressive parascularly. The admission rate for Hispanic men with high test results would fall to zero. “You end up with misleading results,” said Matsudaira.
Another great concern is reporting on the average test results. “It looks like me a trap,” said Matsudaira. “This will mechanically give the administration the claim to claim that there are lower admission standards for black students compared to white students when you know that this is not correct inference.”
The statistical problem is that there are more Asian and White students at the very high end of the distribution of SAT scores, and all these 1600 perfects will make the average for these racial groups. (Like a very large person will falterly falter height of a group.) Even if a college has a high test score threshold that it applies to all racial groups and that no one below a 1400 is admitted, the average SAT score for black students will always be lower than that of white students. (See graphic below.) The only way to avoid this is to admit purely by test score and take only students with the highest scores. In some highly selective universities, there are enough candidates with a 1600 SAT to fill the whole class. But no institution fills its student body with the results of tests alone. This could mean neglecting candidates with the potential to be concert pianists, football players and great writers.
The average score trap
Admission data is a very busy political problem. The Biden administration initially directed the collection of admission data to the college by race and ethnicity. The Democrats wanted to collect this data to show how the country’s colleges and universities became less diverse with the end of positive action. These data should start this fall, following a complete technical and procedural examination.
Now the Trump administration requires what was already underway and adding a multitude of new data requirements – without following normal processes. And instead of following the decline in the diversity of higher education, Trump wants to use admission data to threaten colleges and universities. If the new directive produces poor data that is easy to interpret, it can get its wish.
