Toy Example: AI Early-Warning System for Student Dropout
The university should use the system.
If the system is likely to substantially benefit students and it will be operated lawfully and all morally relevant remaining risks are proportionate, then the university should use the system.
Finance, reputation, etc.
The system is likely to substantially benefit students.
Reasons support that it CAN help. But why "likely to"?
The system will be operated lawfully.
All morally relevant remaining risks are proportionate.
Resource allocation, privacy, autonomy and fairness as only risks?
Timely support can reduce dropout.
Earlier contact enables earlier interventions.
Correct risk flags enable timely support.
The model correctly flags elevated risk before dropout.
Internal validation shows useful predictive performance for dropout.
Bañeres et al. (2023) report reduced dropout and increased engagement after targeted early interventions.
SourceThe privacy and autonomy burden can be limited.
The fairness risk can be reduced to an acceptable level.
The resource-allocation risk can be limited.
Declining support has no adverse consequence.
Students can decline the offered support.
Students may ignore or decline outreach without consequences.
Access to individual risk scores can be restricted.
Only designated advisors can view individual risk flags.
Subgroup monitoring can mitigate the identified disparity.
The subgroup disparity can be identified reliably.
International students have a higher false-positive rate.
The false-positive rate is 18% for international students and 9% for domestic students.
Decision thresholds can be revised after subgroup review.
Validation reports are produced by relevant student subgroup each semester.
Human allocation can prevent the score from becoming the allocation rule.
Counselling allocation requires a separate assessment.
Predicted dropout risk is not identical to need for counselling.
Counselling capacity is limited.
The university can provide only 200 additional counselling appointments per semester.
Advisors make the final allocation decision.
Advisors consider circumstances and student preferences before allocating scarce appointments.
Law and morality are the only relevant moral domains w.r.t. the system.
Finances and reputation are relevant as well. Perhaps even more considerations.
If the system is likely to substantially benefit students, then the university has a pro tanto reason to use it.
If the university has a pro tanto reason to use the system and there are not stronger countervailing reasons against using it, then the university should use the system.
If the system will be operated lawfully and all morally relevant remaining risks are proportionate, then there are not stronger countervailing reasons against using it.
Finance, reputation, etc.
Universities have the pro tanto duty to foster their students' educational successes.
Universities substantially structure students' opportunities to succeed in their studies.
An undefeated pro tanto reason determines what should be done when no equally strong reason supports an alternative.
Does it have to be equally strong?
All applicable legal regimes can be complied with by the university.
The one's in the reasons supporing this claim don't really seem to me like they would cover all relevant reasons...
Applicable data-protection requirements can be met.
I would like more reasons for this. Data protection seems like a complicated issue to me here...
The applicable AI Act requirements can be met.
Applicable higher-education and public-law requirements can be met.
Too underspecified: What laws are these even?
The system is not listed under Annex III of the AI Act.
The system does not qualify as high-risk according to other consideration in Art. 6 AI Act.
The university has legal authority to operate the proposed student-support workflow.
The risk score does not itself determine an academic or disciplinary decision.
The applicable higher-education framework permits the university to provide academic advising and student-support services.
The score is not used for admission, grading, examination, discipline, or exclusion from services.
If universities substantially structure students' opportunities to succeed in their studies, then universities have the pro tanto duty to foster their students' educational successes.
If Universities have the pro tanto duty to foster their students' educational successes, then if the system is likely to substantially benefit students, then the university has a pro tanto reason to use it.
If there are stronger countervailing reasons against using the system, then it will not be operated lawfully or not all morally relevant remaining risks are proportionate.
Finance, reputation, etc.
The system does not have to be considered high-risk.
If the AI Act requirements posed to the system are minimal, then the applicable AI Act requirements can be met.
The AI Act requirements posed to the system are minimal.
If the system does not have to be considered high-risk, then the AI Act requirements posed to the system are minimal.
If the system is not listed under Annex III of the AI Act and it is not the case that the system does qualify as high-risk according to other consideration in Art 6 AI Act, then it is not the case that the system does have to be considered high-risk.
If Human allocation can prevent the score from becoming the allocation rule, then the resource-allocation risk can be limited.
That's not true. Humans can make stupid decisions, too.
If all applicable legal regimes can be complied with by the university, then the system will be operated lawfully.
The university does their best effort to comply with all applicable laws.