This episode of Data Skeptic explores the ethical and societal dimensions of recommender systems, focusing on trust, manipulation, privacy, fairness, and sustainability. Key insights include the importance of explainability for user retention, the prevalence and cost of fake reviews, and the environmental impact of model training, while highlighting the shift toward user control and regulatory mandates like the EU's Digital Services Act.
Summarized by Podsumo
Trust and explainability are critical: Users lose trust quickly if recommendations fail, and providing explanations—even if not always heeded—is a key retention strategy.
Manipulation of recommender systems is widespread and costly: Attacks like 'brushing' (fake purchases) and 'shilling' (fake users) exploit feedback loops, with an estimated 4% of online reviews being fake, driving $800 billion in annual US sales.
Environmental impact is a growing concern: The carbon footprint of a recommendation model depends on hardware and the energy source of its location; simpler models often achieve comparable accuracy with much lower energy consumption.
User control and regulation are becoming central: Platforms like Blue Sky let users choose their own algorithms, and the EU's Digital Services Act mandates explainability and a non-profiling option, shifting power from platforms to users.
"If you lose trust once, the user might just consider it as useless, even after it becomes relevant."
"The identical model is clean in one country and filthy in another, purely because of where the electricity comes from."
"We went from 'predict the number' to 'answer for the consequences.'"