AI voting booth with biased robotic arms

Can AI Truly Represent Us? Unveiling the Biases in AI Voting Systems

Elliot Brynn in Politics & Policy June 2025 • 4 min read.

"Explore how AI language models (LLMs) perform in voting scenarios, revealing biases and limitations in mirroring human choices."

Artificial Intelligence (AI) has made remarkable strides in recent years, particularly in the realm of language processing. Large Language Models (LLMs) like GPT-4 and LLaMA-2 are now capable of understanding and generating human-like text, leading to their integration into various services. However, along with the excitement surrounding these advancements, it's crucial to acknowledge the limitations and potential unforeseen consequences of their widespread use.

One area where LLMs are generating considerable interest is in digital democracy, specifically in assisted real-time voting. The idea of using AI 'digital twins' to represent individual voter preferences has been proposed, raising both enthusiasm and ethical concerns. Proponents believe that AI agents could enable more nuanced and granular voting, but concerns about automation, democratic integrity, and agent bias necessitate careful consideration.

To address these critical questions, a new study investigates the voting behaviors of LLMs, their inherent biases, and how well they align with human voting patterns. By comparing the voting patterns of human participants with those of LLM agents, the study sheds light on the limitations and potentials of integrating LLMs into collective decision-making processes.

Do AI Voting Systems Mirror Human Choices?

AI voting booth with biased robotic arms

The study, titled 'LLM Voting: Human Choices and AI Collective Decision-Making,' explores the voting behaviors of Large Language Models (LLMs), specifically GPT-4 and LLaMA-2. Researchers used a dataset from a human voting experiment to establish a baseline for human preferences. They then conducted a corresponding experiment with LLM agents, analyzing their choices and biases.

The researchers observed that LLM voting outcomes were influenced by the choice of voting methods and the presentation order of candidates. This highlights a key challenge: LLMs are susceptible to biases that can skew their decisions, much like how the arrangement of candidates on a ballot can influence human voters.

Voting Method Matters: The way votes are cast (e.g., approval voting, ranked voting) significantly impacts LLM choices.
Presentation Order: The order in which options are presented affects LLM voting outcomes.
Persona Influence: Adopting different personas can reduce biases and improve alignment with human choices.
Chain-of-Thought Potential: While not improving accuracy, the 'Chain-of-Thought' approach shows promise for AI explainability.

One significant finding was the trade-off between preference diversity and alignment accuracy in LLMs. Different temperature settings, which control the randomness of the LLM's responses, influenced this balance. The study indicates that LLMs may lead to less diverse collective outcomes and biased assumptions when used in voting scenarios. This underscores the need for cautious integration of LLMs into democratic processes.

The Future of AI in Democratic Decision-Making

The study's findings serve as a reminder that while LLMs offer exciting possibilities, their integration into democratic processes requires careful consideration. As AI continues to evolve, addressing these biases and limitations will be crucial to ensuring fair and representative outcomes in collective decision-making.

About this Article -

This article was crafted using a human-AI hybrid and collaborative approach. AI assisted our team with initial drafting, research insights, identifying key questions, and image generation. Our human editors guided topic selection, defined the angle, structured the content, ensured factual accuracy and relevance, refined the tone, and conducted thorough editing to deliver helpful, high-quality information.See our About page for more information.

This article is based on research published under:

DOI-LINK: 10.1609/aies.v7i1.31758,

Title: Llm Voting: Human Choices And Ai Collective Decision Making

Subject: cs.cl cs.ai cs.cy cs.lg econ.gn q-fin.ec

Authors: Joshua C. Yang, Damian Dailisan, Marcin Korecki, Carina I. Hausladen, Dirk Helbing

Published: 31-01-2024

Everything You Need To Know

What are the primary Large Language Models (LLMs) examined in the study regarding their voting behaviors?

The study focuses on the voting behaviors of two prominent Large Language Models (LLMs): GPT-4 and LLaMA-2. These models were chosen to represent the capabilities of current AI in understanding and generating human-like text. The research explores how these specific LLMs perform in mimicking human voting patterns and the biases they exhibit in decision-making processes.

How do different voting methods impact the decision-making of Large Language Models (LLMs), and what are the implications?

The research highlights that the choice of voting methods significantly influences the outcomes generated by LLMs. Different methods, such as approval voting and ranked voting, can lead to varied results, mirroring how these methods affect human voters. This suggests that the specific voting system used can introduce bias and affect the alignment of LLM choices with human preferences. It underscores the necessity of carefully selecting voting methods to ensure fairness and prevent skewed outcomes when integrating LLMs into democratic processes. The study also finds that the presentation order of the candidates also affects the decision of the LLMs.

Can adopting different personas improve the accuracy or fairness of Large Language Models (LLMs) in voting scenarios?

Yes, the study suggests that adopting different personas can reduce biases and improve the alignment of LLMs with human choices. By giving the LLMs different 'personalities' or perspectives, researchers found that the models' voting patterns could be altered to better reflect human preferences. This indicates a potential method for mitigating bias, although it also introduces complexities in determining which personas are most representative or desirable in a voting context. The researchers also explored 'Chain-of-Thought' to understand the decision of LLMs.

What is the trade-off between preference diversity and alignment accuracy when using Large Language Models (LLMs) in voting scenarios?

The study reveals a trade-off between preference diversity and alignment accuracy in LLMs. Different temperature settings, which control the randomness of the LLM's responses, influence this balance. Higher temperatures can lead to more diverse but potentially less accurate outcomes, whereas lower temperatures might result in more aligned but less diverse results. This means that when LLMs are used in voting, there's a risk that the collective outcomes could be less diverse than human preferences, especially if the LLM is tuned for high accuracy. This highlights the need for careful calibration and consideration of how LLMs are used in collective decision-making to ensure fair and representative outcomes.

What are the key challenges and ethical concerns of integrating Large Language Models (LLMs) into digital democracy and voting systems?

The integration of Large Language Models (LLMs) into digital democracy raises several key challenges and ethical concerns. One major concern is the potential for bias in LLMs, which can skew voting outcomes and misrepresent voter preferences. The study highlights that LLMs are susceptible to influences like voting methods, presentation order, and even the adoption of different personas. Other concerns include the need to ensure democratic integrity, prevent automation bias, and carefully consider how LLMs' inherent limitations might affect collective decision-making. Also, the opacity of LLM decision-making, and the potential for manipulation or misuse of these systems, further complicate the ethical considerations, underscoring the need for cautious and thoughtful integration of LLMs into democratic processes.