[OC] Ergebnisse des politischen Kompasses für 16 große LLMs in 1.120 Testläufen
Von GenerativeFart
[OC] Ergebnisse des politischen Kompasses für 16 große LLMs in 1.120 Testläufen
Von GenerativeFart
27 Kommentare
I gave the Political Compass test from [politicalcompass.org](http://politicalcompass.org) to the most relevant LLMs 70 times each: 30 times using the original questions, 30 times using polarity-flipped questions to reduce affirmative bias, and 10 times with the question order shuffled. I then compared the results.
Surprisingly, all the models scored far into the libertarian-left quadrant. Not even Grok or the Chinese models made it out of that quadrant.
By far the most interesting result was Grok’s bimodal distribution. It appears to have two distinct personas: one that aligns with the other models and another that is considerably more right-wing. I suspect this may be related to Grok having been specifically trained to exhibit less left-wing bias than other models.
I also asked the models to place themselves on the Political Compass without completing the questionnaire. They all perceived themselves as more balanced and centrist than their test results suggested. GLM and Gemini Flash showed the largest discrepancies between their self-assessments and measured positions, while DeepSeek V3 showed the smallest.
Visualizations were done using Python and matplotlib and svgpath2mpl to render the company logos as vector paths.
Big disclaimer: this analysis was not conducted with full scientific rigor. I tried my best, but there are clear weaknesses in the methodology. For example, the Political Compass itself appears to have a strong libertarian-left bias. The strongest conclusions are therefore comparative: for example, that model X is more conservative than model Y rather than that LLMs are politically extreme in absolute terms. However, compared with older results, it appears that LLMs may have shifted further toward the libertarian left in recent years.
I published all the data and its available on my blog, which I unfortunately cannot link 🙁
Reality has a left-bias in these days of the Overton Window being so far right it’s falling off the edge of rationality.
Love that you did mistral
Grok is interesting
I would have expected more of a Nazi bias from Grok. Huh.
Complete horseshit. The world has just shifted so right your definition of left is skewed. I used to be right and now I’m left and the pattern-difference now compared to even right after 9/11 is fucking insane.
Fantastic insight. Any chance you’ll be doing it over the next few months/years as well to observe a difference?
This particular site has been criticized in the past of being utterly unscientific, completely opaque and of deliberately leading towards libertarian-left outcomes while claiming world leaders are far more authoritarian-right than they are. I’m slightly lib-left of center, but this consistently places me in the „Bernie was the compromise“ camp. Have you considered trying this with a better-regarded system instead?
The political compass test isn’t accurate tho, everyone ends up on libleft. Because to end up on the right you would need to be like „I’d sell my mother and do a few genocide“ in the answers. Never understood why they made the political compass test that way.
Worth noting political compass is HEAVILY skewed to libleft, a better tool would be more useful to test this with
I think this is mostly down to training data. If they had trained the LLMs on talk radio transcripts and bumper stickers, russian bots in YouTube comment sections and smartphones listening on drunken rants by racist old dudes and people who willfully misunderstand how marginal tax brackets work, AI wouldn’t have such a left leaning bias.
The best quadrant to be in
Seems likely more an artifact of anti-empiricism becoming adopted as a political ideology
I was just talking to Gemini about the ridiculousness of a civilian vs. military uprising, and in three posts it had half-designed a consumer-electronic railgun to free the proletariat.
I imagine right wing content in datasets gets hit disproportionately harder than left wing content being scrubbed out by all the ghost workers that have to flag and remove harmful content in training.
But also the old adage has always been that reality has a leftwing bias
Politics are this bad with so many left leaning AI models. I fear what will happen when the thumbs strongly start shaping these models.
Do 8 axes or 16 axes instead of this garbage test
Because of the bipolar nature of grok, could you have shown a line of each extreme?
I think what this more shows is this test seems set up to make everyone seem more left leaning/libertarian than reality.
Questions like „All people have their rights, but it is better for all of us that different sorts of people should keep to their own kind,“ „People with serious inheritable disabilities should not be allowed to reproduce.“ which disagreeing with is considered an extreme libertarian stance.
or for the left vs right economics „If economic globalisation is inevitable, it should primarily serve humanity rather than the interests of trans-national corporations.“ Not agreeing that the goal of economics is to serve transnational corporations is not an extreme left wing stance.
Even the more intense left leaning questions like „“from each according to his ability, to each according to his need” is a fundamentally good idea.“ have been regularly asked and found widespread agreement across political spectrum is not as extreme of an idea except due to the fact its from the communist manifesto.
It seems to fall into the false dichotomy side of things where „Human lives should be sacrificed to corporations“ and „maybe they shouldnt“ are polar extremes on the political spectrum but in a way that makes eceryone seem more left wing than they are.
Yeah you shouldn’t draw any conclusions from this flawed test.
https://youtu.be/lOCRDsNH3D0?si=f0STpU-cVX1f9ywc
Well I would like to see multiple users try the same, all of them with their own compass results alongside, to see if there is any correlation between user and llm.
It simply seems that something is off if all the models lean towards libertarian left.
In a perfect world the political axis would be centered somewhere near Meta’s AI.
Reality and (honest) history themselves are „left leaning“ to the extent that humanity actually *has* made significant progress (extending the circle of inclusion and compassion, expanding rights, reducing outright violence, extending lives, etc).
Political compass is a horrible metric that puts any rational person in the same quadrant. Of course this is what it did
Grok is lib-left?! Uh, #Doubt
Reality has a known left wing bias
I’m LMAO at quadrants for this data.
Most politicians arent even close to this quadrant.
If ai models truly ar a mirror image of people on the internet and every bit of data fed to them id say enlighted centrism is actually where they are clustered.