The left and political theory: a disaster story

published 15 May 2013

This is the second part of the introduction to my theoretical writings.  In the first part, I described the long decline in socialist beliefs in the West and asked the question of whether it is possible to reinvigorate belief in a socialist alternative.   

The great bulk of leftist political theory can be traced back to Karl Marx.  His theories were products of a life-long attempt to conduct a rigorous scientific analysis of the dynamics of capitalist society.  His influence can still be felt to this day - virtually every leftist political current, from the social democrats to the bolsheviks and even the anarchists, base their understanding of historical dynamics on his theoretical model.  Although these currents differ strongly on methods for getting there, they all share Marx's goal of socialism (or at least they did until the social democrats gave up the ghost and effectively abandoned political theory).

Marx's analysis had much to commend it - his basic approach, of applying materialist structural analysis in order to understand social dynamics, was innovative and is still as valid as ever today.  However, whereas nowadays we have a wide range of techniques that can be applied to eliciting the dynamics of complex non-linear systems - from sophisticated suites of statistics, to stochastic simulations and many more - Marx had to make do with dialectics, which was an extremely rudimentary and semi-mystical approach that he borrowed from Hegel.  Thus, it is not surprising that the model that he produced had many flaws. 

If one goes to the trouble of picking through his turgid prose and bombastic rhetoric and applies a retrospective critique to his hypotheses, one can identify various predictions that he got wrong.  For example: his notion of identity and how it was determined by economic forces was far too simplistic and turned out to be profoundly wrong; his view of historical development was far too mechanistic and linear; his assumption that the problems of social organisation would be trivial to solve once the parasitic capitalistic class were removed from the equation turned out to be spectacularly over-optimistic. 

However, the prestige and growth of mass-based socialist organisations owed much to their intellectual leaderships' claims to a Marxist theory that was 'scientific'. This made it politically difficult for them to admit that there could be errors in his theories. Thus, they resisted evidence that seemed to contradict his predictions and gradually lost the ability to improve their theoretical models.  This created a progressive ossification of socialist theory as it failed to incorporate inconvenient new scientific findings.

Lysenko image: wikipedia

The process can be seen most clearly in the state doctrine of the Soviet Union, where effective research in many scientific disciplines was rendered impossible as certain foundational theories and methods were considered sacrosanct, even though they were useless. Dialectical materialism in the social sciences and Lysenkoism in biology were especially effective ways of preventing theoretical development. Marx's attempts to provide structural descriptions of how modern economies work gradually turned into articles of faith rather than science that could stand on solid empirical foundations.

It took some time for ossification to take hold entirely and for Marxism to become totally isolated from scientific advances. In the pre-WW1 period, Marxist-inspired parties were full of scientists and critical thinkers and reality was yet to diverge strongly from Marx's predictions. Figures such as Gramsci and Kautsky were far from being slavish followers of doctrines handed down from above. However, the mystical status that Marx had gained within the socialist movement meant that theoretical modifications had to be introduced as patches on top of his theories rather than corrections of his mistakes. This led to an increasingly arcane, brittle and rigid theoretical foundation, where even the slightest deviation from orthodoxy required tortuous logical twists. Consequently, even those Marxists who ploughed a furrow independently of Moscow became increasingly isolated from the mainstream of scientific thought and increasingly reliant on faith over evidence. Today, this theological transformation is almost complete - the approaches of contemporary disciples of Marx often resemble medieval scholastics or strange religious cults more than social scientists.  

Bright shining lights upfront: a dark void behind image: Ben Terrett (flickr)

There were also, of course, people who were influenced by Marx yet recognised the inadequacy of some aspects of his model.  As the 20th century progressed it became increasingly obvious that the masses were not converging on a proletarian identity as he had predicted. However, the typical response to such limitations coming to light was to abandon the structural and analytic aspects of his work while retaining the rhetorical and mystical parts - the worst of all worlds.  The post-modernists, post-structuralists and the broad academic movement that emerged around them from the 1960s onwards, have produced a great big morass of vague, impressionstic portraits of various aspects of 'modernity' and "subjectivity" from which it is difficult to extract a single unambiguous proposition, never mind a testable theory. Today they still reign supreme in many university's humanities departments where they often eschew the very idea of 'totalising discourses' and even, in some cases, reject science itself, based upon a straw-man caricature of the worst excesses of the scientific community. 

Wishful thinking writ large image:

Others simply substituted wishful thinking for sound structural analysis - celebrating acts of individual rebellion like smashing windows or fighting police (or even liberating cheese!) or convinced themselves that loose, semi-organised networks of 'activists', televised protest movements and technological developments were somehow game-changing innovations that would transform the global social order despite the total lack of supporting evidence.  Last, but by no means least, are those who abandoned the idea of there being an alternative to the economic status quo and limited their ambitions to winning elections and maybe curbing some of the worst excesses of the current social order when they got a chance - the great majority of parties that were once social democratic eventually took this route.

As time passes, the memory of the Soviet Union will continue to fade from popular consciousness and the left will once again have an opportunity to define itself by its theoretical propositions rather than images of Russian gulags and concrete tower-blocks.  However, if it is ever to rebuild its intellectual base, it needs to be able to convince people that its project is based on a sound theoretical basis.  No matter how disaffected they become with the current social order, smart people will not march into battle without being convinced that there is a realistic prospect of arriving at something better at the end of it.

The underlying problem that led to this disaster story was not that Marx was a bad scientist. His basic approach of materialist structural analysis in order to understand social dynamics, was far ahead of most of his contemporaries.  The problem was that a political movement adopted a particular scientific theory as representing eternal truth.  Political movements are expressions of shared values and all science can do is to provide a method for figuring out the best way to realise those values, and the best answer to the question will continue to change with time. A political movement that cannot accept changes in the answers will find itself rapidly heading down the same cul-de-sac that the left fell into once Marxism became detached from empiricism.  

Theoretical Writings

Danger: Theory! Here is where the fun starts. I predict trouble. Ireland 25 Apr 2013 image: William Hederman

My journey through the left ran in parallel with a journey through science. As my scientific understanding deepened, I found myself increasingly coming across political beliefs and approaches that I knew to be either wrong or, worse still, "not even wrong" because the questions and concepts that were being debated just didn't make any sense.  Many of the problems I encountered are far from exclusive to the left, and many scientific misunderstandings are rife across the political spectrum.  However, I am much more familiar with the intricacies of political discourse on the left, and the left at least aspires to care about truth and reason, so that is my focus.  Over the course of the coming year, I plan to publish a series of articles that look at the various areas where political discourse tends to ignore modern scientific understandings. The following is a list of the themes that I will touch upon:

  1. Modelling – how and why we build simplified models of social systems and what we can do with them.
  2. Social Classification - dividing up populations into groups (social classes, genders, ethnies, nations...), how you do it, what you can say about these groups and why the answers to many common political questions (e.g. "how many classes are there? does the middle class exist? are we all middle class now? ) belong to the "not even wrong" class.
  3. Rationality - why the rational political actor is a bad approximation of how human cognition actually works.
  4. Communication and language - why explicit language-based communication of political ideas is tremendously limited and frequently ineffective in the dissemination of ideas.
  5. Measurement & planning - why simple metrics are so useful and how they relate to centralisation and planning
  6. Science - why critiques of science, empiricism and 'positivism' are wide of the mark.
  7. Revolution – why premising a political project on large-scale institutional and process re-engineering in a short time period is a very bad idea.
  8. Democracy – why democracy works well only in fairly narrow and specific areas.
  9. Deduction – why deductive reasoning is almost useless when it comes to analysing actual social dynamics.
  10. Materialism and culture – why ideas are material things too and why they really do matter.
  11. Elections and political parties – why it’s ineffective to combine policy formation and fighting elections in political parties.   
  12. Economics – why both Marx’s economic theories and neo-classical marginalism are poor models of real world economies.

It’s worth addressing, in advance, some misconceptions that might arise about the scope of this work:

This is not anything like a scientific theory of politics.  I merely want to show how modern scientific understandings on some of these issues contradict some of the basic assumptions of a lot of political discourse on the left.

This is not innovative.  None of what I say is based on novel research.  I am merely distilling various findings that impact upon political analysis.  Almost everything that I say is fairly uncontroversial from a scientific point of view.  Many people have said this stuff before and better than I can.  However, I know from experience that many of the implications of these findings are quite unknown in political circles. Correcting that to whatever extent I can is the limit of my ambition.

I am not an expert in all of the disciplines that I will be talking about.  However, computer science is a very broad discipline and I have been fortunate enough to have worked on interesting problems that required me to read scientific literature in a fairly wide range of relevant areas. I have a reasonably solid understanding of what can be said with confidence about most of the above themes  and have access to experts in others to review my writings.

This is not political propaganda.  Many of the things that I will be saying go against my own wishful thinking.  I do not want them to be true, but they just are.  I think that greater social equality is a desirable goal, which places me on the left of the political spectrum.  I think there’s lots of evidence to suggest that greater social equality would bring with it lots of benefits that most people would consider desirable.  However, that’s not all that important – I would be in favour of greater equality even if the evidence suggested that it created significant social problems.  I prefer societies that are more equal because that’s what I prefer.  I have no more interest in proving that science says that my political values are correct, than I am interested in proving that my fondness for cheese is scientifically correct.  Hence I’m going to try to leave my values out of this stuff altogether. 

I will be publishing these articles in draft form.  If anybody can point out any inaccuracies in what I publish, I will correct them as soon as possible.  Similarly, any suggestions for ways in which the drafts can be improved will be considered.  However, I'm going to avoid getting drawn into philosophical debates with people who have profound disagreements with my approach.   

I will factor these theoretical articles out of the rest of the material that I publish because there’s probably a somewhat different audience for this stuff compared to my more personal, anecdotal political stories.  Still, for anybody who does enjoy the stories, I would encourage you to have a look at this theoretical stuff – not all theory is boring and turgid!  I will make a concerted effort to keep the language accessible and clear to a broad audience, but some of the material will probably be a little bit difficult. I will publish the first article in this theoretical series towards the end of this month (May) and every 2 weeks or so thereafter.

With that introduction out of the way, I will return to the main narrative of my political journey for the moment.  In the next installment, which I’ll publish on Friday, I’ll return to Chomsky and how he relates to my political development...

Comments (8)


Very interesting- looking forward to reading more!


Learning a lot from you - never knew the socialist movement of the past was so doggedly entrenched in Marxist Theory
that it couldn't accept evidence-based changes in social thinking..

Alan MacSimoin

"his assumption that the problems of social organisation would be trivial to solve once the parasitic capitalistic class were removed from the equation turned out to be spectacularly over-optimistic."

Am I misreading you, or are you suggesting that the Russia of Lenin, Stalin or Brezhnev overturned capitalism? My understanding of the form which capitalism took there (as it did in many underdeveloped countries) was state capitalism. Very different to Ireland or the USA, but still capitalism.

Socialism has yet to be tried.

chekov's picture

I'm more referring to the fact that Marx had very little to say about what he meant by "socialism" or "communism" or what sorts of institutions or systems of social organisation he thought would be necessary for moving to whatever he was talking about from his contemporary social order. Meanwhile, he seemed quite certain that socialism / communism would emerge from the overthrow of capitalism by the proletariat. Thus, I think it's fair to say that he assumed that the problems of implementing them would be trivial (or else he could not have been so certain that they would emerge).

When it comes to Russia, I was here talking more about the ideological heritage of Marxism rather than describing the economy that the bolsheviks built. I do think that the early bolsheviks were intent on creating some variety of socialism (whatever they meant by that) and that they ran into many of the practical difficulties of social organisation without having anything much in the way of a plan for dealing with them.

Personally, I don't think that there's much use in giving actual social systems labels like socialism or capitalism or whatever as such labels mean very different things to different people and I don't think it's possible to deduce the basic dynamics of a system from such labels anyway - much better to actually look at how they worked and what problems they had.


A few thoughts on this. First of all, a minor point – having read a fair bit of Marx, I don’t recognise the description of ‘turgid prose and bombastic rhetoric’ as in any way accurate; this is a subjective judgement at the end of the day, of course, but it strikes me as a bit of knee-jerk irreverence for its own sake (and also as having your cake and eating it – if something Marx wrote is easy to read, it’s ‘bombastic’; if it’s not, it’s ‘turgid’).

More importantly, I don’t agree with the account of the way Marxism developed in the twentieth century (even allowing for the fact that a couple of paragraphs are necessarily going to be highly compressed and leave little room for subtlety). You talk about a steady, inexorable decline, from the 1900s, when ‘Marxist-inspired parties were full of scientists and critical thinkers’, towards an ‘increasingly arcane, brittle and rigid’ approach to Marxism, culminating in today’s ‘theological transformation [which] is almost complete’. So it’s a straightforward downward curve from the (relatively) good old days to the contemporary debacle.

I would see things in a different light, always remembering that the development of Marxism as an intellectual theory or tradition is closely linked to its fate as a political movement. The first two decades or so of the twentieth century were very productive, with a lot of fresh thinking by Marxists in Germany, Austria, Poland, Russia etc. Then from the early 20s there was a clear decline. The social-democratic parties no longer cared about Marxism, it was just something they paid lip service to with no connection to their practical activity. Kautsky remained a member of the SPD until his death in the late 30s but his ideas had no bearing on its practice. Even when the SPD’s finance minister in the 20s was a leading Marxist economist, Rudolf Hilferding, there was nothing recognisably ‘Marxist’ about his actions in office.

The Communist parties did care about Marxism, but after the Soviet dictatorship consolidated itself, all theoretical questions had to be subordinated to the interests of the ruling elite in the USSR. Once Stalin had eliminated his rivals, the whole Communist movement had to promote the idea that he was a genius and the greatest living Marxist thinker, on a par with Marx himself, even though he was really a very mediocre, plodding theorist. So that had a crippling effect on the development of Marxist theory (not least because any serious Marxist theoretical work from the 1920s on would have to address the nature of the Soviet Union and ask what kind of society was being constructed there, which was strictly taboo within the CPs).

Consider the fate of a few leading Marxist thinkers during the inter-war period: Kautsky died in obscurity; Trotsky was hounded to his death; Bukharin, Rakovsky and the other leading Russian Marxists were executed in the purges; Luxemburg was posthumously branded a heretic and placed on the Stalinist Index; Lukacs was muzzled and had to retreat into cultural criticism for the best part of 30 years; Korsch was expelled from the German CP and driven out of practical politics. Gramsci was only able to do creative theoretical work in the 30s because he was completely isolated from the Comintern, and his prison writings only became known from the 1950s onwards. Marxism inside the Communist movement became a dead orthodoxy. Most of the useful theoretical work from this period came from people who were outside the movement; but their isolation took its toll, there was a tendency either to retreat into an academic ghetto (the Frankfurt School intellectuals) or a sectarian one (much of the Trotskyist movement, such as it was).

But then from the late 50s, with the emergence of the New Left, things began to loosen up, and the next couple of decades were very productive for Marxism; a lot of dross was produced, but a lot of good work too, in history, economics, sociology, political theory. There was also a rediscovery of people who had been overlooked during the lean years, Luxemburg, Lukacs etc.; there were new editions and translations of their works. I’d say my own political outlook has been shaped more by the work of socialist writers from that period, the 1960s and 1970s, than by the ‘classical’ Marxists of the early 20th century.

Then you had another turn for the worse from the 1980s, which was inseparable from the political defeats of the workers’ movement and the triumph of neoliberalism. To put it very crudely, there were two unhealthy reactions to the rightwards shift in global politics. One was to throw the baby out with the bathwater; the other was to preserve every last drop of filthy bathwater for fear of losing the baby. So on the one hand you have all the ‘post-Marxist’ stuff which became the height of academic fashion, on the other hand you have dogmatic sects clinging onto a stale orthodoxy. It was no easy matter to steer a course between the two.

Even so, the last two or three decades haven’t been barren by any means. One of the writers that I have a lot of time for would be Daniel Bensaid. You identify as one of the central weaknesses of Marx’s original theory that ‘his notion of identity and how it was determined by economic forces was far too simplistic and turned out to be profoundly wrong’. When Bensaid was asked ‘what parts of the Marxist heritage clearly belong in the past’, this was his response: ‘To begin with, I would mention a certain kind of sociological optimism – the idea that capitalist development almost mechanically brings about the growth of an ever-growing, ever-more concentrated, ever-more organized and ever-more conscious working class. A century of experiences has made plain the scale of divisions and differentiations in the ranks of the proletariat. The unity of the exploited classes is not a natural given, but something that is fought for and built.’ But he was very much against ditching the Marxist understanding of class altogether:

‘The impression of a decline or even disappearance of the proletariat is often fed by a restrictive and sometimes workerist definition of social classes on the basis of classificatory sociological categories. For Marx, however, it wasn’t a matter of a positivist sociology of classes but of a dynamic social relationship – since classes only exist in struggle. If you look at the relationship to property in the means of production, the form and level of wages from employment, and location within the social division of labour, the large majority of workers in the so-called tertiary sector (including an ever greater number of women) are proletarians according to the initial meaning that Marx applied to the term. In 1848, the Paris proletariat discussed in The Class Struggles in France was not industrial but engaged in something more along the lines of studio-type craftwork. One can therefore easily mistake a weakening of class organization and consciousness (a consequence of political and social defeats) for an irreversible decline of class struggle. That said, we have to focus on the obstacles that now exist to working-class organization and consciousness: the privatization and individualization of social life; flexibility of work; individualization of work time and forms of payment; the pressure of unemployment and job insecurity; dispersal of industry and changes in the organization of production, to name a few.

‘Still, the capital/labour relationship is a central one within contemporary societies. On the other hand, I wouldn’t use the term “main conflict” since it tends to reduce the other contradictions to a “secondary” place. Rather, there are a series of contradictions that do not fall within the province of the same temporality (the same historical scale), but which are closely intertwined (or “overdetermined” by the prevailing logic of capital, to borrow a term from Althusser’s lexicon): gender (or sex) relations; the relationship between nature and human society; and the relationship between the individual and the collective. The real problem is one of linking these contradictions together.’

That’s not a bad starting-point, and a long way removed from any kind of Marxist fundamentalism. As well as being an academic, Bensaid was for many years a leader of the LCR, one of Europe’s larger far-left groups, and of the USFI, one of its larger far-left currents, so he has as much claim to represent latter-day Marxism as any of the more ossified sects that are incapable of developing new ideas.

chekov's picture

Hi Ed,

Thanks for the thoughtful comment. As you note, mine is a highly compressed account and it attempts to paint the history of Marxist thought in broad brush-strokes and doesn't go into great detail. Having said that, I don't think that there's all that much that I would disagree with you on. In this blog post: I go into a little more detail on the ebb and flow of socialist ideas, which traces a little bit more of the fluctuations, whereas this article really just focuses on the big-picture overall trend.

Most importantly, I hope I didn't give off the impression that I was dismissing the structural importance of the division between capital and labour - I think that the conflict of interest between those who depend on wages to survive and those whose income comes from capital has to be one of the most basic elements of any systemic analysis of how our world works the way it does.

I guess the two points on which I'd disagree with you are on overall flavour - I don't think that the existence of a handful of creative and non-dogmatic thinkers makes much difference to the overall trend - there are always outliers, but we're talking about a political tradition which had absolutely massive scope and the mainstream of it did become ossified under the pressures of the cold war. Secondly, I really am not a fan of Marx's prose - but that's subjective and not really all that important and probably a bit unfair of me to bring it up!


It occurred to me reading back over this that I’m probably influenced in my take on things by the field I specialize in, history. There’s not much room for an approach to Marxist historiography that’s purely based on exegesis, you’d run out of material very quickly. Marx didn’t write any works of history as such (it barely existed as an academic discipline in his lifetime); he left us with some general remarks about the primacy of class conflict and economic forces in history, which can only ever be a starting-point, telling you what to look for; he also wrote a fair bit about contemporary politics, mostly in Europe and the US, which can sometimes be useful if handled with care (often latter-day Marxist historians will start off by summarising what Marx said and showing why he was mistaken). The temptation to lean on the authority of sacred texts just isn’t there—not for any historian worth a damn anyway.

It’s a different story when it comes to economic theory; Marx wrote so much on economic questions that you could spend a long time just writing commentaries on his work, trying to summarize its main themes and debating with others about ‘what Marx really meant’ (and people still do). The way that his economic writings came to be published—two volumes of ‘Capital’ pieced together by Engels from fragments after Marx died; the Grundrisse only coming to light decades later—also encourages a textual approach. I guess it also makes more sense to talk about a steady decline of Marxism from the 1900s to the present day if you’re looking at economics, insofar as Europe circa 1905 was still pretty close to the capitalism Marx had been writing about, but has been moving further and further away from it since then, so the ‘what Marx really meant’ approach was bound to yield diminishing returns. But with Marxist historiography, it would never occur to me to think in terms of a steady decline, since most of the best work has been produced in the last 50 years or so; Marxist economics may still be very much in the shadow of ‘Capital’ (there have been several commentaries on it published in the last few years alone by the likes of Harvey and Jameson) but I would just take it for granted that Marxist historians like Hobsbawm, Perry Anderson, Arno Mayer etc. have produced work that’s way in advance of anything you can find in the Marx/Engels complete works.

chekov's picture

I guess the big difference is that, as you say, historical materialism was left fairly unspecified by Marx and, therefore, there was much less orthodoxy to have to bow to in order to usefully apply it. Consequently you got much less in the way of theoretical contortions to make observations appear to be consistent with Marx. The really major work of Marxist historiography was "Origins of the Family, Private Property, and the State" but it was written by Engels who had far less of a deity status and, therefore, it was relatively easy for latter day Marxists to diverge from it without being branded heretics. Having said that it did have a very significant impact upon anthropology in the early 20th century (David Graeber's "towards an anthropological theory of value" is good on that).