In the aftermath of Donald Trump’s surprising victory over Hillary Clinton in this year’s presidential election, some contended that the Democratic Party chose the wrong candidate. Rather than going with Clinton, whose coziness to financial elites and the party establishment left her vulnerable to Trump’s populist attacks and created the much-discussed “enthusiasm gap” among Democratic voters, they argued that Bernie Sanders would have been the better choice. Unlike Clinton, he offered a left populist vision that could have countered Trump’s nativism, racism, and misogyny, while responding to the profound economic anxieties to which the Democratic Party establishment had proven so tone-deaf.

Without getting into the merits of this argument, what is remarkable about it is the fact that it is a plausible argument to make. Just a year ago, the idea that anyone other than Clinton could be the Democratic Party’s standard-bearer would have been almost unthinkable. Even more inconceivable was the idea that a rumpled, self-described “democratic socialist” from Vermont would galvanize tens of millions of voters and come dangerously close to derailing Clinton’s anointment.

And yet this is precisely what happened. Not only did Sanders give Clinton a serious run for her money, but he pulled the political discussion to the left during the primaries. He forced Clinton to at least mouth support for a fifteen-dollar minimum wage, oppose the Trans-Pacific Partnership (TPP) and Keystone XL pipeline, along with backing some half-measures to guarantee “debt-free” college, put a “public option” for Obamacare back on the table, and “get tough” on Wall Street.

But for all the surprises about Sanders’s candidacy, one thing was entirely predictable: the fact that Sanders ran for president as a Democrat. Although he toyed with the idea of mounting an independent, third-party campaign, he ultimately felt that he had no choice but to contest the Democratic Party nomination. As he explained at an MSNBC town hall meeting in March 2016:

[First] in terms of media coverage, you have to run within the Democratic Party. Number two, that to run as an independent, you need, you could be a billionaire. If you’re a billionaire, you can do that. I’m not a billionaire. So the structure of American politics today is such that I thought the right ethic was to run within the Democratic Party.

Sanders’s points were well taken. Few countries have a two-party system as deeply entrenched as the United States, and it is one of the only industrialized democracies without a well-established labor or socialist party. In almost any other country, Sanders would likely have run as a standard-bearer for such a party. But in the American context, that option was unavailable.

This lack of a US labor party might not seem like such a big deal, especially given the recent track record of such parties in the countries where they do exist. But history shows that for workers the only thing worse than having a sellout, reformist labor party is not having such a party. Many US policy failures — particularly the lack of a public health-care system and its skimpy, punitive social benefits — can be chalked up at least in part to the lack of a labor party. (Some have argued that the Northern wing of the Democratic Party served as a functional equivalent, but the fact that it was tied both to sections of Northern capital and Southern agrarian elites fundamentally undermined its ability to play this role.)

Many people think that this lack of a labor party is baked into American culture and politics. Going back more than a century, the German sociologist Werner Sombart famously argued that economic prosperity undermined efforts to build a socialist party in the United States, which “came to nothing on roast beef and apple pie.” Engels declared the country’s “purely bourgeois” culture (and lack of a feudal past) an obstacle to the formation of a labor party. Decades later, US social scientists like Louis Hartz and Seymour Martin Lipset built on Engels’s insights, contending that the country’s pure “Lockean” liberalism created an exceptionally hostile terrain for a labor party to take root.

On top of its hostile political culture, others have argued that US election rules create insurmountable barriers for labor parties, or any kind of third party for that matter. According to such accounts, “winner-take-all” elections combined with a presidential (as opposed to parliamentary) political system virtually guarantee a two-party system.

It is true that labor parties have long faced an uphill battle in the United States. But focusing on enduring traits like political culture and election rules misses a big part of the story that has direct relevance to the present political moment: prior to the 1930s, there was a much more vibrant independent left party tradition than exists today.

The potential of this lost political tradition becomes even clearer when we compare the US experience to the political trajectories of other countries. Take Canada for instance. Despite being socio-economically similar to the United States, a labor-based party, the New Democratic Party (NDP), is well established in Canada.

Granted, the NDP has tended to serve as a third party at the federal level, and has suffered the same kind of rightward drift that has affected social-democratic parties elsewhere. But its presence in the political sphere has made a difference. Scholars have identified the NDP as a key reason for Canada’s relatively stronger labor laws and more generous social benefits, including its publicly funded health-care system.

Many believe that Canada’s more collectivist political culture and parliamentary political system created more favorable terrain for a labor party to take root than in the United States. But the historical data tells a different story: despite the seemingly more favorable political terrain, prior to the 1930s Canadian labor and left parties had an equally difficult time getting off the ground as their American counterparts. Indeed, it was only in the 1930s that we saw a divergence.

We can see this in Figure 1, which tracks vote shares for “independent left third parties” (ILTPs) in the United States and Canada from 1867 to 2009. Prior to World War I, ILTP support was generally higher in the United States. From the end of World War I through the early 1930s, Canadian ILTP support was volatile, while US support remained small but significant. There was no reason not to expect another spike in ILTP support along the lines of what occurred in the 1890s or pre–World War I period. The decisive divergence occurred in the 1930s when ILTP support collapsed in the United States and took off in Canada.

(Figure 1 might suggest that ILTP support diverged after World War I, but the postwar spike marked a period of volatility in the Canadian party system, not a lasting shift in ILTP support. It represents the meteoric rise of the Progressive Party at the federal level and the United Farmers movement at the provincial level. These agrarian parties’ impressive electoral gains proved fleeting. In no case were Progressive/United Farmer parties able to establish a lasting political presence, and none survived past 1935. It was only in the 1930s with the Cooperative Commonwealth Federation’s [CCF] emergence that ILTP support took hold in Canada.)

Why did this shift occur? What happened in the 1930s that simultaneously bolstered labor party fortunes in Canada, and undermined them in the United States? Arguments about enduring political cultures and election rules cannot answer this question. Instead, the social, political, and economic struggles of that critical turning point — and how the outcome of those struggles fundamentally reshaped the political landscape in both countries — are of central importance.

The Labor Party Promise

Prior to the 1930s, as afterwards, the Democrats and Republicans were the two major US political parties. In Canada, it was the Liberals and Conservatives. But as Figure 1 suggests, the major parties had small but significant competitors to worry about. Amid the economic tumult of the late nineteenth and early twentieth centuries, workers and farmers organized mass protest movements. Some of these took the form of political parties, many of which spanned both sides of the US-Canada border.

Figure 1 also shows that these parties had a tough time surviving over the long haul, rising and falling along with the movements that birthed them. But their presence shaped the political landscape. Even when they lost, as third parties often do, they put key issues on the table, forcing the major parties to address them. At a more symbolic level, they showed that workers and small farmers could be an independent political force.

This is not to say that workers and small farmers were not involved in major party politics. They just were not involved as workers and small farmers. Rather, they were organized into the major parties based on religion, race, ethnicity, region, and other identities. That changed in the 1930s. Many workers and small farmers in both the United States and Canada found political identities as workers and farmers. In so doing, they forged more durable class-based political coalitions than had previously existed in either country.

But these coalitions looked very different, and had different political consequences. In the United States, workers and small farmers were eventually absorbed into the Democratic Party, tying them to a peculiar coalition that included Northern urban liberals and Southern racist reactionaries. In Canada, they allied independently of the mainstream parties to form a new party, the Cooperative Commonwealth Federation, precursor of today’s NDP.

Co-optation Versus Coercion

To understand why these different party-class coalitions emerged, we need to look at how the ruling parties of the time (Democrats in the United States, Liberals and Conservatives in Canada) responded to the worker and farmer protests sparked by the Great Depression.

The short version is that President Roosevelt and the Democrats adopted a co-optive response that absorbed the protests and undercut political challengers, while the Canadian Liberals and Conservatives adopted a coercive response that aggravated the protests and left room for challengers, specifically the CCF, to take root.

In the United States, FDR’s 1932 campaign invoked “the forgotten man at the bottom of the economic pyramid,” and made President Hoover’s handling of the Depression a key election issue. This was part of an effort to appeal to workers’ class identity, and position the Democratic Party as the “natural” home for such voters. Over the course of the 1930s, this strategy succeeded.

In hindsight, this seems obvious. Of course FDR sought to position the Democrats as the “natural” home for working-class voters, and make the incumbent’s handling of the Great Depression a core campaign issue. But at the time it was anything but obvious. Organized labor did not yet have the kind of institutional commitment to the Democrats that exists today, and workers’ party allegiances were governed much more by neighborhood, ethnic, or religious ties.

At the same time, the Democratic Party was hardly a political vehicle for liberalism. Much of the party leadership in the early 1930s was to the right of President Hoover, and attacked his handling of the Depression by accusing him not of inaction, but of wasteful deficit spending. Some like DNC chair John Raskob wanted to downplay economic issues and focus instead on Prohibition.

With Hoover deeply unpopular as the Great Depression ground on, the 1932 election was the Democrats’ to lose. What was missing after decades of defeat was a winning coalition. Some saw Roosevelt as the one who could unite urban progressives, Southern conservatives, and Western populists within the existing party, while also attracting new constituencies. Roosevelt’s advisers recommended a contradictory policy mix of centralized planning and fiscal austerity, including a 25 percent across-the-board cut in federal spending. The hope was that this, combined with his “forgotten man” rhetoric, would solidify a progressive coalition, while remaining vague enough not to alienate conservative Democrats.

It didn’t quite work. Conservative Democrats reacted with alarm to what they perceived as Roosevelt’s class-laden rhetoric, provoking 1928 Democratic presidential nominee Al Smith to mount a primary challenge to Roosevelt. He charged FDR with seeking to “destroy [the masses of working people of this country] by setting class against class and rich against poor.” Roosevelt withstood Smith’s challenge, but the primary fight made the Great Depression and Roosevelt’s class rhetoric far more of a campaign issue than he and his supporters had hoped.

Roosevelt brought his balancing act to the general election against Hoover, blending social welfare liberalism with a conservative defense of states’ rights and balanced budgets. The approached bewildered Hoover, but won over the electorate. In the process, it positioned the Democrats as what one observer termed “the liberal party, the party which will restore the balance of power between the rich and the poor,” and created an opening for Roosevelt to bring labor into the Democratic Party tent.

The extent to which this intra-party conflict and political maneuvering mattered becomes clearer when we compare what happened in Canada. There, neither party grasped the depths of the crisis, and both remained opposed to government intervention in the economy. Conservative leader R. B. Bennett capitalized on dissatisfaction with incumbent Liberal prime minister William Lyon Mackenzie King to defeat him in the 1930 election, but not based on promising to solve the country’s economic problems. Rather, he ran on a law-and-order platform, vowing to crush farmer and worker protest under an “iron heel of ruthlessness.” Upon taking office, he had prominent organizers jailed or deported, the Communist Party banned, radical literature censored, and meetings disrupted. Idled workers were rounded up and shipped to remote work camps.

Bennett’s failure to revive the economy cost him the election in 1935, which returned King’s Liberals to office. But even then, King did little to reach out to aggrieved workers and farmers, and continued to tamp down protest. This left room for a new party to emerge.

That party was the CCF. It started off in 1932 as an odd mixture of agrarian populists, a few union observers, a rump group of “Independent Labour” members of parliament and scattered representatives from local and provincial labor party organizations, along with a group of largely academic socialist intellectuals known as the League for Social Reconstruction. In the United States, similar groups were drawn into the New Deal coalition. But the Canadian ruling parties’ unimaginative and hostile response to the crisis and surrounding unrest left these groups politically excluded, creating a “common foe” to unite them.

The Policy Trap

If ruling parties’ rhetoric and the dynamics of party competition created different possibilities for political coalitions in the United States and Canada, it was their policies while in office that cemented those coalitions. In the United States, New Deal labor and agricultural policy offered enough material benefits to absorb certain farmer and labor constituencies, while simultaneously undermining independent farmer and labor political organizing by exacerbating intra-class divisions. The result was labor party collapse. In Canada, the Liberal and Conservative parties’ repression and neglect of farmer and labor constituencies left them excluded, pushing them toward the CCF.

It’s important to recognize that when Roosevelt took office in 1933, farmers and workers remained politically up for grabs. Building on a legacy of independent political organizing, labor and agrarian populist parties were gaining traction across much of the upper Midwest and West Coast. In Minnesota, the Farmer-Labor Party regularly won state and federal office throughout the 1920s and claimed the governorship in 1931. For his part, Roosevelt made no mention of unions or collective bargaining rights in his 1932 campaign.

New Deal farm and labor policy changed that. Agricultural policy focused on shoring up collapsed farm prices, which privileged large farmers and consolidated a conservative “farm bloc” within the New Deal coalition. Meanwhile, a moratorium on farm foreclosures and relief subsidies benefited family farmers, previously the backbone of agrarian populism. These benefits drew them towards the Democrats while making them uneasy about backing a third party, “unless they are cocksure of something better” as a Wisconsin farmer leader put it.

Meanwhile, since New Deal farm subsidies only went to landowners, they excluded tenant farmers, sharecroppers, and farmworkers. However, given Jim Crow–era voting restrictions and the citizenship status of many immigrant farmworkers, these groups were often disenfranchised and could not serve as a viable base for a farmer-labor party. Additionally, planter and grower violence impeded class-based political organizing.

All told, FDR’s agricultural policies privileged agrarian elites, provided enough benefits to placate small farmers, and excluded tenant farmers, sharecroppers, and farmworkers. This undermined the agrarian constituency for a farmer-labor party and encouraged absorption into the Democratic Party. At the same time, the consolidation of a conservative agrarian bloc within the New Deal coalition limited the reform possibilities of the reorganized Democratic Party.

As for labor policy, Roosevelt’s advisers convinced him to include language guaranteeing unions’ collective bargaining rights in the 1933 National Industrial Recovery Act (NIRA), which was replaced and strengthened by the 1935 National Labor Relations Act (NLRA, or Wagner Act) when it was declared unconstitutional. As union membership exploded, the perceived benefits of the Wagner Act drew labor towards Roosevelt, while his increasingly ambitious “Second New Deal” reforms led business elites to abandon him. Deprived of business support, FDR embraced labor as a key source of funds and votes for his 1936 reelection campaign.

As labor found a more sympathetic ear in the White House, union leaders who had previously supported a labor party began to temper their position. Clothing Workers president Sidney Hillman explained:

The position of our organization is known: that we are for a labor party . . . But in the last two years things have happened . . . We have participated in making the labor policy of this administration . . . We know that [NIRA] meant the revival of our organization . . . We know that the defeat of the Roosevelt Administration means no labor legislation for decades to come . . . The re-election of Roosevelt will not solve all our problems, but it will give us a breathing spell.

Just as the Wagner Act drew potential labor party supporters into the Democratic Party, so too did it undermine independent labor party efforts. It did so by pitting the rival American Federation of Labor (AFL) and Congress of Industrial Organizations (CIO) against each other over the act’s implementation and interpretation. This drew organizational energy away from labor party organizing and sabotaged cross-union political collaboration, particularly at the local level. These divisions solidified the labor-Democrat alliance at a moment when labor seemed poised to organize for independent class politics.

Although workers backed FDR in 1936, labor support for a party of their own grew as post-election frustration with Roosevelt set in. His overly even-handed response to the Flint auto strikes of 1936 to 1937 and then the “Little Steel” strikes in May of 1937 prompted CIO leader John L. Lewis to remark that “it ill behooves one who has supped at labor’s table and who has been sheltered in labor’s house to curse with equal fervor and fine impartiality both labor and its adversaries when they become locked in deadly embrace.”

Many also complained of Roosevelt’s foot-dragging with key labor legislation in the new Congress. By that point Lewis was hinting strongly at forming an independent farmer-labor party. At the same time, state-and local-level labor parties were gaining traction, building on the disconnect between Democratic Party rhetoric defending labor rights and the reality of Democratic governors using state troops to break strikes.

Some sought to coordinate these parties at a national level, but their efforts went nowhere, and by 1938 almost all the parties had collapsed. Why? NLRA provisions (and their interpretation) served to stoke intra-class conflict, pitting AFL leaders who believed the law was biased in the CIO’s favor against CIO leaders, who responded in kind.

These inter-federation attacks might have spurred organizational growth, but they also seriously hampered political organizing. Rival federations had to defend their turf, distracting from efforts to channel workers’ dismay with the Democrats into labor party support. Internecine conflict paralyzed local labor councils, one of labor’s main political vehicles. The AFL purged all CIO affiliates from the councils and withdrew support from CIO-sympathetic candidates. This defunded many local and state-level labor party movements.

Ultimately, the combination of inter-organizational conflict and resource diversion undermined whatever base might have existed for labor party support. The Democrats were left as the only game in town.

Meanwhile in Canada. . .

Ruling parties’ policy responses to the Great Depression in Canada had the opposite political effect that they did in the United States. Whereas the New Deal absorbed key constituencies into the Democratic Party while undermining independent political organizing, Canadian ruling parties’ policies of repression and neglect of farmers and workers foreclosed the possibility of absorbing them, while galvanizing support for the CCF.

For farmers, Bennett’s increased tariffs and meager farm subsidies closed off more markets to Canadian farmers but did little to bolster collapsed farm prices. This pushed many farmer groups to seek an independent political voice. As for workers, Bennett and his provincial counterparts offered paltry unemployment relief along with an ample dose of state repression. King reversed some of Bennett’s most egregious anti-labor policies upon returning to office in 1935, but rebuffed calls for a Canadian Wagner Act. Provincial and federal governments enacted some labor regulations, but nothing approaching the NLRA.

Despite the lack of labor legislation, union organizing surged, with membership spiking by 37 percent between 1935 and 1937. The lack of labor rights might have hampered Canadian unions’ growth relative to the United States, but it also unified Canadian labor. While AFL and CIO unions traded accusations of government favoritism as they fought over the NLRA, Canadian unionists united to protest their lack of legal legitimacy.

The Canadian Trades and Labour Congress (TLC), which included AFL- and CIO-affiliated unions, only expelled its CIO unions in 1939, two years after the US split, and then only under direct pressure from the AFL. Importantly, the TLC did not conduct AFL-style purges of local labor councils when it expelled CIO affiliates, allowing local councils to affiliate with the CCF. Also, unlike in the United States, Canadian CIO unions (who eventually formed the Canadian Congress of Labour (CCL) sought reunification with the TLC from the start. The Canadian state’s refusal to recognize labor rights did not dissolve inter-federation rivalries, but it did mute their political significance.

All this created an opening for the CCF. The problem was that Canadian labor remained divided on the question of its political affiliations. Some sought closer ties to the CCF, while others maintained a non-partisan stance. The non-partisans were aided in their efforts to resist CCF affiliation by supporters of the Communist Party.

World War II brought the question of labor’s political affiliations to the fore. Unlike in the United States, where Roosevelt’s wartime labor policy successfully absorbed the labor leadership and tightened the labor-Democratic Party alliance, King’s wartime labor policy further alienated Canadian labor. He excluded labor representatives from wartime planning agencies, and his efforts at promoting collective bargaining were purely advisory and ignored by employers. Meanwhile, he responded to wartime strikes with increased state repression.

This politicized wartime class conflict, exposing the gap between government promises of promoting collective bargaining and the reality of steadfast government intransigence. As the number of strikes doubled between 1941 and 1943, King’s response was a series of government edicts further restricting picketing and strikers’ civil liberties.

Spiraling class conflict spilled into the political arena, as the CCF surged in industrial Ontario. The party took thirty-four seats in Parliament in the 1943 provincial election, enough to form the Official Opposition. By uniting their agrarian Western base and industrial workers in Ontario, the CCF was now a much more serious threat to the Liberals, and a more viable political option for labor.

This prompted labor’s non-partisan faction to switch sides, backing a resolution at the 1943 CCL convention recognizing the CCF as the “political arm of labour.” The ruling parties’ policies of repression and neglect had prevented labor’s absorption, instead allowing the CCF to take root as a viable labor party in Canada.

The Paradox of Social Democracy

As American leftists recover from the shock of Hillary’s defeat and Canadian leftists take stock of the NDP’s rightward drift, especially in light of last year’s electoral collapse and current leadership vacuum, it becomes clear how the current political terrain has been shaped by these past struggles. While some chalk up the current situation to deeply rooted political cultures and electoral rules, a look at the history shows that the present party system was neither predetermined nor immutable.

That does not mean, however, that it is easy to change. For now, the consequences of those political struggles of the 1930s are dire. In the United States, in the face of the Democratic Party’s apparent collapse and the rise of the neo-populist right, the missing labor party has become a “phantom limb” of American politics, an amputation of the political spectrum whose pain is still felt in its absence.

From being a movement which, though small in size, posed enough of a threat to force the major parties to adopt many of its key demands, today the labor party threat is reversed, with the results of the Democratic Party’s penchant for alternatively bullying and ridiculing those who fail to fall in line behind the latest “lesser evil” candidate made starkly clear in Trump’s victory.

But the Canadian story shows that retaining that political “limb” comes with its own set of problems. From issuing bold statements such as those in Social Planning for Canada (1935) and Democracy Needs Socialism (1938), and accomplishing feats like leading the expansion of collective bargaining to the public sector and establishing the first public health-care system in Canada, the CCF/NDP has retreated from that vision over time, to the point where many on the Left question the value of continued support for the party. Indeed, with many provincial NDP governments presiding over austerity regimes once elected, and the federal party outflanked to its left on more than one occasion by Justin Trudeau’s Liberals, it is hard to recognize today’s NDP as a party of the Left.

To different degrees, the NDP and the generations of activists who have sought in vain to pull the US Democratic Party to the left have been ensnared in what Robert Brenner has called the “paradox of social democracy”: the parties’ rise, expansion, and ability to win reforms have depended on the existence of mass social struggle. And yet, as these parties (or party factions, in the American case) have become established parts of the political landscape, that very base of social struggle has become a liability to their organizational legitimacy and stability. As such, they have sought to contain, co-opt, and even crush the disruptive social forces that have enabled their rise, while simultaneously moderating their party program.

In Canada, this has been the story of the NDP at least since the defeat and expulsion of the socialist “Waffle” faction in the early 1970s, and can be seen in its allergic reaction to most social movements since. In the United States, labor’s absorption into the Democratic Party in the 1930s was an important first step, followed by the incorporation of key layers of the black freedom and feminist movements of the 1960s and 70s. More recently, the challenges have come from #BlackLivesMatter activists and the Sanders insurgency.

Clearly, the Left faces serious challenges to finding its political voice in both countries. But those challenges remain starker in the United States. For all its problems, the NDP has helped prevent the Canadian political spectrum from lurching further to the right than it would have otherwise. More importantly, the NDP retains both nominal commitments and structural connections to labor and some progressive movements. These could potentially be mobilized as part of an effort to reform the party, as we are seeing now with the Leap Manifesto.

The same cannot be said for the United States, where the Democratic Party has never made any pretention of being a working-class party, aside from seeking working-class votes. Repeated efforts at “realigning” the party along more progressive lines have diversified the racial and gender make-up of the party leadership and improved its positions on some social issues, without challenging its fundamentally pro-business outlook. Ultimately, a new left politics in the United States will have to involve recapturing the energy of some of those pre-New Deal labor, agrarian, and socialist radicals who held on to the promise of a party of their own.

---