“They took my shopping cart.”

On Market Street in downtown San Francisco, an old man with long, dirty hair has dropped his collection can. Coins and scraps of paper spill across the sidewalk and into the road. Nobody is helping.

Every day, between two and four in the morning, cleaners with high-powered water jets come out to hose Market Street clean of urine, rubbish and, sometimes, men, women and children who have nowhere else to go. Bags and belongings are confiscated in an effort to discourage one of America’s most intransigent homeless populations from sleeping on sidewalks that are now slap in the centre of the most desirable neighbourhood in one of the nation’s most expensive cities. This is where Twitter has its headquarters. This is where the best and brightest young minds in the world come to work as engineers, designers, visionaries. It’s just a shame that the smell of shit and guilty consciences is hard to soak away.

The story of how the hi-tech industry has brought rapid social change to San Francisco has been written before, by journalists and authors who have lived their entire lives in the Bay Area and watched the city struggle through successive industrial bubbles and survive. That story has been so well covered in recent months that the San Francisco Chronicle published a short guide “to help our brethren who have flocked here to write the latest ‘How the tech boom is changing San Francisco’ opus”: “It is a tale of two cities. It’s a city of very rich people and very poor people . . . A city losing its soul. Sleek, black Uber cars that whisk hipsters from bar to bar . . . Thirteen-dollar sandwiches. Google Glass. That’s your opening paragraph, visiting journalists. You’re welcome.”

The question of whether rampant social inequality is changing San Francisco is a straightforward one, and the answer to it is yes. Yes, tech money is driving the freaks, the queers, the broke artists and ordinary working-class families out of the city, helped along by 30 years of backward housing policies, greedy landlords and inefficient social care. That much is obvious.

What is just as interesting to those with an eye on the digital future is how the local politics of San Francisco is changing technology. The industry that is having the most profound effects on the way all of us live, that is changing what it means to be social, has its geographic hub in a place where social tensions have never been higher.

When “techies” and tech reporters talk about San Francisco, what they usually mean is a small chunk of this already small city – the central bustle of the Mission, the Tenderloin and the sprawling former docklands south of Market Street (SoMa). Twitter is located here, thanks to a generous tax break from the city authorities; it’s where lots of the start-ups are based, and it’s where a great many of the tens of thousands of new tech employees who have moved here in the past few years are choosing to live. It’s also where the tension between the dazzling digital future and the grotty hypocrisy of the present is most cartoonish.

On my first day, walking past a pop-up futurist conference called Quantum Leap, I am obliged to step around several homeless men sleeping in the baking sun. A few yards away, in the line for artisanal coffee whose $20 price tag is apparently justified by a special brewing method, young entrepreneurs discuss their starting salaries while an elderly woman dressed in grimy plastic bags yells, “How dare you?” at nobody in particular. San Franciscans are rightly annoyed when recent visitors boggle at their city’s social contradictions. Unfortunately, it’s the first thing you notice. When you get used to not noticing it – because otherwise you’d never be able to drink your coffee and get on with your day – that’s a different problem altogether.

There is something slightly creepy about the bourgeois cybertopia of Start-up City, all hi-tech doodahs and rustic jumpers. Valencia Street is where good hipsters go when they die. There are a lot of little dogs, but not a lot of little children. There are artisanal cheese parlours, “destination” bakeries and tasteful shops where you can buy a variety of specialist instruments for disappearing up your own backside, or those of others. The weather is always beautiful, a perpetual late spring where it’s never quite cold enough to wear your fanciest overcoat.

It’s all a little bit Pleasantville – if you ignore the vagrants, addicts and angry activists. That’s the sort of ignoring that takes practice. “In downtown SF the degenerates gather like hyenas, spit, urinate, taunt you, sell drugs, get rowdy, they act like they own the centre of the city,” wrote Greg Gopman, a former start-up CEO, on his Facebook wall last December. “There is nothing positive gained from having them so close to us.” One of his friends replied that the presence of so many homeless people was an embarrassment when “we’re supposed to be a gleaming utopia of what a city could be”.

There are half a million long-term homeless people in the US, and a third of them live in California. Still suffering from the stripping of inpatient mental health facilities in the Reagan years, people with nowhere else to go often find themselves in downtown San Francisco, with its clement weather – specifically, they find themselves in SoMa and the Tenderloin, which is where many of the non-profits set up to meet their needs are based.

****

Howard Street, where I’m staying, has many short-term-occupancy hotels and charity hostels. The street reeks of excrement; the lack of public toilets is an ongoing problem, and the many homeless people in the area have no choice but to relieve themselves in the street. Next door to the bedsit hostels is Startup House, a dormitory-style live-work space where hopeful young tech entrepreneurs from outside the city can rent a bunk bed for more than the cost of an apartment in other American cities. Over two weeks I lose count of the times I am warned, as a small white woman, not to walk down the street by myself; there is a sense of mutual hostility, of different communities that, with more or less reason, can’t look one another in the eye.

The place where I’m staying is a repurposed warehouse full of artists and start-up kids, all of them progressive, alternative weirdos who would be weirder still in a town where rainbow-coloured hair, veganism and a fondness for music that goes bleep weren’t entirely mainstream. They have far more in common with me, a foreign reporter from thousands of miles away, than they do with their next-door neighbours. I know them (how else?) from the internet, and they would probably be perplexed to hear anyone describe them as cool kids – like nearly everyone else in this city, they see themselves as outsiders, as people who have fought hard to find their place in a mean and vicious world. They are still fighting.

My friends aren’t millionaires but some of their friends are, and they’re all hustling to make enough money to stay in the city where some of them have lived since they were born. Half of them are running a new company together through Y Combinator, the most prestigious of the start-up “accelerators” that lend seed capital to fledgling companies and that helped launch Airbnb, Dropbox and Scribd, along with hundreds of companies that haven’t made millions.

Bay Area venture capitalism has its own dialect; the frequency with which people employ it is a reasonable indicator of how much of the Kool-Aid they’ve drunk. That’s another buzzword, as it happens; during the last tech boom, the catchphrase “Keep drinking the Kool-Aid” was used by dotcom workers encouraging one another to stay in the game, a reference both to the Jonestown massacre and to Tom Wolfe’s Electric Kool-Aid Acid Test. In the San Francisco of legend, the one I grew up reading about, the drugs were acid, sex and weed. Now the artists are off their heads on calamity capitalism and very expensive coffee.

****

“Make something people want”, the Y Combinator slogan, is one of the unofficial mottoes of Startupland. On Folsom and Ninth, behind a nondescript warehouse door, is Heavybit – a start-up incubator hosting newborn companies that are designing widgets and magic gizmos straight out of Star Trek. “Incubator” is the right word; this is an intensive-care nursery of Bay Area capitalism, where baby start-ups are tended to by mentors and investors and kept functioning with a constant supply of coffee and nutrition bars.

The young people working here are fashionable, multiracial, excited. I’m not allowed to tell you what they’re developing. When I ask, several hardware engineers compete to come up with things they might be making: “America’s first unicorn breeding programme!” “Instagram for cats!” “A fitness tracker for your invisible friend!”

“Start-up kids” aren’t all white and Asian men in their twenties and thirties but that’s mostly who we’re talking about. The conversation about why the tech boom has rewarded a very specific, privileged demographic is everywhere in San Francisco, in cafés and bars and on the blogs and journalism start-ups read by the scene, and the reasons given are always the same: women didn’t learn to code early enough, black and Hispanic families don’t value education in the same way. Tech success is most likely to look like a nerdy white guy in a hoodie – a lot like the Facebook founder, Mark Zuckerberg. Of course, not every Bay Area bazillionaire is a nerdy white, middle-class guy in a hoodie, but enough of them are that the class, gender and racial disparities are uncomfortably obvious, especially if you’re new in town.

SoMa is the Wall Street of the Twenty-Teens, populated by venture capitalists and well-dressed young hopefuls, although that’s likely to mean a slightly more expensive hoodie rather than a suit. Smart, ambitious young people now go into tech in the way that they once went into banking – because their mothers told them to go the hell out there and make some money.

“This new breed, they’re not hackers,” says Jamie Zawinski, who left the tech industry to run the DNA Lounge, one of the most popular music venues downtown. “Not in the true, original sense of the word. They’re not interested in taking things apart, in exploring, in building things. They got into this line of work because someone told them it was a good career, and it’s disgusting – and that’s why I don’t do it any more.”

Zawinski was one of the founders of Netscape, “the first web browser that mattered”, and is often touted as an example of a techie who has worked to keep San Francisco’s vaudevillian weirdscape alive and dancing, rather than just getting out with his pile.

“I’m interested in building things, in how technology can change the world and the way we interact with each other, and how these weird little prosthetics they’re building can improve your life,” Zawinski says. “They’re interested in cashing out. Fifty per cent of their conversation is about finances. I don’t think they even realise it. It’s a dramatic failure of imagination.”

He warms to his theme. “That’s all any of these fuckers talk about. It’s disgusting. They’re just so focused on cashing out and making money, not actually building anything real, just working hard enough to get by. That changed around ’97, ’98, but it got really bad about four years ago. It’s the second bubble.”

There are two things every child knows about bubbles: they are beautiful, and they burst. Time and again, I am told that San Francisco is “a bubble” – referring both to the gorgeous, insular never-neverland where workers in the city’s tech and associated industries live and play and to the localised economic boom that has fuelled the fat years. Those who have been around for a decade or more remember what happened when the dotcom bubble of the 1990s finally popped: a lot of people lost their jobs and homes, and rental prices in the city came down, but not drastically. Rents are now higher than ever – rising more than three times faster than the national average – and despite intra-industry deals of dubious legality to keep tech wages down, they show no sign of dropping.

Zawinski is right: money is more important than ever to San Francisco’s fast-moving youth cohort. But if you lived in the most expensive city for yuppies in America, where you need a $70,000 salary just to pay your debts and make the rent, you might be worried about money, too.

“We want to make money because here, we have to,” says one photographer who now works as an engineer at a start-up, and hence is able to afford lunch. “By here, I mean San Francisco, but also America. Our generation is fucked. We’ll get no social security, we have no retirement, no savings, no job security.”

The young people entering the tech industry today are of a generation that does not necessarily long for riches, but desperately wants to be able to afford a decent standard of living without having to avoid its reflection in the mirror every morning.

“Most of us tech people will never be the 1 per cent,” says the photographer. “We’re aiming for the 10 per cent, which might allow us to have our parents’ lifestyle. We’re just trying to get by in a world that’s not giving us the same chances the baby boomers had. What did I do as an artist? I went and got an engineering degree.”

Sometimes when you’re dying of thirst, you have to drink the Kool-Aid.

If today’s tech industry is what yesterday’s financial sector was, there is one important difference. In finance, making money is the ideology, as well as the effect of that ideology. By contrast, much of the tech sector is driven by an ethos that is liberal, or at least progressively libertarian – it’s supposed to be about making things, breaking things, challenging conventional power and its monopoly over information. It’s about disruption.

There are even decent numbers of self-described anarchists filing on to the unmarked Google buses that have become the default symbol of how tech money is taking over the city. (I won’t go on about them here, in case I fall foul of the San Francisco Chronicle, but the short version is that these buses, which ferry workers from the city to the tech giants’ suburban campuses, have become the focus of the campaign against new money.)

At the heart of the San Franciscan tech industry is a paradox. It’s a paradox about creativity coming to terms with its place in capitalism. Hacking, computer engineering, the entire ethos of changing the world with communications technology, is supposed to be about more than money – but money is what a job for a big tech firm now means. And money changes things. “Don’t be evil” was the mantra of Google, but there is some anxiety that “evil”, or at least callousness, is attaching itself to tech corporations anyway. Google, like other Bay Area firms, is anxious to preserve its benign social reputation; none of its employees was allowed to speak to me on the record about gentrification and cultural anxiety. Tech workers don’t want to see the Mission emptied of poor families. But it’s happening anyway.

Tech still sees itself as an upstart industry challenging power, even when it has become that power. It sees itself as a field of outsiders, which is why it is difficult for tech workers to comprehend that they are now the privileged insiders who are resented by artists and communities of colour. The graffito “Techies go home”, scrawled on the sidewalk outside one business in Oakland, was posted on Twitter by an angry, anguished tech worker who explained that she was “born here”.

San Francisco’s growing inequality has been described, inaccurately, as a culture war; in fact, it’s a simple class war. The cultural anxiety is experienced almost entirely by tech workers, their associates and friends. Most of the tech entrepreneurs you meet in San Francisco are alternately embarrassed and defiant about money, whether they have it or merely hope to have it, and struggling to come to terms with what that money is doing to the city they love. In line for another overpriced coffee – hey, I never said the Kool-Aid wasn’t tasty – another start-up employee tells me he’s thinking about making a video game based on gentrification. The gamer would play a local resident trying to build schools and cheap grocery stores as the incoming bourgeoisie sets up cocktail bars. It’s a joke. It isn’t meant to be cruel.

Bubbles are fragile, but from the outside they can seem hard to breach. I take a drive through the city with my friend Meredith, another artist who is struggling to stay afloat in San Francisco. “Financially, this city just exhausts creative people that don’t have a lot of money supporting their art, somehow. Everybody else is hustling constantly just to break even.” Her current sublet – a small, leaky room with no kitchen on a houseboat in Sausalito, for $600 a month – is up in 30 days. She still hasn’t found another place to live. “I don’t know if I’ll stay.”

Meredith is a fixture on the San Francisco arts scene, the former editor of the underground magazine Coilhouse, and a recording artist on, of all things, the theremin. “I spend a lot of time staring at the city from across the bridge,” she says. “I spend a lot of time wishing I felt like I belonged.”

Artists aren’t the only people locked out of this “gleaming utopia”. Raquel Donoso, of the Latino Community Foundation, tells me that “about 40 per cent of Latino families do not have a computer and internet access at home. That included East Palo Alto. This is in Silicon Valley, where there’s tremendous opportunities and tons of money. People are losing their jobs and they can’t even get unemployment insurance, because everything is online. Our families are basically locked out of that world.”

There is nothing wrong with making things that people want. The problem is that personhood and desire are constrained by capital; money affects whose wants appear to matter. The kids in Startup House may want a pizza delivery drone, but not in the same way low-income families want health care, or the elderly men lying in their own faeces on Howard Street want a safe place to sleep. There is nothing wrong with making things people want. It’s just that too little attention is being paid to the things people need.

The wants and needs of young, healthy, middle-class people with connections and a reasonable amount of spare cash are overrepresented among Start-up City’s priorities. For one thing, those are the problems with solutions that sell. For another, given a few million dollars and a team of semi-geniuses, those problems are easy to solve. Structural social injustice and systemic racism are harder to tackle – and that’s where the tech sector has, until recently, thrown up its hands.

“We can’t do anything about it” is what I hear, time and time again, from business representatives and rich individuals. And no – individuals can’t do a great deal about 30 years of awful housing policies and structural tax exemptions that prevent wealth from being redistributed. But corporations that are now wealthier than many national economies might be able to do something.

Google, in response to the Google bus protests, has just announced a scheme to subsidise public transport for schoolchildren. It’ll cost millions, but that’s a snip for a firm that earned $16.86bn in the last quarter of 2013. “I don’t think that just because gentrification happens, people can sit back and say it’s OK,” Donoso says. “Forcing people out of their homes is not OK. We have to have a discussion about what kind of community we want to build and live in, and how to maintain that community.”

****

“Eviction after 20 years”, reads a hand-drawn sign in an apartment window on Market Street. A few doors down, the shopfronts of the late-night Whole Foods – an expensive organic supermarket which is, apparently, also a major cruising spot – display large, stylish posters of smiling peasant farmers in faraway countries with cheerful motifs about “ending poverty”. Poverty many thousands of miles away can and should be alleviated, preferably with a small donation at the checkout. Poverty that you have to step over to get into your office is disturbing, and then embarrassing, and then – finally – annoying.

The likes of Google and Twitter make much of their commitment to philanthropy. Until recently, however, that generosity has not extended to their own backyard.

“Think about the Carnegies, the Mellons, the Fords. Even here in the city, Levi Strauss started here,” Donoso says. “All of those businesses really made philanthropy a part of their DNA and it was about solving our local issues. I don’t necessarily see that in the [hi-tech] sector. People see themselves as more global.”

A large part of what political theorists term “the Californian Ideology” – a heady brew of progressive techno-libertarianism and laissez-faire capitalism – is based on lack of faith in governments to solve social problems. “The centre of power is here,” said the venture capitalist Chamath Palihapitiya during last year’s US government shutdown, in an interview with the website This Week in Startups. “Where value is created is no longer in New York, it’s no longer in Washington, it’s no longer in LA. It’s in San Francisco and the Bay Area . . . Companies are transcending power now. We are becoming the eminent vehicles for change and influence, and capital structures that matter.”

“This anti-government ethos is really troubling and short-sighted on the part of those techies who hold it,” says Gary Kamiya, a journalist and lifelong San Franciscan and the author of the guidebook Cool Gray City of Love. “They don’t understand certain basic things about how society functions, like the concept of the tragedy of the commons – that if everybody is only out for themselves, you’re unable to come together to create things that can only be done by joint action.”

For the luckier San Franciscans, the argument that techno-capitalism is a good substitute for the state isn’t hard to make – not when their employers provide everything social democrats expect the state to supply, from transport and health care to yoga rooms, video arcades and laundry service at the office. There is little reason to believe in solving local problems collectively when evidence of the failure of collectivism is sleeping in the sun on every sidewalk.

What would the tech industry look like if it weren’t being cannibalised by capitalism? According to Mitch Altman, an inventor and founder of the Noisebridge hackerspace in the Mission, it would be much more exciting. “In the hacker scene all over the world, the vast majority are doing things because they think it’s awesome – they love it, they’re passionate about it,” says Altman, who is a figurehead of the hackerspace movement, a global network of free spaces where people of all abilities can work together on tech, art and engineering projects. “When people do things because they love it, they’re putting things into the world that wouldn’t be there. Not every techie is rich. Some of them are scraping by on a few hundred dollars a month, and some are squatting.”

Noisebridge has been open since 2007, and is tucked away above a taquería in the middle of the Mission, where gentrification is doing its worst cultural violence. Up a dingy flight of stairs is a wide, bright space, stuffed to the rafters with arcane computer components, matchstick sculptures, 3D printers and wired-looking young people typing frenetically on many-stickered laptops. It looks like a scene from a mid-Nineties movie about hackers.

“There’s a reason why Noisebridge started in the Bay Area,” says Altman, whose long, greying hair is dyed pink and blue. “Since the Gold Rush this has been a place which has attracted weirdos. That’s one of the reasons I love it here. I’m a total misfit and I love being surrounded by misfits. The people who live here mostly really want to live here, even though the people who are moving in with lots of money with Google jobs and things like that [also] want to be here.”

There are still a great many people in the Bay Area who believe that information should be free, even when a two-bedroomed flat in the Haight is eye-wateringly expensive. There are still hackers and engineers whose first priority is to take the lid off society and put it back together more efficiently, and those hackers, according to Altman, “need community. Even introverted geeks need community, and it’s hard to find community. It’s always going to be hard work.”

Community is the key. The biggest tech firms in the Bay Area have capitalised on the social value of community. From Google, Facebook and Twitter to Instagram and WhatsApp, the power of networks is being monetised even as the communities in which those businesses are based twist and change. What they are trying to work out now – what everyone in the Bay Area is trying to work out – is what it means to build community. “It isn’t as if there is a template or an accepted rule for how we’re all supposed to be members of a polis,” Kamiya says. “A city is a very peculiar thing.”

San Francisco is more peculiar than most. For better or worse, the class and cultural anxieties of this city are sending ripples around the globe. Tech is not what’s tearing San Francisco apart. America is what’s tearing San Francisco apart – America, and the vanishing chance it offers its citizens to build pleasant lives without brutalising others by proxy. The paradox at the heart of the hi-tech industry, the struggle between idealism and the inhumanities of capital, will leave its mark on the technologies future generations inherit.

---