Software developer at a big library, cyclist, photographer, hiker, reader. Email: chris@improbable.org
17555 stories
·
165 followers

In 1998, these SF men got a tattoo to snag free tacos for life. Here's what happened after.

1 Share

Martha Sanchez, who came up with the deal, arrived at that pricey estimate after the promotion was already underway. Assuming that the 50 or so inked-up people stayed in San Francisco for their lifetimes and got an eight-buck meal every week, the figure would be staggering. That number didn’t even include the second round of people when Casa Sanchez revived the deal; more on that later.

But for at least two of the first people in San Francisco to get inked up in exchange for free meals, the grub was never the point.

Rewind back to 1998: Greg Tietz, then 35, had only been in San Francisco for a couple of years and was looking to plant his roots in his neighborhood when a Mexican restaurant's promotion piqued his interest. 

Get a tattoo, get free Mexican food for life.

"I was just a happy-go-lucky bartender at Bottom of the Hill," Tietz, now 57 and a video logger for ESPN, told SFGATE. "And I was trying to explore areas of the Mission I hadn't been to, and found myself walking past Casa Sanchez."

By that point in time, Tietz had wanted to get his first tattoo but was wracked with indecision — a feeling that many first-time tattoo recipients can attest to.

"Like, boy this really needs to be special, really needs to mean something. I'm nervous about it."

Something about the Casa Sanchez tattoo just stuck for Tietz. The logo, a young boy riding a cob of corn-slash-rocket ship lovingly named “Jimmy the Cornman,” was great. The food was, too. (He only vaguely remembered his first Casa Sanchez meal, a "deluxe" carne asada burrito with all of the fixings.)

"I caught a co-worker friend of mine who had probably a dozen or more tattoos, saying, 'You should come check this place out with me and see what you think of the offer,' and helped me find a good tattoo artist." 

That friend was Guido Brenner, who now runs security at Bottom of the Hill. Brenner told SFGATE that he liked the art, and the food was quite good, even as a lifer in San Francisco who knew the Mexican food scene “like the back of [his] hand.”

That same night, Tietz and Brenner became the second and third people to get the Casa Sanchez logo permanently fixed onto their bodies. 

The actual first person, it turned out, was a local who got the Jimmy tattoo long before the promotion was even announced.

“She loved it so much that she came back with the logo tattooed and said, ‘Hey, what do you think?’ and [Martha] said, ‘Oh, you get free lunch for life,’” Brenner said.

Martha’s sister wasn’t on board with the tats-for-tacos deal, Brenner added, taking the sign off the window whenever Martha would put it up. But Martha reportedly persisted, reprinting the sign until her sister relented. (Martha Sanchez did not respond to a request for comment for the story.)

What happened next: They went viral the old-fashioned way. It made the rounds on the San Francisco alt-weeklies, then KPIX and the San Francisco Chronicle. 

That was Martha's doing, Tietz said. 

"She was savvy with media, trying to publicize the offer, and she was like, 'Here's the first guy to get the tattoo.'"

Brenner recalled at least three or four other people getting the same tattoo within the next 24 hours, compelled by the story and the flash sheet still laying around in the parlor.

Tietz still remembers that first spread with photos of his ink, published in the now-defunct San Francisco Bay Guardian.

And then when the local news cycle died out, the national one came months later: The Associated Press ran a wire on the promo. CNN featured them on their news shows for weeks. "All Things Considered" had them on the air. Tietz went on "Leeza," Leeza Gibbons' talk show, sitting on a panel next to Jennifer Flavin, Sylvester Stallone's wife, and actor D.B. Sweeney. 

"I was just like the guy that they kept teasing," Tietz said of his experience on the talk show, "like, 'This guy gets free food for life and wait till you hear what he had to do.'"

Then, it went global. A German travel show profiled him, Tietz said. He ended up, inexplicably, on a Chinese TV show, he said.

By no means was either of them a celebrity. Tietz only really recalled getting stopped out in public once: At a gas station on 9th Street. Brenner would occasionally get spotted while at work.

But the Casa Sanchez promo felt part and parcel of an idiosyncratic, almost quaint vision of San Francisco, that, only in this city would residents line up in droves to be marked for life with a cute small business’ logo in exchange for eight bucks worth of food.

Then, Casa Sanchez revived the promotion in 2010 as the recession dragged on, branded as a "stimulus special." And, like clockwork, the media cycle began once more.

KPIX did another segment with him on. The Wall Street Journal did a feature, showing off Tietz, Brenner and a new vanguard of people who joined the tattoo-meal pact.

For as long as he lived in Potrero Hill, just a hop-skip away from the Mission, Tietz enjoyed the meals, never forgetting to put a few bucks in the tip jar.

That was big for him. In a 1999 interview with the Chronicle, he said, "You don't want to take undue advantage of something like this. It's karma."

He still stands by that. 

"Generally, servers are good tippers, because we do like karma. I believe in that and it keeps the money coming back in so we can get by as service industry workers. ... I didn't want to go in there and like, you know, picking their pockets, in a sense and always showing up for free food.”

And, within a month, the food paid for itself. The publicity the two of them got for the years that Casa Sanchez was in operation was a nice bonus.

But the thing that stuck with them the most, back in '99 and now, was this idea that the “tacos-for-tattoos” promo did end up in a family of sorts.

Every time a news organization wanted a story, Sanchez would corral all of the tattooed folks at the restaurant and gather them together, and every time would be like a family reunion — with everyone keeping up with each other’s kids and work and relationships.

"It was a little community, of those of us that got the tattoo and also, between us and the Sanchez family,” Tietz said.

The way Tietz explains it, he’s lived in San Francisco for nearly as long as he’s had the Casa Sanchez tattoo. He’s still pals with Brenner, both tied by the bond that the tattoo provided the two of them, and by the Bottom of the Hill “family” they both were a part of. 

And for Brenner, he’s still in touch with Martha Sanchez, who he fondly calls Marty. 

“I have gone on trips [with] her husband. … I’ve gone out to Cuba with him before to interview Harley riders out in Cuba,” he said.

He was also in a band with another tattooed person and her husband.

We'll never find out whether the deal cost millions; the restaurant shuttered in 2012 in a deal with another local restaurant — two years after it revived the tattoo promotion — so that the family could reorient its efforts on its thriving grocery-store snack line.

And, for those disappointed in the meals running out, no fear.

The restaurant that ultimately took over Casa Sanchez’s old lease, the pupusa shop D’Maize, has it in their lease agreement that anyone with the OG tattoo is welcome to a free pupusa. (Tietz has stopped by once or twice, and Brenner’s been three times, at least, but the familial aspect isn’t there, they said.)

“I certainly respect their business decision, they’re still like family to me,” Tietz said. “I don't, I don't have any, you know, problem with them stopping the free food. But what I do miss is that the food was really good, and you know I always got to see family in there and they always made us feel extremely welcome.”

The colors have faded a bit. The cob of corn has gotten a bit stale, faded to a late-day yellow as opposed to the day-glo of the original Casa Sanchez logo he got tatted on him back when he'd get a burrito once a week.

But for all intents and purposes, the tattoo the two of them got meant way more than the food or even the notoriety. The tattoo, the story behind it and the friendships he cultivated — all thanks to Casa Sanchez — will all last in perpetuity. 

Read the whole story
Share this story
Delete

Peter Thiel and Silicon Valley’s Pursuit of Power: Excerpt

1 Comment

Sometime around the spring of 1988, several members of the Stanford University chess team traveled to a tournament in Monterey, California, in an old Volkswagen Rabbit. To get across the Santa Cruz Mountains, they took California’s Route 17, a four-lane highway that is regarded as one of the state’s most dangerous because of its tight curves, bad weather, and wild-animal crossings. The chess team had no particular reason to hurry, but the 20-year-old driver of the Rabbit weaved in and out of lanes, nearly rear-ending cars as he slipped past them. For large portions of the trip, he seemed to be flooring the accelerator.

Peter Thiel was at the wheel. Thin, dyspeptic, and humorless, he had seemed like an alien to his classmates since arriving at Stanford two and a half years earlier. He didn’t drink, didn’t date, didn’t crack jokes, and he seemed to possess both an insatiable ambition and a sense, deeply held, that the world was against him. He was brilliant and terrifying. He was, recalled one classmate, Megan Maxwell, “a strange, strange boy.”

Thiel looked up when, predictably, the lights of a police cruiser appeared in his rearview mirror. He pulled the Rabbit over, rolled down the window, and listened as a state trooper asked if he knew how fast he was going. The other young men in the car — relieved to have been stopped but also afraid of how this might play out — looked at each other nervously.

Thiel addressed the statie coolly in his usual uninflected baritone. “Well,” he said, “I’m not sure if the concept of a speed limit makes sense. It may be unconstitutional. And it’s definitely an infringement on liberty.”

Unbelievably, the trooper seemed to accept this. He told Thiel to slow down and have a nice day. Even more unbelievably: As soon as he drove out of sight, Thiel hit the gas pedal again, just as hard as before. To his astonished passengers, it was as if he believed that not only did the laws of California not apply to him — but that the laws of physics didn’t either. “I don’t remember any of the games we played,” said the teammate who was riding shotgun, a man who is now in his 50s. “But I will never forget that drive.”

Anyone who has followed Thiel’s career will find much to recognize in the Route 17 encounter. The reflexive contrarianism, the unearned confidence, the impossibly favorable outcome — they feel familiar, both in Thiel himself and the companies he helped create. Today, of course, that scrawny chess nerd is the billionaire co-founder of PayPal and Palantir and arguably the greatest venture capitalist of his generation, with a sideline as patron of such far-right causes as the 2016 candidacy of Donald Trump. Thiel (who did not comment for this article, which is adapted from my new biography, The Contrarian) is perhaps the most important influence in the world’s most influential industry. Other Silicon Valley personas may be better known to the general public, including Jeff Bezos, Elon Musk, and even a few who don’t regularly launch rockets into space. But Thiel is the Valley’s true idol — the single person whom tech’s young aspirants and millennial moguls most seek to flatter and to emulate, the cult leader of the cult of disruption.

The blitzscaling strategy he and his employees pioneered at PayPal created the growth playbook for an entire generation of start-ups, from Airbnb to WeWork. His most legendary bet — loaning $500,000 to a socially inept Harvard sophomore in exchange for 10 percent of a website called <a href="http://TheFacebook.com" rel="nofollow">TheFacebook.com</a> — is significant less for the orders-of-magnitude economic return he realized and more for the terms he embedded in the deal. Thiel ensured that Mark Zuckerberg would be the company’s absolute dictator. No one, not even Facebook’s board of directors, could ever overrule him. Similar maneuvers were adopted at many of Thiel’s portfolio companies, including Stripe and SpaceX, and today, across the industry, it’s more the norm than the exception.

Thiel hasn’t just acted in a certain way and left it for others to notice and follow. He taught his methods to founders-in-training at Stanford, codifying the lessons from the fleet of companies founded by his former employees — the so-called PayPal Mafia. He later collected his thinking in a book, Zero to One. It became a best seller, partly because it promised a path to Thiel-scale wealth and partly because it developed the idiosyncrasies that had been present in the college-age Thiel into a full-blown ideology. The book argues, among other things, that founders are godlike, that monarchies are more efficient than democracies, and that cults are a better organizational model than management consultancies. More than anything, it celebrates rule-breaking. Thiel bragged that of PayPal’s six founders, four had built bombs in high school.

The ideas were out there, but they were, undeniably, different. For decades, Silicon Valley had been dominated by the mythology of Steve Jobs. The acid-dropping hippie CEO had argued that technology could be a form of creative expression, and he convinced a generation of entrepreneurs that they should invent products that would improve the lives of their customers. This was the “‘bicycle for the mind’ value system,” says Roger McNamee, the founder of the venture-capital firm Elevation Partners, and it filtered into many of the most successful companies of the ’90s and ’00s.

Thiel despises the counterculture (he dates the precise beginning of American decline to Woodstock) and is contemptuous of the notion of creativity for its own sake. For Thiel, the purpose of founding a company is to control your own destiny. “A startup is the largest endeavor over which you can have definite mastery,” he wrote. A new generation of entrepreneurs, coming of age in the wake of the financial crisis, embraced his ideas. Thiel told them to flout norms and seek lucre, not impact. “Only one thing can allow a business to transcend the daily brute struggle for survival,” he wrote. “Monopoly profits.” Perhaps his single best student has been Mark Zuckerberg, who built a monopoly in his industry and used it to crush competitors and charge progressively higher fees to advertisers — all while telling the world that this essentially predatory behavior was a social good.

With Thiel’s encouragement, tech would “move fast and break things,” as the Facebook motto put it, and executives believed it was better to ask forgiveness than permission. The industry that developed would be defined by these clichés, convincing itself that “disruption” wasn’t just an unfortunate consequence of innovation but an end in itself. Thielism would show up even at companies where he was not an investor: at Juul, the e-cigarette company that marketed to children; at Robinhood, which tempted novice investors with volatile investment products; and at Uber, which paid drivers less than minimum wage and violated statutes with apparent glee.

McNamee, an early adviser to Zuckerberg who turned apostate and published Zucked: Waking Up to the Facebook Catastrophe, sees this recent history as an expression of Thiel’s values. As he put it to me, “The PayPal Mafia philosophy became the founding principle for an entire generation of tech companies.”

Peter Thiel was not a popular boy. In the middle-class San Francisco suburb of Foster City, his high-school classmates were awed by his intelligence but found him inscrutable and haughty — qualities that made him a target for abuse. “It’s obvious in retrospect that what we were doing was bullying,” one of his regular tormentors told me. “I’ve always thought he might have a list of people he’s going to kill somewhere and that I’m on it.” Thiel got more assured as he matured physically, although he was not confident so much as disdainful, walking around with an expression that said, according to a friend, “Fuck you, world.”

Thiel never actually swore, but once, during his freshman year at Stanford, he quoted one of his roommates doing so during an argument. The roommate responded by printing a commemorative sign and taping it to the ceiling. It had the date, January 1986, and declared, “Under this spot, Peter Thiel first said the word fuck.” It stayed there for the rest of the semester, eliciting laughs from the rest of the hall — except for Thiel, who didn’t notice and wasn’t told. In May, he was all packed up and preparing to leave the dorm for the final time when someone pointed to the sign. Wordless, Thiel moved his desk under the paper, stepped up, tore it down, and left for the summer. “God,” a college acquaintance told me. “We were such dicks to him.”

The mockery wasn’t about politics, at least not initially, but that was how Thiel processed it. He had been raised by Evangelical German immigrants and fancied himself an aspiring William F. Buckley. It wasn’t unusual to be conservative at Stanford — it housed the Hoover Institution — but Thiel considered it a hothouse of lefty antagonists. “He viewed liberals through a lens as people who were not nice to him,” said a classmate. “The way people treated him at Stanford had a huge impact. That’s still with him.” Thiel began to embrace a new identity — that of the right-wing provocateur. He joked about starting a fake charity, Liberals for Peace, that would raise money based on a vague agenda and then do absolutely nothing except pay him. And he told classmates that concern about South African apartheid, perhaps the single buzziest issue on American campuses, was overblown. “It works,” he told Maxwell. (Thiel’s spokesman has said that Thiel doesn’t remember being asked his views on apartheid and never supported it.)

In 1987, Thiel poured his sense of grievance into the launch of a right-wing newspaper, the Stanford Review. It was his first entrepreneurial venture and the beginning of a network that would eventually expand and dominate Silicon Valley. Thiel’s primary innovation with the Review was to connect the parochial concerns of a small elite — conservative Stanford undergraduates— to mainstream national politics. Thus the optional $29 per year dues charged by the student senate became a microcosm of tax-and-spend liberalism and a plan to add non-white authors, like Zora Neale Hurston, to Stanford’s Western Culture course became a civilization-level threat. A fundraising letter later sent to older alumni warned that a professor was teaching a course on Black hairstyles. It led to a flood of donations. These sorts of antics helped draw the attention of Ronald Reagan’s secretary of Education, who came to speak at a Review event and made national news recapping it on PBS’s MacNeil/Lehrer NewsHour.

Thiel’s newspaper was also fixated on sex. The first issue featured a satirical column, “Confessions of a Sexual Deviant,” about a young straight man who’d chosen to be celibate. According to the Review, it was almost impossible to visit a men’s restroom without witnessing a gay sex act or to cross the quad without having fistfuls of free condoms pressed into your hand. In 1987, presenting homosexuality as an addiction, a columnist wrote that “unnatural” gay men had “yielded to temptation so many times that the fires of lust burn within them, making it indeed difficult for them to control themselves.” During Thiel’s last year on campus, his close friend and Review collaborator Keith Rabois stood outside the home of a Stanford residential fellow and shouted at the top of his lungs, “Faggot! You are going to die of AIDS! You’re going to get what’s coming to you!”; two days later, the Review published “The Rape Issue,” with an impassioned defense of a student who’d pleaded “no contest” to statutory rape.

Thiel would go on to valorize Rabois as a free-speech martyr in a book, The Diversity Myth, co-written with the architect of the special issue, David Sacks. It’s tempting to psychologize the book, with its lurid complaints about the supposed prevalence of “glory holes” across the Stanford campus. And some who know Thiel speculate, convincingly, that his mid-’90s homophobia was an expression of self-hatred. (Thiel is gay, as is Rabois.) But the book’s incendiary qualities might just have easily been a product of Thiel’s single-minded desire to provoke a reaction. He wanted to make his mark, and he surely knew that the prospect of recent graduates defending the guy who had shouted “Die, faggot!” on the quad of an elite university would get noticed.

But Thiel’s years after Stanford — he got both an undergraduate and a law degree there — were a misfire. A job at a white-shoe firm failed to translate into a Supreme Court clerkship, as he’d expected, and a stint as a derivatives trader at Credit Suisse in Manhattan didn’t pan out. He made plans to move home, to a place brimming with overachieving young men convinced of their own genius and relentless in their determination to get rich: Silicon Valley. “He wasn’t humbled by the experience in New York,” said a friend who was close to him at the time. “He was ready to take things over.”

It started on a sweltering summer day in 1998. Thiel was in a classroom at Stanford’s engineering center, attempting to chat up an awkward but brilliant coder. Max Levchin was 23, and he’d come to a lecture Thiel was delivering on currencies primarily for the air conditioning. The two young men got to talking, and within a day, Thiel, who had been managing a pool of capital he’d raised from friends and family as a hedge fund, told Levchin that he wanted to invest in his embryonic start-up. It made software for Palm Pilots. At the end of that year, they began experimenting with a way for owners of the devices to transmit IOUs to one another. They called the service PayPal, and Thiel, who took over the venture before the year was out, quickly saw its subversive possibilities.

Once you got money via PayPal, you could transfer your balance to a bank. Or you could keep the funds inside PayPal and use them to pay other people. This, Thiel realized, made the service a kind of untrackable digital currency. It was the equivalent of a Swiss bank account in one’s pocket, he believed, boasting to a Wired reporter that it could lead to “the erosion of the nation-state.” Thiel staffed the company with former Stanford Review editors and imposed his libertarian ideas in ways large and small. PayPal employees were free to show up late to meetings as long as they paid $1 for every minute they were tardy, and Ayn Rand was something like required reading.

The company leased its first office above a stationery store and a French bakery in downtown Palo Alto. At the time, the Valley was so full of competing payments companies that there was a second one on the same floor. <a href="http://X.com" rel="nofollow">X.com</a> was better funded than PayPal, with a famous investor — Michael Moritz of Sequoia Capital — and a charismatic founder who’d already sold another start-up for some $300 million. His name was Elon Musk.

Musk didn’t know that the engineers across the landing were also working on digital money transfers. (The sign on their door bore the name of a parent company.) X and PayPal shared a trash bin in the alley behind their building, and PayPal engineers later bragged to a group of X employees that they found documents that described X’s payments scheme, which used the web, rather than Palm Pilots, as well as a system for generating referrals by giving customers cash. They incorporated the ideas into PayPal’s strategy. Some X employees I spoke with took this boast literally, though Musk cast doubt on the story. “It’s possible, I suppose,” he told me. “But it’s a bit like saying, ‘You stole my idea for going to the moon.’” Even though he has long since moved on to running Tesla and SpaceX, Musk has complicated feelings about Thiel, in part because of what happened next.

Shifting PayPal’s focus to the web and paying new users referral fees juiced the company’s growth. Some of Thiel’s coders made a little software app to track how many people had created new accounts, which appeared on his screen as a little box titled “World Domination Index.” Every time a new user joined, the app played the sound of a bell. In November 1999, PayPal’s customer base was a few thousand. By spring, the index was up to 1 million. That was a nearly unprecedented rate of growth, but it meant that PayPal had spent something like $20 million on referral fees out of the $28 million it had raised. The losses, and the similarity of their businesses, persuaded Thiel and Musk to combine their companies.

Thiel left shortly after the merger. “I wouldn’t say we’re oil and water, but there are some pretty big differences,” Musk, who became CEO, told me. “Peter likes the games­manship of investing — like we’re all playing chess. I don’t mind that, but I’m fundamentally into doing engineering and design. I’m not an inves­tor. I feel like using other people’s money is not cool.” A person who has talked to each man about the other put it more succinctly: “Musk thinks Peter is a sociopath, and Peter thinks Musk is a fraud and a braggart.”

It seemed as if Musk won the power struggle, but Thiel had laid a trap, installing most of his deputies — including Levchin and the author of the Review’s “Rape Issue” — in the executive ranks. Musk didn’t realize he was surrounded by a team that was more loyal to Thiel than to him. Later that year, Musk left town for a two-week trip. While he was in the air, a group of Thiel-aligned conspirators confronted the company’s major backer, Moritz, at his office on Sand Hill Road. They demanded their patron be put in charge.

After Moritz reluctantly agreed, Thiel pressed his advantage. At a board meeting, according to several people familiar with what happened, he suggested that PayPal turn over all its cash to Thiel Capital, the hedge fund he was still running on the side, so that he could take advan­tage of the economic upheaval of the post‒dot-com bubble. Moritz assumed Thiel was joking, but Thiel calmly explained that he had a plan to bet on interest rates falling. Thiel’s idea was shot down, but Moritz was furious. Risking a start-up’s limited cash on speculation — particularly speculation that had the potential to personally enrich the CEO — was something no venture capitalist, nor any self-respecting tech entrepreneur, would even consider suggesting. The fact that Thiel would propose it not long after snagging the CEO job in a manner that was not exactly honorable was doubly galling. It sug­gested to Moritz and others on the board a lack of a moral compass.

Thiel and Moritz continued to clash. It may have been partly personal — Moritz had originally invested in Musk’s company, not Thiel’s. But it also reflected the ways in which Thiel was different from Moritz, Musk, and pretty much every important figure in Silicon Valley who’d come before him. “At heart,” Moritz told me, “Peter is a hedge-fund man” — not an entrepreneur. Founders were expected to pour all of themselves into their companies in order to grow as big as possible and, at least if you buy into the mythology of Silicon Valley, to change the world for the better. By this logic, Thiel should’ve been bleeding for PayPal, not scheming to grow his investment portfolio. But Thiel didn’t care about Silicon Valley’s sense of propriety. And it opened up a universe of strategies that his predecessors had never been brazen enough to try.

Under Thiel, PayPal’s willingness to disregard banking rules became a key strategic advantage. Fi­nancial institutions are required to verify that customers are who they say they are by checking identifications, but PayPal, which contended it wasn’t technically a bank, made little effort to do so. (When a reporter noted to Thiel that many of his competitors complied with the regulations, he called them “insane.”) It also did little to stop people from using the money they put into their accounts for illicit purposes. The refund mechanism that PayPal used to return customers’ cash was technically banned by the credit-card companies. When those companies complained, PayPal simply offered an apology and negotiated. Today, the use of unsustainable or ethically dubious tricks to make a start-up insurmountably bigger than its rivals is known as “growth hacking.” It’s widely credited to Thiel and his executives and celebrated by entrepreneurs across the industry. They’re all chasing what Thiel got. The year after he became CEO of PayPal, eBay acquired the company for $1.5 billion.

Twenty years later, Thielism is the dominant ethos in Silicon Valley. That’s partly because Thiel has been effective at seeding the industry with protégés — none of them more prominent than Mark Zuckerberg. Having pursued a grow-at-all-costs, consequences-be-damned expansion strategy, the Facebook CEO is now attempting to create his own coin. Diem (née Libra) is a cryptocurrency that, if all goes according to plan, will function as a sort of replacement for the U.S. dollar inside Facebook (roughly 3 billion users), WhatsApp (2 billion), Instagram (1 billion), and its other apps, as well as those of any other companies that adopt it. The effort has generated considerable concern from the U.S. Federal Reserve, which is understandably committed to the dollar, and from critics across the political spectrum, who express shock at the audacity of a company with Facebook’s track record for privacy violations attempting to put itself at the center of global commerce.

Longtime Thiel associates, though, weren’t surprised at all. “No one seems to have connected the dots to Peter’s original grand vision for PayPal,” a source who’d worked with Thiel for years wrote to me in June 2019, shortly after the currency was announced. This person, and others I spoke with, saw Thiel’s ideology — and his apparent belief that corporate power should subsume the authority of governments — in Facebook’s actions. “There’s a direct influence,” this source said.

Thiel doesn’t control Zuckerberg, and their relationship is complicated, to say the least. Thiel unloaded most of his stock as soon as the company went public. (The shares were falling and morale was low. To rally the staff, Zuckerberg invited Thiel to an all-hands meeting at headquarters, but he just ended up insulting them. “My generation was promised colonies on the moon,” Thiel told them. “Instead we got Facebook.”) But in the years since, Thiel has remained a trusted confidant to Zuckerberg — despite personally cultivating Facebook antagonists, including James O’Keefe, the right-wing provocateur who produced undercover videos attempting to expose Facebook’s supposed bias against conservatives, and Charles Johnson, who helped start the face-recognition company Clearview AI.

Clearview assembled its gargantuan database of faces by scraping photos from Facebook profiles, which Facebook considers a violation of its terms of service. Johnson told me that when he raised money from Thiel, he presented Clearview as both a promising business and as a backdoor way to “destroy” Facebook by exposing its lax privacy standards. Thiel, who as a Facebook board member has a duty to act in the company’s best interests, invested in Clearview anyway. Johnson also says that Thiel used him as a conduit to leak emails between Thiel and Reed Hastings, another Facebook board member, who’d criticized Thiel for backing Donald Trump.

At any normal company, such disloyalty might be grounds for dismissal. But it was Hastings, not Thiel, who resigned from the board, and Zuckerberg never punished his mentor. According to two former Facebook staffers, this was partly because he appreciated Thiel’s unvarnished advice and partly because Zuckerberg saw Thiel as a political ally. Zuckerberg had been criticized by conservative media before the 2016 election and, with Thiel’s encouragement, had sought to cater to them.

In 2019, while on a trip to Washington to answer questions from Congress about his digital currency, Thiel joined Zuckerberg, Jared Kushner, Trump, and their spouses at the White House. The specifics of the discussion were secret — but, as I report in my book, Thiel later told a confidant that Zuckerberg came to an understanding with Kushner during the meal. Facebook, he promised, would continue to avoid fact-checking po­litical speech — thus allowing the Trump campaign to claim whatever it wanted. If the company followed through on that promise, the Trump administra­tion would lay off on any heavy-handed regulations.

After the dinner, Zuckerberg took a hands-off approach to conservative sites. In late October, after he detailed the policy in a speech at Georgetown, Facebook launched a news app that showcased what the company called “deeply reported and well-sourced” outlets. Among the list of recommended publications was Breitbart, Steve Bannon’s site, even though it had promoted itself as allied with the alt-right and had once included a section dedicated to “Black crime.” Facebook also seemed to go out of its way to help the Daily Wire, a younger, hipper version of Breitbart that would become one of the biggest publishers on the platform. Facebook had long seen itself as a government unto itself; now, thanks to the understanding brokered by Thiel, the site would push what the Thiel confidant called “state-sanctioned conservatism.”

Zuckerberg denied that there had been any deal with Trump, calling the notion “pretty ridiculous,” though Facebook’s actions in the run-up to the election would make the denial seem not entirely credible. During Black Lives Matter protests, Twitter hid a post by the president that seemed to condone violence: “When the looting starts, the shooting starts”; Face­book allowed it. In the days leading up to the January 6 insurrection at the U.S. Capitol, Facebook mostly ignored calls to limit the spread of “Stop the Steal” groups, which claimed that Trump had actually won the election.

In the months since, journalists, policy-makers, and even some Facebook employees have struggled to explain why the company remains indifferent to the objections of regulators and lawmakers as well as those raised by common sense. Why is Facebook — and so much of what comes out of what once seemed like the crown jewel of American capitalism — such an obviously malevolent force?

The answers to these questions are partly structural, of course, involving regulatory failures that allowed Zuckerberg to dominate social-media advertising. But they are also ideological. Both figuratively and literally, Thiel wrote the book on monopoly capitalism, and he recruited an army of followers, including Zuckerberg. This is to say that the Facebook founder, like almost every successful techie of his generation, isn’t a liberal or a conservative. He is a Thielist. The rules do not apply.

This article has been updated to include more context on Thiel’s 2019 dinner with Zuckerberg and Kushner.

Excerpted from The Contrarian: Peter Thiel and Silicon Valley’s Pursuit of Power, by Max Chafkin. Published by arrangement with Penguin Press, a member of Penguin Random House, LLC. Copyright © 2021 by Max Chafkin.

Read the whole story
Share this story
Delete
1 public comment
acdha
1 hour ago
reply
“In 2019, while on a trip to Washington to answer questions from Congress about his digital currency, Thiel joined Zuckerberg, Jared Kushner, Trump, and their spouses at the White House. The specifics of the discussion were secret — but, as I report in my book, Thiel later told a confidant that Zuckerberg came to an understanding with Kushner during the meal. Facebook, he promised, would continue to avoid fact-checking po­litical speech — thus allowing the Trump campaign to claim whatever it wanted. If the company followed through on that promise, the Trump administra­tion would lay off on any heavy-handed regulations.”
Washington, DC

Better JIT Support for Auto-Generated Python Code

1 Share

Performance Cliffs

A common bad property of many different JIT compilers is that of a "performance cliff": A seemingly reasonable code change, leading to massively reduced performance due to hitting some weird property of the JIT compiler that's not easy to understand for the programmer (e.g. here's a blog post about the fix of a performance cliff when running React on V8). Hitting a performance cliff as a programmer can be intensely frustrating and turn people off from using PyPy altogether. Recently we've been working on trying to remove some of PyPy's performance cliffs, and this post describes one such effort.

The problem showed up in an issue where somebody found the performance of their website using Tornado a lot worse than what various benchmarks suggested. It took some careful digging to figure out what caused the problem: The slow performance was caused by the huge functions that the Tornado templating engine creates. These functions lead the JIT to behave in unproductive ways. In this blog post I'll describe why the problem occurs and how we fixed it.

Problem

After quite a bit of debugging we narrowed down the problem to the following reproducer: If you render a big HTML template (example) using the Tornado templating engine, the template rendering is really not any faster than CPython. A small template doesn't show this behavior, and other parts of Tornado seem to perform well. So we looked into how the templating engine works, and it turns out that the templates are compiled into Python functions. This means that a big template can turn into a really enormous Python function (Python version of the example). For some reason really enormous Python functions aren't handled particularly well by the JIT, and in the next section I'll explain some the background that's necessary to understand why this happens.

Trace Limits and Inlining

To understand why the problem occurs, it's necessary to understand how PyPy's trace limit and inlining works. The tracing JIT has a maximum trace length built in, the reason for that is some limitation in the compact encoding of traces in the JIT. Another reason is that we don't want to generate arbitrary large chunks of machine code. Usually, when we hit the trace limit, it is due to inlining. While tracing, the JIT will inline many of the functions called from the outermost one. This is usually good and improves performance greatly, however, inlining can also lead to the trace being too long. If that happens, we will mark a called function as uninlinable. The next time we trace the outer function we won't inline it, leading to a shorter trace, which hopefully fits the trace limit.

Diagram illustrating the interaction of the trace limit and inlining

In the diagram above we trace a function f, which calls a function g, which is inlined into the trace. The trace ends up being too long, so the JIT disables inlining of g. The next time we try to trace f the trace will contain a call to g instead of inlining it. The trace ends up being not too long, so we can turn it into machine code when tracing finishes.

Now we know enough to understand what the problem with automatically generated code is: sometimes, the outermost function itself doesn't fit the trace limit, without any inlining going on at all. This is usually not the case for normal, hand-written Python functions. However, it can happen for automatically generated Python code, such as the code that the Tornado templating engine produces.

So, what happens when the JIT hits such a huge function? The function is traced until the trace is too long. Then the trace limits stops further tracing. Since nothing was inlined, we cannot make the trace shorter the next time by disabling inlining. Therefore, this happens again and again, the next time we trace the function we run into exactly the same problem. The net effect is that the function is even slowed down: we spend time tracing it, then stop tracing and throw the trace away. Therefore, that effort is never useful, so the resulting execution can be slower than not using the JIT at all!

Solution

To get out of the endless cycle of useless retracing we first had the idea of simply disabling all code generation for such huge functions, that produce too long traces even if there is no inlining at all. However, that lead to disappointing performance in the example Tornado program, because important parts of the code remain always interpreted.

Instead, our solution is now as follows: After we have hit the trace limit and no inlining has happened so far, we mark the outermost function as a source of huge traces. The next time we trace such a function, we do so in a special mode. In that mode, hitting the trace limit behaves differently: Instead of stopping the tracer and throwing away the trace produced so far, we will use the unfinished trace to produce machine code. This trace corresponds to the first part of the function, but stops at a basically arbitrary point in the middle of the function.

The question is what should happen when execution reaches the end of this unfinished trace. We want to be able to cover more of the function with machine code and therefore need to extend the trace from that point on. But we don't want to do that too eagerly to prevent lots and lots of machine code being generated. To achieve this behaviour we add a guard to the end of the unfinished trace, which will always fail. This has the right behaviour: a failing guard will transfer control to the interpreter, but if it fails often enough, we can patch it to jump to more machine code, that starts from this position. In that way, we can slowly explore the full gigantic function and add all those parts of the control flow graph that are actually commonly executed at runtime.

Diagram showing what happens in the new jit when tracing a huge function

In the diagram we are trying to trace a huge function f, which leads to hitting the trace limit. However, nothing was inlined into the trace, so disabling inlining won't ensure a successful trace attempt the next time. Instead, we mark f as "huge". This has the effect that when we trace it again and are about to hit the trace limit, we end the trace at an arbitrary point by inserting a guard that always fails.

Diagram showing what happens in the new jit when tracing a huge function until completion

If this guard failure is executed often enough, we might patch the guard and add a jump to a further part of the function f. This can continue potentially several times, until the trace really hits and end points (for example by closing the loop and jumping back to trace 1, or by returning from f).

Evaluation

Since this is a performance cliff that we didn't observe in any of our benchmarks ourselves, it's pointless to look at the effect that this improvement has on existing benchmarks – there shouldn't and indeed there isn't any.

Instead, we are going to look at a micro-benchmark that came out of the original bug report, one that simply renders a big artificial Tornado template 200 times. The code of the micro-benchmark can be found here.

All benchmarks were run 10 times in new processes. The means and standard deviations of the benchmark runs are:

Implementation Time taken (lower is better)
CPython 3.9.5 14.19 ± 0.35s
PyPy3 without JIT 59.48 ± 5.41s
PyPy3 JIT old 14.47 ± 0.35s
PyPy3 JIT new 4.89 ± 0.10s

What we can see is that while the old JIT is very helpful for this micro-benchmark, it only brings the performance up to CPython levels, not providing any extra benefit. The new JIT gives an almost 3x speedup.

Another interesting number we can look at is how often the JIT started a trace, and for how many traces we produced actual machine code:

Implementation Traces Started Traces sent to backend Time spent in JIT
PyPy3 JIT old 216 24 0.65s
PyPy3 JIT new 30 25 0.06s

Here we can clearly see the problem: The old JIT would try tracing the auto-generated templating code again and again, but would never actually produce any machine code, wasting lots of time in the process. The new JIT still traces a few times uselessly, but then eventually converges and stops emitting machine code for all the paths through the auto-generated Python code.

Related Work

Tim Felgentreff pointed me to the fact that Truffle also has a mechanism to slice huge methods into smaller compilation units (and I am sure other JITs have such mechanisms as well).

Conclusion

In this post we've described a performance cliff in PyPy's JIT, that of really big auto-generated functions which hit the trace limit without inlining, that we still want to generate machine code for. We achieve this by chunking up the trace into several smaller traces, which we compile piece by piece. This is not a super common thing to be happening – otherwise we would have run into and fixed it earlier – but it's still good to have a fix now.

The work described in this post tiny bit experimental still, but we will release it as part of the upcoming 3.8 beta release, to get some more experience with it. Please grab a 3.8 release candidate, try it out and let us know your observations, good and bad!

Read the whole story
Share this story
Delete

Ethiopia: From Nobel laureate to global pariah, how the world got Abiy Ahmed so wrong - CNN

1 Share
Read the whole story
Share this story
Delete

Child-care workers are quitting rapidly, a red flag for the economy - The Washington Post

1 Comment
Read the whole story
Share this story
Delete
1 public comment
acdha
1 day ago
reply
Imagine if we spent money like we valued families?
Washington, DC

Young climate activists channel anxiety about floods and wildfires - The Washington Post

2 Shares
Read the whole story
Share this story
Delete
Next Page of Stories