Tech and the Fake Market tactic

In one generation, the Internet went from opening up new free markets to creating a series of Fake Markets that exploit society, without most media or politicians even noticing.

1. The open internet markets

American culture loves to use the ideal of competitive free markets as the solution to all kinds of social problems. Though the vaunted Free Market has no incentives to, say, take care of babies with cancer, a well-functioning market can definitely be a great way to see which provider offers the cheapest price for a roll of toilet paper or a bushel of apples.

Given that cultural predilection, some of the first things people made in the early days of the web were new markets. Perhaps the canonical example was eBay; anybody (well, almost anybody) could list their ceramic figurines for sale on eBay and participate in a relatively fair market. On one side, a gaggle of figurine aficionados, enthusiastically searching for the best deals. On the other, a bunch of figurine vendors, competing on price, quality and service. In the middle, a neutral market that just helps connect buyers and sellers through instantly updated information. Everybody’s happy!

Fake Markets: Open

Later, a seller could buy preferred positioning for their products in eBay’s search results, and some product categories started to be dominated by wholesale suppliers, but it still remained a relatively open system. Everybody’s mostly happy!

Not long after eBay started, Google launched, as a sort of market of content, with its PageRank system choosing which pages show up in our search results, ranked by the number of inbound links. On one side were readers, and on the other side we had publishers, and in between was Google using a mysterious but still kind of comprehensible algorithm to create a market where almost everybody felt like they could participate.

But before long, those rankings started to be tainted by spammers, due to the fact that higher ranking in those listings suddenly had monetary value, and making spam links was cheaper than paying for Google’s advertising products. What was an open market to do?

2. The rise of rigged markets

The inevitable automated gaming of the early open digital markets inadvertently catalyzed the start of the next era: rigged markets. Google got concerned about nefarious search engine optimization tricks, and kept changing their algorithm, meaning that pretty soon the only web publishers that could thrive were those who could afford to keep tweaking their technology to keep up in this new arms race. After just a few years, this became a rich-get-richer economy, and incentivized every smaller publisher to standardize on one of a few publishing tools in order to keep up with Google’s demands. Only the biggest content providers could afford to build their own tools while simultaneously following the demands of Google’s ever-changing algorithm.

The problem inevitably became most pronounced in the most valuable markets. Eventually, in lucrative vertical markets like travel, Google started showing its own flight booking tools ahead of the third-party results from travel booking sites, based on the idea that their experience was better for consumers than the confusing and inconsistent results from third parties. This was true, but it was also pretty damn convenient for Google, which now started to make more money on those links.

Fake Markets: Skewed

This was the start of a subtle but critically important pattern on the web: A short-term improvement in user experience helped a single dominant tech company to take over a legacy market in the long term.

Amazon went through a similar process, when it started putting its thumb on the scale, showing its own products first when doing a product search, even if they weren’t the cheapest. We saw a rapid shift where the companies hosting formerly-open markets started to give themselves unfair advantages that couldn’t be countered by the other sellers in the market.

We saw a rapid shift where the companies hosting formerly-open markets started to give themselves unfair advantages that couldn’t be countered by the other sellers in the market.

This shift to rigged markets was perfected in the app stores, where the major players like Apple and Google choose which apps get featured and promoted, and prevent the creation of any apps that would displace or threaten their market dominance. Even if an app does succeed, the app stores promote an ad-supported model that makes app creators dependent on the tech company’s platform for distribution, instead of an app deriving revenues directly from its users.

But even today’s rigged markets have some ways that new players can compete. You could release a new photo-sharing app and theoretically try to compete with Instagram or Snapchat on Apple’s app store. An ordinary shopper could search for “bedsheets” on Amazon’s website and expect to get a list of linens to purchase, both from independent manufacturers and from Amazon’s own Pinzon brand. Even if these markets are skewed, they’re still markets, and that leaves some opportunity.

That’s not to say these systems are fair: the big companies can pick which players in the market get to compete, and issues of network inequality mean people or companies that are privileged enough to be early adopters get unfair advantages. But even with these inequities, we could muddle through and new products or competitors could sometimes emerge.

This has been the status quo for most of the last decade. But the next rising wave of tech innovators twist the definition of “market” even further, to a point where they aren’t actually markets at all.

3. Now: The Fake Markets

Uber‘s promise is simple: you use their app to hail a car, and one driver from a pool of independent drivers agrees to pick you up, and everybody’s happy. In their formulation, they’re a neutral marketplace connecting customers and service providers — kinda like eBay!

But unlike competitive sellers on eBay, Uber drivers can’t set their prices. In fact, prices can be (and regularly have been) changed unilaterally by Uber. And passengers can’t make informed choices about selecting a driver: The algorithm by which a passenger and driver are matched is opaque—to both the passenger and driver. In fact, as Data & Society’s research has shown, Uber has at times deliberately misrepresented the market of available cars by showing “ghost” cars to users in the Uber app.

Fake Markets: Fake

It seems this “market” has some awfully weird traits.

  1. Consumers can’t trust the information they’re being provided to make a purchasing decision.
  2. A single opaque algorithm defines which buyers are matched with which sellers.
  3. Sellers have no control over their own pricing or profit margins.
  4. Regulators see the genuine short-term consumer benefit but don’t realize the long-term harms that can arise.

This is, by any reasonable definition, no market at all. One might even call Uber a “Fake Market”. Yet, by carefully describing drivers in their system as “entrepreneurs” and appropriating the language of true markets, Uber has been welcomed by communities and policymakers as if they were creating a new marketplace. That has serious implications for policy, regulation and even civil rights. For example, we can sincerely laud Uber for making it easier for African American passengers to reliably hail a car when they need a ride, but if persistent patterns of bias from drivers arise again in the Uber era, we’ll have a harder time regulating those abuses because Uber doesn’t usually follow the same policies as licensed taxis.

These pseudo-market patterns also mask patterns of subsidy, like the fact that Uber’s current operations are subsidized by investors to the tune of $2 billion per year. That’s a cost that will be immediately passed along to consumers as soon as Uber succeeds in displacing conventional taxis.

Uber subsidy

The Financial Times states the implications of this economic arrangement very clearly:

[A]ll this equates to is an economic transfer from the working class over to urban metropolitan elites, which benefits one particular corporation over others. This is plainly crazy.

These new False Markets only resemble true markets just enough to pull the wool over the eyes of regulators and media, whose enthusiasm for high tech solutions is boundless, and whose understanding of markets on the Internet is still stuck in the early eBay era of 20 years ago.

Fake markets don’t just happen in traditional products and services — they’re coming to the world of content and publishing, too. Publishers are increasingly being incentivized to use platforms like Facebook’s Instant Articles and Google’s AMP format. Like Uber’s temporarily-subsidized cheaper prices and broader access to ride hailing, these new publishing formats do offer some short-term consumer benefits, in the form of faster loading times and a cleaner reading experience.

But the technical mechanism by which Facebook and Google provide that faster reading experience happens to incidentally displace most of the third-party advertising platforms — the ones that aren’t provided by Facebook and Google themselves. Facebook publishers who use these new distribution channels are incentivized to use Facebook’s advertising platform, where payment rates and profit margins can be unilaterally changed at any time. Just as Uber subsidizes fares during the phase when they’re displacing regulated taxis, Facebook subsidizes publishers’ ad rates during the phase when they’re displacing third-party advertising networks.

In addition to making publishers even more dependent on the two tech titans for revenues, there’s the issue of the algorithms used to discover content. Almost everyone who uses Facebook has become aware that its algorithm for showing content is opaque, to both publishers and readers. As a result, there are fewer understandable tricks that publishers can use to ensure that readers will see their content — and publishing in the Instant Articles format is one of the few that’s known to work. It also happens to require a publisher to invest scarce resources in supporting the Facebook format, with the result of that publisher becoming even more dependent on Facebook for distribution.

So: Neither readers nor publishers know why Facebook shows a particular story in a feed. And media regulators and policymakers can’t see past the short-term benefit of faster-loading stories.

The Fake Market for content looks like this:

  1. Readers can’t trust the information they’re being provided to make a content decision.
  2. A single opaque algorithm defines which readers are matched with which publishers.
  3. Publishers have no control over their own ad rates or profit margins.
  4. Regulators see the genuine short-term reader benefit but don’t realize the long-term harms that can arise.

4. After Markets: Self-Driving News

But wait, it gets worse. Next we replace the sellers in the market.

What we have in ride sharing or content publishing is a rapid move to locked-down systems controlled by one, or at most two, privately-held corporate players. But even in these fake markets, there are currently multiple providers offering their services within the ecosystem. The providers are those Uber drivers or Facebook publishers being lauded as independent entrepreneurs thriving on the platform.

But Uber has already plainly announced its roadmap: Self-driving cars. The much-lauded independent driver-entrepreneurs will be replaced by completely automated service providers as quickly as possible, and not only will those new self-driving cars not have drivers who need to be paid, they will all be owned by Uber itself. When this transition happens over the next decade, we’ll have entire markets of independent contractors displaced by the transition, precisely at the point when the social safety net is being dismantled. In the meantime, politicians across the political spectrum have been presenting these “gig economy” non-jobs as the future of work.

Fake Markets: Closed

Self-driving cars are hard, though. Making a robot that can navigate through a city and deliver a passenger safely and reliably to their destination is an incredibly hard problem that will take a long time to get exactly right.

By contrast, what are the barriers to self-driving news? We’ve already seen that a lot of news consumers aren’t interested in being safely and reliably delivered to accurate news. Success in this case will be much easier: A robotic publisher only has to deliver content that’s emotionally engaging enough to earn a person’s readership for a few moments. That’s even easier to do if the publisher or distributor of the content doesn’t care if the story is true or not. Peter Thiel is on Facebook’s board of directors.

And remember, Facebook tends to subsidize publishers taking advantage of its new platform features just until the point at which those publishers become dependent on them. Publishers are already struggling with overall media industry economics; Facebook’s promised payouts may be an offer they don’t feel like they can refuse.

So what do we do?

Most of the people building these features at these companies don’t mean to undermine markets. The coders and designers at companies like Uber and Facebook and all the others are usually well-intentioned and genuinely see their work as benefiting users. In the immediate term, they’re not even wrong; being able to easily hail a cab or quickly read a story is a real benefit. But most tech workers, including at the biggest tech companies, are blind to the radical political and social agendas of their companies’ owners and investors.

Worse, we’ve lost the ability to discern that a short-term benefit for some users that’s subsidized by an unsustainable investment model will lead to terrible long-term consequences for society. We’re hooked on the temporary infusion of venture capital dollars into vulnerable markets that we know are about to be remade by technological transformation and automation. The only social force empowered to anticipate or prevent these disruptions are policymakers who are often too illiterate to understand how these technologies work, and who too desperately want the halo of appearing to be associated with “high tech”, the secular religion of America.


It’s essential we develop a vocabulary for talking about these issues, and perhaps the single most effective action we can take is to educate our elected officials about the changes that are happening. This stuff is complex, and it’s going to take time to teach all our representatives about why all the changes wrought by these new high-tech apps aren’t necessarily the best thing for our communities in the long term.

But there’s still time to get it right. It’s not inevitable that we have to give over our open markets to new Fake Markets dominated by one or two giant tech companies. And perhaps the single biggest thing we can do is both the hardest and the easiest: We can change our own behaviors. Look at the apps on your phone right now. Are you sure you are comfortable with what’s going to happen when everyone’s running the same apps that you are?

Fake Markets: End

Tags:

It's time to discover Prince

With the return of Prince’s classic 80s and 90s catalog to the most popular streaming services, now’s a great time to (re?)discover the breadth of Prince’s incredible body of work.

Prince circa 1991, by Herb Ritts

The full scale of Prince’s music is probably too much for any unfamiliar listener to just dive into; he released nearly 40 albums under his own name(s), regularly enhanced his single releases with extended versions, remixes that could sometimes comprise an entire EP on their own, and legendary B-sides that were often as strong as the single being released to radio. That’s not even counting the literally hundreds of songs he wrote (and often performed on) for others.

So, here’s an easier way to dive into his catalog, broken down by the type of listener you are, and what genres of music you prefer. I’m assuming little to no familiarity with Prince’s catalog here, beyond staples like the song Purple Rain. The nice thing about Prince’s work is that there are no bad starting points; if you don’t like what you hear at first, he almost certainly made a song in the complete opposite style as well.

The basics

If you’ve never really listened to Prince’s work, there’s a reason his 80s albums are revered. They hold up favorably against the very best albums in pop music.

  • Purple Rain (1984)

    Spotify | Apple Music | Amazon | Tidal

    It really is that good. Half the songs on the album became hit singles, and the other half would have except they were too sexy.

  • 1999 (1982)

    Spotify | Apple Music | Amazon | Tidal

    This one will surprise you. Though Purple Rain has more, bigger hits, this is the album that shaped the sound of 80s radio. And, well, a lot of the Top 40 to this day. The songs really stretch out, and this is the album that turned a lot of casual Prince fans into diehards.

  • Sign O’ The Times (1987)

    Spotify | Apple Music | Amazon | Tidal

    If you want to hear Prince at his experimental best, this is almost every hardcore Prince fan’s favorite album.

The greatest hits

There are a number of Greatest Hits collections for Prince’s work. None of them are terrible, but all of them ignore the second half of his career which, while uneven, still had dozens of truly great songs.

  • Ultimate (2006)

    Spotify | Apple Music | Amazon | Tidal

    The best overall collection of Prince’s work, this includes a number of his best b-sides and extended versions, amply demonstrating why those non-album tracks were essential to understanding his range. And if you like big hits like Little Red Corvette, it shows up here in the full 8-and-a-half-minute glory of its 12" Dance Mix.

  • The Hits/The B-Sides (1993)

    Spotify | Apple Music | Amazon | Tidal

    The first compilation of Prince’s work is still the only one to collect a large number of his b-side recordings. Even if you’ve heard most of his 80s albums, there are almost certainly songs here that you missed.

Specially-crafted starting points

I made a number of playlists that are specifically aimed at people who feel like they’ve never really gotten Prince. I often hear people say, “I know he’s supposed to be super talented, but I never saw him live, and I don’t know what song of his I would love.” This is especially poignant for those of us who were fans because his live shows were amazing, often radically recasting his recorded material, and because his hit pop singles, while brilliant and unique, often didn’t resemble the more obscure works that won us over.

These playlists are necessarily incomplete, because of the inconsistent way Prince’s catalog is made available. Tidal comes closest to including all of these songs, though even at its peak Tidal still omitted hundreds of Prince’s songs from its service. I’ve included Spotify versions of the playlists if most of the songs are available from the service.

  • Discover Prince

    Spotify | Apple Music | Tidal 
    This is a playlist of the most “Prince-sounding” tracks in his catalog, a great way to hear work that a lot of serious fans would say could only have come from Prince.

  • Prince: Guitar Pop

    Spotify | Tidal
    Prince’s most riff-driven rock tracks, showing off both his pop songcraft and his predilection for shredding. This list shows off how his work became more conventionally guitar heavy in this century.

  • Prince: Electronic

    Spotify | Tidal
    The signature sound Prince was known for was his extraordinary and cutting-edge adoption of the latest electronic technologies like drum machines, synthesizers, samplers and sequencers. By bringing all these tools to bear, he changed the sound of popular music. These are some of the songs that caused that change, and some showing off how he kept evolving.

  • Prince: Piano

    Spotify | Tidal
    In his final tour, Prince performed solo at the piano, reaffirming his raw showmanship and the strength of his songwriting. But throughout his career, he showed off his skills on the keys, as these songs amply demonstrate.

More to come

There are, of course, a nearly infinite number of ways to slice and dice a catalog that comprises over a thousand songs. None of these playlists even includes the work that Prince created for other artists. And it’s easy to imagine playlists like “Here are the Prince songs you’ll like if you love Hendrix” or “These are the Prince tracks that Justin Timberlake clearly loved the most”.

But what’s most exciting is the idea that a new wave of listeners can find their own gems in a body of work that offers enough surprises and delights to last for decades to come. If you’re just getting into Prince, I hope these lists form a good starting point, and don’t hesitate to reply to me at @anildash if you’ve got questions or want suggestions of how to get started.

Tags:

It's me, Bike Dad!

Until the Citibike bikeshare program launched here in New York City, I'd ridden a bike perhaps once in the prior twenty years. Since it launched, I ride almost daily.

Because of the massive improvements in quality of life in the city as walkability and cycling affordances were improved, I've become a strong advocate for commuting by bike, and I was excited to talk about my experiences with NY1.

While the Full NY1 series on new commuting trends is unfortunately still a little slanted toward favoring privately-owned, under-utilized automobiles, I think the tide is turning overall because, as I've noted before, our investment in making New York City accessible and safe for pedestrians and bikers has made this the golden era for public space in the entire 400-year history of the city.

Tags:

Design Matters

I was delighted to get to speak with Debbie Millman for her venerable podcast, "Design Matters". If you have an hour to spare, please do check out the conversation — we touched on a ton of topics that are near and dear to my heart.

Tags:

Read more