Bluesky, Threads, and the decentralization dilemma

Two cases on the frontiers of content moderation

Jul 18, 2023

(Rafael Henrique / Getty Images)

One of the more unexpected twists on the consumer internet in 2023 has been the sudden turn toward decentralization: the move away from the walled-garden ethos that defined 2010s social networks toward a world where platforms are exploring ways to give up some control.

To date, the devolution of power to platform users has been more theoretical than actual. Replacing centralized services with federated alternatives comes with difficult trade-offs, and mastering the balance will likely take platforms several years.

Today, let’s talk about two recent cases where these trade-offs are on full display. In the first case, a decentralized platform trips while building a more centralized service. In the second, a centralized service stumbles in its quest to decentralize.

Start with case one. Last week, Bluesky users discovered that at least one account had been able to create a username with the N-word in it. Several others were able to create accounts with other racial slurs, Mashable reported.

While the accounts were removed shortly after Bluesky was alerted, the company — which got its original funding from Twitter co-founder Jack Dorsey — said nothing about the incident publicly until three days later. During that time, some users complained that Bluesky had been too slow to respond and failed to apologize.

Here’s Matt Binder at Mashable:

"They fixed the technology issue but not the people issue," one prominent Bluesky user shared with Mashable. The user, who asked for their identity be withheld from the piece, explained that the company seems to be lacking when it comes to human moderation, trust and safety policies, and communication.

It shouldn’t come as any surprise that Bluesky’s trust and safety mechanisms are lacking: The app was never designed to compete directly with rivals like Twitter or Mastodon. Instead, Bluesky was intended to serve as a proof of concept for the AT Protocol on which it is built — a kind of demonstration app that would spur the development of a large, federated social network.

A pillar of this vision is that users should have “algorithmic choice” — the ability to filter content in their feeds as they see fit. An aspect of this choice is what the company calls “composable moderation.” The basic idea is that Bluesky would label and remove all illegal and some objectionable content, but allow others to build experiences that display posts that others might find objectionable.

In a world where social networks face increasing pressure not to censor content, composable moderation offers a way out. Federated systems like these will allow servers to set defaults that most users would approve of, while also building systems to let edgelords and other interested parties consume more risqué or offensive content.

I really like this approach, which feels like an evolution of Reddit’s mostly successful system of setting a “floor” of guidelines that every forum must follow while letting moderators of individual forums raise the “ceiling” by adding additional rules.

The trouble, as Bluesky found out the hard way, is that even when you are a decentralized social network, users will expect you to maintain the moderation standards of a centralized service. That means strong controls and active enforcement around hate speech, porn, spam, misinformation, and everything else that can harm the user experience.

To their credit, Bluesky executives seem aware of this reality. It’s the chief reason that the company has kept Bluesky in an invite-only beta, sparing itself from the deluge of moderation problems that could come once it opens up the floodgates. In the meantime, it has worked to bring its trust and safety systems up to par.

“Over the past few months, we’ve made significant investments in Trust and Safety, and will continue to do so,” the company said in a statement Saturday. “This includes expanding the T&S team, improving and clarifying policies, and making moderation tools an engineering priority. Our moderation team responds to most reports within 24 hours.”

The problem for Bluesky is that this dynamic could set the company up to fail. Centralized services like Facebook can afford the (flawed!) moderation systems they have because they generate revenue from the data they collect on their users, and then spread the costs out over billions of people. Bluesky’s federated social network will almost certainly be much smaller, and it currently doesn’t generate any revenue.

The worst case scenario for Bluesky, then, is that it feels compelled to build out functions that match those of its biggest and best funded rivals, while stuck with a federated structure and business model that can’t support the costs those functions require. You get most of the downside of building a walled garden, while also getting all the downsides of federation.

In the end, the company’s more existential problem may simply be that the arrival of Threads this month absorbed most of the demand for a Twitter alternative. But the unexpected moderation pressures seem to be taking their toll: company leadership, which is typically quite active on the service, has been mostly silent for days now.

II.

That brings us to case two: Meta’s plans to decentralize Threads.

Threads is built on ActivityPub, a web standard that is functionally similar to Bluesky’s AT. Meta built Threads with the intention of making it interoperable with Mastodon and other services that use ActivityPub, which are collectively known as the Fediverse.

If the services do become interoperable, you should be able to view Mastodon posts within Threads, and Threads in a Mastodon client. Instagram head Adam Mosseri told me Meta wants to make Threads decentralized because the industry is collectively moving in this direction, and over time the best creators will gravitate toward platforms that grant them more freedom. (ActivityPub lets users take their followers with them when they leave a server, for example.)

The move could also have other benefits for Meta: making followers portable could lessen antitrust concerns, for example. And it could reduce political pressure on the company related to content moderation. Hate the community standards on Threads? Go join another server where the rules are more to your liking, while keeping your audience.

Sounds good to me. But in the end, Threads launched without ActivityPub support. “We weren’t able to finish it for launch given a number of complications that come along with a decentralized network, but it’s coming,” Mosseri said.

I take Mosseri at his word. But in the most recent episode of the excellent Moderated Content podcast, co-host Alex Stamos predicted that Meta would ultimately decide not to support ActivityPub. (He also set down his thoughts in a Mastodon thread.)

What’s the issue? Stamos, Meta’s former chief security officer, said that the move would create huge content moderation problems for both Meta and the larger Fediverse.

As Threads removes spammers and other bad actors, they will attempt to evade the bans by setting up new accounts on other Fediverse servers, in the hopes they can still reach Threads’ tens of millions of active useres. For Fediverse servers, that could mean an explosion of new adversarial accounts that few of the mostly volunteer moderators are likely prepared for.

That’s bad for the Fediverse. But it would be bad for Meta, too: the company will have much less metadata on Fediverse users with which to identify bad actors. If you’re on Threads, Meta collects your IP address and other information that can prevent you from repeatedly breaking its rules; if you’re posting to a random Fediverse server, it likely does not. That could create new avenues for harmful content to find its way onto Threads.

Threads, for its part, reiterated its plans to join the Fediverse in a post today. “f you’re a creator with a public account here you’ll soon be able to reach new audiences with no added effort,” the company wrote. “You’ll also have the option to take your content and move to other services like Mastodon at any time.”

Over a long enough time period, the challenges of decentralization still seem solvable to me. But events of the past few weeks suggest taht the cost will be high — and higher, perhaps, than some of the companies that are building this world are prepared to pay.

Talk about this edition with us in Discord: This link will get you in for the next week.