Thing is, I don't think tumblr staff is uniquely transphobic - especially since it's not just trans women's selfies and transmasc art getting censored, but classic queer comics, Black bloggers (especially women), and more, even as our more visible staff members keep talking about trying to fix the problem.

I suspect it's something much bigger and with much worse implications for the entire internet.

I suspect more appeals are being left to the automated system than we think (which they're NOT supposed to do but due to the sheer volume theyre having to deal with, well...) and, between the evergreen problem of report-brigading by bigots, the other evergreen problem of a company internally sucking at determining where justified criticism of their practices ends and harassment begins, and the problems of fighting the porn bots (because every single one you see is a product of survivorship bias) leading the algorithm to become sexist (remember #girlgate?), the automod algorithm is basically collapsing in on itself and learning all the worst biases of society MORE than ever before.

Which, given that these systems are being deployed all OVER the internet and being touted as objective and unbiased because ~computers can't be bigots~, should fucking horrify you.

astraltrickster

To take this a step further, I feel tumblr's moderation problems are a, uh..."lovely" preview of what we might need to be on the lookout for going forward with social platforms on the internet, in MULTIPLE horrifying ways.

Because...look. It's flawed, but from a human cost standpoint it is FAR better for the majority of moderation to be automated. Manual moderation is a PTSD meat grinder. The stuff you don't see that gets filtered out automatically from most platforms includes everything from CSAM, to execution videos, to animal torture, to hate speech so intense and graphically violent that it would make you nauseous, to actionable threats of violence aimed at kids and animals, and more. There is no ethical way to expose people to that 8 hours a day, at the volume demanded by the modern internet. We know this, because that's how it's worked in the past and the trauma it inflicts on people is well known. Never mind that, conditions in domestic cubicle farms are bad enough, but this work is often outsourced to the global south, 8 hours a day is a SHORT shift for a lot of the people working these jobs, which only makes the whole situation even worse...

But for one, we know for a fact that algorithms are...NOT ready to handle that job. For example, here on tumblr, we still see a ton of porn bots and other spam bots, but remember:

For every single one we see, I would hazard a guess that at least 20 get banned before anyone even gets a chance to notice, and they go away pretty fast once reported...

But we still see the ones that break through. Which, to be fair, is even worse under solely manual moderation, since an actual human person has to see the problem in order to stop it, but it is still very There to prove that the system is fallible. Luckily, we have the report button. Unfortunately, it depends on users knowing how to use it...and using it in good faith.

So, in the interest of trying to keep moderation...okay lets be totally honest the corporate reason is to make it CHEAPER; making it less harmful to human workers (because people still need to train the system, but they need to be exposed to a lot less horrible shit to train an algorithm than to actively monitor a website) is just a side effect, but the point is that we end up with...bias automation, which seems to only get worse and worse as time goes on.

For two, let's look at the current tech zeitgeist. As much as I hate to bring up the drawbacks of the AI fad because it makes a lot of people's brains just shut off in vehement rejection of anything that looks like AI might be involved, this really is one of them - too many people are trusting machine learning WAY too much. This is still in very experimental stages. It's FAR from unbiased, as we've just discussed. Yet, because tech companies want to sell it NOW, they're presenting it as Totally Ready To Sell, and The Future, and...Objective. Cold Hard Data.

Imagine how much worse it will be if these algorithms end up getting used in not just internet moderation, but full on law enforcement, before they get better. After all, they're already trying to use facial recognition that way.

And, to make it even more horrifying, let's also look at it from a legal standpoint on the platform's end.

Tumblr, under Verizon, settled a lawsuit over moderation bias. They were ordered to go back to more human oversight and retrain the algorithm, and...for a while things were going okay-ish, the problem never went away but it SEEMED to be doing slightly better?

...but they can't roll back the porn ban, because SESTA/FOSTA are a thing, and payment processors restrict services when it comes to anything they declare too "adult", and KOSA is at risk of actually getting passed (PLEASE keep calling and writing your representatives!!), and so much else...

And we have a wave of book bans and state laws that formally declare queerness to be "sexually explicit" going down...

So, website operators are being put in a HELLISH catch-22 situation. On the one hand, we have nondiscrimination laws. As a business, you CAN'T just declare that a protected class of people is more Inherently Sexual than others. On the other hand, we have legal precedents being set that being queer IS inherently pornographic. In other words, we're approaching a situation where it is LITERALLY IMPOSSIBLE for a platform to stay legally above board. You DON'T censor the queers, you're "providing porn to minors". You DO, you're violating nondiscrimination laws.

I cannot begin to stress how absolutely hellish that will be for EVERYONE - both users and webmasters alike.

Tl;dr: yes tumblr's moderation is biased, and the probable reason makes me WISH it could be attributed to just some absolute raging bigoted fuckweasels on staff. Buckle up and call your representatives and for FUCK'S SAKE DO NOT LET DESANTIS GET ANYWHERE IN 2024.

A Mini Guide to RSS Feeds