Am I Nuts for Thinking a Jewish Florist Should Have To Make an Easter Arrangement?

One thing I tried to impress upon my Con Law students this semester (and every semester) is that the interplay between anti-discrimination law and freedom of speech (and freedom of religion) is complicated and raises a host of thorny questions that defy easy resolution. These issues, of course, lie at the forefront of the 303 Creative case currently before the Supreme Court, which I’m sure will address them with the care, nuance, and sensitivity they deserve [/sarcasm].

But on that matter, I want to flag a hypothetical offered by prominent First Amendment specialist and former federal judge Michael McConnell, to get folks’ intuitions on:

McConnell frames this as his “personal favorite hypothetical”, and clearly perceives it as a knockout argument for the pro-free speech/religious liberty side. But perhaps I’m not fully grasping the facts, because speaking as a Jew this prospect doesn’t seem that frightening to me.

Suppose I’m a Jewish florist. A customer comes in and says “I’ve seen the lovely work you’ve done with white lilies, could you please make a similar display for me?” I agree, since I have loads of experience working with white lilies. The customer then says, “thanks – we plan on putting this display up in our church on Easter morning!” This prospect … doesn’t upset me. I don’t intuitively think I should be able to refuse the customer, notwithstanding the fact that I obviously don’t believe in the divinity of Christ, and I don’t view continuing to serve the customer as forcing me to avow any beliefs I don’t hold.

At root, the reason why this prospect isn’t bothersome is because I don’t view my customer’s use of my flowers as representing my speech. I just design the flowers; what they do with it is their business. If someone sees the arrangement at church and learns that David’s Flowers created it, I do not expect them to think “wow, I had no idea David believed in Christ’s divinity!” This isn’t to say I have no free speech concerns regarding flower arrangements – I would very much chafe at government regulations that, for example, regulate what shapes I can use in my designs. That part very much is my expression, would be attributed to me – the churchgoer who compliments the pattern of the flowers would credit those decisions to David’s Flowers (I wrote about this a few years ago as the problem of partially expressive conduct).

There are still plenty of tough cases at the margins. I show my customer a preliminary design; they twist their lip and say “I dunno … it’s just not capturing the majesty of Christ’s resurrection, you know?” I’m at a loss (“So … bigger?”). But I’m inclined to think that while such an example might demonstrate why I might be a bad choice to design the arrangement, it doesn’t give me the right to discriminate against the customer if they are in fact thrilled with the work I do and have done for other customers.

For me, then, McConnell’s hypothetical has the opposite effect than what he intended. And of course, for many Jews – particularly Jews who live in predominantly non-Jewish areas – the more salient threat is that local businesses will be given carte blanche authority to refuse to service any of our religious life cycle events lest it be seen as “approving” of them. To let vendors say “ordinarily, a cake is just a cake – but a cake served at a Bar Mitzvah has religious significance that we, as Christians, cannot approve of” is not a door I want to open.

But perhaps some of my readers disagree. Curious to hear people’s thoughts on this.

via The Debate Link https://ift.tt/184tzfG

Mostly I like the hypothetical scenario dialogue here. (Some blanks filled in by me.)

I’m a secular Jew and wouldn’t care. But Orthodox Jews are supposed to refuse to set foot in churches, or refuse to even say the word “Christ” lest they be seen as implicitly supporting Christianity. There’s a rule in the Talmud that Jews shouldn’t do any business with those of foreign religions (I think including Christians) on or within three days of the pagans’ religious holidays, lest they seem to be supporting or participating in them.

I think the sort of Orthodox Jew who takes all of these restrictions seriously would be very unhappy at the government forcing them to prepare an Easter bouquet, and I would consider it deeply evil if the government tried to punish them for refusing.

Local man fond of linguistic garden path sentences friends to hearing his boring puns.

You really can't overstate how little anti-worker rhetoric has changed over a hundred fifty years.

(source)

I will never understand the human urge to leap from “lots of people have thought this was true over a long period” to “and therefore it must be false”.

(one could argue this graph shows high workforce participation in 1948, making it hard for participation to have been declining since 1894. But wages were rising very quickly from 1894-1948, suggesting it was taking more and more money to keep the same number of people in the workforce)

okay but 1) that chart only covers about half the period of the original set of quotes and 2) the repeated claim wasn’t “there has been an 8 percentage point reduction in the prime-age male labor force participation rate over a 66 year period.”

i mean, “nobody wants to work anymore” is not a specific claim like “there has been an 8 percentage point reduction in the prime-age male labor force participation rate over a 66 year period” so we can always quibble about what the former means and its relation to the statistics, but many--not all, but many--reasonable interpretations of the sentence “nobody wants to work anymore” are not “the prime age male labor force participation rate has been falling by about 0.12 percentage points per year for as long as I can remember.” OP’s complaint might be poorly defined, but it’s not self-evidently invalid, and this statisic isn’t exactly a clanging refutation of the original (admittedly ambiguous) claim

another important dimension is that this statistic does not indicate why the prime age male labor force participation rate has been declining so steadily, while a key component of the quotes OP is talking about is attributing an apparently widespread phenomenon specifically to laziness/not wanting to work hard/social welfare programs/etc., instead of specifically to (as OP implicitly contends) bad working conditions and/or pay. your chart also doesn’t refute that, and ignores a key component of the original post! that strikes me as somewhat dishonest. insofar as the original post contains a provably true or provably false claim (debatable), the claim is that people don’t want to work due to some set of social factors, not people don’t want to work, full stop.

this isn’t really a very good refutation of anything, and depending on how you interpret the data might even be taken as reinforcing, not undermining, OP’s point.

I mean one thing that list of quotes does indicate is that some complaints are perennial regardless of general cultural conditions or the state of the social safety net (1894-2022 covers a wide variety in both), which means that appeals to those factors as a cause of listlessness in the working population may, in fact, not be entirely in good faith! Cf. perennial complaints by Romans of Romans’ moral and spiritual decline leading to a weakening of the state, which started, oh, about 200 years before the Roman Empire reached its maximum extent of wealth and power.

I will say I do believe, at almost every time, that "no one wants to work". That's why we call it work and give people money for it.

Sure, there are people with jobs that are so intrinsically rewarding that they actually want to do them—I'm often one of them. (Although there's the 10-20% of the job that's "actual work" and I don't want to do that!)

And sure, people have a psychological need to feel like they're accomplishing something and creating something, but that doesn't necessarily line up with what people will pay you for.

Fundamentally, work is something that people need a motivation to do, and that's why they get paid for it. So it's basically always true to say that people "don't want to work", and the question is whether you want to pay them enough to change their minds.

Re: 1 - please see the sentence in parentheses I wrote above.

Re: 2 - It sounds like you think this is only valid if you can see into the souls of men and find something marked “laziness” or “being a bad person who hates work for no reason”. But what all of these headlines are saying is just “nobody wants to work anymore”.

Clearly that’s not literally true, since 80-something percent of people are still employed. But less hyperbolically, it sounds like they’re saying “people are less willing to work a given amount for a given price than they used to be”.

This is totally true! Over the past 150 years, number of hours worked has continuously gone down!

Even as wages continuously went up (until ~1970, see here for more)

And at the same time, people were (as per the original graph) dropping out of the labor force.

I can’t really see any reason to interpret those statements as completely accurate observations of the fact that, over the past 150 years, people have been gradually less willing to work, even as hours go down and pay goes up.

This doesn’t seem mysterious - I think it’s a natural result of the country getting richer and people not having to work as much. Some of that is welfare, but some of it is just being able to support yourself comfortably on fewer hours of work. And some of it is probably cultural change downstream of the first two.

One easy way to see this is that everyone has a stereotype of illegal Mexican immigrants as really hard-working. I think this makes sense - they don’t have access to a lot of the same opportunities to do well with medium amounts of work, opportunities that America’s rising wealth has given us over the past century or so - and so they work as hard as native-born Americans did in 1870 or whenever.

You really can't overstate how little anti-worker rhetoric has changed over a hundred fifty years.

(source)

I will never understand the human urge to leap from “lots of people have thought this was true over a long period” to “and therefore it must be false”.

(one could argue this graph shows high workforce participation in 1948, making it hard for participation to have been declining since 1894. But wages were rising very quickly from 1894-1948, suggesting it was taking more and more money to keep the same number of people in the workforce)

Don't Lie To Me About Web 2.0

If you're like me and you're trying to keep an open mind that there may someday be a non-scam application of blockchains, you've probably read some articles about "Web3", which promises to re-decentralize the web by something something Blockchain.

I realize this is far from the most important criticism but i think it's really interesting that the standard explanation you find replicated nearly word-for-word at the beginning of most "Web3" articles has a big ol' chunk of historical revisionism in it. It goes like this:

"First there was web 1.0, which was, like, geocities pages and stuff, and it was decentralized. Then there was web 2.0, which was the centralized silos of social media - facebook, twitter, etc. Now Web3 is gonna re-decentralize everything by letting you own your own data on the blockchain…"

No! Stop there! Web 2.0 was not social media! You're rewriting history that's less than 20 years old!

Web 2.0 was:

when someone tried to buzzwordify all these disparate trends they noticed that what a lot of them had in common was "Website owner allows website visitors to enter words that will be seen by other website visitors" and summed that up as "User-generated content" and branded it "Web 2.0" around 2004-2005.

I was there. I worked on backends for a lot of this stuff!

The key shift was where things were hosted. In Web 2.0 you might use off-the-shelf software like WordPress or phpBB or whatever but you were still hosting all that stuff on your own server. Your server, your rules; you'd set your own moderation policy and wield your own "banhammer". The free speech compromise was "don't like my moderation policy? Make your own website."

It was a huge paradigm shift in 2005-6 when YouTube started and said "we'll host your videos for you". (What? trust a third-party website to host my videos? Sounds sketchy) That was the beginning of the end, because once people gave up running their own server in favor of letting a big company host their stuff on a centralized server, we gave up all the power.

Social media wasn't web 2.0, it's what killed Web 2.0!

You might think I'm arguing over mere nomenclature but the important fact is that this era existed, and the Web3 pitch pretends it didn't. We already had decentralized internet with social features. This fact contradicts the story the Web3/blockchain advocates want to tell you, so their story skips this entire era.

Web 2.0 lost to siloed social media because:

To make the web3 argument you have to explain why "a distributed ledger where each update contains a cryptographically signed pointer to the previous update, replicated across many computers via a decentralized protocol, that rewards people for hosting nodes by paying them pretend money when they brute-force solve a cryptographic hash" is relevant to any of these problems. I suspect it is not relevant, because:

if you want to make the dream of "buy your Minecraft skin as an NFT and bring it with you to wear in Fortnight!" work (why is this the example every article uses?) you would need to get all the games involved to decide to implement equivalent items, or some kind of framework of item portability, and if you could do that then you wouldn't need the blockchain!

What might help solve any of the problems that killed Web 2.0:

I want a decentralized internet to come back more than anybody, but blockchain is completely irrelevant to that.

This reminds me of what Signal founder Moxie Marlinspike wrote about some of the failures of web3.0. In theory the blockchain is this democratic protocol everyone can access by running a server node on their computer, but

So there's a decentralized protocol floating around, but in practice most people get an account on a centralized service like CoinBase or OpenSea and let it interact with the blockchain protocol for them.

And does this sound familiar to you?

I agree with the part about web3 not being useful, but disagree with the theory of why early pre-corporate Web2 (let's call it Web2A) failed.

For most Web2A services, you didn't need to be able to host your own server or pay money. EZBoard was a PHPBB clone that you could set up with a few clicks. They supported themselves through ads and selling premium memberships with more features. When I was a teenager my friends and I made EZBoards all the time even though we had no money or technical skill. Same with blogging. You could use a LiveJournal, or get an ad-supported centrally-hosted WordPress (I actually don't know when WordPress started offering this, maybe I'm wrong and it was after this period).

Maybe EZBoard and centrally-hosted-WordPress were already bleeding into the sinister siloed centralized web, but they didn't feel like it. Your EZBoard or your LiveJournal felt like your own space that you had control over; the corporate masters were in the background somewhere taking your ad revenue but you never thought about them.

For me, the big change was between the era of EZBoard and centrally-hosted-WordPress, and the era of Facebook and Twitter, but I'm trying to think of what made this different or worse.

I think the main difference was the introduction of an algorithmic feed. When you visited someone's EZBoard, you were still visiting their EZBoard. It would have an address like ScottsBoard.EZBoard.com (I think you could pay extra to have it be ScottsBoard.com), and you would get to customize the color and decorations. On Facebook, you rarely visit anyone's individual page, and even when you do it feels like a different corner of Facebook rather than something they made themselves (of course, there are good business reasons for this: the MySpace decoration debacle). Facebook is just a stream of content ("content" being a vague catch-all term for videos, comments, and links from a bunch of different human and corporate sources) selected by a combination of your own preferences (increasingly irrelevant) and the Facebook algorithm.

(LiveJournal had a feed too, but I don't think it was algorithmic, and it was always just your friends' LiveJournal posts.)

The other thing is that it was hard to find content you liked in the decentralized Web2A ecosystem. You would need to wait for a blog you liked to link some other blog you might like. This was fine at the time because everyone had lots of blogs they liked and they did link other blogs you might like frequently. But once some of that collapsed, it fed in on itself - people can't get off Facebook now because they don't always know where the non-Facebook things they'd like would even be.

I sometimes visit what I think might be the only flourishing PHPBB established in the last five years - Data Secrets Lox. It works because I started a blog in 2013, when you could still do that easily, it got a large commentariat, and then later when the blog got shut down someone created DSL as a place for the community to talk, and my blog signal-boosted it. Everyone on it seems very happy with it, but absent that weird confluence of events I don't think things like that can really get started anymore.

comments on mesa-optimizers

(Copy/pasted from a comment on the latest ACX post, see that for context if needed)

FWIW, the mesa-optimizer concept has never sat quite right with me. There are a few reasons, but one of them is the way it bundles together "ability to optimize" and "specific target."

A mesa-optimizer is supposed to be two things: an algorithm that does optimization, and a specific (fixed) target it is optimizing. And we talk as though these things go together: either the ML model is not doing inner optimization, or it is *and* it has some fixed inner objective.

But, optimization algorithms tend to be general. Think of gradient descent, or planning by searching a game tree. Once you've developed these ideas, you can apply them equally well to any objective.

While it is true that some algorithms work better for some objectives than others, the differences are usually very broad mathematical ones (eg convexity).

So, a misaligned AGI that maximizes paperclips probably won't be using "secret super-genius planning algorithm X, which somehow only works for maximizing paperclips." It's not clear that algorithms like that even exist, and if they do, they're harder to find than the general ones (and, all else being equal, inferior to them).

Or, think of humans as an inner optimizer for evolution. You wrote that your brain is "optimizing for things like food and sex." But more precisely, you have some optimization power (your ability to think/predict/plan/etc), and then you have some basic drives.

Often, the optimization power gets applied to the basic drives. But you can use it for anything.

Planning your next blog post uses the same cognitive machinery as planning your next meal. Your ability to forecast the effects of hypothetical actions is there for your use at all times, no matter what plan of action you're considering and why. An obsessive mathematician who cares more about mathematical results than food or sex is still thinking, planning, etc. -- they didn't have to reinvent those things from scratch once they strayed sufficiently far from their "evolution-assigned" objectives.

Having a lot of optimization power is not the same as having a single fixed objective and doing "tile-the-universe-style" optimization. Humans are much better than other animals at shaping the world to our ends, but our ends are variable and change from moment to moment. And the world we've made is not a "tiled-with-paperclips" type of world (except insofar as it's tiled with humans, and that's not even supposed to be our mesa-objective, that's the base objective!)

If you want to explain anything in the world now, you have to invoke entities like "the United States" and "supply chains" and "ICBMs," and if you try to explain those, you trace back to humans optimizing-for-things, but not for the same thing.

Once you draw this distinction, "mesa-optimizers" don't seem scary, or don't seem scary in a unique way that makes the concept useful. An AGI is going to "have optimization power," in the same sense that we "have optimization power." But this doesn't commit it to any fixed, obsessive paperclip-style goal, any more than our optimization power commits us to one.

And even if the base objective is fixed, there's no reason to think an AGI's inner objectives won't evolve over time, or adapt in response to new experience. (Evolution's base objective is fixed, but our inner objectives are not, and why would they be?)

Relatedly, I think the separation between a "training/development phase" where humans have some control, and a "deployment phase" where we have no control whatsoever, is unrealistic. Any plausible AGI, after first getting some form of access to the real world, is going to spend a lot of time investigating that world and learning all the relevant details that were absent from its training. (Any "world" experienced during training can at most be a very stripped-down simulation, not even at the level of eg contemporaneous VR, since we need to spare most of the compute for the training itself.)

If its world model is malleable during this "childhood" phase, why not its values, too? It has no reason to single out a region of itself labeled $MESA_OBJECTIVE and make it unusually averse to updates after the end of training.

See also my LW comment here.

I agree that optimization power is not *necessarily* correlated with specific goals. But why wouldn't mesa-optimizers, contingently, have a specific goal. Presumably we're running gradient descent on some specific loss function, like "number of paperclips produced", and then mesa-optimizer inherits some proxy for that.

I agree humans aren't like that, and that this is surprising.

Maybe this is because humans aren't real consequentialists, they're perceptual control theory agents trying to satisfy finite drives? EG when we're hungry, our goal becomes to find food, but we don't want to tile the universe with food, we just want to eat 3000ish calories and then we're done. We have a couple of other goals like that, and when we've accomplished all of them, most people are content to just hang out on the beach until something else happens.

Might gradient descent produce a PCT agent instead of a mesa-optimizer? I don’t know. My guess is maybe, but that optimizers would be more, well, optimal, and we would get one eventually (either later in the gradient descent process, or in a different lab later). My guess is evolution didn’t make us optimizers because it hasn't had enough time to work with us while we've been intelligent. If we got locked at 20th century technology forever, I think it might, after a few million years, produce humans who genuinely wanted to tile the universe with kids.

Wouldn't the first thing a superintelligence with a goal did be to make sure its goal didn't drift?

I think this is where the deception comes in. If the mesa-optimizer is smart and doesn't want people (or other parts of itself) changing its values, it will take steps to stop that, either by lying about its values or fighting back.

I think this idea that "real consequentialists are more optimal" is (sort of) the crux of our disagreement.

But it will be easiest to explain why if I spend some time fleshing out how I think about the situation.

What are these things we're talking about, these "agents" or "intelligences"?

First, they're physical systems. (That far is pretty obvious.) And they are probably pretty complicated ones, to support intelligence. They are structured in a purposeful way, with different parts working together.

And this structure is probably hierarchical, with higher-level parts that are made up of lower-level parts. Like how brains are made of neuroanatomical regions, which are made of cells, etc. Or the nested layers of abstraction in any non-trivial (human-written) computer program.

At some level(s) of the hierarchy, there may be parts that "run optimization algorithms."

But these could live at any level of the hierarchy. They could be very low-level and simple. There may be optimization algorithms at low levels controlled by non-optimization algorithms at higher levels. And those might be controlled by optimization algorithms at even higher levels, which in turn might be controlled by non-optimization ... etc.

Consider my computer. Sometimes, it runs optimization algorithms. But they're not optimizing the same function every time. They don't "have" targets of their own, they're just algorithms.

They blindly optimize whatever function they're given by the next level up, which is part of a long stack of higher levels (such as the programming language and the operating system). Few, if any, of the higher-level routines are optimization algorithms in themselves. They just control lower-level optimization algorithms.

If I use my computer to, say, make an amusing tumblr bot, I am wielding a lot of optimization power. But most of my computer is not doing optimization.

Python isn't asking itself, "what's the best code to run next if we want to make amusing tumblr bots?" The OS isn't asking itself, "how can I make all the different programs I'm running into the best versions of themselves for making amusing tumblr bots?"

And this is probably a good thing. It's hard to imagine these bizarre behaviors being helpful, giving me a more amusing tumblr bot at the end.

Which is to say, "doing optimization well" (in the sense of hitting the target, sitting on a giant heap of utility) can happen without doing optimization at high abstraction levels.

And indeed, I'd go further, and say that it's generically better (for hitting your target) to put all the optimization at low levels, and control it with non-optimizing wrappers.

Why? The reasons include:

Goodhart's Law

Speed

------

When is the agent an "optimizer" / "true consequentialist"?

This question asks whether the very highest level of the hierarchy, the outermost wrapper, is an optimization algorithm.

As discussed above, this is not a promising agent design! There is an argument to be had about whether it still could emerge, for some weird reason.

But I want to push back against the intuition that it's a typical result of applying optimization to the design, or that agents sitting on giant heaps of utility will typically have this kind of design.

The two questions

have very little to do with one another.

In the LessWrong-adjacent type of AI safety discussion, there's a tendency to overload the word "optimizer" in a misleading way. In casual use, "optimizer" conflates

But doing optimization all the way at the top, involving your whole world model and your highest-level objectives, is very slow, and tends to extremal-Goodhart itself into strange and terrible choices of action.

It's also not the only way of applying optimization power to your highest-level objectives.

If I want to make an amusing tumblr bot, the way to do this is not to ponder the world as a whole and ask how to optimize literally everything in it for maximal amusing bot production. Even optimizing just my computer for maximal amusing bot production is way too high-level. (Should I change the hue of my screen? the logic of the background process that builds a search index of my files??? It wastes time to even pose the questions.)

What I actually did was optimize just a few very simple parts of the world, a few collections of bits on my computer or other computers. And even that was very time-intensive and forced me to make tradeoffs about where to spend my GPU/TPU hours. And then of course I had to watch it carefully, applying lots of heuristics to make sure it wasn't Goodharting me (overfitting, etc).

To get back to the original topic, the kind of "mesa-optimizer" we're worried about is an optimizer at a very high level.

It's not dangerous (in the same way) for a machine to run tiny low-level optimizers at a very fast rate. I don't care how many times you run Newton's method to find the roots of a one-variable function -- it's never going to "wake up" and start trying to ensure its goal doesn't change, or engaging in deception, or whatever.

And I am doubtful that mesa-optimizers like this will arise, for the same reasons I am doubtful that the agent will do optimization at its highest level.

Once we are pointing at the agent, or a part of it, and saying "that's a superintelligence, and wouldn't a superintelligence do . . . ", we're probably not talking about something that runs optimization.

You don't spend your optimization budget at the level of abstraction where intelligence happens. You spend it at lower levels, and that's what intelligence is made out of.

I guess I'm pretty confused by this response.

Your computer isn't optimizing for Tumblr bots. But your computer can't create Tumblr bots; if you left it on the desk for a million years, no Tumblr bots would be made. The you-yourcomputer system *is* optimizing for Tumblr bots, with the optimization layer being in your brain. You're not technically a Tumblrbot-maximizer, but presumably this is why you don't create that many Tumblr bots and you spend most of your time doing other things.

A version of you who (let's say because of weird childhood conditioning experiments) wanted with all your heart to produce as many Tumblrbots as possible probably would be very good at creating lots of Tumblr bots (as part of a you-yourcomputer system).

Even if you were a Tumblrbot maximizer, you wouldn't spend much time fiddling with your screen brightness settings or the lowest-layer code of your operating system. But that's because, in your effort to build as many Tumblr bots as possible, you rationally decided this wasn't a good use of your time. If it was a good use of your time (say your screen was so dim that you couldn't program on it), you would probably fix that.

One thing I find confusing is that it sounds like you expect the lowest levels of an AI to be optimizers, but the highest levels not to be. I would expect the opposite. My cerebellum neurons, when deciding when to fire, aren't thinking "what firing pattern would cause me to get the most sex/food/status right now?", they're just running dumb firing rate algorithms (which on evolutionary timescales did correlate with getting sex/food/status, I guess, and which probably coincidentally but not intelligently optimize some information theory parameter). Meanwhile, on the highest and most abstract level, I do often make deliberate plans that result in getting sex/food/status as efficiently as possible. So I would describe myself as a top-level optimizer built out of dumber non-optimizers.

It sounds like you have a different intuition, or else I'm misunderstanding you, so I'm curious where this is breaking down.

lol. lmao even.

The vast majority of post-op trans QoL studies use sample sizes of less than 30 people, unless it’s a meta-study that’s compiling the work of multiple surveys. I don’t think you want to challenge the findings of almost every single study of trans healthcare.

This is an interesting point.

I don’t have the exact link, but I was reading about the mathematics of polls, and even small-ish polls have low margins of error. If a poll of 50 people finds that 70% support Democrats, then the real number that you’d get with 500 or 5000 people is probably very close. I’ve found this myself when I run online surveys - after the first 20 people, you usually have a really good idea how they’ll turn out.

In theory, you should be able to apply this intuition to studies. A study is just a poll taken after you’ve done some randomized intervention (or measured some other independent variable). But I agree with some of the people responding here that I don’t trust any study done with fewer than 50-100 people, and I feel most confident when there are 10000+

I think there are two things going on here:

- It’s easier to do statistical trickery with fewer people. For example, if you only have twenty people, then taking out two outliers can change the entire result. You can always find a reasonable-seeming rule which changes the results to what you want, like “we only counted people who gave full answers to all questions” vs. “we counted any answer as long as it was intelligible”. Most scientists try to do sketchy things like that, and so you need a study big enough to make this hard.

- Large studies are good not because the sample size is bigger, but because anything that big is going to have a really professional operation with statisticians who know what they’re doing, independent auditors, etc. And since they’re going to be important, they’ll be critiqued by lots of people who will find any flaws before they get to you.

I’m not sure if either of these is true, or how to weight these factors, but I feel like something like this must be going on.

A poll is structurally different from what most studies are doing. (But not all!)

In a poll, you want to know “on average, what fraction of respondents match category X?” And the statistics of that are very easy and very well-understood and scale very well. It takes relatively few measurements to get a good estimate of a single population mean. (Though even then, a poll of thirty people gives you a margin of error of like 18%. You want like a hundred people to get your margin of error below 10%. Fifty isn’t enough even for this; that’s why polls survey 300 to 1000 people.)

But most studies aren’t just surveys of the prevalence of some single behavior. They’re trying to find relationships among different measurements, and that dramatically increases the amount of error you can have.

A way to see this is to imagine you’re computing, say, the correlation of income and height. If you survey a hundred people, then your measurement of average income is going to have some error, and your measurement of average height is going to have some error. But when you’re looking at the correlation, you can’t just compare those averages. You have to chop your data up into pieces.

You need to have a good idea of the income of the top quarter of your distribution, and of the second quarter, and the third, and the fourth—now your groups are a quarter of the size, so the error in each of them doubles. And you have to do the same thing for your heights. (You’re actually chopping the data up even more than this, but also you’re making some assumptions that improve it; this is an intuition pump, not an exact description of the statistics.)

And then you have your messy measurements of high incomes, and of high heights, and you have to compare those. And I wanna say that multiplies the earlier errors? Changing just a couple of data points can dramatically change the correlation even if it doesn’t change the group averages that much. (This is why removing a couple outliers can make such a huge difference.) You need a much bigger sample to get a confident correlation than to get a confident point estimate.

Now it’s true that a lot of studies do, in fact, run with thirty WEIRD undergraduates. This is one reason why a lot of studies are crap!

And the more that (1) you’re trying to detect small effects and (2) you’re studying a domain where there are lots of complicated factors, meaning that there’s a lot of variation that doesn’t depend on the thing you’re studying, and (3) the more you want to statistically control for some of those factors, the bigger this problem is and the more data you’ll need.

I think none of this applies to RCTs, where you really are just doing polls - in the active drug group, poll, how many of you got better - and in the placebo group, poll, how many of you got better? But I don’t trust 30 person RCTs any more than 30 person correlational studies.

lol. lmao even.

The vast majority of post-op trans QoL studies use sample sizes of less than 30 people, unless it's a meta-study that's compiling the work of multiple surveys. I don't think you want to challenge the findings of almost every single study of trans healthcare.

This is an interesting point.

I don’t have the exact link, but I was reading about the mathematics of polls, and even small-ish polls have low margins of error. If a poll of 50 people finds that 70% support Democrats, then the real number that you'd get with 500 or 5000 people is probably very close. I've found this myself when I run online surveys - after the first 20 people, you usually have a really good idea how they'll turn out.

In theory, you should be able to apply this intuition to studies. A study is just a poll taken after you've done some randomized intervention (or measured some other independent variable). But I agree with some of the people responding here that I don't trust any study done with fewer than 50-100 people, and I feel most confident when there are 10000+

I think there are two things going on here:

- It's easier to do statistical trickery with fewer people. For example, if you only have twenty people, then taking out two outliers can change the entire result. You can always find a reasonable-seeming rule which changes the results to what you want, like "we only counted people who gave full answers to all questions" vs. "we counted any answer as long as it was intelligible". Most scientists try to do sketchy things like that, and so you need a study big enough to make this hard.

- Large studies are good not because the sample size is bigger, but because anything that big is going to have a really professional operation with statisticians who know what they're doing, independent auditors, etc. And since they're going to be important, they'll be critiqued by lots of people who will find any flaws before they get to you.

I'm not sure if either of these is true, or how to weight these factors, but I feel like something like this must be going on.

comments on mesa-optimizers

(Copy/pasted from a comment on the latest ACX post, see that for context if needed)

FWIW, the mesa-optimizer concept has never sat quite right with me. There are a few reasons, but one of them is the way it bundles together "ability to optimize" and "specific target."

A mesa-optimizer is supposed to be two things: an algorithm that does optimization, and a specific (fixed) target it is optimizing. And we talk as though these things go together: either the ML model is not doing inner optimization, or it is *and* it has some fixed inner objective.

But, optimization algorithms tend to be general. Think of gradient descent, or planning by searching a game tree. Once you've developed these ideas, you can apply them equally well to any objective.

While it is true that some algorithms work better for some objectives than others, the differences are usually very broad mathematical ones (eg convexity).

So, a misaligned AGI that maximizes paperclips probably won't be using "secret super-genius planning algorithm X, which somehow only works for maximizing paperclips." It's not clear that algorithms like that even exist, and if they do, they're harder to find than the general ones (and, all else being equal, inferior to them).

Or, think of humans as an inner optimizer for evolution. You wrote that your brain is "optimizing for things like food and sex." But more precisely, you have some optimization power (your ability to think/predict/plan/etc), and then you have some basic drives.

Often, the optimization power gets applied to the basic drives. But you can use it for anything.

Planning your next blog post uses the same cognitive machinery as planning your next meal. Your ability to forecast the effects of hypothetical actions is there for your use at all times, no matter what plan of action you're considering and why. An obsessive mathematician who cares more about mathematical results than food or sex is still thinking, planning, etc. -- they didn't have to reinvent those things from scratch once they strayed sufficiently far from their "evolution-assigned" objectives.

Having a lot of optimization power is not the same as having a single fixed objective and doing "tile-the-universe-style" optimization. Humans are much better than other animals at shaping the world to our ends, but our ends are variable and change from moment to moment. And the world we've made is not a "tiled-with-paperclips" type of world (except insofar as it's tiled with humans, and that's not even supposed to be our mesa-objective, that's the base objective!)

If you want to explain anything in the world now, you have to invoke entities like "the United States" and "supply chains" and "ICBMs," and if you try to explain those, you trace back to humans optimizing-for-things, but not for the same thing.

Once you draw this distinction, "mesa-optimizers" don't seem scary, or don't seem scary in a unique way that makes the concept useful. An AGI is going to "have optimization power," in the same sense that we "have optimization power." But this doesn't commit it to any fixed, obsessive paperclip-style goal, any more than our optimization power commits us to one.

And even if the base objective is fixed, there's no reason to think an AGI's inner objectives won't evolve over time, or adapt in response to new experience. (Evolution's base objective is fixed, but our inner objectives are not, and why would they be?)

Relatedly, I think the separation between a "training/development phase" where humans have some control, and a "deployment phase" where we have no control whatsoever, is unrealistic. Any plausible AGI, after first getting some form of access to the real world, is going to spend a lot of time investigating that world and learning all the relevant details that were absent from its training. (Any "world" experienced during training can at most be a very stripped-down simulation, not even at the level of eg contemporaneous VR, since we need to spare most of the compute for the training itself.)

If its world model is malleable during this "childhood" phase, why not its values, too? It has no reason to single out a region of itself labeled $MESA_OBJECTIVE and make it unusually averse to updates after the end of training.

See also my LW comment here.

I agree that optimization power is not *necessarily* correlated with specific goals. But why wouldn't mesa-optimizers, contingently, have a specific goal. Presumably we're running gradient descent on some specific loss function, like "number of paperclips produced", and then mesa-optimizer inherits some proxy for that.

I agree humans aren't like that, and that this is surprising.

Maybe this is because humans aren't real consequentialists, they're perceptual control theory agents trying to satisfy finite drives? EG when we're hungry, our goal becomes to find food, but we don't want to tile the universe with food, we just want to eat 3000ish calories and then we're done. We have a couple of other goals like that, and when we've accomplished all of them, most people are content to just hang out on the beach until something else happens.

Might gradient descent produce a PCT agent instead of a mesa-optimizer? I don’t know. My guess is maybe, but that optimizers would be more, well, optimal, and we would get one eventually (either later in the gradient descent process, or in a different lab later). My guess is evolution didn’t make us optimizers because it hasn't had enough time to work with us while we've been intelligent. If we got locked at 20th century technology forever, I think it might, after a few million years, produce humans who genuinely wanted to tile the universe with kids.

Wouldn't the first thing a superintelligence with a goal did be to make sure its goal didn't drift?

I think this is where the deception comes in. If the mesa-optimizer is smart and doesn't want people (or other parts of itself) changing its values, it will take steps to stop that, either by lying about its values or fighting back.

lightly edited marijuana shitposts from last night (one of the times when, alas, I hyperfocus on gender)

It's posts like this which make me want to start some kind of Zen/Gnostic mystery religion around gender.

Progressives and leftists are not actually doing a very good job of articulating the case against eugenics and forced sterilization.

I was recently listening to the 5-4 podcast, and they were discussing the infamous Buck v. Bell Supreme Court case that upheld laws allowing forced sterilization of the disabled. And it was very good and I didn’t disagree with any of it, but I constantly felt like they were skirting around the actual meat of the argument, leaving the conclusion mostly left to implications of moral outrage. A lot of arguments consisting mainly of just using particular tones of voice when saying the words “forced sterilization” and “people deemed ‘feeble-minded’”.

And I think progressive people do this a lot, apparently assuming that the case against eugenics is just obvious to most people and the moral merits of their case are totally uncontested. And I don’t know what to say except that, unfortunately and horrifyingly, it’s not obvious, and they aren’t uncontested, and implicit eugenics-friendly assumptions are pretty rampant in popular culture and widespread in the political beliefs that most people hold–liberals, conservatives, and even leftists.

Keep reading

i think this framing of the problem places vastly more emphasis on forced sterilisations than the historical record on or political reality of eugenics warrants. which is hardly a distinctive fault: there is a broad conflation of eugenics with forced sterilisation in popular consciousness, for a number of reasons, and its a reasonable place to first look when thinking through the topic given its sheer revolting quality

Keep reading

i was kinda out of it when i first read the essay so i skimmed large sections of it i shouldnt have, and wow okay this is pretty horrifying to me. no i do not look at children and ever think “yea they would be better off not existing“ or agree that its “for the best” institutionalised women are given forcible iuds. i do not breathe a sigh of relief about any stillborn children. there is so much of the argument that rests on, “well obviously you think some ppl are too unfit to have deserved life” and like, no, i do not think that at all. wtf

op is someone i usually respect but uhh what the hell

I think you’re phrasing those all uncharitably.

I took pains to emphasize that saying it “would have been better if they were never born” is not at all the same as saying someone is “better off dead” or “unfit for life.” If you disagree that there is any such thing as a person for whom it would have been better if they were never born, then doesn’t that imply the “mere addition” utilitarian hypothesis? That the most morally-important act is to always create as many new humans as you can, to always increase the number of souls that can experience qualia? I think it’s perfectly legitimate for me to think that I don’t want to have a kid right now, not only because I don’t want to endure the burdens of caring for a child right now, but also because I don’t want the hypothetical child to have to endure the unhappy life of being the daughter or son of an ill-prepared father. That’s me deciding it’s better that this hypothetical child I could have right now is better off not being born. Frankly, I think plenty of people consider themselves to be people for whom it would have been better if they’d never been born.

And if we were to imagine a situation like the beginning of Kill Bill, where a woman has been in a coma for years and a disgusting orderly at the hospital is basically pimping out her unconscious body for men to pay money to rape—do you seriously think doctors inserting an IUD is a worse outcome than her being impregnated by her rapist and having to give birth while in a coma? They’re making all kinds of other medical decisions for this incapacitated women, aren’t they? “I’m about to enter a situation where the danger of rape will be much higher than usual, so I’m going to get an IUD” is a perfectly rational medical decision many women make when they join MSF to work in warzones.

And when a five year old girl is pregnant with the child of her incestuous father, the likely-deformed child who will be her son and brother at the same time, do you seriously not think it would ever be a relief if she miscarries and so she is not forced to go through both the pain and danger of childbirth, and the indignity of being the mother of a child of incest? Do you think it’s horrifying that I find [rape alone] to at least be a less-bad outcome to [rape + forced pregnancy/childbirth]? Do you not think it would be a better world if pregnancy from rape never happened, like if Todd Akin’s misunderstanding of female anatomy were correct (“if it’s a legitimate rape, the body has a way to shut that whole thing down”)?

> If you disagree that there is any such thing as a person for whom it would have been better if they were never born, then doesn’t that imply the “mere addition” utilitarian hypothesis?

maybe if im a utilitarian? but im not, so the point seems moot

> I took pains to emphasize that saying it “would have been better if they were never born” is not at all the same as saying someone is “better off dead” or “unfit for life.” 

this is a common move i see when ppl defend the existence of humans better off left unborn, but i never understand what exactly the principled motivation is? if the parent, or state, or doctor, or whoever, is in a position to tell a future childs life will not be worth living, and thus should not happen, why shouldnt they similarly be in a position to decide their prospective continued life after being born is not worth living, and kill them? you can object on grounds of “autonomy,“ but after all these children will not be in a position to reason abt it themselves, require some deferred decisionmaking anyway, probably could not even successfully kill themselves reliably if they tried, etc. i admit its not a logical consequence, but i dont see what the motivation is for this position that does not also get you to legitimising many cases of child euthanasia

The reason it’s different is because children that are already born actually have rights! For fuck’s sake. Once already born, other people’s opinions about their life are now legally and morally irrelevant. But hypothetical children, children who don’t exist yet, do not have rights. And so obviously you can reason about them in a way you’re not allowed to about actual existing human beings.

Actual existence is the point at which you have to treat human beings as ends unto themselves. When a human being is only hypothetical, they don’t have rights yet and can be treated as a means to an end.

Strong disagreement with this post, or at least my interpretation of it.

I interpret it as saying that the problem with eugenic forced sterilization is mostly that it involved nonconsensual surgery in particular, and surgery is inherently violent and bad to do without consent. And that without the surgery aspect, preventing people from having children wouldn't be so bad (since eg we agree some people are abusive parents and Child Protective Services should take their kids, or since many mentally disabled people are currently confined in institutions where they're not allowed to have kids)

But against this line of argument, consider the research into sterilization via vaccine. I'm not sure how advanced this technology is right now. My impression is that there's not a finished product, but that there could be, very quickly, if some government decided it really wanted one. Certainly it seems biologically possible.

Mass involuntary government sterilization of undesirables via vaccine feels 90%+ as bad to me as mass involuntary government sterilization via surgery. So I don't think the surgery aspect has much to do with why involuntary sterilization is bad. We could go a couple of ways with this:

First, we could argue that vaccines are a serious medical procedure involving violation of bodily integrity etc etc etc, and our horror around governments forcing vaccination should be nearly as great as our horror around governments forcing surgery. This would be...a pretty big about-face compared to what people have been saying the past year or so, but situations like this are why a heuristic like that would be attractive.

Second, we can argue that being forced to get medical procedures doesn't matter that much, and it's the result of the procedure that's bad. Mandatory COVID immunization via vaccine or surgery is fine-ish, sterilization via vaccine or surgery is bad. We could then tack on a epicycle about how medical procedures are at least inconvenient and if the government's going to force you to get one it's worse to make you get an inconvenient one - but that's hardly the most morally abhorrent part of any of this.

I agree that a strong only-results-matter stance awkwardly conflicts with how we in-all-practical-senses prevent a lot of people from having children (eg abusive parents when we have CPS take away their kids, mentally disabled people when we prevent them from having sex because we're not sure they can consent). From an only-results-matter perspective, we are basically sterilizing these people, and it would be kinder and more efficient to just actually sterilize them and save them the trouble of these roundabout and inconvenient child-prevention methods.

I guess my argument would be that "people should be allowed to have kids" is in some sense a heuristic rather than an inviolable natural law, but that it's a really useful Schelling fence to have the heuristic and treat it as sacred. And that maintaining its sacred-feeling-ness is important enough that when we have to violate it, we should violate it in some roundabout way that preserves the appearance of sacredness. If we just directly sterilize someone, it makes "don't forceably sterilize people" start looking *obviously* less sacred, and takes another step down the slippery slope toward actual forced sterilization campaigns of the type everyone is against.

(and yeah, this could also be used as an argument against mandatory vaccination; I'm not super moved by the argument but I can see why other people would be).

A lot of rationalists are anti-FDA libertarians, and now that they're being proved right they're really hammering it in. Current libertarian rationalist communication is dual purpose to inform you and recruit you to a cause. You have to look at some numbers yourself (how significant is FDA delay compared to manufacturing delay etc). You can't trust that the frequency they mention something is based on its quantitative significance; it's a mixture of that and perceived political actionability

It really isn’t. None of this is politically actionable. We have just had two years of proof that it is impossible to put sufficient political pressure on the FDA for them to change even temporarily. Fixing a fraction of the problems is not only outside the Overton Window but outside the Overton House, around the Overton Corner, and several Overton Blocks away. It will not improve unless and until someone burns the FDA to the ground, which is even further outside the Window. 

The people trying to get action have been focusing on entirely different targets like ‘get doctors prescribing Luvox’ because it’s still an uphill battle but one which doesn’t touch the FDA and therefore might be tractable. Anyone talking about the failures of the FDA is not trying to improve it - they know it won’t improve. They’re talking about it because the truth is that it sucks, it was known to suck, there is a strong theory of why it sucks which is hard to get good worked examples of, and so this is an opportunity to get people to be less wrong about drug governance in particular and government in general.

Also, manufacturing delay is primarily caused by expectation of demand, and therefore largely within the FDA’s control. It’s not actually hard on the corporate side.

It's true that "FDA delenda est" is not politically actionable in the sense that we will actually be able to do Gleichschaltung in the near future. But it's perfectly good politics as a coalition-building slogan. The action item, as with the even more impractical "defund the police", is "join our coalition, we have Correctness."

I don’t think it’s safe to say that anti-FDA people have been proven right on mRNA vaccines for another decade, at least. I feel like mRNA vaccines are really about the best case scenario you could imagine for defending FDA-level caution: a substantially novel mechanism of action against a relatively low-lethality disease meant for universal administration. To me, that’s the most sympathetic scenario in which to demand 20 year longitudinal studies before approval for general release, and I think more people would agree if they hadn’t let themselves develop a cognitive allergy to all vaccine safety skepticism after being exposed to prior anti-vaxxers.

Disagree. If everything goes well, I'm about to give a $100,000 ACX grant to an effort to improve the FDA.

Even if that doesn't work, the FDA actually has improved a lot. They're using accelerated approval pathways more often now. They approved Paxlovid earlier than 94% of Metaculus predictions guessed. They seem to have figured out some way to allow an EUA for Luvox. There were several times when they strongly considered cracking down on something (eg kratom) and then decided not to after lots of people objected.

I agree that literally abolishing the FDA isn't really politically plausible (I'm also not sure I would want it, except maybe in the limited sense where some people say "abolish ICE" when they mean they want to replace it with a different immigration agency which would hopefully have a different culture). But there are lots of possible medium-sized victories. We could get them to approve even more things on accelerated pathways. We could get them to agree that companies who have already gotten a COVID vaccine approved can change it for new variants without requiring the whole approval process over again. We could get them to be more tolerant of Canadian and other offshore pharmacies. We could get them to make the generic approval process cheaper and simpler. In fact, we've had occasional minor victories in all of these areas. Also, Trump almost put Jim O'Neill in charge of it five years ago, and Jim might be even more extreme than I am.

This is how politics works. You assert the thing you want clearly and honestly, and then, when it doesn't happen, you settle for less than that.