Hmm very interesting oh hey why is the x axis like that ?

Ahhh okay that’s why we’re doing this whole thing

I thought the issue was with the Y-axis being stretched out to make the increase seem larger... Didn't realize the fucking X-axis is literally backwards first time seeing this

Friendly reminder that the US violent crime has decreased for 30 years while most americans delusionally thought it rose

First, yes, original graph is extremely silly. And it's important that violent crime has been decreasing for basically my entire life and people are often unaware of that.

But it's also very true that we had a huge uptick in homicide last year. I think it's still way lower than it was in, say, 2000, but a 30-50% increase in one year is both significant and bad. (It's especially bad if it keeps going.)

Now I think we didn't record an increase in violent crime overall. But first, homicide is bad even if it's not accompanied by robberies and muggings. And second homicide numbers are generally more reliable than other violent crime numbers; it's reasonably likely that other violent crime also ticked up but we can't tell as easily.

(Recording on crime in the U.S. is really bad and it's a real problem for making informed policy decisions.)

It’s even more specific.  We had a huge uptick in shooting specifically.  I wonder if the run on guns during the pandemic has created more just from some more guns -> more shootings effect. 

“capitalists don’t work, just take a cut of profits” is about monopolopy capitalism, “capitalism requires constant growth and is therefore doomed to destroy the planet” is about competitive capitalism

monopolists don’t need growth, competitive capitalists have to work

Do you have examples of monopolists that don’t need growth/to work? My understanding of e.g. US Steel and Standard Oil was that they were constantly working to improve efficiency and keep prices down, both to increase profits and ensure competitors couldn’t cut in?

Remembering that monopoly versus competition is a continuum and not a strict binary

But if you thin about, like, a legally enforced monopoly where competition is forbidden, there’s not a ton of pressure to innovate or improve. Which doesn’t mean it doesn’t happen anyway sometimes, but there’s not a lot of pressure to make it happen.

But also I like the original tension, if we leave aside the “monopoly/competitive” aspect. The more capitalists are just sitting on their asses collecting rents, the less we should be worried about capitalism driving them towards endless destructive growth. (If we were worried about that in the first place.)

I think one of the stories of the modern financial economy is the old money, which was content to collect rents, has been overthrown by new financial innovation, which is creating boom/bust cycles with destructive growth.  Almost every financial innovation seems to spin out into a boom/bust/decades to rebuild cycle.  It happened with junk bonds, housing derivatives, it’s happening with auto loan derivatives, it’s happening in insurance spaces, etc.  

I made the students in my cryptography class each pick a topic to write a paper on, and like four (out of fourteen) of them chose Bitcoin.

And now I have to read four extremely credulous papers about how amazing Bitcoin is.

I guess this is my penance for making math students write papers?

So the list of suggestions I gave them was:

I would love to read a really good paper on the Quadratic Sieve but that's probably a bit ambitious for the course I'm teaching.

Same with the lattice-based cryptography topic; we cover knapsack, the New Hope Ring-LWE algorithm, and the Gentry bootstrap of of Ring-LWE to get fully homomorphic encryption. I'd love to read a good paper on NTRU or something but it's tricky.

But it's really more about whether the paper addresses math well than the topic. I'd love to read a good paper on Bitcoin. But the papers I actually get start from "Bitcoin is a revolutionary new financial tool that will solve all the problems with modern finance" and then maybe throw in a bit of explaining what a hash function is.

They are setting themselves up for a lucrative career in consulting.  

That particular mundane variant on l’appel du vide where you don’t fixate on falling or jumping off a high place, but rather on how easy it would be to drop your keys or phone from there

This is exactly how I feel especially about standing near water. I am convinced that some water-damageable object on my person is going to wind up in there if I don’t keep at least like five feet away.

I get it a lot riding the elevator in my apartment complex, looking at that little crack between the floor of the elevator and the outside

Around long drops I do worry more about me falling, though also about my stuff.

Near bodies of water, my brain can’t convince me that me falling in will hurt me, but it’s still convinced I will, so I move on to worrying about my stuff.

It never occurred to me to worry about dropping my keys into the crack between the elevator and the fixed floor until a coworker actually did drop their keys down the elevator shaft a few years ago.  Now I always have my hand on my key pocket when I step into an elevator.  

The worst part about this GME thing is that it’s introduced a whole pile of Millennials and Zoomers to the wonders of casual day-trading. Don’t they remember how it ended the last time we went through this in the 90s?

Becoming a degenerate gambler to pwn the casinos. 

It occurs to me that one way the GME nonsense is like Occupy Wall Street is that there’s no, like, exit strategy.

One of the fundamental problems with OWS is that it didn’t actually have goals. They were all very angry at Wall Street (and, you know, fair!), and so they were going to get together and protest until—

Well, until…

And that was the problem. People went to them and said “we see your protest. What are you asking for?” And often they didn’t have an answer, beyond “we want to register our opposition to the entire capitalist system.” And that means they can never win, and there’s no answer to when they should go home.

And so inevitably the movement slowly deflated.

GME trading is kind of the same at this point. The WSB people who invested in October had a clear thesis: this company is undervalued at $10, so we will buy some shares and hold them until they reach their equilibrium value of 20 or whatever.

The WBS people who invested in early January had a clear thesis: we’re going to squeeze the Melvin Capital short position to run the stock price up. That’s a risky move, but they made it work. Their 20 USD investment is now a 200 USD investment. Good for them.

The thesis for people getting now seems to be “Wow, this is making some big Wall Street investment firms really mad. Buy more to make them madder!” And there are a few issues with this (the firms that are taking short positions now are quite happy for you to buy more!) but one of them is, when do you plan to sell?

People talk about the short squeeze, but that’s dead. And even if you believe the short squeeze is still alive, you have to have some sense of when you’d think it was dead and sell. If you don’t have that, you’re setting yourself up to get screwed eventually.

I’ve seen some posts to the effect of, if you sell now you’re betraying the revolution. You are bad and disloyal if you have an exit strategy. If you’re listening to that argument, then you’re a mark.

I haven’t, and couldn’t find it easily, but that seems really implausible to me.

It seems implausible that it would work; it’s really hard to hold the price of a security above what people think it’s worth.

It seems implausible that they could hold the line that long; each individual investor is under pressure to sell.

It seems implausible that they could coordinate that, given the messaging right now abhors the idea of exiting at all.

And it seems implausible that they could coordinate that without going to jail, because that is literally 100% exactly what the rules against “market manipulation” are for.

https://yudkowsky.medium.com/r-wallstreetbets-is-trying-something-unprecedented-in-history-and-the-medias-not-reporting-it-7ab507e4a038 is the link, since no one’s put it in the notes yet.

(Sometimes reblogging things fixes formatting)

Thanks for the link.

That’s not an awful essay, but it’s not really a good one.

The big issue with the strategy at the end (other than possible legal risk) is that it is very much not happening. People are selling GME stock constantly. The trading volume has been enormous! No one is holding the line against anything.

(And I suspect that WSB doesn’t own a non-trivial fraction of the company, but that’s just a guess.)

They managed to land a short-term short squeeze by driving the price up so high that Melvin couldn’t afford the cost of credit to maintain its position. But at this point there’s a well-diversified collection of short sellers at prevailing prices, and also a diversified collection of buyers (and both of these groups include hedge funds and institutional investors, probably to similar degrees).

But the big thing is, yeah, a distributed collection of shareholders who are all selling at different prices is nothing like a single shareholder who can hold out for huge sums of money. Nor should it be; it’s a good thing that the market isn’t cornered by one actor! (Usually we don’t like market manipulation by monopsonists, you know?)

I feel like he more or less agrees with you on what’s going on, and is just interested in a different aspect than you? The thing he cares about is that this is a really hard coordination problem people are trying to solve (possibly without realizing it which bodes ill for them), and you’re pointing out that they’re failing at it. It sounds like you’re pointing out that we can see they’re failing in a way he didn’t realize when he wrote it. But I don’t think he’s surprised by them failing, he just thinks the attempt is exciting and they did get further than he expected.

His essay was easily in the top 20% of takes I’ve read in terms of correctness. That doesn’t make it a good piece exactly.

First, because it spent a ton of time vigorously arguing that it wasn’t possible to get to the point, instead of just getting to the fucking point. Which is a common EY vice but was really grating here.

Second, I think the emphasis was really misplaced. The most important thing to communicate about the whole GameStop thing is please don’t get involved. If you buy into GameStop, you will lose a large pile of money. You shouldn’t speculate on individual stocks in general, and you specifically shouldn’t buy this stock right now.

They’re not going to solve that coordination problem he talks about, there was never any way for them to do that, and it’s a good thing they can’t; we put a lot of effort into setting up a system to prevent that, because it undermines the purpose of the system.

I read EY as rooting for the WSB short squeeze, which I think is bad on 2-3 levels.

Also misses the point that while WSB might have initiated the short squeeze, major funds almost certainly actually snapped up the majority of the stock.  The profit individual retail investors will make will be a rounding error compared to the funds that actually own the largest chunks of gamestop.  

[repeating to myself] you turn it clockwise to tighten the screw and increase dnd-type order, you turn it widdershins to to loosen the screw and increase dnd-type chaos

righty-tighty lefty-loosey doesn’t work for me because i was always like

“right/left from the perspective of a point on the top or the bottom of the circle?”

(cuz it’s a circular motion not a linear motion, obviously)

and when i asked people just got frustrated at me

I always thought of it as a right hand rule vs left hand rule thing

…but the right-hand rule is widdershins? like the positive rotational direction is counterclockwise. so it untightens.

see this is why this is confusing. just talk about clocks, everyone knows clocks.

I think “right-hand rule” might mean different things to different people? Here’s what was taught to me as “right hand rule” in (belarusian) high school:

Think about screwing IN something. Take your right hand (hence the rule). Which way do you need to rotate it so it would “naturally” go forward (the “screwing IN” motion)? “

Somehow for me (and i guess the person who taught us that, and my class) the “go forward via rotating ccw” feels unnatural?…

I still use this to this day. Like this is literally how i determine the rotation direction when i use a screwdriver: just stand there rotating my hands

Helpful answer (to most Americans): “right” is the way you’d turn a steering wheel to turn the car right.

Possibly more constructive answers: you’re moving the top to the right. Or if you think of the top as “forward”, you’re turning it to the right to tighten it.

Reason I, a mathematician, actually made a post: holy shit it is the right-hand rule. If you screw it in the direction that corresponds to “up” by the right-hand rule, it moves up. (That is, counterclockwise rotation will move towards you, and clockwise rotation will move away from you, which is the right-hand rule exactly.)

For those who don’t know what the right-hand rule means in this context, it means “point your thumb in the direction you want the thing to go; the direction your fingers curve is the direction to turn it.” At least, for right-handed threading, but that’s almost all of it.

Most useful when screwing something in, but towards your or upwards, as with a water hose to a faucet.

So, that’s the thing. I know the right-hand rule. The right-hand rule is very important in calculus. I have never heard it related to screws before.

In math the right-hand rule gives the orientation of the cross product. Point your fingers in the direction of u, then curul them in the direction of v, and your thumb will point in the direction of u×v.

(And we always draw our coordinate systems to respect this rule, so if you point your fingers toward x and curl them towards y, your thumb will point towards z.)

I had not realized until today that this describes the righty-tighty rule for screws.

Semi-related, NY used to teach the left hand rule for electric motors to highschool kids, thus creating perpetual confusion on “when do I use a left hand rule, and when do I use a right hand rule.” 

I am genuinely frightened by the housing situation in the US.

With most political crises…you can imagine that, somehow, at least in theory, someone will find the magic perfect solution that can fix everything.  Maybe the disputing sides aren’t really as far apart as they think they are, in terms of their actual desires – maybe they’re just hopped up on short-sighted culture-war – and maybe they’ll realize that eventually.  Maybe it’s an empirical dispute, and at some point we’ll all understand that one side is just right and the other side is just wrong.  Maybe it’s a dispute over fuzzy conceptual values, and eventually some of the cultural norms driving the conflict will quietly fade away.  Maybe technology will multiply the loaves and fishes and make the whole issue moot. 

But with housing, as far as I can tell, it really is a pure head-to-head conflict of irreconcilable material interests.  Houses and apartments are fantastically expensive, relative to what they “should” be in a sane market.  Millions and millions of people are un-housed or under-housed, and miserable, because of it.  Millions more have invested pretty much everything they have into owning real estate, and would be completely ruined if housing prices ever dropped substantially across the board.  Any state of affairs that satisfies one group will necessarily be catastrophic to the other.  The battlefield is critical to basically everyone’s life; no one short of the ultra-wealthy can afford not to care about this.

(Yes, yes, obviously it’s better in some locales than in others, not every place is Manhattan or San Francisco.  But, basically everywhere, it’s much worse than it should be.)

I have no useful ideas, no economic genius that can square the circle, just dread.

You can square it somewhat by distinguishing house prices and real estate prices.

Zoning a neighborhood to allow more dense development should make housing cheaper, but land more valuable; it’s not clear to me that this would reduce the value of the real estate investments that most of the homeowners there have made.

I don’t think it’s true that “basically everywhere it’s much worse than it should be.”  In the vast majority of the country, houses are very affordable, but people write those areas off for some reason.  

Starter homes in Cincinnati or Cleveland might be 70k-80k.  You can get a home in St. Louis or Kansas City for even cheaper.  Median home prices in most of the midwest are probably 150k.  In the small college town where I went to grad school, 3 bedroom-1 bathroom type houses typically cost $45-50k and it was common for grad students to buy homes and rent a room out to cover the mortgage.  

Lots of US cities are down from their peak sizes in the 50s and have lots of cheap housing stock as a result.  It’s just people don’t seem to want to move there.  

wait parts of manhattan are zoned so the buildings can't be tall?

people spend so much on rent, and building tall apartment buildings to rent to them is literally illegal?

like, whether this is the intent or not, the effect is the same as a cartel. It's just like what OPEC tries to do: to keep prices high, let's make an agreement to produce less. Except unlike OPEC's agreements, this law can actually be enforced

in general, it actually makes sense for powerful people to support regulations on their own industry, because it's functionally the same as a cartel agreement, except it can actually be enforced.

rustingbridges said:

Yeah, I mean you just have to look at the cost of rent. It's got to be fucked up somehow but I don't know the details

So I'm just going from "why does my refridgerator smell bad", to taking out some plastic container in the back, to "oh my lord jesus christ that's fucking disgusting"

In basically every major city, to first approximation, the height that buildings _are_ is the maximum height they're _allowed_ to be. The number of apartments that gets built is the maximum number that developers can convince the government to let them build. But yeah, it surprises people that in large swaths of Manhattan, it's illegal to build higher than two or three stories.

And even in places where it’s perfectly legal to build higher, there might be a neighborhood planning committee or home owners association that can still block you.   People seem determined that local owners can decide nothing can ever change. 

I should maybe clarify that sometimes it's _de jure_ legal to build higher, but it is not _de facto_ legal. If it were actually possible, in basically any major city, people would do it. Sometimes it's illegal because the zoning code explicitly says "no buildings more than three stories" or whatever. Sometimes it's illegal because you would need approvals from seven different boards and three different bodies of concerned citizens, and also build five parking spaces for each unit, and promise that each unit will have three thousand square feet per occupant, but not charge more than five hundred dollars in rent, or something. In the second case, you maybe can't point to a specific provision that says "you can't build higher or denser". But you can't.

My area is nicely streamlined in that there is only one committee and everything else is by right. 

Even still, one elderly person retired from the board due to health reasons last year and suddenly we have new apartment buildings a few stories taller than older apartment buildings.  Some of my neighbors are up in arms and talking about getting on to the board to make sure such things never happen again. 

wait parts of manhattan are zoned so the buildings can't be tall?

people spend so much on rent, and building tall apartment buildings to rent to them is literally illegal?

like, whether this is the intent or not, the effect is the same as a cartel. It's just like what OPEC tries to do: to keep prices high, let's make an agreement to produce less. Except unlike OPEC's agreements, this law can actually be enforced

in general, it actually makes sense for powerful people to support regulations on their own industry, because it's functionally the same as a cartel agreement, except it can actually be enforced.

rustingbridges said:

Yeah, I mean you just have to look at the cost of rent. It's got to be fucked up somehow but I don't know the details

So I'm just going from "why does my refridgerator smell bad", to taking out some plastic container in the back, to "oh my lord jesus christ that's fucking disgusting"

In basically every major city, to first approximation, the height that buildings _are_ is the maximum height they're _allowed_ to be. The number of apartments that gets built is the maximum number that developers can convince the government to let them build. But yeah, it surprises people that in large swaths of Manhattan, it's illegal to build higher than two or three stories.

And even in places where it’s perfectly legal to build higher, there might be a neighborhood planning committee or home owners association that can still block you.   People seem determined that local owners can decide nothing can ever change. 

A few really bad tools have risen to ubiquity in data science, and they’re an immense drag on the productivity of almost everyone in the field.

Someday someone is going to create, and then successfully promote, a serious competitor to these tools, and I will be so happy.  It won’t actually be that hard, because the tools are so bad.

The tools I’m thinking of are

- Jupyter Notebook (which is such an inherently bad idea it feels like a mean joke)

- Pandas (which is much less actively harmful than Jupyter Notebook, but is a very cumbersome and confusing way of doing some very basic and foundational tasks)

- “Jupyter + Pandas,” the synergetic combination of these two tools (pandas clearly expects you to use Jupyter so you can see its HTML output) that has data science in a tighter grip than either bad tool could manage on its own

—-

Everyone knows Jupyter Notebook is bad.  People talk about it with amused shame, like it’s candy or an addictive drug.  Here’s Joel Grus ranting about it for an hour, for example.

What is Jupyter Notebook?  It’s basically an interactive interpreter that looks like an IDE.  You can write long blocks of code at once easily, and you can go back and edit/delete/rewrite your code … and all the while you are in the same interpreter session, with the same global state, which was produced by code you ran earlier and then rewrote or deleted.

The state of the session is the context in which your code executes, yet it quickly diverges from anything your code could ever have produced!  Indeed, any Jupyter Notebook quickly develops a mysterious state which is impossible for anyone to reproduce perfectly.  A huge fraction of all code written by data scientists is first executed inside one of these phantom, inexplicable states.

Yet we develop our code in this nightmare joke IDE anyway, because nothing else has the same (fairly simple, but essential) visualization tools.  And because we like doing computations that take a while, and doing all of them in a single, convoluted, stateful process running alongside development is a simple (albeit horrible) way to avoid doing them more than once.

Some people embrace this tool to an extent I do not understand, seeing some untapped potential in it.  For example, Google made Colaboratory/Colab, and Netflix built some vast complex system around it so they could … so they could do … honestly, I watched that whole video and I’m still not sure.

—-

Pandas is … okay, I guess, it’s just very un-Pythonic.  Python is great!  That’s why these ubiquitous add-ons to python are so frustrating.

Python likes having one conceptually simple way to do each things.  Pandas has a huge, inconsistent API with 5 different ways to do everything.

Quick, do you want `pd.read_sql` or `pd.read_sql_query` or `pd.read_sql_table`?  Do you want `is_na` or `is_null`?  `join` and `merge` do the overlapping things with different argument syntax.  There is no concept of a field/column with nullable type, so the moment you add a null value to a typed field, its type degrades to “object.”  Everything is fuzzy and squishy and changes from version to version.

But it prints the outputs of SQL queries in a pretty way that everyone loves.  … except only if you’re in a Jupyter Notebook.  You’re in a Jupyter Notebook, right?  You’re using pandas, right?  Right???

Interesting, this is the first criticism of Jupyter I’ve read.

It feels to me like a reimplementation of the Mathematica interface, which I’m familiar with, so it made immediate sense—but of course Mathematica has all the same issues you identified, and the only safe way to run it is to nuke the kernel and start over from scratch.

When I was teaching Calculus with a Mathematica lab, one of the things I had to tell everyone on Day 1 was that it was totally possible to get Mathematica into an invisible internal state that would cause some bits of code to just Not Work, and sometimes I would have no way of recovering from them without just rebooting the kernel.

And these are problems you can generate in like thirty minutes of awkward flailing by freshmen calling only one-line commands. (The most common one was trying to define a function with the syntax f[x] = x^2 rather than f[x_]:=x^2; this defines an array object over f, which doesn’t get undefined when you fix the syntax, and takes precedence over the function interpretation in some contexts.)

But obviously if you can screw the state up that badly with thirty minutes of playing, you can get it more screwed up in a big program. If you’re consistently careful you can avoid screwing up the state, but a tool with the property “you have to always be very careful to avoid causing critical invisible problems for yourself” is in retrospect obviously not a great tool.

I think jupyter is great at ad-hoc data exploration, it’s basically mathematica.  And mathematica is basically built around the idea of lab notebooks.  So for maybe 50-60% of the work I do, it’s the perfect tool for the job, which is replicate a lab-notebook style of thinking.   I can iterate ad-hoc analysis quickly, fix plots,etc.  Pandas seems built for the same reason “wouldn’t it be great if we could ad-hoc manipulate data in python the same way we do in R.”   The problem becomes ad-hoc projects sometimes become the basis of new analysis pipelines and if you don’t get them out of jupyter right away you are screwing yourself. 

It’s also way too easy to abuse.  I had a summer intern who left me with a jupyter notebook that requires you to run code out of order to work at all.  You can imagine how it happened, he made a cell it didn’t run because he forgot to define something, so then he made a new cell below it and ran that and tried again.  

I defer to both of your experience, but the best respones I’ve seen to that suggestion is that in actual lab notebooks, you can’t go back and edit the page that said what you did in the last experiment. Jupyter doesn’t (naturally) keep a record of what you tried; it keeps a record of whatever the final version of everything was.

(Same with Mathematica et al., obviously.)

Yeah, that is fair.  The way I do my work flow works well, but it maybe isn’t the actual workflow that a notebook style system would lead a new user to.  And also a lot of my habits are workarounds to avoid natural problems the notebooks create.  

It would be nice to have the markdown comment/code mix + inline plotting tied together in an environment that forced you to go more linearly top to bottom.    

A few really bad tools have risen to ubiquity in data science, and they’re an immense drag on the productivity of almost everyone in the field.

Someday someone is going to create, and then successfully promote, a serious competitor to these tools, and I will be so happy.  It won’t actually be that hard, because the tools are so bad.

The tools I’m thinking of are

- Jupyter Notebook (which is such an inherently bad idea it feels like a mean joke)

- Pandas (which is much less actively harmful than Jupyter Notebook, but is a very cumbersome and confusing way of doing some very basic and foundational tasks)

- “Jupyter + Pandas,” the synergetic combination of these two tools (pandas clearly expects you to use Jupyter so you can see its HTML output) that has data science in a tighter grip than either bad tool could manage on its own

—-

Everyone knows Jupyter Notebook is bad.  People talk about it with amused shame, like it’s candy or an addictive drug.  Here’s Joel Grus ranting about it for an hour, for example.

What is Jupyter Notebook?  It’s basically an interactive interpreter that looks like an IDE.  You can write long blocks of code at once easily, and you can go back and edit/delete/rewrite your code … and all the while you are in the same interpreter session, with the same global state, which was produced by code you ran earlier and then rewrote or deleted.

The state of the session is the context in which your code executes, yet it quickly diverges from anything your code could ever have produced!  Indeed, any Jupyter Notebook quickly develops a mysterious state which is impossible for anyone to reproduce perfectly.  A huge fraction of all code written by data scientists is first executed inside one of these phantom, inexplicable states.

Yet we develop our code in this nightmare joke IDE anyway, because nothing else has the same (fairly simple, but essential) visualization tools.  And because we like doing computations that take a while, and doing all of them in a single, convoluted, stateful process running alongside development is a simple (albeit horrible) way to avoid doing them more than once.

Some people embrace this tool to an extent I do not understand, seeing some untapped potential in it.  For example, Google made Colaboratory/Colab, and Netflix built some vast complex system around it so they could … so they could do … honestly, I watched that whole video and I’m still not sure.

—-

Pandas is … okay, I guess, it’s just very un-Pythonic.  Python is great!  That’s why these ubiquitous add-ons to python are so frustrating.

Python likes having one conceptually simple way to do each things.  Pandas has a huge, inconsistent API with 5 different ways to do everything.

Quick, do you want `pd.read_sql` or `pd.read_sql_query` or `pd.read_sql_table`?  Do you want `is_na` or `is_null`?  `join` and `merge` do the overlapping things with different argument syntax.  There is no concept of a field/column with nullable type, so the moment you add a null value to a typed field, its type degrades to “object.”  Everything is fuzzy and squishy and changes from version to version.

But it prints the outputs of SQL queries in a pretty way that everyone loves.  … except only if you’re in a Jupyter Notebook.  You’re in a Jupyter Notebook, right?  You’re using pandas, right?  Right???

Interesting, this is the first criticism of Jupyter I’ve read.

It feels to me like a reimplementation of the Mathematica interface, which I’m familiar with, so it made immediate sense—but of course Mathematica has all the same issues you identified, and the only safe way to run it is to nuke the kernel and start over from scratch.

When I was teaching Calculus with a Mathematica lab, one of the things I had to tell everyone on Day 1 was that it was totally possible to get Mathematica into an invisible internal state that would cause some bits of code to just Not Work, and sometimes I would have no way of recovering from them without just rebooting the kernel.

And these are problems you can generate in like thirty minutes of awkward flailing by freshmen calling only one-line commands. (The most common one was trying to define a function with the syntax f[x] = x^2 rather than f[x_]:=x^2; this defines an array object over f, which doesn’t get undefined when you fix the syntax, and takes precedence over the function interpretation in some contexts.)

But obviously if you can screw the state up that badly with thirty minutes of playing, you can get it more screwed up in a big program. If you’re consistently careful you can avoid screwing up the state, but a tool with the property “you have to always be very careful to avoid causing critical invisible problems for yourself” is in retrospect obviously not a great tool.

I think jupyter is great at ad-hoc data exploration, it’s basically mathematica.  And mathematica is basically built around the idea of lab notebooks.  So for maybe 50-60% of the work I do, it’s the perfect tool for the job, which is replicate a lab-notebook style of thinking.   I can iterate ad-hoc analysis quickly, fix plots,etc.  Pandas seems built for the same reason “wouldn’t it be great if we could ad-hoc manipulate data in python the same way we do in R.”   The problem becomes ad-hoc projects sometimes become the basis of new analysis pipelines and if you don’t get them out of jupyter right away you are screwing yourself. 

It’s also way too easy to abuse.  I had a summer intern who left me with a jupyter notebook that requires you to run code out of order to work at all.  You can imagine how it happened, he made a cell it didn’t run because he forgot to define something, so then he made a new cell below it and ran that and tried again.  

Okay, so Comcast really is as bad as everyone says, and I’m looking at jumping ship to Verizon Fios (which is the same price and maybe I should have just done that to begin with).

Because I spent so much of my time these days talking to people who are not physically present, I was debating getting an actual phone line—very amillenial of me, I know. But there are a couple of questions that I just cannot get answers out of the internet on, because too many people are trying to sell me things

I had a VOIP phone from a cable company for many years, it was no better than a google voice call.  In particular, when I was having internet issues, the same issues plagued the voip phone.  

I expect a traditional landline is a better experience, but I haven’t had one since college. 

so basically i dont really get index funds because i thought the insight behind passive investing is “the current price is your best estimate of the value” and therefore “you don’t really need to like make stock picks and trade based on news, you can just buy stock in bunch of arbitrary companies”

so…. the implication is that you don’t really need… fund managers? Like you’re saying their expertise is not really valuable.

So….. then you invest in an index fund… and you’re still giving a commission to a fund manager?

if you’re going to invest $10,000 you can easily just like randomly select a hundred companies and invest on average 100 in each, right? or…. invest in 500. But instead some people invest in a S&P 500 index fund and pay a small fee. Isn’t the whole point that there’s nothing special about heir list… so what are you paying them for??

You’re paying them for actually executing the trades.

Like, first, it costs (a small amount of) money to execute trades on the stock market. So a bit goes there.

But also, if you’re investing in the S&P500, you want to do a certain amount of balancing: stocks join the 500, they leave the 500, sometimes you want to reweight from one of th stocks to another (like if one of the stocks does really well you don’t want it to now be 20% of your portfolio).

You could do all that yourself, and handle the shitload of paperwork and tax statements involved, or you could pay someone else to do that for you.

Okay, I have a different question - why aren’t there more plagiarism funds?

A non-index mutual fund is supposed to have higher fees because smart people chose the stocks according to some set of characteristics. So maybe Goldman Sachs hires a dozen finance PhDs to pick stocks that will still go up in a slow economy or something. Then they sell it to you and charge extra to pay those dozen PhDs’s salaries.

Plausibly a retail investor could just look at which stocks are in the fund and buy those stocks and not pay the fees, but as you mention this would be annoying.

But why doesn’t some other company sell the Goldman Sachs Plagiarism Fund, which will always buy and sell exactly the same stocks Goldman Sachs does, one second after Goldman Sachs buys or sells them? And charge a fee which is less than a dozen PhDs’ salaries but still high enough to make a profit? Are good baskets of stocks protected intellectual property, or is there something else preventing this?

I’m pretty sure the answer is that Goldman Sachs doesn’t have to tell you what it’s buying or selling.

I mean, it has to tell you eventually, but that’s like once a quarter. And if they’re doing anything interesting, they definitely don’t want to tell you until then.

Before explaining the thing, make sure it is a thing. There are 2800 companies listed on the NYSE; the US has a little under 10000 mutual funds. Allowing that many of those are passive, still, are they all going to have a unique and original strategy?

That said, I suppose that “following the geniuses at Goldman Sachs, but cheaper and with a quarterly delay” is not the sort of thing people like to put in their advertising materials.

A lot of them aren’t executing unique strategies because they’re executing very blunt and obvious strategies: there are a ton of “The S&P 500” or “A mix of S&P 500 and bonds, with the mix shifting over time to target a 2050 retirement date” type funds.

Really, the “geniunses at Goldman Sachs” stuff tends to be less the actively managed mutual funds and more the hedge fund sort of stuff. But if you’re doing any of that, you’re probably trying to time things a lot more precisely than “in the same quarter”.

Also, we know from 2008 and from the occasional high frequency flash crash that there are a lot of funds following the same strategies (leading to occasional run-away problems when correlated movements feed off each other).  

I guess like... I’ve spent more time on hold with AT&T this month dealing with an internet outage then I’ve spent dealing with the DMV in my entire life.

In terms of unpleasant bureaucracies I’ve had to deal with in my life, in order of time actually spent (in my personal, non-professional life)

1. insurance companies - easily hundreds of hours 2. hospitals - several dozen hours 3. cable and phone companies - 10s of hours. 4. other utility companies, 10s of hours.   Have people complaining about DMVs really never had to deal with an insurance company?  Maybe I’ve had an unlucky life, but like... one fairly minor car accident with a commercial vehicle on the other end (and at fault), and I’ve got a lien from my health insurance company on me in case I get a payout from either the other driver’s car insurance or his company’s general liability policy once they decide which one should pay. 

I’d hate to imagine how bad dealing with the bureaucracy would be if someone had been seriously injured.  

Hey, I know several people in my orbit have opinions on care/activities for autistic children. I have a friend who’s doing stopgap care for a nonverbal autistic child and is looking for ways to engage them productively. Optimum thing is games for the child to play with their sibling that are handlable for nonverbal autistic children; apparently they’ll play catching games but we want more than one activity for them to do.

Does anyone have pointers to good resources for this sort of thing? Names for good treatment modalities or else just sources of resources or activities etc for nonverbal autistic kids would be great. The only group I can name off the top of my head is ABA and I have been assured that I don’t want to direct people there. :) But who has good resources to provide for this?

Thanks! And feel free to pass this on to people who might know more.

How old is the child?  

One game/activity is to play follow the leader with blocks- building something and having them try to copy it.  

They’re nine.

Thanks, I’ll pass that suggestion along.

My experience is limited to one person, but around that age the child I’m thinking of really liked making rubbings and drawing.  He was shown how to do a rubbing with a plaque on a park bunch and started doing it everywhere.  Coins, stamped metal of any kind, brick textures,etc.  

We’d also play a different follow-the-leader game taking turns adding to a drawing.  It mostly went ok, but a few times the child got frustrated with it and melted down. 

Hey, I know several people in my orbit have opinions on care/activities for autistic children. I have a friend who’s doing stopgap care for a nonverbal autistic child and is looking for ways to engage them productively. Optimum thing is games for the child to play with their sibling that are handlable for nonverbal autistic children; apparently they’ll play catching games but we want more than one activity for them to do.

Does anyone have pointers to good resources for this sort of thing? Names for good treatment modalities or else just sources of resources or activities etc for nonverbal autistic kids would be great. The only group I can name off the top of my head is ABA and I have been assured that I don’t want to direct people there. :) But who has good resources to provide for this?

Thanks! And feel free to pass this on to people who might know more.

How old is the child?  

One game/activity is to play follow the leader with blocks- building something and having them try to copy it.