I remember Brian Wansink’s post about his grad student, how proud he was of her for torturing the data for weeks before it confessed, working late instead of having a social life. Well, she was an exchange student, so it wasn’t like she could go home to her family in the evening. Wansink thought this post would look good, but it looked like he saw academia as a cut-throad competition to publish the most outlandish claims and outdo the other, less reckless scientists.
Last week I saw this post on the StackOverflow blog: https://stackoverflow.blog/2020/06/08/how-an-interview-code-submission-that-wasnt-even-submitted-changed-our-process/.
The similarities to Wansink’s post are disturbing.
This is an insane hiring process, and the blog post about it is worse. In order to be invited to the interview, you have to write a program, but the code doesn’t matter, your workflow does, but also they don’t really tell you what they want from your code. I could write such a program in one sitting, with extensive comments in the code. Tic-tac-toe is not a difficult thing to write, and the data structures don’t need to be fancy because the state space is so small. Even without symmetry-breaking and removal of impossible board states, there are 19683 total possible configurations of black and white stones (or noughts and crosses) on a three-by-thee grid. As a cherry on top, you have a limited maximum recursion/search depth of nine!
This is a toy problem, and you can take many more shortcuts you couldn‘t take in chess, checkers, or connect four. This wouldn’t be very extensible - especially to bigger boards - but it would be fast and easily understandable, and you could still implement the move selection algorithm based on an interface that lets you swap out the handling of the board state.
The key insight, the only insight you need to understand in order to write this program is this: You don’t even need to come up with any heuristic to write that program. You can walk the whole tree every time.
Hell, this would be a fun candidate for something like “brainfucking the technical interview“, or training a neural network.
But correctness is not what the focus was on. This was some kind of weeder question, to filter out people who were not technical enough, but also to filter out people who were technical but not familiar with software engineering tools and practices like source control and unit tests. I totally get it.
This was a question to get the people who use cmake or autotools and deliver .deb package, an installer in .msi format, and a nice .pdf manual.
It was also a question that easily wasted a couple of hours and made applicants over-engineer their solutions in order to stand out. This is a toy problem after all. This is a weeder question with just enough subjectivity to reject anybody for either over-engineering or not putting enough work in.
You can solve this problem in 80 lines of code if you’re golfing slightly and skimp on comments, or 200 with comments, test cases, and a GUI. You can be super wasteful with CPU cycles, and you don’t need to use all that much RAM.
The problem is this: This problem doesn’t need any fancy software engineering, and it doesn’t even need data structures, algorithms, or big-O notation.
It’s not merely one step up from FizzBuzz, I grant you that. It’s three or four steps up from FizzBuzz. There’s recursion, there’s combinatorics, you have to at least choose a representation for the game state, even if it ultimately doesn’t matter which one.
But it’s a piss-poor target for source control and software engineering. I didn’t have to look up minimax, I just put everything in a single source file, and apart from missing a semicolon, my program worked on the first try. Then I tweaked it slightly by introducing a time preference factor, so it chooses a winning move faster. I did not look up anything. My version one was fully formed, with a simple representation, and no test cases except for one line of code that runs minimax on an empty board and prints the result. It should be a draw. If I made this into a git repository, I’d commit the initial version here, and introduce a slightly better API (maybe more OOP), GUI, more test cases for edge cases, symmetry-breaking, caching (should benchmark the shit out of this, probably not worth it), and alpha-beta pruning in separate commits. But is this actually a weeder question or am I supposed to add these flourishes?
Worth noting: This is also an especially poor target for test-first TDD with unit tests. You need to get your data structures and algorithms right before you can even start writing test cases. Test-Driven Development is appropriate for a GUI for GNU Chess, but not for a chess engine, or you’ll fall into a trap where your rigid test suite prevents you from refactoring your code.
Honestly, I’d be at a loss because I wouldn’t know how to “show my work“ as well. This is a problem I used to have in maths class, but not in the real world. In the real world, you just have to explain things so people with a certain amount of prior knowledge can understand you, so you pick a target audience like engineering undergrads or yourself in six months and explain it so either can understand what’s going on.
But showing your work is different: You must explain something to an arbitrary standard, so somebody who knows more than you, or who thinks he knows more than you, can tick off boxes. I’m sure my maths teachers must have hated it when I added a drawing helpfully labelled “geometric/graphical intuition“ to my assignments in probability, analysis, vector algebra, or linear optimisation. I know they hated it. They told me so.
So what does “commit early“ mean in this case?
Back to the StackOverflow post: They asked a guy to an interview who couldn’t complete the assignment, but dropped some buzzwords and demonstrated that he knew all the software engineering process bullshit. Yet he couldn’t be arsed to turn this rough outline into working code, and sent it back, and the people at [redacted company that is not StackOverflow] were so dazzled they invited him for an interview anyway, and scrapped the whole test.
This is exactly the wrong lesson. You can‘t just drop a few links into the design document and act like the implementation should be trivial, and then just not do it. This guy had written some skeleton code with multiple commits - The blog posts praises the clear commit messages. So the applicant is familiar with the tools and the process, but he can’t implement a recursive search, and he wonders if he should use a formulation of the algorithm that uses an additional if statement or a more concise one? Is branch prediction really a concern? Can‘t this guy be arsed to look up the algorithm on Wikipedia? It’s probably right there, written in Pascal.
The fucking blog post is over two times as long as my implementation of tic-tac-toe, and I didn’t even cheat by writing it in Prolog.
But instead, this applicant makes several commits that don’t result in any working code, drops some buzzwords about a meaningless implementation difference, and dazzles the company into an in-person interview, gets the job, but declines their offer.
Why the fuck would you consider somebody who writes several commits worth of bullshit when the actual program is 100 lines, not counting empty lines or lines with only curly braces?
And why the fuck would you write about this on your fucking blog?
And why the fuck am I so angry?
Because people like this are running StackOverflow now. They care about bullshitting people with things you just Googled but didn’t understand, rather than demonstrating your understanding through working code. If you understand, if you truly understand, if you grok, then the code should be easy. StackOverflow is run by people who don’t believe in working software over comprehensive documentation.
StackOverflow is run by people who don’t believe the answers have to be correct as long as people act professional when they give them, and that’s a problem if you are running a question-and-answer site.
Instead of listening to their moderators and the people who produce the answers, they give them dark mode.
Why would you brag about this on your blog?
Why did StackOverflow content decline?
The fish rots from the head.