Many hard LeetCode problems are easy constraint problems
(buttondown.com)484 points by mpweiher 16 hours ago
484 points by mpweiher 16 hours ago
The LC interviews are like testing people how fast they can run 100m after practice, while the real job is a slow arduous never ending jog with multiple detours and stops along the way.
But yeah that's the game you have to play now if you want the top $$$ at one of the SMEGMA companies.
I wrote (for example) my 2D game engine from scratch (3rd party libs excluded)
https://github.com/ensisoft/detonator
but would not be able to pass a LC type interview that requires multiple LC hard solutions and a couple of backflips on top. But that's fine, I've accepted that.
>The LC interviews are like testing people how fast they can run 100m after practice
Ah, but, the road to becoming good at Leetcode/100m sprint is:
>a slow arduous never ending jog with multiple detours and stops along the way
Hence Leetcode is a reasonably good test for the job. If it didn't actually work, it would've been discarded by companies long ago.
Barring a few core library teams, companies don't really care if you're any good at algorithms. They care if you can learn something well enough to become world-class competitive. If you can show that you can become excellent at one thing, there's a good chance you can become excellent at another thing.
That's basically also the reason that many Law and Med programs don't care what your major in undergrad was, just that you had a very high GPA in whatever you studied. A decent number of Music majors become MDs, for example.
But why stop there? Why not test candidates with problems they have never seen before? Or problems similar to the problems of the organization hiring? Leetcode mostly relies on memorizing patterns with a shallow understanding but shows the candidates have a gaming ability. Does that imply quality in any way? Some people argue that willing to study for leetcode shows some virtue. I very much disagree with that.
> If it didn't actually work, it would've been discarded by companies long ago
You're assuming that something else works better. Imagine if we were in a world where all interviewing techniques had a ton of false positives and negatives without a clear best choice. Do you expect that companies would just give up, and not hire at all, or would they pick based on other factors (e.g. minimizing the amount of effort needed on the company side to do the interviews)? Assuming you accept the premise that companies would still be trying to hire in that situation, how can you tell the difference between the world we're in now and that (maybe not-so) hypothetical one?
I never made any claims about optimality. It works (for whatever reason) hence companies continue to use it
If it didn't work, these companies wouldn't be able to function at all.
It must be the case that it works better than running a RNG on everyone who applied.
Does it mean some genius software engineer who wrote a fundamental part of the Linux kernel but never learned about Minimum Spanning Trees got filtered out? Probably. But it's okay. That guy would've been a pain in the ass anyway.
> If it didn't actually work, it would've been discarded by companies long ago.
This that I've singled out above is a very confident statement, considering that inertia in large companies is a byword at this point. Further, "work" could conceivably mean many things in this context, from "per se narrows our massive applicant pool" to "selects for factor X," X being clear only to certain management in certain sectors. Regardless, I agree with those who find it obvious that LC does not ensure a job fit for almost any real-world job.
> Hence Leetcode is a reasonably good test for the job. If it didn't actually work, it would've been discarded by companies long ago.
I see it differently. I wouldn't say it's reasonably good, I'd say it's a terrible metric that's very tenuously correlated with on the job success, but most of the other metrics for evaluating fresh grads are even worse. In the land of the blind the one eyed man is king.
> If you can show that you can become excellent at one thing, there's a good chance you can become excellent at another thing.
Eh. As someone who did tech and then medicine, a lot great doctors would make terrible software engineers and vice versa. Some things, like work ethic and organization, are going to increase your odds of success at nearly any task, but there's plenty other skills that are not nearly as transferable. For example, being good at memorizing long lists of obscure facts is a great skill for a doctor, not so much for a software engineer. Strong spatial reasoning is helpful for a software developer specializing in algorithms, but largely useless for, say, an oncologist.
> Hence Leetcode is a reasonably good test for the job. If it didn't actually work, it would've been discarded by companies long ago.
This is an appeal to tradition and a form of survivorship bias. Many successful companies have ditched LeetCode and have found other ways to effectively hire.
> If you can show that you can become excellent at one thing, there's a good chance you can become excellent at another thing.
My company uses LeetCode. All I want is sane interfaces and good documentation. It is far more likely to get something clever, broken and poorly documented than something "excellent", so something is missing for this correlation.
5 years ago you'd have a project like that, talk to someone at a company for like 30m-1hr about it, and then get an offer.
Did you mean to type 25? 5 years ago LC challenge were as, if not more, prevalent than they are today. And a single interview for a job is not something I have seen ever after 15 years in the space (and a bunch of successful OSS projects I can showcase).
I actually have the feeling it’s not as hardcore as it used to be on average. E.g. OpenAI doesn’t have a straight up LC interview even though they probably are the most sought after company. Google and MS and others still do it, but it feel like it has less weight in the final feedback than it did before. Most en-vogue startup have also ditched it for real world coding excercices.
Probably due to the fact that LC has been thoroughly gamed and is even less a useful signal than it was before.
Of course some still do, like Anthropic were you have to have a perfect score to 4 leetcode questions, automatically judged with no human contact, the worst kind of interview.
> 5 years ago you'd have a project like that, talk to someone at a company for like 30m-1hr about it, and then get an offer.
Based on my own experiences, that was true 25 years ago. 20 years ago, coding puzzles were now a standard part of interviewing, but it was pretty lightweight. 5 years ago (covid!) everything was leet-code to get to the interview stage.
And nowadays people are blatantly using AI to answer questions like this (https://www.finalroundai.com/coding-copilot). Even trying to stumble through design questions using AI
100%. I just went through an interview process where I absolutely killed the assignment (had the best one they'd seen), had positive signal/feedback from multiple engineers, CEO liked me a lot etc, only to get sunk by a CTO who thought it would be cool to give me a surprise live test because of "vibe coding paranoia". 11 weeks in the process, didn't get the role. Beyond fucking stupid.
This was the demo/take-home (for https://monumental.co): https://github.com/rublev/monumental
It's funny because this repo really does seem vibe-coded. Obviously I have no reason not to believe you, but man! All those emojis in the install shell script - I've never seen anyone other than an AI do that :) Maybe you're the coder that the AI companies trained their AI on.
Sorry about the job interview. That sucks.
There's even a rocket emoji in server console.logs... There are memes with ChatGPT and rocket emojis as a sign of AI use. The whole repo looks super vibe-coded, emojis, abundance of redundant comments, all in perfect English and grammar, and the readme also has that "chatty" feel to it.
I'm not saying that using AI for take-home assignments is bad/unethical overall, but you need to be honest about it. If he was lying to them about not using any AI assistance to write all those emojis and folder structure map in the repo, then the CTO had a good nose and rightfully caught him.
I used AI for the Docker setup which I've already done before. I'm not wasting time on that. Yeah you can vibe code basic backend and frontend and whatnot, but you're not going to vibe code your way to a full inverse kinematics solution.
I'm not a math/university educated guy so this was truly "from the ground up" for me despite the math being simple. I was quite proud of that.
Hah I feel you there. Around 2 years ago I did a take home assignment for a hiring manager (scientist) for Merck. The part B of the assignment was to decode binary data and there were 3 challenges: easy, medium and hard.
I spent around 40 hours of time and during my second interview, the manager didn't like my answer about how I would design the UI so he quickly wished me luck and ended the call. The first interview went really well.
For a couple of months, I kept asking the recruiter if anyone successfully solved the coding challenge and he said nobody did except me.
Out of respect, I posted the challenge and the solution on my github after waiting one year.
Part 2 is the challenging part; it's mostly a problem solving thing and less of a coding problem: https://github.com/jonnycoder1/merck_coding_challenge
Part 2 is the challenging part; it's mostly a problem solving thing and less of a coding problem
That doesn't look too challenging for anyone who has experience in low-level programming, embedded systems, and reverse engineering. In fact for me it'd be far easier than part 1, as I've done plenty of work similar to the latter, but not the former.
That sucks so hard man, very disrespectful. We should team up and start out own company. I tried checking out your repo but this stuff is several stops past my station lol.
A surprise live test is absolutely the wrong approach for validating whether someone's done the work. IMO the correct approach is to go through the existing code with the applicant and have them explain how it works. Someone who used AI to build it (or in the past had someone else build it for them) wouldn't be able to do a deep dive into the code.
We did go into the assignment after I gently bowed out of the goofy live test. The CTO seemed uninterested & unfamiliar with it after returning from a 3 week vacation during the whole process. I waited. Was happy to run him through it all. Talked about how to extend this to a real-world scenario and all that, which I did fantastically well at.
I feel your pain. This isn't a question about AI or not. It's about if you can do the work and do it well. This kind of nonsense happened before AI. If you can't win the game of Jeapordy you don't get the job which has nothing to do with being a Jeapordy contestant!
I put absolutely every egg into that basket. The prospect of working in Europe (where I planned to return to eventually) working on cool robot stuff was enticing.
The fucking CTO thought I vibe-coded it and dismissed me. Shout-out to the hiring manager though, he was real.
That is an insane amount of work for a job application. Were you compensated for it at all?
The other comments here note that, and the author even stated it directly, that it was vibe-coded.
No. Should I invoice them? I'm still livid about it. The kicker is the position pays a max of 60-120k euros, the maximum being what I made 5 years ago.
Yes. I put a ton of work into it. I had about 60 pages worth of notes. On inverse kinematics, FABRIK, cyclic algorithms used in robotics, A*/RRT for real-world scenarios etc. I was super prepared. Talked to the CEO for about two hours. Took notes on all videos I can find of team members on youtube and their company.
Luckily the hiring manager called me back and levelled with me, nobody kept him in the loop and he felt terrible about it.
Some stupid contrived dumbed down version of this crane demo was used for the live test where I had to build some telemetry crap. Nerves took over, mind blanked.
Here's the take-home assignment requirements btw: https://i.imgur.com/HGL5g8t.png.
Here's the live assignment requirements: [1] https://i.imgur.com/aaiy7QR.png & [2] https://i.imgur.com/aaiy7QR.png.
At this rate I'm probably going to starve to death before I get a job. Should I write a blog post about my last 2 years of experiences? They are comically bad.
This was for monumental.co - found them in the HN who's hiring threads.
60k-120k euros. The upper 20k probably being entirely inaccessible so in reality probably like 70-100k euros.
Its not really memorizing solutions. Yes you can get quite far by doing so but follow ups will trip people up. However if you have memorized it and can answer follow ups, I dont see a problem with Leetcode style problems. Problem solving is about pattern matching and the more patterns you know and can match against, the better your ability to solve problems.
Its a learnable skill and better to pick it up now. Personally I've solved Leetcode style problems in interviews which I hadnt seen before and some of them were dynamic programming problems.
These days its a highly learnable skill since GPT can solve many of the problems, while also coming up with very good explanations of the solution. Better to pick it up than not.
It is and isn't. I'd argue it's not memorizing exact solutions(think copy paste) but memorizing fastest algos to accomplish X.
And some people might say well, you should know that anyways. The problem for me is, and I'm not speaking for every company of course, you never really use a lot of this stuff in most run of the mill jobs. So of course you forget it, then have to study again pre interview.
Problem solving is the best way to think of it, but it's awkward for me(and probably others) to spend minutes thinking, feeling pressured as someone just stares at you. And that's where memorizing the hows of typical problems helps.
That said, I just stopped doing them altogether. I'd passed a few doing the 'memorizing' described above, only to start and realize it wasn't at all irrelevant to the work we were actually doing. In that way I guess it's a bit of a two way filter now.
I’d say it’s not even problem solving and it’s more pattern recognition.
I actually love LC and have been doing a problem a week for years. Basically I give myself 30 minutes and see what I can do. It’s my equivalent to the Sunday crossword. After awhile the signals and patterns became obvious, to me anyway.
I also love puzzlerush at chess.com. In chess puzzles there are patterns and themes. I can easily solve a 1600 rated problem in under 3 seconds for a chess position I’ve never seen before not because I solve the position by searching some move tree in my mind, I just recognize and apply the pattern. (It also makes it easier to trick the player when rushing but even the tricks have patterns :)
That said, in our group we will definitely have one person ask the candidate a LC style question. It will probably be me asking and I usually just make it up on the spot based on the resume. I think it’s more fun when neither one of us know the answer. Algorithm development, especially on graphs, is a critical part of the job so it’s important to demonstrate competency there.
Software engineering is a hugely diverse field now. Saying you’re a programmer is kinda like saying you’re an artist. It does give some information but you still don’t really know what skill set that person uses day to day.
The only part of memorizing fastest algorithm the vast majority needs is whatever name that goes by in your library. Generic reusable code works very well in almost any language for algorithms.
Even if you are an exception either you are writing the library meaning you write that algorithm once for the hundreds of other users, or the algorithm was written once (long ago) and you are just spending months with a profiler trying to squeeze out a few more CPU cycles of optimization.
There are more algorithms than anyone can memorize that are not in your library, but either it is good enough to use a similar one that already is your library, or you will build it once and once again it works so you never go back to it.
Which is to say memorizing how to implement an algorithm is a negative: it means you don't know how to write/use generic reusable code. This lack is costing your company hundreds of thousands of dollars.
> memorizing fastest algos
I don't think most LC problems require you to do that. Actually most of them I've seen only require basic concepts taught in Introduction to Algorithms like shortest path, dynamic programming, binary search, etc. I think the only reason LC problems stress people out is time limit.
I've never seen a leetcode problem that requires you to know how to hand code an ever so slightly exotic algorithm / data structure like Fibonacci heap or Strassen matrix multiplication. The benefit of these "fastest algos" is too small to be measured by LC's automatic system anyway. Has that changed?
My personal issue with LC is that it has a very narrow view of what "fast" programs look like, like most competitive programming problem sets. In real world fast programs are fast usually because we distribute the workload across machines, across GPU and CPU, have cache-friendly memory alignment or sometimes just design clever UI tricks that make slow parts less noticeable.
> you never really use a lot of this stuff in most run of the mill jobs. So of course you forget it, then have to study again pre interview.
I'm wondering how software devs explain this to themselves. What they train for vs what they actually do at their jobs differ more and more with time. And this constant cycle of forgetting and re-learning sounds like a nightmare. Perhaps people burn out not because of their jobs but the system they ended up in.
I'm fine with that in an interview... I'm not fine with that, in a literally AI graded assignment where you cannot ask clarifying questions. In those cases, if you don't have a memorized answer a lot of times I cannot always grasp the question at hand.
I've been at this for 30+ years now, I've built systems that handle millions of users and have a pretty good grasp at a lot of problem domains. I spent about a decade in aerospace/elearning and had to pick up new stuff and reason with it all the time. My issue is specifically with automated leetcode pre-interview screening, as well as the gamified sites themselves.
I'd say that learning to solve tough LeetCode problems has very little (if not precisely zero) value in terms of you as a programmer learning to do something useful. You will extremely rarely need to solve these type of tougher select-the-most efficient-algorithm problems in most real-world S/W dev jobs, and nowadays if you do then just as AI.
Of course you may need to pass an interview LeetCode test, in which case you may want to hold your nose and put in the grind to get good at them, but IMO it's really not saying anything good about the kind of company that thinks this is a good candidate filter (especially for more experienced ones), since you'd have to be stupid not to use AI if actually tasked with needing to solve something like this on the job.
If a position needs low-level from-scratch code so performance-critical, and needs it so quickly that the developer must recall all of this stuff from memory, any candidate likely wouldn’t be asked to give a technical interview, let alone some gotcha test.
That was because the parent complained about not having good write ups. You can use GPT which has already been trained on publicly available solutions to generate a very good explanation. Like a coaching buddy. Keeping in mind there are paid solutions that charge 15k USD for this type of thing, being able to upskill at just 20bucks a month is an absolute steal.
Been in software development for 30 years. I have no idea what "Leetcode" is. As far as I know I've never been interviewed with "Leetcode", and it seems like I should be happy about that.
And when someone uses "leet" when talking about computing, I know that they aren't "elite" at all and it's generally a red flag for me.
Leetcode with no prep is a pretty decent coding skill test
The problem is that it is too amenable to prep
You can move your score like 2stddev with practice, which makes the test almost useless in many cases
On good tests, your score doesn't change much with practice, so the system is less vulnerable to Goodharting and people don't waste/spend a bunch of time gaming it
I think LC is used mostly as a metric of how much tolerance you have for BS and unpaid work: If you are willing to put unpaid time to prepare for something with realistically zero relevance with the day-to-day duties of the position, then you are ripe enough to be squeezed out.
> On good tests, your score doesn't change much with practice, so the system is less vulnerable to Goodharting and people don't waste/spend a bunch of time gaming it
This framing of the problem is deeply troubling to me. A good test is one that evaluates candidates on the tasks that they will do at the workplace and preferably connects those tasks to positive business outcomes.
If a candidate's performance improves with practice, then so what? The only thing we should care about is that the interview performance reflects well on how the candidate will do within the company.
Skill is not a univariate quantity that doesn't change with time. Also it's susceptible to other confounding variables which negatively impact performance. It doesn't matter if you hire the smartest devs. If the social environment and quality of management is poor, then the work performance will be poor as well.
leetcode just shows why interviews are broken. As a former senior dev (retired now, thanks to almost dying) I can tell you that the ability to write code is like 5% of the job. Every interview I've ever attended has wasted gazillions of dollars and has robbed the company of 10X that amount.
Until companies can focus on things like problem solving, brainstorming, working as a team, etc. the situation won't improve. If I am wrong, why is it that the vast majority of my senior dev and dev management career involved the things I just mentioned?
(I had to leave the field, sadly, due to disability)
Oh and HR needs to stop using software to filter. Maybe ask for ID or something, however, the filters are flagging everyone and the software is sinking the ship, with you all with it.
> My biggest problem with leetcode type questions is that you can't ask clarifying questions.
What is there to clarify? Leetcode-type questions are usually clear, much clearer than in real life projects. You know the exact format of the input, the output, the range for each value, and there are often examples in addition to the question. What is expected is clear: given the provided example inputs, give the provided example outputs, but generalized to cover all cases of the problem statement. The boilerplate is usually provided.
One may argue that it is one of the reasons why leetcode-style questions are unrealistic, they too well specified compared to real life problems that are often incomplete or even wrong and require you to fill-in the gaps. Also, in real life, you may not always get to ask for clarification: "here, implement this", "but what about this part?", "I don't know, and the guy who knows won't be back before the deadline, do your best"
The "coin" example is a simplification, the actual problem statement is likely more complete, but the author of the article probably felt these these details were not relevant to the article, though it would be for someone taking the test.
> you can't ask clarifying questions
> My biggest problem with leetcode type questions is that you can't ask clarifying questions. My mind just doesn't work like most do, and leetcode to some extent seems to rely on people memorizing leetcode type answers. On a few, there's enough context that I can relate real understanding of the problem to, such as the coin example in the article... for others I've seen there's not enough there for me to "get" the question/assignment.
The issue is that leetcode is something you end up with after discovery + scientific method + time, but there's no space in the interview process for any of that.
Your mind slides off leetcode problems because it reverses the actual on-the-job process and loses any context that'd give you a handle on the issue.
IMO leetcode has multiple problems.
1. People can be hired to take the test for you - surprise surprise 2. It is akin to deciding if someone can write a novel from reading a single sentence.
Hiring people for the test is only valid for online assessment. For an onsite, its very obvious if the candidates have cheated on the OA. I've been on the other side and its transparent.
> It is akin to deciding if someone can write a novel from reading a single sentence.
For most decent companies, the hiring process involves multiple rounds of these challenges along with system designs. So its like judging writing ability by having candidates actually write and come up with sample plots. Not a bad test.
If they are on site why not interview them? If the purpose of these online assessments is to be the mouth of the funnel that process is starting to fail.
https://www.reddit.com/r/leetcode/comments/1mu3qjt/breaking_...
There are funded companies set up just to help you get past this stuff.
https://www.reddit.com/r/leetcode/comments/1iz6xcy/cheating_...
Personally I feel software development has become more or less like assembly line work. If I was starting out today I would seriously consider other options.
> My biggest problem with leetcode type questions is that you can't ask clarifying questions.
Huh? Of course you can. If you're practicing on leetcode, there's a discussion thread for every question where you can ask questions till the cows come home. If you're in a job interview, ask the interviewer. It's supposed to be a conversation.
> I wouldn't even mind the studying on leetcode types sites if they actually had decent explainers
If you don't find the hundreds of free explanations for each question to be good enough, you can pay for Leetcode Pro and get access to editorial answers which explain everything. Or use ChatGPT for free.
> It's not a matter of skill, it's just my ability to take in certain types of problems doesn't work well.
I don't mean to be rude, but it is 100% a matter of skill. That's good news! It means if you put in the effort, you'll learn and improve, just like I did and just like thousands and thousands of other humans have.
> Without any chance of additional info/questions it's literally a setup to fail.
Well with that attitude you're guaranteed to fail! Put in the work and don't give up, and you'll succeed.
> My biggest problem with leetcode type questions is that you can't ask clarifying questions.
Yeah this one confused me. Not asking clarifying questions is one of the sureshot ways of failing an interview. Kudos if the candidates ask something that the interviewers havent thought of, although its rare as most problems go through a vetting process (along with leak detection).
Many interviews now involve automated exercises on websites that track your activity (don't think about triggering a focus change event on your browser, it gets reported).
Also, the reviewer gets an AI report telling it whether you copied the solution somewhere (expressed as a % probability).
You have few minutes and you're on your own.
If you pass that abomination, maybe, you have in person ones.
It's ridiculous what software engineers impose on their peers when hiring, ffs lawyers, surgeons, civil engineers get NO practical nor theorical test, none.
The major difference between software devs and lawyers, surgeons, and civil engineers is that the latter three have fairly rigorous standards to pass to become a professional (bar, boards, and PE).
That could exist for software too, but I'm not sure HN folks would like that alternative any better. Like if you thought memorizing leetcode questions for 2 weeks before an interview was bad, well I have some bad news.
Maybe in 50-100 years software will have that, but things will look very different.
At least in the US, lawyers, surgeons, & civil engineers all have accredited testing to even enter the profession, in the form of the bar exam, boards, and FE & PE tests respectively. So they do have such theoretical tests, but only when they want to gain their license to practice in a given state. Software doesn't have any such centralized testing accreditation, so we end up with a mess.
Switching to devtools also triggers a focus change and is detectable by other means (such as repeatedly invoking a debugger statement).
The one's i've gotten have all seemed more like tests of my puzzle solving skills than coding.
The worst ones i've had though had extra problems though:
one i was only told about when i joined the interview and that they would be watching live.
One where they wanted me streaming my face the whole time (maybe some people people are fine with that)
And one that would count it against me if i tabbed to another page. So no documentation because they assume i'm just googling it.
Still it's mostly on me to prepare and expect this stuff now.
You can make up API calls which you can say you'd implement later. As long as these are not tricky blocks, you'll be fine.
For Google, Facebook and Amazon, yes. At least last I interviewed there a few years ago. They're more interested in the data structure/algorithm
But I have also been to places that demand actual working code which is compiled and is tested against cases
Usually there the problem is simpler, so there's that
I feel like if I'm being asked this in an interview, they're not asking me to use a constraint solver, they're asking me to _write_ a constraint solver. Just for a specific constraint problem, not a more general constraint solver.
You're right, but that just shows how fundamentally silly this interview approach is.
In any real engineering situation I can solve 100% of these problems. That's because I can get a cup of coffee, read some papers, look in a textbook, go for a walk somewhere green and think hard about it... and yes, use tooling like a constraint solver. Or an LLM, which knows all these algorithms off by heart!
In an interview, I could solve 0% of these problems, because my brain just doesn't work that way. Or at least, that's my expectation: I've never actually considered working somewhere that does leetcode interviews.
I was told to use ANY language in an interview. I asked them if they were sure, so I solved it with J. They were not too pleased and asked me if I could use another language, so I did prolog and we moved on to the next question. Then the idiot had the audacity to say I should not use "J and Prolog" but any common known language. I asked if assembly was fine, and they said no. Perhaps python or javascript. I did the rest in python, needless to say I didn't get the job. :-)
Reminds me of https://aphyr.com/posts/340-reversing-the-technical-intervie... (and the follow-ups to it)
I find it hilarious when people brag about stupid shit like that. Congrats on sabotaging your own interview process I guess??
I haven't been asked leetcode questions in a while and when I was asked, it was an easy level problem. I don't know where they ask hard leetcode problems, I also never solved a hard leetcode problem on my own.
The purpose of coding questions should be a problem that you can solve in about 20 minutes, then they ask another, and then you get 20 minutes to either finish or talk about other things. If you ask questions where either someone knows the trick and they pass, or they don't and fail you don't learn much. You need to watch the person write code to see if they are reasonable about it.
I interviewed at an investment bank in London and they asked me pretty hard questions. One was to implement some multithreaded producer consumer thing in C++. I can't remember the details but it was... well you know how writing multithreaded C++ is. I was allowed to look up references at least. Took me maybe 20 minutes and the whole time the interviewer was just sitting on his phone while I wrote it.
Weird experience. Didn't get that job (probably for the best tbf).
I was once asked fizz buzz in an interview and it made me sad that some people don't pass it.
I'm routinely asked LC Hard questions in interviews. Sometimes more than one in one 45 minute interview.
That said, I interview in silicon valley and I'm a mixed race American. (extremely rare here) I think a lot of people just don't want me to pass the interview and will put up the highest bar they can. Mind you, I often still give optimal solutions to everything within good time constraints. But I've practiced 1000+ problems and done several hundred interviews.
More exactly, you can't invent algorithms on a spot which took who knows how many years for others to invent. I.e. the question ends up being more if you know about a specific algorithm, which results in "invent it if you don't know about it". It's absolutely silly to test for ability to invent one on the spot, so it's a pretty pointless interview question really.
This. Literally every problem in NP can be cast as a constraint problem. The question of whether a solver is the right solution varies a lot depending on the application, and in an interview , it’s almost by definition not the right solution.
They can also be dreadfully slow (and typically are) compared to just a simple dynamic program.
LeetCode problems typically are in P. The challenge is finding out why.
This will be true in some interviews, but not in all.
I'm generally against using leetcode in interviews, but wherever I've seen it used it's usually for one reason & one reason alone: known dysfunctional hiring processes. These are processes where the participants in the hiring process are aware of the dysfunction in their process but are either powerless or - more often - too disorganised to properly reform the process.
Sometimes this is semi-technical director level staff leveraging HR to "standardise" interview techniques by asking the same questions across a wide range of teams within a large corp. Other times this is a small underresourced team cobbling together interview questions from online resources in a hurry, not having the cycles to write a tailored process for themselves.
In these cases, you're very likely to be dealing with a technical interviewer who is not an advocate of leetcode interviewing & is attempting to "look around" the standardised interview scoring approach to identify innovative stand out candidates. In a lot of cases I'd hazard even displaying an interest in / some knowledge of solvers would count significantly in your favour.
If someone solves a leetcode hard with a constraint solver and you don't hire them, you are an idiot.
Do you know how few people in this world even know what a constraint solver is, let alone how to correctly define the problem into one?
I used a constraint solver to solve a homework problem once in my CS degree 3rd year. My god just writing the damn constraints was a huge cognitive load!
I did this, wrote an Essence-prime program to generate Minion solver code for a simple instance of the knapsack problem, as part of a startups "solve one of these and get an interview" challenges. Because I had used those tools recently for a contract job (and wrote/presented a paper on invitation of the solver authors,) I thought it would be fun and didn't really want the job. Got an interview but every dev was like "why did you use a cannon to swat a fly?" and were clearly concerned that without strict supervision I would create baroque towers of garbage for them to clean up.
I would like to believe that most people capable of writing a solver would appreciate simple code. It's like when looking at ffmpeg or some physic engine code. You know you'll forget the details easily so you make sure everything is as simple as they can be.
> If someone solves a leetcode hard with a constraint solver and you don't hire them, you are an idiot.
I do hope you're exagerating here, but in case you aren't: this is an extremely simplistic view of what (software) engineers have to do, and thus what hiring managers should optimize for. I'd put "ability to work in a team" above "raw academic/reasoning ability" for the vast majority of engineering roles, any day.
Not that the latter doesn't matter, of course, but it's by no means the one and only measure.
OK, but obviously this presupposes a job where the hiring process is focused on leetcode.
Hey I'm with you 100% about the idea of code-interviews/leetcode being a problem and the importance of culture-fit and ability to work on a team.
I should have said "if you deemed this a fail on the code interview, you are an idiot".
I've won a couple hackathons with just CP-SAT & Linear Programming which led to my first jobs. I'm surprised not more people know/use it. Very inefficient compared to the "correct" answer but the development speed is much faster.
> If someone solves a leetcode hard with a constraint solver and you don't hire them, you are an idiot
Sometimes you just don't want someone that takes these shortcuts. I think being able to solve the problem without a constraint solver is much more impressive
This - the only downside to a constraint solver is it's usually slower. If you want them to write a fast algorithm, then specify that. Have an actual metric for it, if they can pass it with the declarative language, then great. If not, they should have written a more complicated algorithm.
Yes and no: I've asked questions like this in interviews, and I'd count it as a plus if the candidate reached for a constraint solver. They're criminally underused in real-world software engineering and this would show the candidate probably knows how to get the right answer faster instead of wasting a bunch of time.
Now, if they did answer with a constraint solver, I'd probably ask some followup whiteboard questions to make sure they do actually know how to code. But just giving a constraint solver as an answer definitely wouldn't be bad.
Yes, especially if the interviewee said something like 'this may not be asymptomatically optimal, but if it's not a known bottleneck, then I might start with constraint solver to get something working quickly and then profile later.' Especially if it's a case where even the brute-force solution is tricky.
Otherwise penalizing interviewees for suggesting quick-and-dirty solutions reinforces bad habits. "Premature optimization is the root of all evil," after all.
Using a bad algorithm when a good algorithm that is known to exist is premature pessimization and should be avoided.
There is some debate about what premature optimization is, but I consider it about micro optimizations that often are doing things a modern compiler will do for you better than you can. All too often such attempts result in unreadable code that is slower because the optimizer would have done something different but now it cannot. Premature optimization is done without a profiler - if you have a profile of your code and can show a change really makes a difference then it isn't premature.
On the other hand job interviews imply time pressure. If someone isn't 100% sure how to implement the optimization algorithm without looking it up brute force is faster and should be chosen then. In the real world if I'm asked to do something I can spend days researching algorithms at times (though the vast majority of the time what I need is already in my language's standard library)
It’d be a positive in my book if they used a constraint solver.
General constraint solver would be terribly inefficient for problems like these. It's a linear problem and constraint solver just can't handle O(10^6) variables without some beefy machine.
FWIW, the OP's problem is not linear. It's an integer programming problem.
A trick if you can't do a custom algorithm and using a library is not allowed during interview could be to be ready to roll your own DPLL-based solver (can be done in 30 LOC).
Less elegant, but it's a one-size-fits-all solution.
Okay, but who says you need to use a simple constraint solver? There are various sophisticated constraint solvers that know how to optimize.
At this point, job interviews are so far removed from actual relevance. Experience and aptitude still matter a lot, but too much experience at one employer can ground people in rigid and limiting ways of thinking and solving problems.
Great insight. But this is sadly not applicable to interviews.
> It's easy to do in O(n^2) time, or if you are clever, you can do it in O(n). Or you could be not clever at all and just write it as a constraint problem
This nails it. The point of these problems is to test your cleverness. That's it. Presenting a not-clever solution of using constraint solvers shows that you have experience and your breadth of knowledge is great. It doesn't show any cleverness.
>The point of these problems is to test your cleverness.
In my experience, interviewers love going to the Leetcode "Top Interview 150" list and using problems in the "Array String" category. I'm not a fan of these problems for the kind of jobs I've interviewed for (backend Python mostly), as they are almost always a "give me a O(n) runtime O(1) memory algorithm over this array" type challenge that really doesn't resemble my day to day work at all. I do not regularly do in-place array algorithms in Python because those problems are almost always handled by other languages (C, Rust, etc.) where performance is critical.
I wish interviewers would go to the "Hashmap" section for interviews in Python, JavaScript, etc., type of languages. They are much less about cleverness and more about whether you can demonstrate using the appropriate tools in your language to solve problems that actually do resemble ones I encounter regularly.
There's also the problem of difficulty tuning on some of these. Problem 169 (Majority Element) being rated "Easy" for getting a O(n) runtime O(1) memory solution is hilarious to me. The algorithm first described in 1981 that does it (Boyer–Moore majority vote algorithm) has a Wikipedia page. It's not a difficult to implement or understand algorithm, but its correctness is not obvious until you think about it a bit, at which point you're at sufficient "cleverness" to get a Wikipedia page about an algorithm named after you. Seems excessive for an "Easy" problem.
Interviews should not be about cleverness. They should test that you can code. I almost never write an algorithm because all the important algorithms are in my standard library already. Sure back in school I did implement a red-black tree - I don't remember if it worked, but I implemented it: I can do that again if you need me to, but it will take me several days to get all the details right (most of it looking up how it works again). I use red-black trees all the time, but they are in the language.
You need to make sure a candidate can program so asking programing question make sense. However the candidate should not be judged on if they finish or get an optimal or even correct answer. You need to know if they write good code that you can understand, and are on a path that if given a reasonable amount of time on a realistic story would finish it and get it correct. If someone has seen the problem before they may get the correct answer, but if they have not seen it they won't know and shouldn't expected to get the right answer in an hour.
These tests are programming tests, but also effectively IQ and conscientiousness tests in the same way that most of what people learn in college is pointless, but graduating with a 4.0 GPA is still a strong signal.
I will say, IME, it's pretty obvious when people have seen a problem before, and unless you work at a big company that has a small question pool, most people are not regurgitating answers to these questions but actually grappling with them in realtime. I say this as someone who has been on both ends of this, these problems are all solvable de novo in an hour by a reasonable set of people.
Leetcode ability isn't everything, but I have generally found a strong correlation between Leetcode and the coding aspects of on the job performance. It doesn't test everything, but nothing in my experience of hiring has led me to wanting to lower the bar here as much as raise the bar on all other factors that influence job performance.
Yeah it just depends on whether your interviewer considers that "solved". To test this out, I wrote a one liner in Python (after imports) that solves it with a hashmap (under the hood for Counter, which uses a heap queue to find the most common one):
return Counter(nums).most_common(1)[0][0]
And that's 50th percentile for runtime and memory usage. Doing it with another one liner that's 87% percentile for time because it uses builtin Python sorting but is 20th percentile for memory:
return sorted(nums)[len(nums) // 2]
But the interviewer might be looking for the best approach, which beats "100%" of other solutions in runtime per Leetcode's analysis:
m, c = -1, 0
for x in nums:
if not c:
m = x
c = 1
elif m == x:
c += 1
else:
c -= 1
return m
I guess my point is that interviewers should be flexible and view it as a dialogue rather than asking for the "right answer". I much prefer "identify the bug in this self contained code snippet and fix it" type problems that can be completed in <15-30 minutes personally, but Leetcode ones can be fine if you choose the right problems for the job.
Honestly in day to day programming I find data types & associated APIs are so so much more important than algorithms.
I would rather work with a flexible data type with suboptimal performance than a brittle data type that maybe squeezes out some extra performance.
Your example of in-place array mutation feels like a good example of such a thing. I feel like there should be a category of interviewing questions for "code-safety" not just performance.
> The point of these problems is to test your cleverness.
Last round I did at Meta it was clearly to test that you grinded their specific set of problems, over and over again, until you could reproduce them without thinking. It's clear because the interviewers are always a bit surprised when you answer with whatever is not the text-book approach on both leetcode and on the interview guide they studied.
Cleverness is definitely not high on the list of things they're looking for.
Cheekily using counting sort ended things the one and only time I agreed to interview with Meta. Definitely improved my inbox for a couple years though.
Bottom up dynamic programming algorithms require some cleverness.
All of the ones listed can be solved with a top down dynamic programing algorithm. Which just means "write recursive solution, add caching to memoize it".
For some of these, you can get cleverer. For example the coin change problem is better solved with an A* search.
Still, very few programmers will actually need these algorithms. The top thing we need is to recognize when we accidentally wrote a quadratic algorithm. A quick scan of https://accidentallyquadratic.tumblr.com/ shows that even good people on prominent projects make that mistake on a constant basis. So apparently being able to produce an algorithm on the test, doesn't translate to catching an algorithmic mistake in the wild.
For the love of me I still can't consistently solve dynamic programming problems. Because "write a clever brute force solution that can be cached" is so broad that there are tons of variations out there, and a slight twist can bring you out of the loop fast.
When I interview with problem solving problems, the point is to understand how the candidate thinks, communicates, and decomposes problems. Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".
Interviewers learn nothing from an instant epiphany, and they learn next to nothing from someone being stumped.
Unfortunately, this is why we can't have nice things. Problem solving questions in interviews can be immensely useful tools that, sadly, are rarely usefully used.
> the point is to understand how the candidate thinks, communicates, and decomposes problems.
100% and it's a shame that over time this has become completely lost knowledge, on both sides of the interview table, and "leetcode" is now seen as an arbitrary rote memorization hurdle/hazing ritual that software engineers have to pass to enter a lucrative FAANG career. Interviewees grind problems until they've memorized every question in the FAANG interview bank, and FAANG interviewers will watch a candidate spit out regurgitated code on a whiteboard in silence, shrug, and say "yep, they used the optimal dynamic programming solution, they pass."
If somebody writes the optimal algorithm that should be a negative unless their resume indicates they are writing that algorithm often. The only reason you should know any algorithm well enough to get it right is if your job is implementing the optimal version for every single language. Of course nobody maintains one algorithm in many different languages/libraries (say libc++, python, rust, ada, java - each has different maintainers), so I can safely safe the number is zero who should be able to implement your cleaver algorithm. Now if your cleaver algorithm is in the language standard library (or other library they often use) that should be able to call/use it, though even then I expect them to look up the syntax in most languages.
> Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".
Absolutely agree. When I interview, I start with a simple problem and add complexity as they go. Can they write X? Can they combine it with Y? Do they understand how Z is related?
> the point is to understand how the candidate thinks, communicates, and decomposes problems
Interviewers always say this, but consider: would you endorse a candidate who ultimately is unable to solve the problem you've presented them, even if they think, communicate, and decompose problems well? No interview in this industry prizes those things over getting the answer right.
Note how I structure my problem solving questions to be progressive and adjustable, both up and down. This gives me room to simplify and get the candidate to a place where they can show me something (candidates who truly come up goose eggs on everything functional but still show solid fundamentals may be showing that the interview is for the wrong job family). It also means that it is virtually impossible to get all the way to "the end" and "finish" the problem, as I leave room for extension and modification. I had one question that I thought was long enough, and, of maybe ~120 interviews with it, exactly two people dunked on it, one writing out code for solutions with and without libraries. That guy was a complete jerk, and I wasn't at all surprised when the entire panel came back not-inclined.
My first boss (a CTO at a start-up) drilled this into us. What you know is far less valuable than how you learn/think and how you function on a team.
Interesting. Sounds like you and other HN commentators from firms that interview better than the industry Leetcode convention oughta be on one of those workplace lists on GitHub (like this one: https://github.com/poteto/hiring-without-whiteboards) for applicants who want to go through a more interesting process.
Every interview I know is severely time limited. I don't care if you can solve the problem, so long as your are clearly making progress and have proven you could solve the problem if given longer.
Now I give you problems I expect to take 20 minutes if you have never seen them before so you should at least solve 1. I have more than once realized someone was stuck on the wrong track and redirection efforts were not getting them to a good track so I switched to a different problem which they were then able to solve. I've also stopped people when they have 6 of 10 tests passing because it is clear they could get the rest passing but I wouldn't learn anything more so it wasn't worth wasting their time.
In the real world I'm going to give people complex problems that will take days to solve.
Would a good answer be "I can do it as a constraint problem, but since I guess you are not asking for this, the solution is..." and then proceed as usual?
Id probably stop the candidate, dig into how they’d using constraint based solvers, and how they might expect that to fall apart. Applicability and judgment is worth way more than raw algorithmic questions.
One way to think about this is:
Is a fresh graduate more likely to provide a solid answer to this than a strategic-thinking seasoned engineer? If so, just be conscious of what your question is actually probing.
And, yes, interview candidates are often shocked when I tell them that I’m fine with them using standard libraries or tools that fit the problem. It’s clear that the valley has turned interviewing into a dominance/superiority model, when it really should be a two-way street.
We have to remember that the candidate is interviewing us, too. I’ve had a couple of interviews as the interviewee where the way the interview was conducted was why I said “no” to an offer (no interest in a counter, just a flat “no longer interested” to the recruiter, and, yes, that surprises recruiters, too).
Constraint solvers are also often not applicable to the real world either.
Many formulations scale in a way that is completely unusable in practice.
Knowing how to get tools like Z3 or Gurobi to solve your problems is it's own skill and one that some companies will hire for, but it's not a general purpose technology you can throw at everything.
This post is the unironic version of "FizzBuzz in TendorFlow", where just because you have a big hammer doesn't mean everything is a nail. And I say that as an enjoyer of bug hammers including SMT solvers.
>The point of these problems is to test your cleverness.
No it's just memorization of 12 or so specific patterns. The stakes are too high that virtually everyone going in will not be staking passing on their own inherent problem solving ability. LeetCode has been so thoroughly gamified that it has lost all utility of differentiability beyond willingness to prepare.
Yeah, it tests if the candidate enjoys the programming-adjacent puzzle game of LeetCode, which is a perfectly decent game to play, but it is just a signal.
If somebody grinds LeetCode while hating it, it signals they are really desperate for a job and willing to jump through hoops for you.
If somebody actually enjoys this kind of stuff, that is probably a signal that they are a rare premium nerd and you should hire them. But the probably play Project Euler as well (is that still up?).
If somebody figures out a one-trick to minmax their LeetCode score… I dunno, I guess it means they are aware of the game and want to solve it efficiently. That seems clever to me…
Given this consider that LeetCode solving is rarely ever part of your work. So then, what are they selecting for with the habit?
But what is it differentiating? And is it really the best evidence of willingness to prepare? My MSc and BA on the topics, my open source contributions, two decades of industry experience... Those aren't evidence of not only willingness but execution of preparation?
That they would ask me to prepare for that is a signal as well.
In no case is it a useful signal on if I can do my job better than someone else. Some people like this type of problem and are good at it anyway which is a good signal compared to average - but there are also above average people who don't enjoy this type of problem and so don't practice it. Note that both cases the people I'm talking about did not memorize the problem and solution.
That willingness to prepare doesn't reconcile with the realities of parenthood and all of the other responsibilities someone in their thirties may have. Consistently finding that time will be a huge ask, especially if you haven't worked on those problems in a while.
No its not a measure of cleverness. Its about whether you can break down problems and apply common methods. Thats the entire job. Its a learnable skill and honestly resisting learning because of personal biases is a red flag in my book.
The point is to test whether or not you put in the time to sharpen common patterns and also to test your communication ability
Yes. Common LC patterns such as 1D and 2D dynamic programming. I'm not defending leetcode style interviews, in fact I think they are actually bad, I'm simply stating their intent as observed by me.
In my notes I have roughly 30 patterns to leetcode questions btw.
Most interviews are based on the premise that if a diabetic can't synthesize their own insulin in their basement, they are somehow cheating at the game of life.
If my wife's blood sugar is high, she takes insulin. If you need to solve a constraint problem, use a constraint solver.
If your company doesn't make and sell constraint solving software, why do you need me to presume that software doesn't exist and invent it from scratch?
It’s explicitly not testing if you can synthesize insulin in a crisis, it’s a general aptitude test for “if we tell you you need to cram this textbook on how to synthesize insulin by next week and then ask you how to do it on a call, can you coherently repeat that back to us?”
In defense of coding tests, most people who can't solve simple dynamic programming problems generally turn out to be pretty poor programmers IRL.
At least that's been my experience. I'm sure there are exceptions.
Whenever constraint programming languages come up, you can’t miss mentioning Håkan Kjellerstrand. He’s put together an amazing collection of problems and examples—including plenty for MiniZinc—on his site: https://www.hakank.org/minizinc/
> Now if I actually brought these questions to an interview the interviewee could ruin my day by asking "what's the runtime complexity?"
This completely undermines the author's main point. Constraint solvers don't solve hard leetcode problems if they can't solve large instances quickly enough.
Many hard leetcode problems can be solved fairly simply with more lax runtime requirements -- coming up with an efficient solution is a large part of the challenge.
> coming up with an efficient solution is a large part of the challenge
More of my work tends to be "rapidly adopting solution to additional and changing requirements" than "come up with most efficient solution", so why are we interviewing for something where in practice we just throw a couple extra instances at it? (Your specific job role may vary, of course, but I usually just increase the scaling factor)
Author's point is that coming up with the most efficient solution might not actually be a good measure of your real-world performance.
And that's been a longrunning critique of leetcode, of course. However, this is a neat framing where you can still use the same problems but give solutions that perform better when measured by "how adaptable is this to new requirements?"
A loonnngggg time ago when I was green, and wasn't taught about constraint solving in my State University compsci program, I encountered the problem when trying to help a friend with his idea.
He wanted to make an app to help sports club owners schedule players for the day based on a couple simple rules. I thought this was going to be easy, and failed after not realizing what I was up against. At the time I didn't even know what I didn't know.
I often look back on that as a lesson of my own hubris. And it's helped me a lot when discussing estimates and timelines and expectations.
This might be a dumb question (as I'm not familiar with constraint solvers) but would a linear optimization approach be better? I've used linear optimization for scheduling in the past. The nice thing is that linear optimization handles rule conflicts well, because you just set weights on all your rules and the optimizer will find the "least bad" solution to the conflicts.
This is what major sports leagues use for season scheduling (source: https://mathstodon.xyz/@j2kun/108975072813565989)
Well if your using MiniZinc you're free to use a CP solver, MIP solver, SAT solver, CP-SAT-LP solver. In general the model is roughly the same, even though some formulations work better for some solvers than others.
But CP (and CP-SAT) solvers tend to do very well on scheduling problems
> I thought this was going to be easy, and failed after not realizing what I was up against. At the time I didn't even know what I didn't know
This reminds me of high school ~25 years ago when I just started learning TI-Basic on my calculator and was dabbling in VB6 on my PC, and I was flipping burgers at Steak n Shake as my part time job. The manager moaned about how hard it was to write the employee schedules out each week (taking into account requested days off, etc) and I thought “ooh, I know how to write software now, I’ll make a scheduling program!” I told the manager I bet I could do it.
… it took a very short time for 16 year old me to realize writing scheduling software to solve for various constraints is pretty damned hard. I never brought it up after that.
I would be blown away if a candidate solved it using DP and then said “but let me show you how to use a constraint solver”. Immediate hire.
Long time ago, just for fun, I wrote a constraint solver problem that could figure out which high yield banks to put money into that were recommended on doctor of credit(https://www.doctorofcredit.com/high-interest-savings-to-get/) based on <= `X` money and <= `Y` # of transactions on debit cards maximize the yield and other constraints(boolean and real valued)
I played it for a while when interest rates were really low and used the thing for my own rainy day savings(I did get tired changing accounts all the time)
- Constraint solvers? That's a nice concept, I heard about this once. However, for the purposes of the interview, let's just write some Python code, I wanna see your way of thinking...
(I think it's almost impossible to convince your interviewer into constraint solvers, while the concept itself is great)
I find this post interesting independent of the question of whether leetcode problems are a good tool for interviews. It's: here are some kinds or problems constraint solvers are useful for. I can imagine a similar post about non-linear least squared solvers like ceres.
Yeah, especially for learning how to use a solver!
> Most constraint solving examples online are puzzles, like Sudoku or "SEND + MORE = MONEY". Solving leetcode problems would be a more interesting demonstration.
He's exactly right about what tutorials are out there for constraint programming (I've messed around with it before, and it was pretty much Sudoku). Having a large body of existing problems to practice against is great.
> The real advantage of solvers, though, is how well they handle new constraints.
Well said. One of the big benefits of general constraint solvers is their adaptability to requirements changes. Something I learned well when doing datacenter optimization for Google.
My biggest problem with leetcode type questions is that you can't ask clarifying questions. My mind just doesn't work like most do, and leetcode to some extent seems to rely on people memorizing leetcode type answers. On a few, there's enough context that I can relate real understanding of the problem to, such as the coin example in the article... for others I've seen there's not enough there for me to "get" the question/assignment.
Because of this, I've just started rejecting outright leetcode/ai interview steps... I'll do homework, shared screen, 1:1, etc, but won't do the above. I tend to fail them about half the time. It only feels worse in instances, where I wouldn't even mind the studying on leetcode types sites if they actually had decent explainers for the questions and working answers when going through them. I know this kind of defeats the challenge aspect, but learning is about 10x harder without it.
It's not a matter of skill, it's just my ability to take in certain types of problems doesn't work well. Without any chance of additional info/questions it's literally a setup to fail.
edit: I'm mostly referring to the use of AI/Automated leetcode type questions as a pre-interview screening. If you haven't seen this type of thing, good for you. I've seen too much of it. I'm fine with relatively hard questions in an actual interview with a real, live person you can talk to and ask clarifying questions.