Comment by kccqzy

Comment by kccqzy 13 hours ago

49 replies

Great insight. But this is sadly not applicable to interviews.

> It's easy to do in O(n^2) time, or if you are clever, you can do it in O(n). Or you could be not clever at all and just write it as a constraint problem

This nails it. The point of these problems is to test your cleverness. That's it. Presenting a not-clever solution of using constraint solvers shows that you have experience and your breadth of knowledge is great. It doesn't show any cleverness.

viccis 12 hours ago

>The point of these problems is to test your cleverness.

In my experience, interviewers love going to the Leetcode "Top Interview 150" list and using problems in the "Array String" category. I'm not a fan of these problems for the kind of jobs I've interviewed for (backend Python mostly), as they are almost always a "give me a O(n) runtime O(1) memory algorithm over this array" type challenge that really doesn't resemble my day to day work at all. I do not regularly do in-place array algorithms in Python because those problems are almost always handled by other languages (C, Rust, etc.) where performance is critical.

I wish interviewers would go to the "Hashmap" section for interviews in Python, JavaScript, etc., type of languages. They are much less about cleverness and more about whether you can demonstrate using the appropriate tools in your language to solve problems that actually do resemble ones I encounter regularly.

There's also the problem of difficulty tuning on some of these. Problem 169 (Majority Element) being rated "Easy" for getting a O(n) runtime O(1) memory solution is hilarious to me. The algorithm first described in 1981 that does it (Boyer–Moore majority vote algorithm) has a Wikipedia page. It's not a difficult to implement or understand algorithm, but its correctness is not obvious until you think about it a bit, at which point you're at sufficient "cleverness" to get a Wikipedia page about an algorithm named after you. Seems excessive for an "Easy" problem.

  • bluGill 11 hours ago

    Interviews should not be about cleverness. They should test that you can code. I almost never write an algorithm because all the important algorithms are in my standard library already. Sure back in school I did implement a red-black tree - I don't remember if it worked, but I implemented it: I can do that again if you need me to, but it will take me several days to get all the details right (most of it looking up how it works again). I use red-black trees all the time, but they are in the language.

    You need to make sure a candidate can program so asking programing question make sense. However the candidate should not be judged on if they finish or get an optimal or even correct answer. You need to know if they write good code that you can understand, and are on a path that if given a reasonable amount of time on a realistic story would finish it and get it correct. If someone has seen the problem before they may get the correct answer, but if they have not seen it they won't know and shouldn't expected to get the right answer in an hour.

    • Eridrus 29 minutes ago

      These tests are programming tests, but also effectively IQ and conscientiousness tests in the same way that most of what people learn in college is pointless, but graduating with a 4.0 GPA is still a strong signal.

      I will say, IME, it's pretty obvious when people have seen a problem before, and unless you work at a big company that has a small question pool, most people are not regurgitating answers to these questions but actually grappling with them in realtime. I say this as someone who has been on both ends of this, these problems are all solvable de novo in an hour by a reasonable set of people.

      Leetcode ability isn't everything, but I have generally found a strong correlation between Leetcode and the coding aspects of on the job performance. It doesn't test everything, but nothing in my experience of hiring has led me to wanting to lower the bar here as much as raise the bar on all other factors that influence job performance.

  • Anon1096 12 hours ago

    Majority Element is rated easy because it can be trivially solved with a hashmap in O(N) space and that's enough to pass the question on Leetcode. The O(1) space answer is probably more like a medium.

    • viccis 11 hours ago

      Yeah it just depends on whether your interviewer considers that "solved". To test this out, I wrote a one liner in Python (after imports) that solves it with a hashmap (under the hood for Counter, which uses a heap queue to find the most common one):

      return Counter(nums).most_common(1)[0][0]

      And that's 50th percentile for runtime and memory usage. Doing it with another one liner that's 87% percentile for time because it uses builtin Python sorting but is 20th percentile for memory:

      return sorted(nums)[len(nums) // 2]

      But the interviewer might be looking for the best approach, which beats "100%" of other solutions in runtime per Leetcode's analysis:

        m, c = -1, 0
        for x in nums:
            if not c:
                m = x
                c = 1
            elif m == x:
                c += 1
            else:
                c -= 1
        return m
      
      If I were interviewing, I'd be happy with any of these except maybe the sorted() one, as it's only faster because of the native code doing the sort, which doesn't change that it's O(n log n) time and O(n) space. But I've had interviews where I gave answers that were "correct" to the assumptions and constraints I outlined but they didn't like them because they weren't the one from their rubric. I still remember a Google interview, in which we're supposed to "design to scale to big data", in which they wanted some fiddly array manipulation algorithm like this. I gave one that was O(n log n) but could be done in place with O(1) memory, and the interviewer said it was "incorrect" in favor of a much simpler O(n) one using dicts in Python that was O(n) memory. Had the interviewer specified O(n) memory was fine (not great for "big data" but ok) I would have given him the one liner that did it with dicts lol

      I guess my point is that interviewers should be flexible and view it as a dialogue rather than asking for the "right answer". I much prefer "identify the bug in this self contained code snippet and fix it" type problems that can be completed in <15-30 minutes personally, but Leetcode ones can be fine if you choose the right problems for the job.

  • 3vidence 12 hours ago

    Honestly in day to day programming I find data types & associated APIs are so so much more important than algorithms.

    I would rather work with a flexible data type with suboptimal performance than a brittle data type that maybe squeezes out some extra performance.

    Your example of in-place array mutation feels like a good example of such a thing. I feel like there should be a category of interviewing questions for "code-safety" not just performance.

roadside_picnic 12 hours ago

> The point of these problems is to test your cleverness.

Last round I did at Meta it was clearly to test that you grinded their specific set of problems, over and over again, until you could reproduce them without thinking. It's clear because the interviewers are always a bit surprised when you answer with whatever is not the text-book approach on both leetcode and on the interview guide they studied.

Cleverness is definitely not high on the list of things they're looking for.

  • AlotOfReading 6 hours ago

    Cheekily using counting sort ended things the one and only time I agreed to interview with Meta. Definitely improved my inbox for a couple years though.

Eridrus 38 minutes ago

Constraint solvers are also often not applicable to the real world either.

Many formulations scale in a way that is completely unusable in practice.

Knowing how to get tools like Z3 or Gurobi to solve your problems is it's own skill and one that some companies will hire for, but it's not a general purpose technology you can throw at everything.

This post is the unironic version of "FizzBuzz in TendorFlow", where just because you have a big hammer doesn't mean everything is a nail. And I say that as an enjoyer of bug hammers including SMT solvers.

btilly 11 hours ago

Bottom up dynamic programming algorithms require some cleverness.

All of the ones listed can be solved with a top down dynamic programing algorithm. Which just means "write recursive solution, add caching to memoize it".

For some of these, you can get cleverer. For example the coin change problem is better solved with an A* search.

Still, very few programmers will actually need these algorithms. The top thing we need is to recognize when we accidentally wrote a quadratic algorithm. A quick scan of https://accidentallyquadratic.tumblr.com/ shows that even good people on prominent projects make that mistake on a constant basis. So apparently being able to produce an algorithm on the test, doesn't translate to catching an algorithmic mistake in the wild.

chaboud 12 hours ago

When I interview with problem solving problems, the point is to understand how the candidate thinks, communicates, and decomposes problems. Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".

Interviewers learn nothing from an instant epiphany, and they learn next to nothing from someone being stumped.

Unfortunately, this is why we can't have nice things. Problem solving questions in interviews can be immensely useful tools that, sadly, are rarely usefully used.

  • mjr00 12 hours ago

    > the point is to understand how the candidate thinks, communicates, and decomposes problems.

    100% and it's a shame that over time this has become completely lost knowledge, on both sides of the interview table, and "leetcode" is now seen as an arbitrary rote memorization hurdle/hazing ritual that software engineers have to pass to enter a lucrative FAANG career. Interviewees grind problems until they've memorized every question in the FAANG interview bank, and FAANG interviewers will watch a candidate spit out regurgitated code on a whiteboard in silence, shrug, and say "yep, they used the optimal dynamic programming solution, they pass."

    • bluGill 11 hours ago

      If somebody writes the optimal algorithm that should be a negative unless their resume indicates they are writing that algorithm often. The only reason you should know any algorithm well enough to get it right is if your job is implementing the optimal version for every single language. Of course nobody maintains one algorithm in many different languages/libraries (say libc++, python, rust, ada, java - each has different maintainers), so I can safely safe the number is zero who should be able to implement your cleaver algorithm. Now if your cleaver algorithm is in the language standard library (or other library they often use) that should be able to call/use it, though even then I expect them to look up the syntax in most languages.

      • kragen 7 hours ago

        What if we just really enjoy clever algorithms?

        I've probably implemented first-order Markov-chain text generation more than a dozen times in different languages, and earlier this week I implemented Newton–Cotes adaptive quadrature just because it sounded awesome (although I missed a standard trick because I didn't know about Richardson extrapolation). I've also recently implemented the Fast Hadamard Transform, roman numerals, Wellons–NRK hash tries, a few different variants of Quicksort (which I was super excited to get down to 17 ARM instructions for the integer case), an arena allocator with an inlined fast path, etc. Recently I wrote a dumb constrained-search optimizer to see if I could get a simpler expression of a word-wrap problem. I learned about the range-minimum-query algorithm during a job interview many years ago and ad-libbed a logarithmic-time solution, and since then I've found a lot of fascinating variants on the problem.

        I've never had a job doing this kind of thing, and I don't expect to get one, just like I don't expect to get a job playing go, rendering fractals, reading science fiction, or playing video games. But I think there's a certain amount of transferable skill there. Even if what I need to do this week is figure out how to configure Apache to reverse proxy to the MediaWiki Docker container.

        (I know there are people who have jobs hacking out clever algorithms on different platforms. I even know some of them personally. But there are also people who play video games for a living.)

        I guess I'd fail your interview process?

        • Eridrus 24 minutes ago

          It's usually fairly obvious when people have just seen the solution before.

          But also, interviews are fuzzy and not at all objective, false negatives happen as well as false positives.

          If you want people to know about these things you should put them in your resume though. People can't read your mind.

  • Peritract 11 hours ago

    > Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".

    Absolutely agree. When I interview, I start with a simple problem and add complexity as they go. Can they write X? Can they combine it with Y? Do they understand how Z is related?

    • gorbachev 8 hours ago

      Same. I'm never doing a fail/pass type interview. Instead I try to assess where the candidate is on the beginner/intermediate/expert axis and match that with the expectations of the role I'm interviewing for.

  • Apocryphon 10 hours ago

    > the point is to understand how the candidate thinks, communicates, and decomposes problems

    Interviewers always say this, but consider: would you endorse a candidate who ultimately is unable to solve the problem you've presented them, even if they think, communicate, and decompose problems well? No interview in this industry prizes those things over getting the answer right.

    • bluGill 9 hours ago

      Every interview I know is severely time limited. I don't care if you can solve the problem, so long as your are clearly making progress and have proven you could solve the problem if given longer.

      Now I give you problems I expect to take 20 minutes if you have never seen them before so you should at least solve 1. I have more than once realized someone was stuck on the wrong track and redirection efforts were not getting them to a good track so I switched to a different problem which they were then able to solve. I've also stopped people when they have 6 of 10 tests passing because it is clear they could get the rest passing but I wouldn't learn anything more so it wasn't worth wasting their time.

      In the real world I'm going to give people complex problems that will take days to solve.

  • StefanBatory 11 hours ago

    Would a good answer be "I can do it as a constraint problem, but since I guess you are not asking for this, the solution is..." and then proceed as usual?

    • chaboud 10 hours ago

      Id probably stop the candidate, dig into how they’d using constraint based solvers, and how they might expect that to fall apart. Applicability and judgment is worth way more than raw algorithmic questions.

      One way to think about this is:

      Is a fresh graduate more likely to provide a solid answer to this than a strategic-thinking seasoned engineer? If so, just be conscious of what your question is actually probing.

      And, yes, interview candidates are often shocked when I tell them that I’m fine with them using standard libraries or tools that fit the problem. It’s clear that the valley has turned interviewing into a dominance/superiority model, when it really should be a two-way street.

      We have to remember that the candidate is interviewing us, too. I’ve had a couple of interviews as the interviewee where the way the interview was conducted was why I said “no” to an offer (no interest in a counter, just a flat “no longer interested” to the recruiter, and, yes, that surprises recruiters, too).

corimaith 13 hours ago

>The point of these problems is to test your cleverness.

No it's just memorization of 12 or so specific patterns. The stakes are too high that virtually everyone going in will not be staking passing on their own inherent problem solving ability. LeetCode has been so thoroughly gamified that it has lost all utility of differentiability beyond willingness to prepare.

  • bee_rider 12 hours ago

    Yeah, it tests if the candidate enjoys the programming-adjacent puzzle game of LeetCode, which is a perfectly decent game to play, but it is just a signal.

    If somebody grinds LeetCode while hating it, it signals they are really desperate for a job and willing to jump through hoops for you.

    If somebody actually enjoys this kind of stuff, that is probably a signal that they are a rare premium nerd and you should hire them. But the probably play Project Euler as well (is that still up?).

    If somebody figures out a one-trick to minmax their LeetCode score… I dunno, I guess it means they are aware of the game and want to solve it efficiently. That seems clever to me…

  • erikerikson 12 hours ago

    Given this consider that LeetCode solving is rarely ever part of your work. So then, what are they selecting for with the habit?

    • cratermoon 12 hours ago

      Selecting for people like themselves.

      • erikerikson 11 hours ago

        I think this is one of the more true answers but can you be more specific?

        Like in race? Like in wealth? Like in defection willingness? Like in corruption?

        Asking for a friend who is regularly identified as among the most skilled but feels their career has been significantly derailed by this social phenomenon.

  • jkubicek 12 hours ago

    In defense of questions like this, “willingness to prepare” is a significant differentiator

    • erikerikson 12 hours ago

      But what is it differentiating? And is it really the best evidence of willingness to prepare? My MSc and BA on the topics, my open source contributions, two decades of industry experience... Those aren't evidence of not only willingness but execution of preparation?

      • JonChesterfield 6 hours ago

        The papers and open source indicate that you can build stuff. That's not what it's testing for.

        Will you put up with very long hours of insane grindy nonsense in the spirit of being a team player for a team that doesn't really remember what game they're playing?

        Are you sufficiently in need of income to be fighting through this interview dance in preference to other things, such that once you join you'll be desperate to stay?

        Those are extremely important questions, and a willingness to have spent a thousand hours memorising leetcode correlates strongly with the attributes sought.

      • lubujackson 10 hours ago

        It is a differentiator when you are hiring straight from college. The fact we use this beyond entry level roles is a sign the company has lost the thread and is cargo culting.

    • bluGill 11 hours ago

      That they would ask me to prepare for that is a signal as well.

      In no case is it a useful signal on if I can do my job better than someone else. Some people like this type of problem and are good at it anyway which is a good signal compared to average - but there are also above average people who don't enjoy this type of problem and so don't practice it. Note that both cases the people I'm talking about did not memorize the problem and solution.

    • avgDev 12 hours ago

      It also means "I don't have money for food, and at this point I am desperate".

    • tjpnz 12 hours ago

      That willingness to prepare doesn't reconcile with the realities of parenthood and all of the other responsibilities someone in their thirties may have. Consistently finding that time will be a huge ask, especially if you haven't worked on those problems in a while.

      • LordDragonfang 12 hours ago

        I mean, it would be illegal for them to state it outright, but most companies would prefer not to hire people with kids and other responsibilities. That's the whole reason there are specific discrimination laws for that.

        • cratermoon 12 hours ago

          LeetCode questions neatly solve the problem of not wanting to hire people who won't, or can't, spend hours of their free time doing things they hate for a goal they don't care about except to the extent that will feed and house them.

[removed] 10 hours ago
[deleted]
another_twist 11 hours ago

No its not a measure of cleverness. Its about whether you can break down problems and apply common methods. Thats the entire job. Its a learnable skill and honestly resisting learning because of personal biases is a red flag in my book.

theflyinghorse 13 hours ago

The point is to test whether or not you put in the time to sharpen common patterns and also to test your communication ability

  • ebiester 12 hours ago

    Super common patterns like dynamic programming?

    • theflyinghorse 7 hours ago

      Yes. Common LC patterns such as 1D and 2D dynamic programming. I'm not defending leetcode style interviews, in fact I think they are actually bad, I'm simply stating their intent as observed by me.

      In my notes I have roughly 30 patterns to leetcode questions btw.