Comment by the__alchemist

Comment by the__alchemist 5 days ago

That's because you're doing web stuff. (I/O limited). So much of our computing experience has been degraded due to this mindset applied more broadly. Despite a steady improvement in hardware, my computing experiences have been stagnating and degraded in terms of latency, responsiveness etc.

I'm not going to even go into the comp chem simulations I've been running, or that about 1/3 the stuff I do is embedded.

I do still use python for web dev, partly because as you say, it's not CPU-bound, and partly because Python's Django framework is amazing. But I have switched to rust for everything else.

makestuff 5 days ago

As a java backend dev mainly working on web services, I wanted to like python, but I have found it really hard to work on a large python project because the auto complete just does not work as well as something like java.

Maybe it is just due to not being as familiar with how to properly setup a python project, but every time I have had to do something in a django or fast api project it is a mess of missing types.

How do you handle that with modern python? Or is it just a limitation of the language itself?

Reply View 3 replies

kstrauser 5 days ago

That's 100% an IDE thing. I use Zed (or Emacs or anything else supporting an LSP) and autocomplete is fast and accurate.

Reply View | 0 replies
JodieBenitez 4 days ago

Pycharm has been fine. Just disable the AI stuff and you get accurate completion. It even has completion for Django ORM stuff, which is heavily dynamic.

Reply View | 1 reply
- a96 5 hours ago
  
  Pycharm has "AI stuff" now?
  
  Reply View | 0 replies

kstrauser 5 days ago

I won’t completely argue against that, and I’ve also adopted Rust for smaller or faster work. Still, I contend that a freaking enormous portion of computing workloads are IO bound to the point that even Python’s speed is Good Enough in an Amdahl’s Law kind of way.

Reply View 5 replies

morganherlocker 5 days ago

I hear this a lot, but can you really say that you're consistently saturating a 1Gbps line for netcode or 6+ Gbps nvme for disk data? In my experience this doesn't really happen with code that isn't intentionally designed to minimize unnecessary work.
A lot of slow parsing tends to get grouped in with io, and this is where python can be most limiting.

Reply View | 1 reply
- kstrauser 5 days ago
  
  I don't personally use Python directly for super IO intensive work. In my common use cases, that's nearly always waiting for a database to return or for a remote network API to respond. In my own work, I'm saturating neither disk nor network. My code often finds itself waiting for some other process to do that stuff on its behalf.
  
  Reply View | 0 replies
lenerdenator 5 days ago

It's been said that Python's greatest superpower is that it's the second-best language at the most stuff.
No one's really developed an ecosystem for a language that's more performant that can match it, and that's all it needs to assert dominance.

Reply View | 2 replies
- azkalam 5 days ago
  
  I've never understood this. Python cannot be optimized like C, C++ or Rust. It cannot do advanced functional things like OCaml, Haskell or Scala. It cannot run in browsers like TypeScript. It cannot do games programming like C# and it can't do crazy macro stuff like Clojure. I don't think it's even second best at those things.
  
  Reply View | 1 reply
  
  lenerdenator 4 days ago
  
  I'm reading this as, "It cannot do things the best", and that's correct. It can't.
  But it can do them well enough, and enough people know it that they can drag a solution across the line in most domains.
  
  Reply View | 0 replies

pdonis 5 days ago

> That's because you're doing web stuff.

I guess you didn't notice where he talked about running numpy?

Reply View 0 replies

liuliu 5 days ago

And 300ms for a DB call is slow, in any case. We really shouldn't accept that as normal cost of doing business. 300ms is only acceptable if we are doing scrypt type of things.

Reply View 6 replies

kstrauser 5 days ago

> in any case.
In some cases. Are looking up a single indexed row in a small K-V table? Yep, slow. Are you generating reports on the last 6 years of sales, grouped by division within larger companies? That might be pretty fast.
I'm not sure why you'd even generalize that so overly broadly.

Reply View | 5 replies
- liuliu 5 days ago
  
  To put in perspective, 300ms is about looping over 30GiB data from RAM, loading 800MiB data from SSD, or doing 1TFLOPS on a single core computer.
  300ms to generate a report would be able to go through ~100M rows at least (on a single core).
  And the implicit assumption that comment I made earlier, of course is not about the 100M rows scan. If there is a confusion, I am sorry.
  
  Reply View | 4 replies
  
  kstrauser 5 days ago
  
  That's all true, so long as you completely ignore doing any processing on the data, like evaluating the rows and selectively appending some of them into a data structure, then sorting and serializing the results, let alone optimizing the query plan for the state of the system at that moment and deciding whether it makes more sense to hit the indexes or just slurp in the whole table given that N other queries are also executing right now, or mapping a series of IO queries to their exact address in the underlying disks, and performing the parity checks as you read the data off the RAID and combine it into a single, coherent stream of not-block-aligned tuples.
  There's a metric boatload of abstractions between sending a UTF-8 query string over the packet-switched network and receiving back a list of results. 300ms suddenly starts looking like a smaller window than it originally appears.
  
  Reply View | 3 replies