Writing secure Go code
(jarosz.dev)380 points by gus_leonel 8 days ago
380 points by gus_leonel 8 days ago
If the code cannot be reached, what is the point of having it as a dependency?
Does it know which part of a dependency has a vulnerability and check, if the execution reaches _that_ part? Then it would make sense.
> Does it know which part of a dependency has a vulnerability and check, if the execution reaches _that_ part?
Yes, govulncheck does symbol-level reachability static analysis, and the vulndb is manually annotated with affected symbols for each vulnerability.
(So glad to see a comment about this at the top, I have sometimes feared we made a mistake in designing a low-noise vulnerability scanner, because I've often seen complaints that "it doesn't work" because it doesn't show as many vulnerabilities as its more popular, less accurate alternatives.)
My understanding is that the primary goal is to determine that if a program is pulling in a dependency, and only using a small part of it, to determine if that part is vulnerable or not. This allows a program owner to know if they need to do an emergency version bump in the face of a CVE or something like that. For some businesses doing emergency deployments is a massive deal.
> hipster language
Funny, I always considered Go a hipster language for Google fanboys.
It kinda is if you're thinking about the manual-coffee-grinder-french-press hipster who eschews automatic coffee makers. Rob Pike doesn't believe in syntax highlighting and to date the Go website / interactive editor doesn't have any. "When I was a child, I used to speak like a child, think like a child, reason like a child; when I became a man, I did away with childish things."
Anyway, that's fine, I like Go and I like grinding coffee manually on occasion.
Funny, I have a similar analogy when it comes to mice: Small children lacking verbal communication skills can only point at things, which is the equivalnet of using a"pointing device". When they grow up, they learn to speak meaningful sentences to express themselves. Which is equvalent to learning to use the command line...
Perhaps, but all I really care about is having a complied, strongly-typed language with a fully-featured modern stdlib and good cross-compilation support that includes wasm. If that comes with an automatic admission to the Google Fanboy Club, then sign me up.
What other well-established languages do we have that meet this criteria? I know .net is a strong contender but do we have other options?
Rust & Java also come to mind (yes, Java can be AOT compiled). Erlang too if you want more fearless concurrency if you’re OK with JIT languages. There’s lots of alternatives to Go in its space but it does have mindshare and there’s nothing wrong with staying on the well trodden path even if it’s full of if err != nil instead of sane error chaining built into the language.
You either die a hipster or live long enough to become mainstream.
Great tips in here - I was not aware of `go vet` nor `go test -race`.
FWIW, while go is not memory safe, I do find that it's much easier to be safe in go than it is in other languages. Its verboseness lends to a very clear understanding of what's happening in any given function. I absolutely hated this at the start, but now ~3 years into maintaining a go codebase, I find it quite nice both for debugging as well as editing old code. I know exactly what each function does, and what the structure of data is in any given context.
Another interesting side effect is that AI tools seem to work amazingly well with golang, given how context is often local to the function.
Go very much is memory safe in the absence of data races.
Data races cause issues in all languages, though it's fair to say that Go is affected slightly more than languages like Java. Rust is a bit special by making data races hard to trigger (impossible in safe code IIUC), but this is not typical.
Kind of, regarding Rust.
It is impossible in the context of having all threads accessing in-process memory.
If the data can be accessed externally, regardless of the guarantees being uphold on the Rust side, there are no guarantees from third parties accessing the same data.
It also doesn't prevent other race issues with external data.
Memory like that needs to be wrapped with unsafe for access, there is the volotile crate to mark stuff like that so the compuler won't optimize it away.
Other than rust haskell seems like the other primary candidate for memory safety even across threads.
> I absolutely hated this at the start, but now ~3 years into maintaining a go codebase, I find it quite nice
I've heard this so often. Thanks for sharing :)
I find going back to other languages and trying to read other people's code is a trial. There's always the temptation to write "smart" code that is terse but takes a lot of parsing to understand.
I love that I can pick up anyone's Go code and it's going to make sense almost immediately because everything is explicit, and laid out the same way (including that rhythm of "do the thing, test the error, do the thing, test the error")
Go is memory-safe. It's not the definition of "memory-safe language" that it's impossible to write memory-unsafe code, only that ordinary code is memory-safe by default.
> ordinary code is memory-safe by default
What does that mean? What constitutes "ordinary"? I'm not sure there is any official definition of memory safety, but I would consider it to mean that aside from code that is explicitly marked as unsafe it is impossible to write code that has undefined behavior.
Good definition. I've seen Go beginners trying to append to a slice from multiple goroutines. It works as well as calling push_back on the same vector from multiple threads in C++. It can easily corrupt GC state and lead to segfaults. The beginner didn't use any advanced trickery or the unsafe package. Therefore Go is not a memory safe language.
Go lets you use `unsafe.Pointer` (or indeed, assembly intrinsics) if you really want to, but those are certainly not used "ordinarily".
It's not just about that. Data races can expose an object in a state that was never written from any thread in Go, potentially corrupting even internal details not exposed. Simply writing a struct value from two different threads can expose this.
An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.
Another example would be code specifically contrived to highlight a soundness problem in the language.
I used the term "extraordinary" to avoid exactly this kind of bickering over corner cases that aren't relevant to day-to-day software development (or at least, not in ways that aren't immediately evident when they come up.)
> An example of extraordinary code would be code that interfaces with and/or pulls in non-memory-safe legacy C code.
That's my point though. Of course calling non-memory safe native code over FFI can lead to memory-safety problems in any language. Likewise using the "unsafe" subset that basically every language has. But none of that is required in Go. It is only required that you mutate shared state from different threads, which is something that I would imagine happens in a lot of Go code codebases since it is an extremely easy mistake to make.
To be clear I think:
1. Go is mostly a memory safe language because it does in fact prevent the most common memory safety issues in C/C++ (UAF, buffer overflows, etc)
2. It is LESS memory safe than other modern memory-sage languages (Rust, Java, C#, Python, etc....)
3. The memory safety issues in Go are very difficult to exploit in code that is not specifically crafted to surface them
Generally.
That is as it does not have pointer arithmetic, unlike C, and arrays / slices are bounds checked. So one will get a crash from a null pointer deref.
The other risk with null pointer access is struct member access via such a pointer, but again due to lack of pointer arithmetic, that can't be easily triggered. The one way would be to have a massive struct, say much greater than the page size, and deref through that - fairly unlikely.
The other reference types (slices, maps, interface values, channels) are safe unless subject to data race issues (multi goroutine update). However channels are safe there, as their role is to be used from multiple goroutines.
So the path to lack of memory safety would be a data race, leading to type misinterpretation, hence type unsafety, then incorrect access and/or spatial and temporal unsafety as a consequence.
Apart from poor design / implementation of explicit multi threaded apps, the most likely data race strikes me as accidental lexical capture by a goroutine, hence movement to the heap, and a resultant race. The sort of thing which was mentioned in a paper (by Uber?). Those should be amiable to detection by linters.
The other case of races from poor threading design would be harder to automatically detect, but also harder to trigger. Probably avoidable by correct use of mutexes around access to the shared types (slices and maps), or simply by following an Actor or CSP design model.
Don’t forget about capslock: https://github.com/google/capslock
Assess your 3P modules for dangerous capabilities
Semgrep is another great option to get value out of static analysis checks against both the language and a few common frameworks. It remains a popular choice for security folks writing static detection rules (and contributing them to the commons).
You can check the open rules here; https://github.com/semgrep/semgrep-rules/tree/develop/go
Does go have a bad security reputation?
I get that anything can be insecure and its a constant battle as this article suggests, but i thought it was quite secure and stable generally (say on a par with .net or any other tool you may use to make a web app at least?)
It has essentially the same security properties of all the modern non-C-languages (ie, C, C++, ObjC), with the added bonus of largely being designed after the deserialization pandemic that especially hit Java, Python, and Ruby. ~All these modern languages are fine for security (though: be careful with serialization formats in anything but Go and Rust).
Arguably, Rust and Go are the two "most secure" mainstream languages, but in reality I don't think it much matters and that you're likely to have approximately the same issues shipping in Python as in Rust (ie: logic and systems programming issues, not language-level issues).
Be wary of anyone trying to claim that there are significant security differences between any of the "modern" or "high-level" languages. These threads inexorably trend towards language-warring.
I'd point out that one advantage Go has over Rust in terms of security are the coverage of standard libraries. Go has great support for HTTP clients/servers, cryptography primitives, SSH, SQL, JSON, secure RNG, etc. all in officially maintained standard libraries. The Rust ecosystem has some standards here but the most widely used HTTP client, just as an example, is mostly maintained by one guy[1]. I think that adds considerable security risk vs Go's net/http.
My own experience is that the Go stdlib has resulted in worse security than, for example, rust.
The reason for that is that both the Rust and Go stdlib have a stability promise, so anything built into them can't change if it's insecure.
For example, the 'tar' package in go by default returns unsanitized paths, and has led to a bunch of CVEs: https://github.com/golang/go/issues/55356
The go stdlib can't change the tar package to make it secure by default because it would be a breaking change to do so.
Rust, on the other hand, has a tar package outside of the stdlib, and so it can evolve to be more secure and over time find a better interface.
We've seen that with various other packages, where the Go stdlib HTTP implementation defaults to no timeouts, and thus makes it easy to DoS yourself. Ditto for tcp. The tls package has similar backwards compatibility warts that make it less secure by default.
Forcing backwards compatibility with network protocols by baking them into the stdlib has largely not been a security win in my experience.
You can argue that people can build packages outside of the Go stdlib too, like if the stdlib "image/draw" package is so bad it can't be used, they can make "golang.org/x/image/draw", or if the stdlib crypto package is bad, they can make "golang.org/x/crypto"... and they did, but people still reach for the stdlib because it's easier to, which makes it an active security trap.
Good point. If you consider the size of your dependency graph as a risk, especially for languages that encourage large dependency graphs like JS and Rust, then Go has a very clear advantage.
> Be wary of anyone trying to claim that there are significant security differences between any of the "modern" or "high-level" languages. These threads inexorably trend towards language-warring.
Hm, I think this is a reasonable take but taken too far. Presumably this out of a desire to avoid people arguing about this-language-feature vs. that-language-feature, but in practice "the language" also gets conflated with the tooling and the ecosystem for that language, and having good tooling and a good ecosystem actually does matter when it comes to security vulns in practice. Indeed, anyone can write SQL injection in any language, but having a culture of finding, reporting, and disseminating those vulnerabilities when they happen, and then having mature tooling to detect where those vulnerable packages are being used, and then having a responsive ecosystem where vulnerable packages get swiftly updated, those are all things that make for more secure languages in practice, even among languages with near-identical feature sets.
What is the "deserialisation pandemic"? It doesn't have obvious web search results, and I'm struggling to imagine what about deserialisation what be common between Java and Python (except that, in both cases, I'd surely just use protobuf if I wanted binary serialisation).
In the early 2000/2010s there was a popular idea that it'd be neat to have (de)serialization functionality that could perfectly roundtrip your language's native objects, without requiring that the objects be whatever the language uses as plain old data storage. In the happy case it worked super well and basically every language sufficiently dynamic to support it got a library which let you take some in memory objects, write them to disk, then restore them exactly as they were at some later time.
This had the obvious-in-retrospect major problem that it meant that your deserialization was functionally equivalent to eval(), and if an attacker could ever control what you deserialized they could execute arbitrary code. Many programmers did not realize this and just plain called deserialization functions on untrusted data, and even when people did become aware that was bad it still turned lots of minor bugs into RCE bugs. It was often a long and painful migration away from insecure deserialization methods because of how darn convenient they were, so it continued to be a problem long after it was well understood that things like pickle were a bad idea.
See https://en.wikipedia.org/wiki/Log4Shell , but also historically the mess that is pickling/unpickling in Python (see the big scary warning at the top of https://docs.python.org/3/library/pickle.html#pickle-python-... ), and more broadly any dynamic language that exposes `eval` in any capacity.
Elixir is "more secure" than Go due to its isolated processes, functional processing, and immutable data.
Given the enormity of Elixir's runtime, that seems extremely unlikely. The kinds of bugs you expect to see in interpreted/VM code are different than those in compiled languages like Rust; someone is going to find memory corruption, for instance, when you index exactly the right weird offset off a binary, or do something weird with an auto-promoted bignum. We still find those kinds of bugs in mainstream interpreted languages built on memory-unsafe virtual machines and interpreters.
I'm not saying Elixir is insecure; far from it. It's a memory-safe language. Just, it would be a weird language slapfight to pick with a compiled language.
My comment isn't about compiled vs. bytecode languages. It's about memory management. For example:
• In Elixir, each process runs in isolation, has its own heap, and prevents one process from directly accessing or corrupting the memory of another process. In contrast, Goroutines share the same address space, which means that a bug in one goroutine can potentially corrupt the shared memory and affect other code.
• Elixir uses immutable data structures by default, so nothing can be changed in place. Go, on the other hand, allows mutable state, which can lead to race conditions if not managed correctly. In other words, Elixir is inherently thread safe and Go is not.
• Elixir uses a generational garbage collector with per-process heaps, meaning that the garbage collection of one process can't impact another process. In contrast, Go uses a mark-sweep garbage collector across its entire memory space. This can cause global pauses that can open a window for denial-of-service attacks.
• Elixir uses supervisor processes to monitor operational processes and restart them if they crash. Go's error handling can lead to memory leaks and other undefined behavior if not carefully managed.
• Elixir inherently protects against race conditions, whereas Go relies on tools like the race detector and developer onus to avoid them.
No.
Ironically, a flip side of the complaints about how Go lacks power is that a lot of the "standard" security vulnerabilities actually become harder to write. The most obvious one is lacking the "eval" that a dynamic language has; more subtle ones include things like, there is no way to take a string and look up a type or a method in the runtime, so things like the Ruby YAML vuln are not assisted by the language level. To write something like that into Go, you'd have to actually write it in. Though you can, if you try hard enoough.
But, as sibling comments point out, nothing stops you from writing an SQL injection. Command injections are inhibited by the command only taking the "array of strings" form of a command, with no "just pass me a string and we'll do shell things to it" provided by the language, but I've dispatched multiple questions about how to run commands correctly in Go by programmers who managed to find []string{"bash", "-c", "my command with user input from the web here"}, so the evidence suggests this is still plenty easy enough to write. Putting the wrong perms or no perms on your resources is as easy as anything else; no special support for internal security (compare with E lang and capabilities languages). And the file access is still based on file names rather than inodes, so file-based TOCTOUs are the default in Go (just like pretty much everywhere else) if you aren't careful. It comes with no special DOS protection or integrated WAF or anything else. You can still store passwords directly in databases, or as their MD5 sums. The default HTML templating system is fairly safe but you can still concatenate strings outside of the template system and ship them out over an HTTP connection in bad ways. Not every race condition is automatically a security vulnerability, but you can certainly write race conditions in Go that could be security vulnerabilities.
I'd say Go largely lacks the footguns some other languages have, but it still provides you plenty of knives you can stab yourself with and it won't stop you.
I've been running govulncheck against my repos for a while, and I have seen some real vulnerabilities go by that could have affected my code, but rather than "get arbitrary execution" they tend to be "didn't correctly escape output in some particular edge case", which in the right circumstances can still be serious, but is still at least less concerning than "gets arbitrary execution".
> I'd say Go largely lacks the footguns some other languages have
With the glaring exception of "I forgot to check the error code", which you need a linter (e.g. as provided by golangci-lint) for. It's critically important for security that you know whether the function you just called gave you a meaningful result! Most other languages either have sum types or exceptions.
No it's not. This is what I meant, cross-thread, when I suggested being wary of arguments trying to draw significant distinctions between memory-safe-language X and memory-safe-language Y. Error checking idioms and affordances have profound implications for correctness and for how you build and test code. Programmers have strong preferences. But those implications have only incidental connections to security, if any. Nevertheless "security" is a good claim to throw into a "my language is better" argument.
Mmm, that's fair. I tend to forget about it because it's not something I personally struggle with but that doesn't mean it's not a problem.
I'd still rate it well below a string eval or a default shell interface that takes strings and treats them like shell does. You assert down below that you've seen this lead to a critical vulnerability and I believe you, but in general what happens if you forget to check errors is that sooner or later you get a panic or something else that goes so far off the rails that your program crashes, not that you get privs you shouldn't. As I say in another comment, any sort of confusing bit of code in any language could be the linchpin of some specific security vulnerability, but there are still capabilities that lead to more security issues than some other capabilities. Compared to what I've seen in languages like Perl this is still only "medium-grade" at best.
And I'm not trying to "defend" Go, which is part of why I gave the laundry list of issues it still has. It's just a matter of perspective; even missing the odd error check here or there is just not the same caliber problem as an environment where people casually blast user-sourced input out to shell because the language makes it easier than doing it right.
(Independent of language I consider code that looks like
operation = determineOperation()
if !canIDoOperation(operation) {
// handle failures
}
doOperation(operation)
> "I forgot to check the error code"
How is it that people "forget to check errors" but not other types, even though they are all just 1s and 0s? Or, to put it another way, why do programmers forget how to program as soon as they see the word "error"?
It seems to be a real phenomenon, but I can't make sense of how it can happen. It is not some subtle thing like misspelling a word in a string constant. You are leaving out entire functionality from your application. It is almost on the order of forgetting to add the main function.
I would think it's a mix of not being sure exactly what to do on error and not wanting to undergo the effort of writing error logic. You have to switch from "basic skeletal structure of the program" to "cover all bases", which isn't simple. So it's easy to have no or rudimentary error handling, and by the time you want to change it, it's hard to change. Like, "malloc can fail, but it would be a lot easier right now if I assume it won't".
One thing to note about data races in Go is that the safe Go subset is only memory-safe if you do not have data races. The original post alludes to that because it mentions the race detector. This situation is different from Java where the expected effect of data races on memory safety is bounded (originally due to the sandbox, now bounded effects are more of QoI aspect). Data races in Java are still bad, and your code may go into infinite loops if you have them (among other things), but they won't turn a long into an object reference.
The good news is that the Go implementation can be changed to handle data races more gracefully, with some additional run-time overhead and some increase in compiler complexity and run-time library complexity, but without language changes. I expect this to happen eventually, once someone manages to get code execution through a data race in a high-profile Go application and publishes the results.
These arguments would be more compelling if they came with actual exploitable vulnerabilities --- in shipped code, with real threat models --- demonstrating them, but of course the lived experience of professional programmers is that non-contrived Go memory safety vulnerabilities are so rare as to be practically nonexistent.
About footguns, I'd like to mention an important one: in Go, it's hard to deserialize data wrongly. It's not like python and typescript where you declare your input data to be one thing, and then receive something else. It's a feature that makes server code, which after all is Go's niche, considerably more reliable.
Safety isn't 0% or 100%, and the more a language offers, the better the result. Go is performant, safe, and fairly easy to read and write. What else do you need (in 99.9% of the cases)?
> It's not like python and typescript where you declare your input data to be one thing, and then receive something else
In Python that's likely to lead to a runtime TypeError, not so much in TS since at runtime it's JS and JS is weakly typed.
Besides, Python has Pydantic which everyone should really should be using. :-)
Only if you use a deserializer that's tied to your classes, and not put everything in a dict. And then only if the data encounters an operation that doesn't accept it. But many operations accept e.g. strings, arrays, ints and floats. Is there even an operation that throws a TypeError when using a float instead of int?
Pydantic only helps (AFAIK) when you're letting it help, and you actually use the correct type information. It's not difficult to use, but it's optional, and can be faulty.
I was referring specifically to security footguns like having a string eval. While one can construct code in which that is the critical error that led to a security vulnerability, that can be said about any confusing bit of code in any language, and I would not judge that to especially lead to security issues.
> i thought it was quite secure and stable generally
It is, but security isn't a "given" anywhere. XSS, SQL Injection, Dependency etc can be done by any language, regardless of how "secure" it claims to be.
The headings are all pretty general (versioning, tooling, scanning, testing) but the contents are Go-specific.
It's a pretty good article IMO and could/should be replicated for other languages as well.
Reminds me of when SQL injection was the hot security problem, which was mainly caused by PHP, but not the language itself but reams and reams on low quality online tutorials trying to keep things simple by just concatenating GET parameters straight into an SQL query.
I use VSCodium when I am programming in Go, using the extension, because it has everything I need, and that includes gosec.
I've been maintaining a Go app for about 9 years now and I can just upgrade the Go version + mod for vulnerabilities (GitHub tells me about them automatically idk) and it works with no changes 99% of the time. I can't overstate how this makes maintaining it very stress-free.
My JS apps on the other hand...
Tbf those are development deps rather than production server deps, and the vuln will be something like "DOS possible if you let users craft their own regex string as input to lib.foo(re) in a server ctx" rather than "by using this in development to build your static js app, people get remote access to your dev machine."
It is a bit silly then that it reports them as vulnerabilities by default.
Worse CRA goes from saviour to deprecated, "use nextjs or vite instead" in a blink. Meta should maintain it. Nextjs will probably morph again in the future so you hope investing in learning vite is the answer. JS has this way.
Meanwhile Rails is so old it is thinking it needs to find a partner, settle down and buy a picket fenced house.
I've had similar experiences, but I've noticed my Node.js applications which have few-to-no dependencies behave in the same way as my Go apps in that regard. I might get some deprecation logs from Node letting me know about future changes, but generally they do just work. The apps with a heavy dependency graph are a different story however.
This is still a feather in Go's cap given the fact that the standard library offers so much out of the box. I don't find myself reaching for dependencies that often.
I shudder to think the amount of thousands of engineering hours are spent in my FAANG to keep our Java services just running as-is with updates.
And now we're moving more to Typescript on Node...UGH.
I've been working with Java for the last decade and for the past 5Y used the latest LTS versions in a very regulated environment (we have very strict patch deadlines for most CVEs). Rarely we hit issues with migrating to different versions of our dependencies. The most painful one was a small API change in Spring that revealed that we were doing something very bad so it took me 1-2D in between meetings to investigate. It is true though that every few weeks we are hit by a new CVE and we have to patch a lib version, but TBH this is what I expect from a language that has so many eyes on it's ecosystem.
A lot of BigCo people's (myself included) perception of Java is tainted by the challenges of old, inherited code bases. Java has been ubiquitous for a long time, and it's not surprising to accumulate code bases that have been underserved maintenance-wise over the years. Updating dependencies on a Java 8 codebase isn't much fun, especially because semvar wasn't widely followed back in those days.
I'm still stuck in JS world - it's difficult to get a Go job if it's not already your day job - and I hate it.
Currently I'm adding a React Native component library to an NX monorepo where I want it to work with Storybook for which I need to add Expo but I can't just run the generator, I need to extract the relevant bits from a template project and cross my fingers it works.
I long to go back to the simplicity of my Go project where I'd start my day by running `make watch` and it would just work. (mind you, it took me a while to find a file watcher that worked properly)
You can run govulncheck as part of your CI pipeline too
Don't get it. Is it because your Go app relies in fewer dependencies? If so, it's just a matter of numbers I believe. JS apps tend to rely on more dependencies on average... but that doesn't need to be that way. I have plain JS apps that still work like the first day (even better than Go apps, since there's no compilation step involved).
TypeScript apps on the other hand, yeah, they tend to be more fragile (at least from my perspective: the tsc package has dozen of dependencies, so anything can go wrong)
You can do that in practically any language however that doesn’t mean it’s easy nor the norm.
JavaScript has a culture of move fast and break things. Whereas Go has a culture of moving slow and backwards compatibility.
It also helps that Go has a pretty extensive stdlibs whereas JavaScript is really more like several distinct language ecosystems wrapped around a common specification. So what works on one JavaScript runtime might not even work on another.
> but that doesn’t need to be that way
It kind of does though. If you need to do something with security implications, reinventing the wheel is usually higher risk than using a popular dependency. So it’s not like you can realistically avoid this issue. At least not without causing bigger problems.
It’s also not just a coincidence that Go apps have far fewer dependencies. The comprehensiveness of the std lib (along with officially maintained /x/ packages) means that you need fewer direct dependencies. And just as importantly for the overall size of the tree, all the dependencies that you do need themselves have fewer dependencies. This can easily mean an order of magnitude difference in total transitive dependencies for a significant project.
Go is nice, but the recent trend of using generics for many stuff is making harder and harder to keep Go code readable imho. See an example here https://eli.thegreenplace.net/2024/ranging-over-functions-in...
I'm not saying it's hard to read, but it's harder than previous Go code that used little or no generics at all.
Your example of go code that's harder to read is iterators, and I agree with you. There's no denying that code like this places a high cognitive load on the reader:
func (al *AssocList[K, V]) All() iter.Seq2[K, V] {
return func(yield func(K, V) bool) {
for _, p := range al.lst {
if !yield(p.key, p.value) {
return
}
}
}
}
On further reflection, I think what makes this example particularly difficult to understand is not so much its use of generics, but the way it uses functions. It's a function that returns a function that takes another function as an argument. The generic [K,V] type arguments are actually pretty straightforward.
I often feel this way about heavy use of typescript generics. The more you lean into the crazy (and awesome) world of generics, the more inscrutable the code becomes to anybody who isn’t a generics wiz. It’s really like an extra language stacked on top of JS. I’ll come back to code I wrote a year ago, and it’ll take me a full day to figure out the types.
But the simplicity of using a library or set of functions that have really nice generics? So awesome. The intellisense and type errors alone can almost be a decent form of documentation.
The source becomes hard and weird to change, but the end result is a very nice DX
That's because nobody has yet solved the side effect problem of the sugar.
All the proposals that have ever been given have ultimately boiled down to essentially `return err`, which, while suitable for meme comments on an internet forum, cannot be used in a real production application for many obvious (and some not immediately obvious) reasons.
At least under the direction of rsc (the new leadership is still settling into the role so that is less clear), the will to add such sugar was there if a good solution was found. But the solution has yet to be found.
I don't know what the syntax should look like.
But the most common pattern is a sequence of calls to functions that return an optional error plus the happy path value, followed by a short circuiting check of the error, followed by a call to another function with the happy path value as an argument. It's very common to have a chain of these kinds of calls making up the body of a function.
It seems like "return err" is very useful for this pattern, if I understand you correctly. A function returning the error from the first call it makes that fails, or the happy path value if all the calls succeed. Seems like it should be possible to bake that pattern into the language, but its tricky doing it a way that doesn't obfuscate the underlying semantics, which is very important to many Go developers.
> I don't know what the syntax should look like.
I'm not sure the syntax is all that significant. There have been numerous proposals, but the syntax was never the reason for rejection. It is that the entire concept is unusable in the state that it is understood.
That's not to say the problems can't be solved, but nobody has yet.
> It's very common to have a chain of these kinds of calls making up the body of a function.
Yes, like in Rust, for example. But it also has defined traits and other features on top of the chaining to deal with the same problems Go would suffer from it had such syntax. Theoretically Go could introduce the same, but it remains unclear how to do that in a way that makes sense in the Go language.
Again, there is probably a solution out there, but nobody has come up with it yet. Surprisingly, these kind of things aren't sent down from the heavens by a magical deity. It takes human effort, which isn't there because they are busy ranting on HN.
> It seems like "return err" is very useful for this pattern
Where would you find it useful (memes aside)?
There were many proposals but none of them were an actual improvement over the simplicity and straightforwardness of the existing. `if (err != nil) {` is simple, short and to the point, and adding language features for only this use case wasn't deemed worth the cost in the end.
The problem with this syntax is that it's not required anywhere, any time. It also makes the logic extraordinarily complex for what it is. You can very quickly get into branch hell. I hate to say this, but often the control flow is much simpler and easier to understand with exceptions. The "if" works fine for one level, but any deeper than that and it's no fun.
I'm curious about your objection to the proposal. Sure, generics mean that libraries need a bit more syntax - that's true in all languages - but the actual consumption of the AssociationList type here is clean and readable.
Most types don't need to be generics. Containers do, and I prefer a bit of generics syntax to copy/pasting the container ten times for ten types.
Generics are, IMO, necessary for even a semi-modern language. Okay, you don't need a turing complete templating sublanguage like C++, but you do need at least a way to take generic functions and create generic containers.
In application code you will almost never write generics. To me, it's always been a non-issue.
Somewhat related, I learned a surprising fact recently: Go is not actually memory safe. In particular because atomicity is only guaranteed for word size values, double word values(interface pointers, slices) can introduce memory unsafety in the presence of concurrency[0].
It's one of those things that feels obvious when you see it.
0: https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
Here is code to circumvent Go's memory safety without importing unsafe.
get() reads a byte at an arbitrary address and set() writes a byte at an arbitrary address.
This is excerpted from BUGFIX 66 ("Hack This Site"):
func racer() {
var (
ptr1 *uintptr
ptr2 *byte
race any
done = make(chan struct{})
)
put := func(x any) {
for {
select {
case <-done:
return
default:
race = x
}
}
}
go put(ptr1)
go put(&ptr2)
for {
var ok bool
ptr1, ok = race.(*uintptr)
if ok && ptr1 != nil {
close(done)
break
}
}
get := func(addr uintptr) byte {
*ptr1 = addr
return *ptr2
}
set := func(addr uintptr, to byte) {
*ptr1 = addr
*ptr2 = to
}
if get(0xdeadbeef) == 111 {
set(0xbaaaaaad, 222)
}
}"Without importing unsafe" is doing a lot of work for examples like this.
This comes from a webpage where the challenge is to compromise the site, despite the fact that Go imports are disallowed (including unsafe). It's a puzzle game.
To clarify, I think Go is magnificent and I use it for everything. The racer() code is just a curiosity.
You shouldn't be modifying any variable concurrently without a mutex. The only exception to this is if the variable is (1) less than or equal to the CPU word size; (2) is at a CPU word size aligned address; and (3) atomic memory access functions are used to read and write the variable.
Isn't this part of the Go memory model (https://go.dev/ref/mem)?
Memory safety as long as you don't violate certain rules is what C and C++ also have. The problem is that programmers make mistakes because we're all human.
No, the "mistakes" we talk about with C/C++ are so common that it's hard to think of a major C/C++ project not to have them, and the "mistakes" we're talking about with Go or "unsafe" Rust are contrivances built to demonstrate things an actively malicious programmer could do. Equating the two is, obviously, a sleight of hand.
> Memory safety as long as you don't violate certain rules is what C and C++ also have
There are numbers between 0% and 100%, thus it's possible that Go can be less than 100% memory safe and still far safer than C or C++.
It's easy to demonstrate contrived abuses of Go concurrency that break memory safety, but despite the enormous popularity of the language, actual shipping vulnerabilities --- mistakes in concurrency, not deliberately-engineered pathological cases, that yield attacker-controlled control over memory --- are basically nonexistent (I can't think of a single one; there must be one somewhere!).
Basically this is about as credible an argument as claiming that Rust isn't memory safe because its libraries have so much `unsafe` code. And that claim: not super credible.
Basically, the takeaway in both cases is that it's not safe to allow an attacker to write code for you in the language. But everybody assumes that's the case anyways, because it's the case with virtually every other language (with one very notable, fraught, and well-understood exception), too.
Instead there’s a whole host of subtle footguns which while not leading to true memory unsafety will lead to complete junk data.
I don't care to litigate program correctness and ergonomics. Those are extremely subjective, and I don't feel like I ever get anywhere useful in those kinds of conversations. The most popular backend programming language in the industry is almost certainly Python, and it barely even has types. I still wouldn't dunk on it.
This thread is about a much narrower question, which is code security. There, I feel like I'm on much firmer ground drawing and defending conclusions, and my conclusion is that there isn't a mainstream general-purpose modern language that is meaningfully more secure than Go (or than Rust, or than Python, etc).
I believe UB without unsafe is considered a bug by the Rust language team.
I should’ve said in my original comment, but I don’t mean to dunk on Go. In practice the issues illustrated in the blog post I linked seem unlikely to cause problems in practice, they are interesting nevertheless.
That would mean it, yes. And yeah there is a bug in rust's borrow checker which can trigger something like that for some very special, "no human will ever write code like that" case. But this is an implementation detail for a semantically memory safe language, while in go's case having UB is a language primitive here.
Please note, currently, there are no tools to detect the new footguns created by the new semantics of 3-clause "for;;" loops: https://github.com/golang/go/issues/66156
> The second step is to keep the Go versions in our projects current. Even though we don’t use the latest and greatest language features, bumping the Go version gives us all security patches for discovered vulnerabilities.
It is not always a good strategy to use the latest toolchain version. There are often some fresh bugs in it. From the security perspective, it is better to use the previous version, which is also still being maintained.
It indeed is. Please read https://go101.org/blog/2024-03-01-for-loop-semantic-changes-... and https://github.com/golang/go/issues/66156
There might be some bugs in .2: https://github.com/golang/go/issues/70035
The examples in that ticket are convoluted, who would write code like that? Has this issue been spotted in the wild?
I agree that there is some issue and a lint should probably warn you about these, but I doubt a lot of people will run into it.
You should read that article carefully.
Some Go core team members don't agree with you: https://github.com/golang/go/issues/66156
As the article also mentions: instead of checking if your program has a dependency on something that contains vulnerabilities, govulncheck checks if vulnerable code is actually reached. I find that so awesome. (And I know, someone is going to point out that hipster language foo does this too and better — it’s not the norm).