The Sovereign Tech Fund invests in Scala
(scala-lang.org)116 points by bishabosha 3 days ago
116 points by bishabosha 3 days ago
Scala is too complicated. Most scala code bases I have worked on have no enforced structure, the language allows for all sorts of unconventional programming paradigms
^ This meme is from 10+ years ago when Scala was at the peak of its hype driven by the FP craze. Nobody seriously writes cryptic-symbolic-operator code like that nowadays. Scalaz, the FP library most notorious for cryptic operator/method names, hasn't been relevant for many years. Today everyone uses Cats, ZIO, or plain Tapir or Play, all of which are quite ergonomic.
Clojure is an exception to pretty much everything in the category. I really wish I had the opportunity to use it in my professional career.
Part of that I think is the culture and not the language. Personally I try to use the least powerful method that gets the job done and that usually keeps me unblocked. In practice that usually means using it as a better Java and not going down the functional monad path. I know scala has gone through a rough patch and maybe migrating from 2 to 3 is painful. But if you try starting a new project now with the latest Scala 3, I think you'll find that its pretty nice. Even IDE support is pretty good.
I think its because we dont have too many established paradigms for functional programming. Having said that I think Scala is just marvellous. I had to work with a Scala codebase written by a set of very mature devs and it was an absolute joy. It influenced how I write Java.
Scala’s downfall is the culture of writing libraries with the most esoteric syntax imaginable. It’s the opposite of Ruby’s “program with pleasure”—-it is “program with maximum cleverness” and “if it doesn’t look like an obscure math equation or symbol soup, it’s probably not idiomatic Scala.” Java’s designers were opposed to operator overloading due to potential for abuse, and did Scala deliver living proof/nightmare scenario of that.
Now I still want operator overloading, because I favor enabling instead of disabling approaches, and I want, for example, new numeric types to have natural syntax that blends in with built-ins, or an equality operator that works consistently. I’ll even settle for += append to strings and collections.
But even Scala standard libraries take operator use to wild extremes. Apparently it’s not enough to use + to append collections, you must use ++. Except Vec you can add with +. There’s ++= and ++=:, and /: and :: and ::: and ?^ and ?^.
Spark Scala interface introduces =!= for inequality comparison. There’s |@| from Cons library. Don’t get me started on Akka. SBT introduces <++=
Surprisingly you can’t override ==, so Spark implements ===. And specs2 testing library implements ====.
I've onboarded experienced and inexperienced developers to fairly big Scala codebases for the best part of a decade and I can't think of a single time symbolic operators were brought up as a source of complexity of confusion. Sbt deprecated <++= 10 years ago.
Agreed. There was a period of time roughly 10-15 years ago where symbolic operators in Scala were very en vogue. That fell out of style in a big way and I haven't encountered symbol soup in a very long time.
Most of the conversations I have with folks about Scala issues these days center around implicits, tooling, and its decline/lack of popularity.
You got it. Scala had a shot being an early mover in the JVM functional programming space, but they really shot themselves in the foot with their version transition problems and tooling issues you allude to. Java is probably "good enough" for most shops now, and if you are not bound to the JVM I really don't understand why you would go with Scala today.
Scala's decline started before Scala 3, which brought its share of breakage (sometimes for dubious reasons, like the new syntax) but also fixed many warts. Tooling has improved a lot lately, but it's too late.
> if you are not bound to the JVM I really don't understand why you would go with Scala today.
Scala's metaprogramming abilities coupled to a powerful type system are still unmatched. Among mainstream languages, only TypeScript gets somewhat close. For your typical service oriented architecture, libraries such as Tapir or ZIO HTTP are pretty nice. I haven't found anything as pleasant in other languages.
That said if an LLM can write 95% of your code today, this point is a bit moot, sadly.
It's not really moot though. The primary benefit of concise but robust frameworks like ZIO is that they are easy to read (like the program is mostly business logic with minimal syntax noise/programming language bookkeeping), not that they're easy to write (though that's true too). Metaprogramming also works nicely with LLMs because you get the expressiveness of something like Python (or better) while retaining a strong type checking layer to give a feedback loop to the LLM. In fact, it kind of shines with an LLM because you largely don't care if things like macros are ugly as long as they produce correct code to present to the typechecker/compiler, so it makes more sense to vibe code the metaprogramming layer to give yourself whatever you need to have straightforward business logic. Conciseness and direct encoding of business logic also helps to keep context focused.
Really this is all true with humans too, but IMO it's multiplied with LLMs because they are insanely capable at dealing with the guts of metaprogramming wizardry if they need to, so you don't end up in a world where that one guy that understands it leaves and then no one can possibly maintain it.
I agree. I took Martin's fantastic online Scala class way back when and I was a fan. Now Clojure is a great JVM alternative, and as you say Java has kicked it up a notch.
I learned recently that one of the killer apps for Scala seems to be in hardware design. Chisel [0] is the core technology of the best open source RISC-V chips. Chipyard [1] is designing leading edge type OOE and AI chips and all of the code is written in Scala. Personally, I can't wait for some of these designs to start being mass produced and put in laptops and phones.
[0] https://en.wikipedia.org/wiki/Chisel_(programming_language)
So, as a justification for support of scala, the thing that seems lacking to me is that Chisel seems to still be centered around scala 2? E.g. their recommended template for getting started still uses scala 2 ... so without support to motivate them to use scala 3, it's not obvious that Chisel benefits much from current work on the scala language? I have not fully understood the Chisel project but I see they have a "compiler plugin" which suggests to me that moving fully off scala 2 requires a meaningful redesign.
https://github.com/chipsalliance/chisel-template/blob/main/b...
Chisel picked up Scala 3 support in 7.0.0 released last September (https://github.com/chipsalliance/chisel/releases/tag/v7.0.0), right now they cross-compile between Scala 2 and 3 but from what I heard they should eventually move to Scala 3 only to start taking advantage of new features like named tuples (https://docs.scala-lang.org/scala3/reference/other-new-featu...).
Chisel absolutely isn’t the type of software that benefits from upgrading because it’s largely standalone. They could be the last project still stuck on 2.x a decade from now and it wouldn’t make much of a difference to its users.
I’ve only used Chisel for a few projects but I’ve never used anything but Chisel in those codebases. Simulation, verification, and all the painful stuff in FPGA/ASIC development depends on non-Scala tooling and all of the inputs (parametrization) are just read in from JSON files produced by scripts in other languages.
It would be nice to be up to date but the hardware NRE is so damn high that working around any limitations in Scala support is a rounding error. Chisel’s outputs are sent out for $X00,000 fab production runs so no one gives a damn whether it’s Scala 2 or 3 as long as it ships a working IC. The last time I used Chisel I was working on a mixed signal design where the Synopsys Fusion Compiler (maybe Custom compiler?) licenses alone ran into the hundreds of thousands per year (iirc it was per seat, so we must have spent over a million per year on Synopsys alone).
I’m not super plugged into scala but I work with Spark quite a bit and my observation has been the whole scala 2.13 -> 3 transition is a huge mess for just about everyone who touches it. I don’t have enough hands-on context to understand why it’s so painful but it seems to be similar or worse to the python 2.7 -> 3 transition in terms of sticking friction.
It is a mess. I've spent some time trying to convert some academic oss projects and some removed features really force large redesigns. I think rather than funding the stuff on this announcement, I wish they would fund a team of experts to work on migration of a prioritized list of projects. This would both provide example patterns of migrating substantial projects and unblock projects who have been saying "we would like to try migrating but library X we use still hasn't"
Well, a more optimistic take here is that if future development on the Scala language was funded explicitly by/for people who are current using Scala 2, that means that the developers would more clearly understand their requirements in terms of making an easier transition for users moving from Scala 2 -> 3
This is great news, nice win for Scala.
It's a great language, I've been working with it for 10 years now. Full stack Scala with Scala.js on the frontend is so very nice. My experience is mostly in fintech & healthcare startups where the language helped us get correctness, refactorability, clarity, and high velocity at the same time without blowing up the team size.
Initially I learned Scala on the job, but I've been writing open source Scala for years since then. It's a cool language to learn and explore ideas in, since it has lots of elegantly integrated features (especially FP + OOP).
Scala may not be the #1 most popular language, and that's fine. Popular stuff surely gets the benefits of funding and attention, and sometimes lacking such support is really annoying, like a few years ago when Scala 3 was first released, the IDEs took a looong time to catch up. But I still choose Scala despite those occasional annoyances, even though I also have years of experience in JS / TS and other languages. It's just a much better tool for my needs.
Why would the governments invest money on such a niche language? "Scala is widely used to build and operate essential systems across multiple industries." - very bold statement.
I don't know what it looks like on the ground now, but Scala was the defacto language of data infrastructure across the post-Twitter world of SV late stage/growth startups. In large part, this was because these companies were populated by former members of the Twitter data team so it was familiar, but also because there was so much open source tooling at that point. ML teams largely wrote/write Python, product teams in JS/whatever backend language, but data teams -- outside of Google and the pre-Twitter firms -- usually wrote Scala for Spark, Scalding etc in the 2012-2022ish era.
I worked in Scala for most of my career and it was never hard to get a job on a growth stage data team or, indeed, in finance/data-intensive industries. I'm still shocked at how the language/community managed to throw away the position they had achieved. At first I was equally shocked to see the Saudi Sovereign Wealth fund investing in the language, but then realized it was just 300k from the EU and everything made sense.
It's still my favorite language for getting things done so I wouldn't be upset with a comeback for the language, but I certainly don't expect it at this point.
Did the Scala 3 changeover blunt its momentum, you think? Or just Python won out in a sort of worse-is-better way?
Mostly the latter. Scala 3 is almost completely irrelevant to the big data space so far. Databricks took six years before upgrading their proprietary Spark runtime to Scala 2.13. Flink dropped the Scala API before even moving to 2.13. I don't know if Scio will seriously attempt the move to Scala 3. All of them suffer from Twitter libraries being abandoned, which isn't insurmountable, but an annoyance still.
And I don't think it matters anymore. I predict that the JVM will eventually be out of the equation. We're already seeing query engines being replaced by proprietary or open source equivalents in C++ or Rust. Large scale distribution is less of a selling point with modern cloud computing. Do you really need 100 executors when you can get a bare metal instance with 192, 256 or 384 cores?
People want a dataframe API in Python because that's what the the ML/DS/AI crowd knows. Queries and processing will be done in C++ or Rust, with little or even zero need for a distributed runtime. The JVM and Scala solve a problem that simply won't exist anymore.
It wouldn't have to be used in very many places to justify a 377k investment. A few big European banks alone would be worth it. Their website says "we invest globally in the open software components that underpin Germany's and Europe's competitiveness and ability to innovate". The fact that Scala is used at a university could also be classified as innovation. This is a minor amount of money if you're going to compare it with a STEM or medical research grant.
Is it niche? Scala is arguably the single most successful functional language. It interoperates with the whole JVM ecosystem. It's probably the #3 JVM language after Java and Kotlin.
Spark is Scala, Twitter was (is?) Scala https://sysgears.com/articles/how-and-why-twitter-uses-scala...
You're being downvoted but you aren't necessarily wrong. Javascript is probably one of the most approachable functional-like languages without being dogmatically functional.
Scala may have fallen out of favor but was quite popular few years ago. And perhaps still is the most popular EU-designed language (developed by EPFL).
Python. Python is the most popular language designed in EU. EPFL is not even in EU
> EPFL is not even in EU
For people who don't get this, EPFL is the Swiss Federal Technical Institue in Lausanne. Switzerland isn't part of the EU or EEA but has instead integrated itself with the EU very closely via a mindboggling number of bilateral agreements with the EU members and the Schengen agreement which allows for free, borderless movement. This has the effect of making it seem very much like they are part of the EU without actually being as such.
https://en.wikipedia.org/wiki/Switzerland%E2%80%93European_U...
True to both. My brain not braining. Was thinking Europe-based/driven. Python started in CWI but PSF is USA-based.
https://www.chisel-lang.org/community
second link - 404
third link - achieved project on github
fourth link - educational project
Perhaps it's a very know and useful project, yet indeed seems very niche to me.
Probably because it's an amazing language for people who know what they're doing
As a longtime Scala lover, I’m so happy to see this. Everyone in here hemming and hawing about version incompatibilities, build tooling and such conveniently forget the warts of other languages and their ecosystems. Scala is an incredible language, especially for the language being so flexible, which is a strength, not a weakness.
Happy to see investment in sbt and the stdlib
Sad to see code coverage tooling called out as something they’re spending money on
Happy to see scala get sponsorship
Very on brand for Germany to invest tech 15 years past its prime.
More like 10. The Scala ecosystem was still really rough in 2011, remember how many companies got burnt trying to follow Twitter's footsteps (cf. the infamous Yammer testimony).
This was a good joke. Surprised it got downvotes. Must be all the Germans who can't take a joke.
(In case you Germans still don't get it -- that was a joke, but feel free to downvote :) )
Hmm, I wonder when Scala will be possible to bootstrap from source.
https://bootstrappable.org/projects/java.html https://bootstrappable.org/projects/java-tools.html https://bootstrappable.org/projects/jvm-languages.html
I am very happy for Scala. So many people taking the time to rant on it. Yes, you can do anything with Scala in a million different ways. So what? So can you do it in C++, Python, Rust, etc. I agree that the whole "Category Theory" libraries are way over the top, but so are libraries in Java using "factories" everywhere. Every language has its pros and cons.
I'd say category theory concepts are not over the top - but category theory libraries could be! I just about never define or use explicit categoric typeclass instances - even though I almost always define categoric methods, e.g. `++` and `empty` for everything that looks like a monoid, `map` for everything that produces values, `contramap` for everything that consumes values. That's usually more than enough - and making structure explicit that way prevents your data structures from semantic drift while they're being handled by other engineers.
I've always liked Scala as a language, but it's challenging to write high-performing and memory-efficient code on the JVM in general. Whenever you raise this issue, you'll encounter a horde of JVM fanboys who insist that it’s not true, giving you all kinds of nonsense excuses and accusing you of not measuring performance or memory consumption properly. If you genuinely want to produce well-performing JVM code, you're essentially writing C-style Java. As soon as you introduce abstraction, performance issues inevitably arise – largely due to the fact that features and modernizations from Project Valhalla haven’t yet been implemented/shipped. Scala proponents will suggest using macros and opaque types, but at scale this approach becomes incredibly cumbersome and even then you won't be able to completely prevent boxing that would actually be unnecessary; you could just as well be writing Rust.
Happy to throw some of mine for that. Scala may not be as popular as some other languages, but it's relevant for use cases as diverse as those for which you would otherwise use Java/C#/Go, JS/TS, C/C++/Rust (via scala-jvm, scala-js and scala-native), and it's a very nice language to work with at that.
Scala isn't as hot as it used to be. I think the rough Scala 2->3 transition, coupled with improvements in the base Java language, emergence of Kotlin + Android support, and popularity of Python in data science and data pipelines (lets just do everything in one language became popular) kind of made Scala not quite as popular as it could have been. Plus the long compile times are a pain. However it seems to have a really high coolness ratio for a language. The few jobs I do see in Scala are very cool looking. Very few boring looking jobs.