They had no idea how their own product worked. They didn’t even know it used encrypted storage.
This was either outsourced or done by some junior engineer who was putting pieces together like it was another Raspberry Pi project that just needed to kind of work.
That’s my entire experience in embedded. Everything I get from other companies basically looks like an internship project right down to the pointer arguments with unspecified bounds on the function calls. One of the companies we bought hardware from keeps representing things are working when they only work on devices in the lab. Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
> Almost nobody in the space produces anything professional and everything uses Yocto even for two person projects where Multistrap would be more productive.
While I agree with your sentiment that there's a lot of poor software engineering in embedded space (especially in consumer-oriented novelty products, less so in established fields like industrial or telco), I can't but wonder what's wrong with Yocto? In my experience, it's quite the opposite: Yocto is the quickest path to get the firmware for a new device assembled, once you have climbed its pretty steep learning curve. I have built a few homebrew firmware build systems out of Debian and make/shell scripts (not my choice), you pretty quickly find yourself reinventing half of the stuff that Yocto does out of the box, but it's all bespoke, janky and hard to maintain. While with Yocto you just take the vendor's meta layer for BSP, put your application in another, and it bakes you a set of flashable images on the other end, complete with SDKs and other goodies for your dev workflow, reproducibly. It doesn't get significantly more sophisticated once you start to need kernel customizations, firmware updates with A/B partition layout, readonly rootfs, manage board- or customer-specific variants and other features that are very common in embedded systems but poorly or not at all supported in standard distros.
The problem isn’t OE per se but every vendor pulls in their own janky patches for everything and you find yourself fighting the build system to cut packages and reduce the filesystem size. It’s also really slow and it’s frankly massive, requiring tens or hundreds of gigabytes of stuff to yield less than a gigabyte of system image. And at the end of the day if a vendor breaks something from another vendor you will find yourself struggling to figure out why. Older versions of their documentation make statements that small teams should avoid Yocto for these reasons.
If you don’t already know anything about how to do these things in Linux Yocto is good for it. But if you try to swim against the current of the way the layer authors want you to do any of those things you will find it very challenging.
Take read-only root, for example. Usually it’s a one-line change in a config file plus an additional mount should you want an overlay in volatile memory. I don’t see how pulling in layers and config files does anything other than obscure how that works.
For a good example of what I hate about it, try building from Meta-Intel without graphics drivers.
What kind of hardware projects do you work on?
I'm mostly in the software space but in the past few years I've been doing a lot more embedded stuff, and the trend I notice is that companies are making great hardware, and then completely ruining its usefulness with bad software and firmware. It's kind of mind blowing to me because I always considered software to be the easy part of making a product, compared to, you know, etching microscopic patterns onto sand to make magical transistors appear in just the right way to do the task you want.
The longer I last in this world the more products I realize are the result of telling a few people who don't know what they are doing to "make it kind of work."