Edge AI for Beginners
(github.com)172 points by bakigul 21 hours ago
172 points by bakigul 21 hours ago
I work at an industrial plant, we use "edge" to refer to something inside the production network.
As an example the control system network is air-gapped so to use ML for instrument control or similar the model needs to run on some type of "edge" compute device inside the production network all of the inferencing would need to happen locally (i.e. not in the cloud).
No, edge is just poorly defined. Plenty of companies call their servers “edge” because they’re collocated with ISPs. Even ISPs when they talk about edge compute aren’t talking about your laptop but about compute in their colo.
I don’t think that’s true. Lambda@Edge has been a thing for over 8 years.
In GPU compute land, "edge" means on the consumer device. The latency of delivery is negligible in comparison to the wall clock compute demands, so it doesn't make much sense to park your GPUs near the consumer.
IoT is "edge".
The only place I've seen "edge" used otherwise is in delivery of large files, e.g. ISP-colocated video delivery.
maybe a decent definition could be compute as close to the user latency-wise as practically possible while having full access to the necessary data.
For certain things this will be able to go as far as the device if you're only ever operating on data the user fully owns, other things will need data centers still but just decentralised and closer to the user via fancier architectures ala the Cloudflare model.
May be this one: https://hanlab.mit.edu/courses/2024-fall-65940
Thank you Microsoft, my llm phishing agents have never been more profitable. Scamming w/ "AI" is the future my friends!
I remember when we bought and installed, among the first in the world, the AWS Outpost, sold as an "edge" (of in between cloud and on prem) infrastructure product. Same term has been previously (ab)used also in the security space, at - again - the confluence between cloud and on-prem. And then - yet one more time - the "edge" was a closer data center for localized delivered cloud services.
One of the most common uses for edge AI not listed in this course is computer vision. You similarly want real-time inference for processing video. Another open source project that makes it easy to use SOTA vision models on the edge is inference: https://github.com/roboflow/inference
This looks like AI slop to me. The first two modules repeat the benefits of Edge AI five times. The "Practical Implementation Guide" https://github.com/microsoft/edgeai-for-beginners/blob/main/... ends with a Pre-course checklist. The whole article is just mini-examples without enough context to understand anything.
MS GitHub seems to be featuring a lot of beginners courses all at the same time. Wonder if they're just pumping them out with AI at this point.
Seems to be. There’s little chance this was written by a human.
I would say this is a poor beginners guide for quantization/compression, it's mostly an API guide for tf/keras quantization APIs it doesn't tell the beginner why or when or which layers (and why) they should apply it to.
But the modules that compare the different model families are quite good. As are the remaining modules that are "How to deploy to $platform 101", including microsoft's, of course ;)
Not that I have a better resource at hand for quantization/compression _for beginners_, and I am probably a bad judge for how beginner friendly Song Han's TinyML course was...
Let me ask the same with: - runs on a laptop CPU - decide if a long article is relevant to a specified topic. Maybe even a summary of the article or picking the interesting part as specified in prompt instructions. - no fine tuning please.
Thank you for any response!
Oh this is hilarious, it is like they used Google Lens like method of translating (overlay the translation, you can see the text blocks). In the Dutch one, the cpu AI text just reads: ‘een’ aka ‘a’ in English
Interestingly the French is completely different.
https://github.com/microsoft/edgeai-for-beginners/blob/main/...
TL;DR
This is a course on how to use Microsoft compute to maximise their profits
The very first sentence:
> Welcome to EdgeAI for Beginners – your comprehensive...
Em dash and the word "comprehensive", nearly 100% proof the document was written by AI.
I use AI daily for my job, so I am not against its use, but recently if I detect some prose is written by AI it's hard for me to finish it. The written word is supposed to be a window into someone's thoughts, and it feels almost like a broken social contract to substitute an AI's "thoughts" here instead.
AI generated prose should be labeled as such, it's the decent thing to do.
I said em dash _and_ the word comprehensive. If you work with LLM generated text enough it gets very easy to see the telltale signs. The emojis at the start of each row in the table are also a dead giveaway.
I am guessing you are one of those people who used em dashes before LLMs came out and are now bitter they are an indicator of LLMs. If that's the case, I am sorry for the situation you find yourself in.
Yes, it’s become a tired trope of a particular kind of LLM luddite to me.
Especially given that there are so many linguistic tics one could pick on instead! “Not x, but y”, the bullseye emoji etc., but instead they get hung up on a typographic character actually widely used, presumably because they assume it only occurs on professionals’ keyboards and nobody would take enough care to use it in casual contexts.
> The emojis at the start of each row in the table are also a dead giveaway.
What's up with the green checks, red Xs, rockets, and other stupid emoji in AI slop? Is it an artifact from the cheapest place to do RLHF?
> AI generated prose should be labeled as such, it's the decent thing to do.
The decent thing to do is to prefix the slop with the prompt, so humans don't waste their time reading it.
I don’t really care if it was.
It’s also documentation for an AI product, so I’d kinda expect them to be eating their own dogfood here.
It seems this is focused on on-device computation - as distinct from, say, Cloudflare's definition of the "edge" as a smart CDN with an ability to run arbitrary code and AI models in geographically distributed data centers (https://workers.cloudflare.com/).
Per Microsoft's definition in https://github.com/microsoft/edgeai-for-beginners/blob/main/...:
> EdgeAI represents a paradigm shift in artificial intelligence deployment, bringing AI capabilities directly to edge devices rather than relying solely on cloud-based processing. This approach enables AI models to run locally on devices with limited computational resources, providing real-time inference capabilities without requiring constant internet connectivity.
(This isn't necessarily just Microsoft's definition - https://www.redhat.com/en/topics/edge-computing/what-is-edge... from 2023 defines edge computing as on-device as well, and is cited in https://en.wikipedia.org/wiki/Edge_computing#cite_note-35)
I suppose that the definition "edge is anything except a central data center" is consistent between these two approaches, and there's overlap in needing reliable ways to deploy code to less-trusted/less-centrally-controlled environments... but it certainly muddies the techniques involved.
At this rate of term overloading, the next thing you know we'll be using the word "edgy" to describe teenagers or something...