Comment by jazzyjackson

Comment by jazzyjackson 4 days ago

25 replies

View on Hacker News

This reminds me I need to publish my write up on how I've been converting digitized home video tapes into clips using scene detection, but in case anyone is googling for it, here's a gist I landed on that does a good job of it [0] but sometimes it's fooled by e.g. camera flashes or camera shake so I need to give it a start and end file and have ffmpeg concatenate them back together [1]

Weird thing is I got better performance without "-c:v h264_videotoolbox" on latest Mac update, maybe some performance regression in Sequoia? I don't know. The equivalent flag for my windows machine with Nvidia GPU is "-c:v h264_nvenc" . I wonder why ffmpeg doesn't just auto detect this? I get about 8x performance boost from this. Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !

[0] https://gist.githubusercontent.com/nielsbom/c86c504fa5fd61ae...

[1] https://gist.githubusercontent.com/jazzyjackson/bf9282df0a40...

jack_pp 4 days ago

> Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !

Issue with cloud CPU's is that they don't come with any of the consumer grade CPU built-in hardware video encoders so you'll have to go with the GPU machines that cost so much more. To be honest I haven't tried using HW accel in the cloud to have a proper price comparison, are you saying you did it and it was worth it?

Reply View 9 replies
  • radicality 4 days ago

    Are the hardware encoders even good? I thought that unless you need something realtime, it's always better to spend the cpu cycles on a better encode with th software encoder. Or have things changed ?

    Reply View 7 replies
    • jmb99 3 days ago

      They still suck compared to software encoders. This is true for both H.264 and H.265 on AMD, Nvidia, and Intel GPUs. They’re “good enough” for live streaming, or for things like Plex transcoding, or where you care only about encoding speed and have a large bandwidth budget. They’re better than they used to be, but not worth using for anything you really care about.

      Reply View 0 replies
    • zos_kia 3 days ago

      That's my experience too. I transcode a lot of video for a personal project and hardware acceleration isn't much faster. I figure that's because on CPU I can max out my 12 cores.

      The file size is also problematic I've had hardware encodes twice as large as the same video encoded with CPU.

      Reply View 4 replies
      • jazzyjackson 3 days ago

        Thanks for that datapoint, I was a little bummed to see ffmpeg not using any of my Macs GPUs, but the CPUs ain’t no slouch so I’ll just go with software encoding on Mac

        Reply View 0 replies
      • siscia 3 days ago

        Would you, or anyone else, be interested in ffmpeg in the cloud?

        Connect credit card, open a web UI, send the command, the files, and eventually get the output?

        Reply View 2 replies
    • jack_pp 4 days ago

      I know they used to be worse, haven't tested the newest ones

      Reply View 0 replies
  • jazzyjackson 3 days ago

    We were a quick and dirty R&D team that had to do a lot of video processing quickly, we were not very cost sensitive and didn’t have anything other than AWS to work with, so I can’t speak to whether it was worth it :)

    Reply View 0 replies
Trixter a day ago

In your snippets, you don't appear to be deinterlacing. If your pre-digitized clips are already deinterlaced, that's fine, but if they're not, you're encoding interlaced material as progressive, and mangling the quality. Try adding a bwdif filter so that your 30i content gets encoded as 60p (which will look more like the original videotapes).

Reply View 0 replies
dekhn 4 days ago

I used ffmpeg for empty scene detection- I have a camera pointed at the flight path for SFO, and stripped out all the frames that didn't have motion in them. You end up with a continuous movie of planes passing through, with none of the boring bits.

Reply View 3 replies
  • hnuser123456 4 days ago

    Then can you merge all the clips starting when motion starts and see hundreds of planes fly across at once?

    Reply View 2 replies
    • dekhn 4 days ago

      Interesting. Yes, I assume that's possible although I'm not sure how you handle the background- I guess you find an empty frame, and subtract that from every image with a plane.

      One of the advantages of working with image data is that movies are really just 3d data and as long as all the movies you work with are the same size, if you have enough ram, or use dask, you could basically do this in a couple lines of numpy.

      Reply View 1 reply
      • hnuser123456 4 days ago

        You could do an add operation if the background is dark enough, or yeah, take a local sample of the background to subtract from each clip, and choose a specific background...

        Reply View 0 replies
Gormo 2 days ago

> I wonder why ffmpeg doesn't just auto detect this?

Hardware encoding is often less configurable and involves greater trade-offs than using sophisticated software codecs, and don't produce exactly equivalent results even with equivalent parameters. On top of that, systems often have multiple hardware APIs to choose from that often different features.

FFMpeg is a complex command-line tool intended for users who are willing to learn its intricacies, so I'm not sure it makes sense for it to set defaults based on assumptions.

Reply View 0 replies
rahimnathwani 4 days ago

  -c:v h264_nvenc
This is useful for batch encoding, when you're encoding a lot of different videos at once, because you can get better encoding throughput.

But in my limited experiments a while back, I found the output quality to be slightly worse than with libx264. I don't know if there's a way around it, but I'm not the only one who had that experience.

Reply View 8 replies
  • ziml77 4 days ago

    IIRC they have improved the hardware encoder over the generations of cards, but yes NVENC has worse quality than libx264. NVENC is really meant for running the compression in real-time with minimal performance impact to the system. Basically for recording/streaming games.

    Reply View 0 replies
  • icelancer 4 days ago

    So counterintuitive that nvenc confers worse quality than QSV/x264 variants, but it is both in theory and in my testing as well.

    But for multiple streams or speed requirements, nvenc is the only way to fly.

    Reply View 1 reply
    • Gormo 2 days ago

      Why's that counterintuitive? It makes intuitive sense to me that an approach optimized for throughput would make trade-offs that are less optimized for quality.

      Reply View 0 replies
  • xnx 4 days ago

    Co-signing. Encode time was faster with nvenc, but quality was noticeably worse even to my untrained eye.

    Reply View 4 replies
    • jazzyjackson 4 days ago

      Fascinating, it didn't occur to me quality could take a hit, I thought the flag merely meant "perform h264 encoding over here"

      Edit: relevant docs from ffmpeg, they back up your perception, and now I'm left to wonder how much I want to learn about profiles in order to cut up these videos. I suppose I'll run an overnight job to reencode them from Avi to h264 at high quality, and make sure the scene detect script is only doing copys, not reencoding, since that's the part I'm doing interactively, there's no real reason I should be sitting at the computer while its transcoding.

      Hardware encoders typically generate output of significantly lower quality than good software encoders like x264, but are generally faster and do not use much CPU resource. (That is, they require a higher bitrate to make output with the same perceptual quality, or they make output with a lower perceptual quality at the same bitrate.)

      [0] https://trac.ffmpeg.org/wiki/HWAccelIntro

      Reply View 3 replies
      • norgie 4 days ago

        Cutting videos with copy can result in some weird playback issues if the cuts aren't on keyframes.

        For manually cutting up videos, I use LosslessCut, which I think uses ffmpeg under the hood and is really helpful for finding and cutting on keyframes.

        Reply View 0 replies
      • pertymcpert 4 days ago

        Yes. The reason is that there's many ways to encode a video, you can choose to spend a lot of compute in order to get the best quality, or the best size efficiency. Fixed function hardware encoders by nature have much less freedom in how they encode, and additionally are expected to be faster, or even real time.

        Reply View 0 replies
      • throwaway4PP 4 days ago

        It’s crude, but you can overcome the degraded quality imparted by gpu accelerated encoding by specifying a higher video bitrate.

        Find a complex short scene in your cpu encoded video, extract it, ffprobe it to get average video bitrate, and take the same clip in raw and try gpu accelerated encoding at +20% bitrate. From there, iterate.

        For a friend’s use-case that I helped with, +30% video bitrate bump overcame the degraded vquality.

        Edit: strangely enough, if memory serves, after the correcting +30% was applied the actual ffprobe bitrates between the videos were very similar, maybe a 10% or less difference. Someone smarter than me can work that logic out.

        Reply View 0 replies