Computer Programming

My Workflow To Converting And Optimizing Video Files Between Servers

There has been a command that I've wanted in my command line fu arsenal for many years and only have gotten around to do it today.  I figured I would share.  Quick rundown about my setup and what in turn creates my problem.  I have one server which is Ubuntu and acts more as a local web and file server for my "internal" projects.  It has libav (ffmpeg or avconv) installed and it works, but it is mighty slow.  Hardware related, I'm sure, the computer was free.

Then I have an old Power Mac G5 that is older than the Ubuntu, but works about 4 times faster than the Ubuntu in processing power.  It also has 4 times the cores so that would make some mathematical sense.  Ubuntu will slow down encoding to about 9 frames a second, while the mac will cruse at 40 or 50 depending on bitrate.

If you know me or have seen any of my outlets, you'll know I have too many hobbies and I try to film them all.  This leaves me with a lot of mixed video files all over my local Ubuntu server.  I have network access (samba, smbd) to a drive on the Ubuntu server, so I can use premiere and connect to the drive.  If the video files are small enough, that I don't have to move any video around.  This is very helpful!  But if the video files are too large, editing is almost impossible, and when it comes time to mixdown, I mixdown choppy video.  Which is not awesome.

I wanted a command line command that will push one file over, encode it, move it back to my fileserver, remove it from the encoding server.

Quick note on the choice I've made: I'm sure you're thinking, why not just move everything to the better server, duh?  Two reasons, number one, I'm too lazy.  I'm having a hard enough time just going through the server's 4TB of drives to see what I want to keep and want to trash.  Number two, security.  The mac is so old it's version locked to an older version of iOS, so I don't really want that directly connected to the interwebs.

Server Connections

First thing's first, I want to be able to make this an automagic process so you need to set up SSH to be able to use your user to access the other machine without prompting for a password.  Basically, you can create a key for your user to use to connect using ssh-keygen and then send your newly created key to the server using ssh-copy-id.  To test if this is working, you can do something like this:

$ ssh [user]@[server] 'whoami'
[user]
$

The above command will work when you replace your user and server in those spaces and it should return the username.  For instance, mine is admin@192.168.1.20.

Sending the file over

If you didn't know, the linux command sftp uses ssh as its protocol, so by setting up one, you set up the other!  I use this one command to send a file over:

$ sftp admin@192.168.1.20:/path/to/remote/directory <<< 'put /path/to/video'
Connected to [server].
Changing to: /path/to/remote/directory
sftp> put /path/to/video
Uploading [video name] to /path/to/remote/directory
...

Encoding on the remote server and bring it back

We have ffmpeg on Ubuntu, but on the mac, it was easier to use homebrew to get handbrakeCLI on the server.  Now that the video file is transferred over, we can send a couple more commands off to the server to get it encoded:

$ ssh admin@imac '/path/to/handbrakeCLI/handbrakeCLI -i \
/path/to/remote/directory/video -b 2000 -o /path/to/remote/directory/encodedvideo.mp4 \
-e x264 -O' && sftp admin@imac:/path/to/remote/directory \
<<< 'get encodedvideo.mp4'

This command does two things, first it fires off the handbrakeCLI call with -i as the input video file, -b as the bitrate, -o as the output, -e as the encoding, which is x264 for H.264 encoding - very popular for .mp4 and .mkv containers, and an -O for optimize.  Quick note on bitrate, my Android phone that I use a lot for filming records at an impressive +4 megabits per second which means there's a lot of info is smaller amounts of time.  This will give you very good picture quality, but will take longer to stream/move around in a network, or even on a computer.  I tried encoding at 3, 2, and 1 Mb/sec and found that 2 was just right.  Little red riding hood reference there.

Then after the encoding is finished, on the local machine, I add && which will concatenate another command after the first one.  This is another sftp call to get the file back from the server using "get" instead of "put".

Conclusion

This is a quick rundown or starting point for you.  Steps that you can do from here is make a bash script that runs these consecutively, or you can use && to chain everything together into one command.  You can also add && ssh [user]@[server] rm non-encodedfile.mp4 if you'd like, just be sure to wait until the move is finished back to the first server or you'll have to start all over again!

---

Comments