I'll not dilly dally any longer. Here is my mostly complete how-to.
The following instructions are for FEDORA 29 only. Your mileage may vary on other Fedora versions.
First start off by installing dependencies needed by Fedora 29.
sudo dnf install gcc gcc-c++ gcc-objc cmake git libX11-devel mesa-libGL-devel libv4l-devel pulseaudio-libs-devel x264-devel freetype-devel fontconfig-devel libXcomposite-devel libXinerama-devel qt5-qtbase-devel qt5-qtx11extras-devel libcurl-devel systemd-devel ffmpeg luajit-devel swig python3-devel ffmpeg-devel qt5-qtsvg-devel ImageMagick-devel mbedtls-devel libva-devel ImageMagick-libs libva-vdpau-driverAfter you have your dependencies in order you'll want to download the source for OBS Studio via git.
git clone --recursive https://github.com/obsproject/obs-studio.gitAfter you have your source downloaded you'll want to enter the source directory and create your cmake build directory.
cd obs-studio ; mkdir build ; cd buildWith the build directory made and entered you'll want to use the following cmake command to configure the OBS Studio source.
cmake -DCMAKE_PREFIX_PATH=/usr -DUNIX_STRUCTURE=1 ..Hopefully cmake finishes without error so you'll be ready to compile. As far as I remember the above dependencies should be enough to get you going. Next you'll want to build the source.
make -j<the number of CPU threads you have>After the 'make -j' you will want to put the number of CPU threads you have. As an example if you have a 4 core plus hyperthreading system you'll have 8 threads. So you will use 'make -j8'. This will speed up the compilation time. If you don't care about speed you can use plain old 'make' with no options.
When the build finishes without error you'll be ready to install OBS Studio into it's final resting place. To do so type in the following command.
sudo make installAt this point you are mostly ready to go. All that remains is configuring OBS Studio to use the FFMPEG VAAPI plugin for hardware encoding. To do so you'll want to start up OBS and click on the 'Settings' button in the lower right corner of the main window. Next click on 'Output' from the left hand column. This is where you'll set the FFMPEG VAAPI plugin to encode.
First under 'Output Mode' you'll select 'Advanced' from the drop down menu. After 'Advanced' is selected it'll open up more options than before. Next refer to the below image for the settings I used to enable stable hardware encoding. I run a 2560x1440 monitor resolution so I needed to set the 'Rescale Output' to 1920x1080. Pay attention to the 'Encoder' drop down menu, this is where you toggle between hardware and software encoding.
|Click for bigger view|
If any of the above doesn't work for you please feel free to leave me a comment and I'll do my best to help you through your issues.