https://dabrain34.github.io/feed.xml

UK calling

Long time no see this beautiful grey sky, roast beef on sunday and large but full packed pub when there is a football or a rugby game (The rose team has been lucky this year, grrr).

It was a delightful journey in the UK starting with my family visiting London including a lot (yes a lot…) of sightviews in a very short amount of time. But we managed to fit everything in. We saw the changing of the guards, the Thames river tide on a boat, Harry Potter gift shops and the beautiful Arsenal stadium with its legendary pitch, one of the best of England.

It was our last attraction in London and now it was time for my family to go to Standsted back home and me to Cambridge and its legendary university.

To start the journey in Cambridge, first I got some rest on Monday in the hotel to face the hail of information I will get during the conference. This year, Vulkanised took place on Arm’s campus, who kindly hosted the event, providing everything we needed to feel at home and comfortable.

The first day, we started with an introduction from Ralph Potter, the Vulkan Working Group Chair at Khronos, who introduced the new 1.4 release and all the extensions coming along including “Vulkan Video”. Then we could start this conference with my favorite topic, decoding video content with Vulkan Video. And the game was on! There was a presentation every 30 minutes including a neat one from my colleague at Igalia Ricardo Garcia about Device-Generated Commands in Vulkan and a break every 3 presentations. It took a lot of mental energy to keep up with all the topics as each presentation was more interesting than the last. During the break, we had time to relax with good coffee, delicious cookies, and nice conversations.

The first day ended up with a tooling demonstrations from LunarG, helping us all to understand and tame the Vulkan beast. The beast is ours now!

As I was not in the best shape due to a bug I caught on Sunday, I decided to play it safe and went to the hotel just after a nice indian meal. I had to prepare myself for the next day, where I would present “Vulkan Video is Open: Application Showcase”.

Vulkan Video is Open: Application showcase !

First Srinath Kumarapuram from Nvidia gave a presentation about the new extensions made available during 2024 by the Vulkan Video TSG. It started with a brief timeline of the video extensions from the initial h26x decoding to the latest VP9 decode coming this year including the 2024 extensions such as the AV1 codec. Then he presented more specific extensions such as VK_KHR_video_encode_quantization_map, VK_KHR_video_maintenance2 released during 2024 and coming in 2025, VK_KHR_video_encode_intra_refresh. He mentioned that the Vulkan toolbox now completely supports Vulkan Video, including the Validation Layers, Vulkan Profiles, vulkaninfo or GFXReconstruct.

After some deserved applause for a neat presentation, it was my time to be on stage.

During this presentation I focused on the Open source ecosystem around Vulkan Video. Indeed Vulkan Video ships with a sample app which is totally open along with the regular Conformance Test Suite. But that’s not all! Two major frameworks now ship with Vulkan Video support: GStreamer and FFmpeg.

Before this, I started by talking about Mesa, the open graphics library. This library which is totally open provides drivers which support Vulkan Video extensions and allow applications to run Vulkan Video decode or encode. The 3 major chip vendors are now supported. It started in 2022 with RADV, a userspace driver that implements the Vulkan API on most modern AMD GPUs. This driver supports all the vulkan video extensions except the lastest ones such as VK_KHR_video_encode_quantization_map or VK_KHR_video_maintenance2 but this they should be implemented sometime in 2025. Intel GPUs are now supported with the ANV driver, this driver also supports the common video extensions such as h264/5 and AV1 codec. The last driver to gain support was at the end of 2024 where several of the Vulkan Video extensions were introduced to NVK, a Vulkan driver for NVIDIA GPUs. This driver is still experimental but it’s possible to decode H264 and H265 content as well as its proprietary version. This completes the offering of the main GPUs on the market.

Then I moved to the applications including GStreamer, FFmpeg and Vulkan-Video-Samples. In addition to the extensions supported in 2025, we talked mainly about the decode conformance using Fluster. To compare all the implementations, including the driver, the version and the framework, a spreadsheet can be found here. In this spreadsheet we summarize the 3 supported codecs (H264, H265 and AV1) with their associated test suites and compare their implemententations using Vulkan Video (or not, see results for VAAPI with GStreamer). GStreamer, my favorite playground, can now decode H264 and H265 since 1.24 and recently got the support for AV1 but the merge request is still under review. It supports more than 80% of the H264 test vectors for the JVT-AVC_V1 and 85% of the H265 test vectors in JCT-VC-HEVC_V1. FFMpeg is offering better figures passing 90% of the tests. It supports all the avaliable codecs including all of the encoders as well. And finally Vulkan-Video-Samples is the app that you want to use to support all codecs for both encode and decode, but its currently missing support for mesa drivers when it comes to use Fluster decode tests..

Vulkanised on the 3rd day

During the 3rd day, we had interesting talks as well demonstrating the power of Vulkan, from Blender, a free and open-source 3D computer graphics software tool switching progressively to Vulkan, to the implementation of 3D a game engine using Rust, or compute shaders in Astronomy. My other colleague at Igalia, Lucas Fryzek, also had a presentation on Mesa with Lavapipe: a Mesa’s Software Renderer for Vulkan which allows you to have a hardware free implementation of Vulkan and to validate extensions in a simpler way. Finally, we finished this prolific and dense conference with Android and its close collaboration with Vulkan.

If you are interested in 3D graphics, I encourage you to attend future Vulkanised editions, which are full of passionate people. And if you can not attend you can still watch the presentation online.

If you are interested in the Vulkan Video presentation I gave, you can catch up the video here:

Or follow our Igalia live blog post on Vulkan Video:

https://blogs.igalia.com/vjaquez/vulkan-video-status/

As usual, if you would like to learn more about Vulkan, GStreamer or any other open multimedia framework, please feel free to contact us!

Vulkan Video encoder in GStreamer

During the last months of 2023, we, at Igalia, decided to focus on the latest provisional specs proposed by the Vulkan Video Khronos TSG group to support encode operations in an open reference.

As the Khronos TSG Finalizes Vulkan Video Extensions for Accelerated H.264 and H.265 Encode the 19th of December 2023, here is an update on our work to support both h264 and h265 codec in GStreamer.

This work has been possible thanks to the relentless work of the Khronos TSG during the past years, including the IHV vendors, such as AMD, NVIDIA, Intel, but also Rastergrid and its massive work on the specifications wording and the vulkan tooling, and of course Cognizant for its valuable contributions to the Vulkan CTS.

This work started with moving structures in drivers and specifications along the CTS tests definition. It has been a joint venture to validate the specifications and check that the drivers suppports properly the features necessary to encode a video with h264 and h265 codecs. Breaking news, more formats should come very soon…

The GStreamer code Building a community

Thanks first to Lynne and David Airlie, on their RADV/FFmpeg effort, a first implementation was available last year to validate the provisional specifications of the encoders. This work helped us a lot to design the GStreamer version and offer a performant solution for it.

The GStreamer Vulkan bits have been also possible thanks to the tight collaboration with the CTS folks which helped us a lot to understand the crashes we could experience without any clues in the drivers.

To write this GStreamer code, we got inspiration from the code written by the GStreamer community along the GstVA encoders and v4l2codecs plugins. So great kudos to the GStreamer community and especially to He Junayan from Intel, Collabora folks paving the way to stateless codecs and of course Igalia’s past contributions.

And finally the active and accurate reviews from Centricular folks to achieve a valid synchronized operation in Vulkan, see for example https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5079 in addition to their initial work to support Vulkan.

Give me the (de)coded bits

The code is available on the GStreamer MR and we plan to have it ready in the 1.24 version.

This code needs at least the Vulkan SDK version 1.3.274 with KHR Vulkan Video encoding extension support. It should be available publicly when the specifications and the tooling will be available along the next SDK release early 2024.

To build it, nothing is more simple than (after installing the LunarG SDK and the GST requirements):

$ meson setup builddir -Dgst-plugins-bad:vulkan-video=enabled
$ ninja -C builddir

And to run it, you’ll need an IHV driver supporting the KHR extension, which should happen very soon, or the community driven driver such as Igalia’s Intel ANV driver. You can also try the AMD RADV driver available here when KHR will be available.

To encode and validate the bitstream you can run this transcode GStreamer pipeline:

$ gst-launch-1.0 videotestsrc ! vulkanupload ! vulkanh265enc ! h265parse ! avdec_h265 ! autovideosink

As you can see a vulkanupload is necessary to upload the GStreamer buffers to the GPU memory, then the buffer will be encoded through the vulkanh265enc which will generate a byte-stream AU aligned stream which can be decoded with any decoder, here the FFMpeg one.

We are planning to simplify this pipeline and allow to select an encoder from a multiple device configuration.

It supports most of the Vulkan video encoder features and can encode I, P and B Frames for better streaming performance. See this blog post

Challenges

One of the most important challenge here was to put all the bits together and understand when the drivers tell you that you are doing something wrong with only an obscure segfault.

The vulkan process to encode stateless a raw bitstream is quite simple as it will be:

• Initialize the video session with the correct video profile (h264,h265 etc…)
• Initialize the sessions paramaters such as SPS/PPS which will be used by the driver during the encode session
• Reset the codec state
• Encode the buffers:
  • Change the quality/rate control if necessary
  • Retrieve the video session parameters from the driver if necessary
  • Set the bitstream standard parameters such as slice header data.
  • Set the begin video info to give the context to the encoder for the reference
  • Encode the buffer
  • Query the encoded result
  • End the video coding operation
  • repeat ..

The challenge here was not the state machine design but more the understanding of the underlying bits necessary to tell the driver what and how to encode it. Indeed to encode a video stream, a lot of parameters (more than 200..) are necessary and a bad use of these parameters can lead to obscure crashes in the driver stack.

So first it was important to validate the data provided to the Vulkan drivers through the Validation layer. This part in implementing an application using Vulkan is the first and essential step to avoid headaches, see here

You can checkout the code from here to get the latest one and configure it to validate your project. Otherwise it should come from the Vulkan SDK.

This step helped us a lot to understand the mechanics expected by the driver to provide the frames and their references to encode those properly. It also helped to use some API such as the rate control or the quality level.

But Validation Layers cant do everything and that’s why designing, reviewing a solid codebase in CTS as a reference is crucial to implement the GStreamer application or any other 3rd party application. So a lot of time in Igalia has been dedicated to ease and simplify the CTS tests to facilitate its usage.

Indeed the Validation Layers do not validate the Video Standard parameters for example such as the one provided for the format underlying specific data.

So we spent a few hours checking some missing parameters in CTS as in GStreamer which was leading to obscure crash, especially when we were playing with references. A blog post should follow explaining this part.

To debug this missing bits, we used some facilitating tools such as GfxReconstruct Vulkan layer or the VK_LAYER_LUNARG_api_dump which allows you to get an essential dump of your calls and compare the results with other implemetations.

Conclusion

It was a great journey providing a cross platform solution to the community to encode video using largely adopted codec such as h264 and h265. As said before other codecs will come very soon, so stay tuned!

I mentioned the team work in this journey as without the dedication of all, it wouldn’t have been possible to achieve this work in a short time span.

And last but not least, I’d like to special thank also Valve to sponsor this work without whom it wouldn’t have been possible to go as far and as fast!

We’ll attend the Vulkanised 2024 event where we’ll demonstrate this work. Feel free to come and talk to us there. We’ll be delighted to hear and discuss your doubts or brilliant ideas !

As usual, if you would like to learn more about Vulkan, GStreamer or any other open multimedia framework, please contact us!

GstPipelineStudio

As it’s not always convenient to use the powerful command line based, gst-launch tool and also manage all the debug possibilities on all the platforms supported by GStreamer, I started this personal project in 2021 to facilitate the adoption to the GStreamer framework and help newbies as confirmed engineers enjoy the power of it.

Indeed a few other projects, such as Pipeviz (greatly inspired from…) or gst-debugger, already tried to offer this GUI capability, my idea with GPS was to provide a cross-platform tool written in Rust with the powerful framework, gtk-rs.

The aim of this project is to provide the GUI for GStreamer but also being able to remote debug existing pipeline while offering a very simple and accessible interface as back in the days I discovered DirectShow with the help of graphedit or GraphStudioNext

Project details

The interface includes 5 important zones:

• (1) The registry area gives you an access to the GStreamer registry including all the plugins/elements available on your system. It provides you details on each elements. You can also access to a favorite list.
• (2) the main drawing area is where you can add elements from the registry area and connect them together. This area allows you to have multiple independent pipelines with its own player for each drawing area.
• (3) The control playback area, each pipeline can be controlled from this area including basic play/stop/pause but also a seekbar.
• (4) the debug zone where you’ll receive the messages from the application.
• (5) The render zone where you can have a video preview if a video sink has been added to the pipeline. Future work includes to have tracers or audio analysis in this zone.

The project has been written in Rust to offer more stability and thanks to the wonderful work to use the GTK framework, it was perfectly fitting to this project as it gives an easy way to use it over the 3 platforms targeted such as GNU/Linux, MacOS and Windows. On this last platform which is quite well “implanted” in the desktop eco-system, the use of GStreamer can lead to difficulties, that’s why GstPipelineStudio Windows MSI will be a perfect match to test the power of the GStreamer framework.

This project has been written under the GPL v3 License.

How it works under the hood

The trick is quite simple as it uses the power of gst-parse-launch API to build a pipeline as a transformation of the visual pipeline to a command line.

So its a clearly a sibling of gst-launch.

Right now its directly linked to the GStreamer installed on your system but future work could be to connect it over daemons such as GstPrinceOfParser or gstd

What’s new in 0.3.4

The main feature of this release is the cross platform ready state. These are beta versions but the CI is now ready to build and deploy Flathub (Linux), Mac OS and Windows version of GstPipelineStudio.

You can download the installers from the project page or with:

• Linux: flatpak install org.freedesktop.dabrain34.GstPipelineStudio
• MacOS: DMG file
• Windows: MSI file

Here is a list of main features added to the app:

• Open a pipeline from the command line and it will be drawn automatically on the screen. This feature allows you to take any command line pipeline and draw it on the screen to allow any new play tricks with the pipeline, such as change of elements, properties etc.

• Multiple graphview allows you to draw multiple independent pipeline in the same instance of GstPipelineStudio. The playback state is totally independent for each of the views.

• Capsfilter has been added to the links allowing to add this crucial feature of GStreamer pipelines.

• gstreamer-1.0 wrap support to the build system. So you can build your own version of GPS using a dedicated GStreamer version.

What’s next in the pipeline

Among multiple use case, key and debug features, the most upcoming features are:

• Support the zoom on the graphview. As a pipeline can be quite big, the zoom is a key/must features for GPS. See MR
• Debug sections such as receiving events/tags/messages or tracers and terminal support, see MR
• Elements compatibility to check if an element can connect to the previous/next one.
• Remote debugging: A tracer wsserver is currently under development allowing to send over websocket the pipeline events such as connections, properties or element addition. A MR is under development to connect this tracer and render the corresponding pipeline.
• Auto plugging according to the rank of each compatible elements for a given pad caps.
• Display the audio signal in a dedicated render tab.
• Translations
• Documentation
• Unit tests

Here is a lighning talk, I gave about this release (0.3.3), during the 2023 GStreamer conference.

Hope you’ll enjoy this tool and please feel free to provide new features with an RFC here or merge requests here.

As usual, if you would like to learn more about GstPipelineStudio, GStreamer or any other open multimedia framework, please contact us!

How to embed statically your own tailored version of GStreamer in your application

Since the gstreamer-full effort, it was possible to create a shared library which will embed the GStreamer framework in addition to its set of plugins.

Within this effort, it was also possible to register the selected plugins/features automatically by calling the gst_init method in your application linking with gstreamer-full.

This method was offering a gstreamer-full package with library, headers and pc files but it was not possible to embed GStreamer statically in your application and use it transparently.

GstVkVideoParser: a standalone solution

In the journey to bring an open source solution for a video parser to the Vulkan Conformance Test Suite, we chose first to use GStreamer as it was bringing all the parsing facilities necessary to support the needed codecs such as H26x or VPx. This solution was supposed to be also cross platform and dragging as less as possible system dependencies. Seen that GStreamer is usually dragging its own dependencies such as glib or orc and as we wanted to have a standalone GstVkVideoParser library supported on Windows, a little bit of work and love was necessary to add this to GStreamer.

Unfortunately this solution has not be retained by the Vulkan Video TSG, not because it was not working but another parser has been made available and easy to integrate to the CTS at source level avoiding binary linkage, see Vulkan Video change.

GStreamer as a full static library

With the gstreamer-full work, everything was almost ready to be used except to have gstreamer-full as a real static library and be able to link with it in any application.

Here is the MR merged and the challenges taken up:

Adding gst-full-target-type=static

To generate the gstreamer-full dependency which will be statically linked into the application, we decided to introduce a new gst meson options, gst-full-target-type.

By default the gstreamer-full will be built as a shared library as before.

By passing gst-full-target-type=static, only static object will be generated and a package config file will be generated for gstreamer-full allowing the application to avoid to know what static library it needs to add the link line. The GStreamer build system will take care of enabling/disabling the features/libraries you (do/don’t) need.

Initialize the plugins/features automatically

To avoid multiple call necessary to initialize GStreamer, it was also necessary to call the gst_init_static_plugins along with gst_init only in full-static mode but it was leading to a build issue.

Indeed most of tools/examples/tests are linking with libgstreamer-1.0 which owns gst_init () but to faciliate the plugins registration, it was necessary to move all the tools build after the gstreamer-full stage. A first MR has been performed to let gstreamer tools be built against gstreamer-full but additional work was necessary for some core tools or helpers such as gst-transcoder or gst-plugin-scanner to avoid a linking issue.

Disable tests and examples

In a future work all the tools/examples/tests should support the full-static mode but as GStreamer aims to be a shared object framework, we decided to leave this work for later and disable all the examples/tests in full static mode as most of the application using a tailor build won’t need the examples and tests.

Windows support

One of the goal of this work was to provide a Windows library to the Vulkan CTS free of dependency, which has been achieved but some additional work might be necessary to support all of the use case, the GStreamer framework offer, especially on supporting library-dependent plugins.

Give me an example …

In the GstVkVideoParser project, various jobs are building Linux and Windows versions generating a library without any GStreamer/glib dependencie, everything is embedded inside the library, as you can see in this GitHub’ Actions.

In this project, GStreamer is used as a meson subproject/wrap which allows to build GStreamer along of GstVkVideoParser. This can be possible easily by adding the following file to your meson project

subprojects/gstreamer-1.0.wrap

[wrap-git]
directory=gstreamer-1.0
url=https://gitlab.freedesktop.org/gstreamer/gstreamer.git
revision=main

[provide]
dependency_names = gstreamer-1.0, gstreamer-base-1.0, gstreamer-video-1.0, gstreamer-audio-1.0

and then add the following lines to your meson.build to depend on gstreamer-full

meson.build

gstreamer_full_dep = dependency('gstreamer-full-1.0', fallback: ['gstreamer-1.0'], required :true)

In order to build a project, library or application which is using a tailored version of GStreamer you can follow this configure example:

$ meson buildfull-static --default-library=static --force-fallback-for=gstreamer-1.0,glib,libffi,pcre2 -Dauto_features=disabled -Dglib:tests=false -Djson-glib:tests=false -Dpcre2:test=false -Dvkparser_standalone=enabled -Dgstreamer-1.0:libav=disabled -Dgstreamer-1.0:ugly=disabled -Dgstreamer-1.0:ges=disabled -Dgstreamer-1.0:devtools=disabled -Dgstreamer-1.0:default_library=static -Dgstreamer-1.0:rtsp_server=disabled -Dgstreamer-1.0:gst-full-target-type=static_library -Dgstreamer-1.0:gst-full-libraries=gstreamer-video-1.0, gstreamer-audio-1.0, gstreamer-app-1.0, gstreamer-codecparsers-1.0 -Dgst-plugins-base:playback=enabled -Dgst-plugins-base:app=enabled -Dgst-plugins-bad:videoparsers=enabled -Dgst-plugins-base:typefind=enabled

In this case we are disabling everything in GStreamer by using -Dauto_features=disabled and some enabled features such as ges, libav, etc. and enable only what we need as plugins, playback, app, videoparsers and typefind.

And finally we are enabling the static build with --default-library=static and -Dgstreamer-1.0:gst-full-target-type=static_library.

Next …

As you can see, it’s quite easy now to build an application and depends on gstreamer-full static build, but there is still some issues to address such as the plugins dependencies which might be not static and some other platform specific issue such as the gstreamer-full symbols export on Windows.

You can follow some open issues such as:

• Static build is failing on Windows platform

• gstfull: Build only static library on Windows

As usual, if you would like to learn more about Vulkan Video, GStreamer or any other open multimedia framework, please contact us!

ESExtractor, how to integrate a dependency-free library to the Khronos CTS

Since the Vulkan CTS is now able to test and check Vulkan Video support including video decoding, it was necessary to define the kind of media container to be used inside the test cases and the library to extract the necessary encoded data.

In a first attempt, the FFMpeg media toolkit had been chosen to extract the video packets from the A/V ISO base media format chosen as a container reference. This library was provided as a binary package and loaded dynamically at each test run.

As Vulkan video aims to test only video contents, it was not necessary to choose a complex media container, so first all the videos were converted to the elementary stream format for H264 and H265 contents. This is a very elementary format based on MPEG start codes and NAL unit identification.

To avoid an extra multimedia solution integrable only with binaries, a first attempt to replace FFmpeg was, to use GStreamer and an in-house helper library called demuxeres. It was smaller but needed to be a binary still to avoid the glib/gstreamer system dependencies (self contained library). it was a no-go still because the binary package would be awkward to support on the various platforms targetted by the the Khronos CTS.

So at Igalia, we decided to implement a minimal, dependency-free, custom library, written in C++ to be compliant with the Khronos CTS and simple to integrate into any build system.

This library is called ESExtractor

What is ESExtractor ?

ESExtractor aims to be a simple elementary stream extractor. For the first revision it was able to extract video data from a file in the NAL standard. The first official release was 0.2.4. In this release, only the NAL was supported with both the H264 and H265 streams supported.

As Vulkan Video aims to support more than H264 and H265 including format such as AV1 or VP9, the ESExtractor had to support multiple format. A redesign has been started to support multiple format and is now available in the version 0.3.2.

How ESExtractor works

A simple C interface is provided to maximise portability and use by other languages.

es_extractor_new

ESExtractor extractor = es_extractor_new(filePath, "options"));

This interface returns the main object which will give you access to the packets according to the file path and the options given in the arguments. This interface returns a ready to use object where the stream has been initially parsed to determine the kind of video during the object creation.

Then you can check the video format with:

ESEVideoFormat eVideoFormat = es_extractor_video_format(extractor);

or the video codec with:

ESEVideoCodec eVideoCodec = es_extractor_video_codec(extractor);

It supports H264, H265, AV1 and VP9 for now.

es_extractor_read_packet

This API is the main function to retrieve the available packets from the file. Each time this API is called, the library will return the next available packet according to the format and the specific alignment (ie NAL) and a status to let the application decide what to do next. The packet should be freed using es_extractor_clear_packet.

Has CI powered by github

To test the library usage, we have implementing a testing framework in addition to a CI infrastructure As github offers a very powerful worklow, we decided to use this platform to test the library on various architectures and platforms. The CI is now configured to release packages for 64 and 32 bits on Linux and Windows.

As usual, if you would like to learn more about Vulkan Video, ESExtractor or any other open multimedia framework, please contact us!

UPDATE: gst-build is now at the root of the main GStreamer git repository In September 2021, the GStreamer community merged most of the git repositories into a single one. The build system referred in this post as “gst-build” is now in the root of this combined/mono repository.

GStreamer is a powerful multimedia framework with over 30 libraries and more than 1600 elements in 230 plugins providing a wide variety of functionality. This makes it possible to build a huge variety of applications, however it also makes it tricky to ship in a constrained device. Luckily, most applications only use a subset of this functionality, and up until now there wasn’t an easy way to generate a build with just enough GStreamer for a specific application.

Thanks to a partnership with Huawei, you can now use gst-build to generate a minimal GStreamer build, tailored to a specific application, or set of applications. In this blog post, we’ll look at the major changes that have been introduced in GStreamer to make this possible, and provide a small example of what can be achieved with minimal, custom builds.

gst-build

gst-build is the build system that GStreamer developers use. In previous posts, we described how to get started on Linux or Windows and how to use it as a daily development tool.

Since GStreamer 1.18, it is possible to build all of GStreamer into a single shared library called gstreamer-full. This library can include not only GStreamer’s numerous libraries, but also all the plugins and other GStreamer dependencies such as a GLib. Applications can then either dynamically or statically link with gstreamer-full.

Creating the gstreamer-full combined library

By providing -Ddefault_library=static -Dintrospection=disabled to the Meson configure command line, it will generate a static build of all the GStreamer libraries which support the static scheme. This will also produce a shared library called gstreamer-full containing all of GStreamer. For now the GObject introspection needs to be disabled as the static build is not ready to support it (see gst-build-167).

Tailoring GStreamer

Generating a combined library doesn’t by itself reduce the total size. To achieve this goal, we need to select which libraries and plugins are included.

gst-build is a highly configurable build system that already provides options to select which plugins are built. But using the gstreamer-full mechanism, one can select exactly which libraries are included in the final gstreamer-full library by passing the -Dgst-full-libraries= argument to meson. The plugins are then automatically included according to the configuration and the dependencies available.

Lets have an example:

$ meson build-gst-full \
  --buildtype=release \
  --strip \
  --default-library=static \
  --wrap-mode=forcefallback \
  -Dauto_features=disabled \
  -Dgst-full-libraries=app,video,player \
  -Dbase=enabled \
  -Dgood=enabled \
  -Dbad=enabled \
  -Dgst-plugins-base:typefind=enabled \
  -Dgst-plugins-base:app=enabled \
  -Dgst-plugins-base:playback=enabled \
  -Dgst-plugins-base:volume=enabled \
  -Dgst-plugins-base:videoconvert=enabled \
  -Dgst-plugins-base:audioconvert=enabled \
  -Dgst-plugins-good:audioparsers=enabled \
  -Dgst-plugins-good:isomp4=enabled \
  -Dgst-plugins-good:deinterlace=enabled \
  -Dgst-plugins-good:audiofx=enabled \
  -Dgst-plugins-bad:videoparsers=enabled

In this example, we generate a gstreamer-full library using only the feature we explicitly specify. The first step to do that is to disable the automatic selection of features based on which dependencies are already installed (-Dauto_features=disabled). Then we explictly enable the features of each subpackage that we want (ie -Dgst-plugins-base:typefind=enabled -Dgst-plugins-base:app=enabled) and we use -Dgst-full-libraries=app,video,player to tell gst-build to bundle and expose only those specific libraries (app, video, player) in gstreamer-full. We also force meson to build as many dependencies itself by using the fallback with (--wrap-mode=forcefallback), this way, those dependencies will be included in the gstreamer-full library.

Tailoring it further

In our collaboration with Huawei, we decided to push further the idea of tailoring GStreamer to increase the granuality beyond the plugin to be able to select individual element or other features inside each plugin.

GStreamer plugins contain different kind of features. The most common type of plugin features are elements, but you can select other type of loadable features such as device provider, typefind and dynamic-type. For example, the ALSA plugin includes a device provider and various elements such as alsasrc and alsasink.

One key goal of the project was to be able to build only the features needed, reducing the binary file size. For example if the user selects only one element such as flacparse from the audioparsers plugin, the code used by the other parsers should not be included in the final binary.

Note that for now, this project has been focused on Linux platforms and has not been tested on other platforms such as Windows or macOS.

We first experimented with the linker option to garbage collect sections (--gc-sections). This option removes code sections which are not used by any part of the final program, except the public library methods. With the standard compiler options, there is normally a section per C source file, but using the -ffunction-sections and -fdata-sections compiler options, the compiler will generate one section per function and per data symbol.

But according to the documention, this feature must be used with care as it can bring an overhead of code if no sections can be garbage-collected and also because of incompatibility with debugger and slowness ( See -fdata-sections comment in gcc man page).

As GStreamer is a very widely used project, we decided to avoid this solution as it could possibly lead to inconsistent results.

Splitting the code inside GStreamer

Instead of creating new sections automatically, we decided to play with linker rules. The linker (ld) already only pulls in object files if they are called by other objects. So this has the effect of entirely omitting any code that isn’t used by the current program.

Up until now, every plugin had a function that would call the registration function for each feature present in the plugin. This function is called when the plugin is loaded. This plugin initialization function was the only one in each plugin with a predictable name. To be able to select plugins, we needed to expose a registration function for each feature. We were very careful to put these in the same file where the feature is implemented.

The plugin initialization function is now in its own file that can be ignored when linking features one by one. This work required modifying every single GStreamer plugin. A registration method has been declared through a set of macro for each of feature available in GStreamer official repositories.

To declare an element, you should use the macro GST_ELEMENT_REGISTER_DEFINE(element, "name", TANK, GST_TYPE). From any plugin you can register the element calling GST_ELEMENT_REGISTER(element, plugin).

The final part of this work was to create a “static” plugin in gstreamer-full which will contain all the features (element etc.) selected by the gst-build configure with -Dgst-full-elements=.

With these in place, all the features which are not selected don’t get included.

Compose your GStreamer feature(s) menu:

Five new options have been added to gst-build:

• gst-full-plugins: Select the plugin you’d like to include. By default, this is all the plugins enabled by the build process. At least one must be passed or all will be included.
• gst-full-elements: Select the element(s) using the plugin1:elt1,elt2;plugin2:elt1 format
• gst-full-typefind-functions: Select the typefind(s) using the plugin1:tf1,tf2;plugin2:tf1 format
• gst-full-device-providers: Select the decide-provider(s) using the plugin1:dp1,;dp2;plugin2:dp1 format
• gst-full-dynamic-types: Select the dynamic-type(s) using the plugin1:dt1,;dt2;plugin2:dt1 format

You can find more information about the work achieved in the gst-build-199 merge request.

A light menu

Let’s start with a default build with all features and get some metrics to see the benefits. The gstreamer-full library will embed as much as possible its external dependencies such as glib. Plugin dependencies will be kept dynamic but as soon as we’ll select one or another plugin/feature, the dependency will be removed.

The build has been performed using an official GStreamer Fedora Docker image.

$ docker pull registry.freedesktop.org/gstreamer/gst-plugins-bad/amd64/fedora:2021-03-30.0-master

$ meson build-gst-full \
  -Ddefault_library=static \
  -Dintrospection=disabled \
  --buildtype=release \
  --strip \
  --wrap-mode=forcefallback

$ ninja -C build-gst-full

After a successful build, we can reconfigure the last gstreamer-full by providing new options to the meson comamnd line through --reconfigure. In this use-case, we’ll enable only 3 elements from the coreelements plugin in GStreamer.

$ meson build-gst-full --reconfigure -Dgst-full-plugins=coreelements '-Dgst-full-elements=coreelements:filesrc,fakesink,identity' '-Dgst-full-libraries=[]'
$ ninja -C build-gst-full

In this example we are first passing the plugin(s) you’d like to enable, coreelements and then passing the elements we’d like to include filesrc, etc. We remove all additional GST libraries except the core library.

lib (stripped) default tailored ligstreamer-full.so 49208656 (49.2 M) 3250256 (3.2M)

This library can now be used through its pkg-config as gstreamer-full file to build a custom GST application.

Use of a linker script

As a final touch, we have also added an option to provide gst-build with a linker script to select exactly what gets included in the final gstreamer-full library. With this linker script you are now able to drop all the public code which is not used by your application and keep only the necessary code. See gst-build-195

The option is: gst-full-version-script=path_to_version_script

Wrapping up

Some interesting merge requests:

• gst-build-199
• gstreamer-661
• gst-plugins-base-1029
• gst-plugins-good-876
• gst-plugins-bad-2038
• gst-plugins-bad-2110
• gst-plugins-bad-2116
• gst-plugins-ugly-79

All this work is now available upstream (1.19.0) and should available in the next 1.20 release of GStreamer.

As usual, if you would like to learn more about meson, gst-build or any other parts of GStreamer, please contact us!

Generating a MPEG DASH stream with GStreamer A brief introduction to adaptive streaming and MPEG DASH

Adaptive streaming is a technique to provide flexibility and scalability by offering variable bit-rate streams to the client. Designed to work over HTTP, it provides media content as separate streams with media type and various bit-rates, the client will be able to select, according to its network bandwidth or its CPU power.

The most popular adaptive streaming systems are:

• HLS (Apple HTTP Live Streaming)
• MSS (Microsoft Smooth Streaming)
• ADS (Adobe HTTP Dynamic Streaming)
• MPEG-DASH (MPEG Dynamic Adaptive Streaming over HTTP)

MPEG DASH is the most complete adaptive streaming technique. This format is based on an XML description file called MPD as Media Presentation Description. This format describes a set of representations which has a media type (audio, video or subtitles) and provides various bit-rate or media format.

This solution is an open standard and is widely supported by the industry. For more information about it, you can visit the DASH-IF website

In the example below, the MPD describes a static content with three media content type (adaptation sets). Each adaptations sets contains representations. The video has 5 different representations, which allows to switch to 5 different bit rates according to the playback constraints.

<MPD mediaPresentationDuration="PT634.566S" minBufferTime="PT2.00S" profiles="urn:hbbtv:dash:profile:isoff-live:2012,urn:mpeg:dash:profile:isoff-live:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 DASH-MPD.xsd">
 <BaseURL>./</BaseURL>
 <Period>
  <AdaptationSet id="1" mimeType="video/mp4" contentType="video" subsegmentAlignment="true" subsegmentStartsWithSAP="1" par="16:9">
   <SegmentTemplate duration="120" timescale="30" media="$RepresentationID$/$RepresentationID$_$Number$.m4v" startNumber="1" initialization="$RepresentationID$/$RepresentationID$_0.m4v"/>
   <Representation id="bbb_30fps_1280x720_4000k" codecs="avc1.64001f" bandwidth="4952892" width="1280" height="720" frameRate="30" sar="1:1" scanType="progressive"/>
   <Representation id="bbb_30fps_320x180_200k" codecs="avc1.64000d" bandwidth="254320" width="320" height="180" frameRate="30" sar="1:1" scanType="progressive"/>
   <Representation id="bbb_30fps_480x270_600k" codecs="avc1.640015" bandwidth="759798" width="480" height="270" frameRate="30" sar="1:1" scanType="progressive"/>
   <Representation id="bbb_30fps_640x360_800k" codecs="avc1.64001e" bandwidth="1013310" width="640" height="360" frameRate="30" sar="1:1" scanType="progressive"/>
   <Representation id="bbb_30fps_3840x2160_12000k" codecs="avc1.640033" bandwidth="14931538" width="3840" height="2160" frameRate="30" sar="1:1" scanType="progressive"/>
  </AdaptationSet>

  <AdaptationSet id="2" mimeType="audio/mp4" contentType="audio" subsegmentAlignment="true" subsegmentStartsWithSAP="1">
   <Accessibility schemeIdUri="urn:tva:metadata:cs:AudioPurposeCS:2007" value="6"/>
   <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main"/>
   <SegmentTemplate duration="192512" timescale="48000" media="$RepresentationID$/$RepresentationID$_$Number$.m4a" startNumber="1" initialization="$RepresentationID$/$RepresentationID$_0.m4a"/>
   <Representation id="bbb_a64k" codecs="mp4a.40.5" bandwidth="67071" audioSamplingRate="48000">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
   </Representation>
  </AdaptationSet>

  <AdaptationSet id="3" mimeType="image/jpeg" contentType="image">
    <SegmentTemplate media="$RepresentationID$/tile_$Number$.jpg" duration="100" startNumber="1"/>
    <Representation bandwidth="12288" id="thumbnails_320x180" width="3200" height="180">
      <EssentialProperty schemeIdUri="http://dashif.org/thumbnail_tile" value="10x1"/>
    </Representation>
  </AdaptationSet>

 </Period>
</MPD>

DASH in GStreamer

Since 2012, GStreamer includes only a DASH client called dashdemux whereas, for HLS, it provides the both elements, demuxer and sink.

DASH Demuxer

This element landed in the GStreamer repository in 2012 and had evolved a lot since that time to support the various use-cases DASH offers in its specifications and its applications. Indeed it is able to support multiple streams (video/audio/subtitles) and allows the user to select from the available streams or automatically select the best representation according to the network.

DASH Sink

A first attempt to propose a DASH sink was also in 2012. The design was not mature enough to be landed and the author never had the time to complete it.

In 2014, a new element called splitmuxsink was introduced. It handles the most complex part of creating a fragmented stream, it cuts files with synchronized audio and video. Based on this element, a new HLS sink called hlssink2 was created in 2017. I decided to finally create a DASH sink based on this approach to fill in the gap in GStreamer.

MPD Parser

In order to unify the MPD support, a first task has been to unify the base classes to read, and write, an MPD file. Based on XML, the Media Presentation Description scheme is composed with multiple nodes owning children and properties as described above. A first important work item was to split the code in ‘objects’ each identifying an XML node from the MPD schema, including the root node, periods, adaptation sets, etc. An object oriented approach has been selected to unify the work regarding the parsing, object property manipulation and the XML format generation.

The sink

Inspired from the work on hlssink2, the dash sink is a “super bin” and includes a splitmuxsink to provide the multiple media segments. Most of the challenge, here, was to write the MPD compliant with the DASHIF conformance test here with usable and suitable media segments.

This plugin is now capable of:

• Multiple input audio/video streams
• Multiple periods
• TS segment supported (MP4 support is not yet complete), need additional work for the segment transition in short segment scheme.
• Fragment segment with given duration
• Static/Dynamic MPD (Passing the DASH-IF conformance test)

An example is better than hundred words

In the following pipeline a static MPD file is created in /tmp along with one single segment long for a single video stream during a period of 60s. The segment will be encoded as H.264 and encapsulated in MPEG transpor stream files.

$ gst-launch-1.0 -m dashsink name=dashsink mpd-root-path=/tmp target-duration=60 dynamic=false period-duration=60000 muxer=ts  v4l2src ! video/x-raw,framerate=30/1,width=320,height=240 ! videoconvert ! queue ! x264enc bitrate=400 ! dashsink.video_0

A further work item is to implement support for creating MP4 fragment files following the CMAF (MPEG-A Part 19) specification.

If you would like to learn more about DashSink or any other parts of GStreamer, please contact us

UPDATE: gst-build is now at the root of the main GStreamer git repository In September 2021, the GStreamer community merged most of the git repositories into a single one. The build system referred in this post as “gst-build” is now in the root of this combined/mono repository.

Cross-compiling with gst-build

gst-build is one of the two build system used by the community to hack into the whole GStreamer solution. A previous blogpost has been released to present gst-build and how to get started with it.

So I will go straight to the point regarding the cross compilation.

Here is my experience to perform a cross-build which can be very useful when you want to save precious build time or be able to work on both host and target with the same base code.

In this post, I will target an aarch64 CPU for the Xilinx reference board: Zynq UltraScale+ MPSoC ZCU106 Evaluation Kit

Prerequisites

• Toolchain (aarch64-linux-gnu-gcc) + sysroot (optional)
• Meson cross file
• Meson > 0.54

First we’ll need here to have a proper toolchain to cross-build. In my case I used the regular toolchain provided by Ubuntu installing the packages:

$ sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu libc6-dev-arm64-cross

This is installing a minimal toolchain in /usr/aarch64-linux-gnu/

Cross file generated with generate-cross-file.py

Below the cross file used to build for aarch64, this file has been generated with this helper script allowing to generated the cross file for other target as well. As you can see, here, we don’t use a dedicated rootfs because gst-build will build all that we need for the GStreamer essentials (including libffi, glib etc.).

$ ./generate-cross-file.py

[host_machine]
system = 'linux'
cpu_family = 'aarch64'
cpu = 'arm64'
endian = 'little'

[properties]
c_args = []
cpp_args = []
objc_args = []
objcpp_args = []
c_link_args = []
cpp_link_args = []
objc_link_args = []
objcpp_link_args = []
pkg_config_libdir = ['__unknown_sysroot__']


[binaries]
c = ['aarch64-linux-gnu-gcc']
cpp = ['aarch64-linux-gnu-g++']
ar = ['aarch64-linux-gnu-ar']
pkgconfig = 'pkg-config'
strip = ['aarch64-linux-gnu-strip']

Here Meson will use aarch64-linux-gnu-gxx to compile with the given arguments setup above. As Meson does not recommend to use environment variables, the cross file contains hard-coded path to the sysroot to provide package config. Indeed since Meson > 0.54, you can define pkg_config_libdir which will help pkg-config to search for the package configuration files for the given target. You can also tell the path to the pkg-config wrapper by modifying the pkgconfig variables as well. Predefined cross file can also be found in gst-build/data/cross-files

Configuring the project for Zynq UltraScale+ MPSoC ZCU106 Evaluation Board

When the cross file ready, we can now configure gst-build in order to have a dedicated build for our platform. Here I’m disabling some unnecessary options of gst-build such as libav, vaapi or gtk_doc.

⚠ Note: Make sure that you are running Meson 0.54.1 which has the necessary patch for complete support of cross-compilation, otherwise gst-build will take glib from the system (pkg_config_libdir prerequisite).

Notice that on this platform, we use gst-omx, so we also give some options specific to this platform, in particular the path to the OpenMAX headers from Xilinx.**

$ /path/to/meson_0_54 build-cross-arm64 --cross-file=my-meson-cross-file.txt -D omx=enabled -D sharp=disabled -D gst-omx:header_path=/opt/allegro-vcu-omx-il/omx_header -D gst-omx:target=zynqultrascaleplus -D libav=disabled -D rtsp_server=disabled -D vaapi=disabled -D disable_gst_omx=false -Dugly=disabled -Dgtk_doc=disabled -Dglib:libmount=disabled -Dintrospection=disabled

After this step, you should be able to build with ninja.

$ ninja -C build-cross-arm64

Installing

Last but not the least, you need to install the artifacts in a folder to deploy on the device, for example, by mounting it your target with NFS. You have to provide a DESTDIR variable to ninja and it will install in $DESTDIR/usr/local/ as install prefix.

$ DESTDIR=/opt/gst-build-cross-artifacts/linux_arm64 ninja -C build-cross-arm64 install

Running the binaries on target

After mounting the folder or copying it to your target, you have to set up a few variables to be able to run GStreamer pipelines:

• PATH=$DESTDIR/usr/local/bin:$PATH
• LD_LIBRARY_PATH=$DESTDIR/usr/local/lib:$LD_LIBRARY_PATH
• GST_PLUGIN_PATH=$DESTDIR/usr/local/lib/gstreamer-1.0
• GST_OMX_CONFIG_DIR=$DESTDIR/usr/local/etc/xdg

A python script is also available to setup the correct environment variables for your target.

$ /path_to/cross-gst-uninstalled.py /opt/gst-build-cross-artifacts/linux_arm64

Building wavpack in gst-plugins-good

To build a plugin such as wavpack which depends on the 3rd party wavpack library. You’ll need to get a proper sysroot with this new library and its dependencies (if needed).

Regarding a root file-system with wavpack, I generated one with cerbero where cross compiling could be described in a next blog post :) But you should normally have it as part of your sysroot.

$ cd /opt
$ git clone https://gitlab.freedesktop.org/gstreamer/cerbero
$ cd cerbero
$ ./cerbero-uninstalled -c config/cross-lin-arm64.cbc bootstrap
$ ./cerbero-uninstalled -c config/cross-lin-arm64.cbc build wavpack

This should have generated a minimal root file-system in /opt/cerbero/build/dist/linux_arm64 which can used then with gst-build as a base root file-system.

You can now generate a new cross file with the given root file-system as parameter.

$ ./generate-cross-file.py --sysroot /opt/cerbero/build/dist/linux_arm64/ --no-include-sysroot

Here I define a sysroot to be be used but I’m disabling the use of sys_root in the cross file to avoid Meson to tell pkg-config to prefix every path with this value. cerbero is generating pkg-config files with the sysroot path already in each pc files.

[host_machine]
system = 'linux'
cpu_family = 'aarch64'
cpu = 'arm64'
endian = 'little'

[properties]
c_args = []
cpp_args = []
objc_args = []
objcpp_args = []
c_link_args = ['-L/opt/cerbero/build/dist/linux_arm64', '-Wl,-rpath-link=/opt/cerbero/build/dist/linux_arm64']
cpp_link_args = ['-L/opt/cerbero/build/dist/linux_arm64', '-Wl,-rpath-link=/opt/cerbero/build/dist/linux_arm64']
objc_link_args = ['-L/opt/cerbero/build/dist/linux_arm64', '-Wl,-rpath-link=/opt/cerbero/build/dist/linux_arm64']
objcpp_link_args = ['-L/opt/cerbero/build/dist/linux_arm64', '-Wl,-rpath-link=/opt/cerbero/build/dist/linux_arm64']
pkg_config_libdir = ['/opt/cerbero/build/dist/linux_arm64/pkgconfig:/opt/NFS/cerbero_rootfs/linux_arm64//usr/share/pkgconfig']


[binaries]
c = ['aarch64-linux-gnu-gcc']
cpp = ['aarch64-linux-gnu-g++']
ar = ['aarch64-linux-gnu-ar']
pkgconfig = 'pkg-config'
strip = ['aarch64-linux-gnu-strip']

Now you should be able to go back to the configure/build/install step and get the wavpack in your plugins registry.

I hope you’ll enjoy the use of gst-build, which is for me a very powerful and flexible tool. A lot of options can be found in the gst-build README such as the update or the use of GStreamer branches.

If you would like to learn more about gst-build or any other parts of GStreamer, please contact us

UPDATE: gst-build is now at the root of the main GStreamer git repository In September 2021, the GStreamer community merged most of the git repositories into a single one. The build system referred in this post as “gst-build” is now in the root of this combined/mono repository.

How to add a new log in GStreamer using gst-build The State of the Art:

GStreamer relies on multiple repositories such as base and good to build its ecosystem, and now owns more than 30 projects in Gitlab. So, a unified tool/build system has always been necessary to build a specified version.

For over a decade, a script named gst-uninstalled was present in the gstreamer/scripts directory to build the whole solution. Although this tool was not very flexible and was missing some options in the command line, it was good enough if you wanted to tackle a surprising bug in our favorite framework. But it was not as good at providing a real swiss-army knife approach to build GStreamer and its dependencies.

Another build system called Cerbero, created a few years ago, provides a standalone solution to build GStreamer packages. This solution offers a wide range of options in addition to a proper sandbox to avoid system dependencies and to be able to prepare packages that include third party software dependencies for a given version. Cerbero is written in Python and can create builds for the host machine like gst-uninstalled but also for various common targets depending on the host. Indeed a Linux regular desktop host will be capable to cross-build GStreamer for x86(32/64bits) but also for architecture such ARM and system such as Microsoft Windows. It can also create builds for Android and iOS. Despite a shell environment allowing artifacts testing, Cerbero is not really convenient for a day to day development related to GStreamer as a new plugin development or a bug fix as it is not easy to update to the last revision without loosing a current work, or to test another branch of GStreamer

The Rise of gst-build:

In order to improve this situation, gst-build was born. Taking advantage of the flexibility of the rising build system, meson, gst-build has been implemented to replace gst-uninstalled and provide a quick and smooth environment to hack into GStreamer and its dependencies.

Autotools Is Dead, Long Live Meson:

Since GStreamer 1.18, meson has been chosen as the only build system for the official GStreamer repositories. For its simplicity, speed and flexibility, meson replaced autotools, so it is also perfect to use with gst-build. Indeed gst-build is just a meson project including GStreamer sub-projects with options to enable/disable selected sub-projects.

So lets take a look on how to get started with gst-build:

A first step using gst-build: A few ‘bits’ about it:

gst-build is mainly a meson.build project. It reads .wrap files which are located in the subprojects folder to determine the elements of the project such as GStreamer or gst-plugins-base. These subprojects use the meson build system as well. gst-build comes with the essential projects you need to start using GStreamer and build it almost without system dependencies. gst-build bundles libffi or glib in the subprojects directory. It can also gather dependencies using pkg-config from the system to build the GStreamer plugins such as flac, for example, which needs libflac to build.

Environment

As we have to choose a real development environment, a 64 bit machine has been selected:

• Ubuntu 18.04
• Bash Shell

Prerequisites

• build-essential (gcc)
• python3
• git
• meson
• ninja

Install meson and ninja

Here are the essential dependencies you need to install before running meson and ninja.

$ sudo apt install build-essential python3 git ninja python3-pip

You can now install meson from the pip repository.

$ pip3 install --user meson

This will install meson into ~/.local/bin which may or may not be included automatically in your PATH by default.

Fetch and Configure

This step will download the GStreamer repositories including some dependencies such as glib etc. into the subprojects folder. Basically it tries to download as many mesonified third party libraries as possible, and breaking news the cmake ones, as a bridge has been implemented recently if necessary.

$ git clone https://gitlab.freedesktop.org/gstreamer/gst-build
$ cd gst-build
$ meson build --buildtype=debug

...

All GStreamer modules 1.17.0.1

  Subprojects
                        FFmpeg: YES
                         dssim: YES
                    gl-headers: YES
                      graphene: YES
                  gst-devtools: YES
          gst-editing-services: YES
                  gst-examples: YES
    gst-integration-testsuites: YES
                     gst-libav: YES
                       gst-omx: YES
               gst-plugins-bad: YES
              gst-plugins-base: YES
              gst-plugins-good: YES
                gst-plugins-rs: NO
              gst-plugins-ugly: YES
                    gst-python: NO
               gst-rtsp-server: YES
                     gstreamer: YES
               gstreamer-sharp: Feature 'sharp' disabled
               gstreamer-vaapi: YES
                         gtest: NO
                   libmicrodns: YES
                       libnice: YES
                        libpsl: YES
                       libsoup: NO
                      openh264: YES
                           orc: YES
                     pygobject: NO
                        sqlite: YES
                      tinyalsa: NO
                          x264: YES
Option buildtype is: debug [default: debugoptimized]
Found ninja-1.8.2 at /usr/bin/ninja

After this step, a newly created folder named build should be ready to be used by ninja to build the binaries.

As you may notice, --buildtype=debug has been added to the command line to get a fully debugable result without optimization. I invite you to this page if you want to fine-tune the build.

Build gst-build

This step will build all GStreamer libraries in addition to the plugins from base/good/bad/ugly/libav if their dependencies have been met or built by gst-build (ie glib, openh264 etc.).

$ ninja -C build

Test gst-build

This command will create an environment where all tools and plugins built previously are available in the environment as a superset of the system environment with the right environment variables set.

$ ninja -C build devenv

A prefix to your prompt should be shown as

[gst-master] bash-prompt $

[gst-master] bash-prompt $ env | grep GST_

From this environment you are now ready to use the power of GStreamer, and even implement new features in it without the fear of using out of date version.

From this shell, you are also able to compile without exiting the environment except when a configure step is necessary. This feature is very convenient to test a branch or fix a bug. Go to the subprojects folder and modify the code directly and then call ninja -C ../../build.

[gst-master] bash-prompt $ gst-inspect-1.0

Let’s add a log line in gst-plugins-base

In this tutorial, I will explain how to add a log line in videotestsrc element, gst-plugins-base’s plugin, rebuild using gst-build and test that the new log is now displayed.

Edit the file

vim subprojects/gst-plugins-base/gst/videotestsrc/gstvideotestsrc.c

Go to the method gst_video_test_src_start and add the line:

GST_ERROR_OBJECT (src, ""Starting to debug videotestsrc, is there an error ?");

This will add a runtime log with the ERROR level. For more information about debugging facilities in GStreamer, visit the following page

Then close the editor.

Build with gst-build

$ ninja -C build

You should see that only the file gstvideotestsrc.c rebuilt.

Test the changes

In order to enable the logs, you have to export the environment variable GST_DEBUG.

Let’s start the playback and display the result in the terminal. The following command will display all the log from videotestsrc with the category ERROR(1).

GST_DEBUG=videotestsrc:1 gst-launch-1.0 videotestsrc num-buffers=1 ! fakevideosink

You should have this output:

Setting pipeline to PAUSED ...
0:00:00.225273663 21743 0x565528ab7100 ERROR           videotestsrc gstvideotestsrc.c:1216:gst_video_test_src_start:<videotestsrc0> Starting to debug videotestsrc, is there an error ?
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.033464391
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ..

Update gst-build

This command will update all the repositories and will reissue a build.

$ ninja -C build_dir update

Adding a new repository

Better to be outside of devenv env. If you want to add a new repository and work in this environment. Very simple and handy way, you’ll have to:

$ cd subprojects
$ git clone my_subproject
$ cd ../build
$ rm -rf * && meson .. -Dcustom_subprojects=my_subproject

And then you can go in your subproject, edit, change, remove even stare at his beauty.

$ ninja -C ../../build
$ ninja -C ../../build uninstalled

If you would like to learn more about gst-build or any other parts of GStreamer, please contact us