diff options
Diffstat (limited to 'docs/pwg/advanced-qos.xml')
| -rw-r--r-- | docs/pwg/advanced-qos.xml | 283 |
1 files changed, 283 insertions, 0 deletions
diff --git a/docs/pwg/advanced-qos.xml b/docs/pwg/advanced-qos.xml new file mode 100644 index 0000000..a6561bc --- /dev/null +++ b/docs/pwg/advanced-qos.xml @@ -0,0 +1,283 @@ +<chapter id="chapter-advanced-qos"> + <title>Quality Of Service (QoS)</title> + + <para> + Quality of Service in &GStreamer; is about measuring and ajusting + the real-time performance of a pipeline. The real-time performance is + always measured relative to the pipeline clock and typically happens in + the sinks when they synchronize buffers against the clock. + </para> + <para> + When buffers arrive late in the sink, i.e. when their running-time is + smaller than that of the clock, we say that the pipeline is having a + quality of service problem. These are a few possible reasons: + </para> + <itemizedlist> + <listitem> + <para> + High CPU load, there is not enough CPU power to handle the stream, + causing buffers to arrive late in the sink. + </para> + </listitem> + <listitem> + <para> + Network problems + </para> + </listitem> + <listitem> + <para> + Other resource problems such as disk load, memory bottlenecks etc + </para> + </listitem> + </itemizedlist> + <para> + The measurements result in QOS events that aim to adjust the datarate + in one or more upstream elements. Two types of adjustments can be + made: + </para> + <itemizedlist> + <listitem> + <para> + Short time "emergency" corrections based on latest observation in + the sinks. + </para> + <para> + Long term rate corrections based on trends observed in the sinks. + </para> + </listitem> + </itemizedlist> + <para> + It is also possible for the application to artificially introduce delay + between synchronized buffers, this is called throttling. It can be used + to limit or reduce the framerate, for example. + </para> + + <sect1 id="section-measuring"> + <title>Measuring QoS</title> + <para> + Elements that synchronize buffers on the pipeline clock will usually + measure the current QoS. They will also need to keep some statistics + in order to generate the QOS event. + </para> + <para> + For each buffer that arrives in the sink, the element needs to calculate + how late or how early it was. This is called the jitter. Negative jitter + values mean that the buffer was early, positive values mean that the + buffer was late. the jitter value gives an indication of how early/late + a buffer was. + </para> + <para> + A synchronizing element will also need to calculate how much time + elapsed between receiving two consecutive buffers. We call this the + processing time because that is the amount of time it takes for the + upstream element to produce/process the buffer. We can compare this + processing time to the duration of the buffer to have a measurement + of how fast upstream can produce data, called the proportion. + If, for example, upstream can produce a buffer in 0.5 seconds of 1 + second long, it is operating at twice the required speed. If, on the + other hand, it takes 2 seconds to produce a buffer with 1 seconds worth + of data, upstream is producing buffers too slow and we won't be able to + keep sycnhronization. Usually, a running average is kept of the + proportion. + </para> + <para> + A synchronizing element also needs to measure its own performance in + order to figure out if the performace problem is upstream of itself. + </para> + <para> + These measurements are used to construct a QOS event that is sent + upstream. Note that a QoS event is sent for each buffer that arrives + in the sink. + </para> + </sect1> + + <sect1 id="section-handling"> + <title>Handling QoS</title> + <para> + An element will have to install an event function on its source pads + in order to receive QOS events. Usually, the element will need to + store the value of the QOS event and use them in the data processing + function. The element will need to use a lock to protect these QoS + values as shown in the example below. Also make sure to pass the + QoS event upstream. + </para> + <programlisting> +<![CDATA[ + [...] + + case GST_EVENT_QOS: + { + GstQOSType type; + gdouble proportion; + GstClockTimeDiff diff; + GstClockTime timestamp; + + gst_event_parse_qos (event, &type, &proportion, &diff, ×tamp); + + GST_OBJECT_LOCK (decoder); + priv->qos_proportion = proportion; + priv->qos_timestamp = timestamp; + priv->qos_diff = diff; + GST_OBJECT_UNLOCK (decoder); + + res = gst_pad_push_event (decoder->sinkpad, event); + break; + } + + [...] +]]> + </programlisting> + <para> + With the QoS values, there are two types of corrections that an element + can do: + </para> + + <sect2 id="section-handling-short"> + <title>Short term correction</title> + <para> + The timestamp and the jitter value in the QOS event can be used to + perform a short term correction. If the jitter is positive, the + previous buffer arrived late and we can be sure that a buffer with + a timestamp < timestamp + jitter is also going to be late. We + can thus drop all buffers with a timestamp less than timestamp + + jitter. + </para> + <para> + If the buffer duration is known, a better estimation for the next + likely timestamp as: timestamp + 2 * jitter + duration. + </para> + <para> + A possible algorithm typically looks like this: + </para> + <programlisting> +<![CDATA[ + [...] + + GST_OBJECT_LOCK (dec); + qos_proportion = priv->qos_proportion; + qos_timestamp = priv->qos_timestamp; + qos_diff = priv->qos_diff; + GST_OBJECT_UNLOCK (dec); + + /* calculate the earliest valid timestamp */ + if (G_LIKELY (GST_CLOCK_TIME_IS_VALID (qos_timestamp))) { + if (G_UNLIKELY (qos_diff > 0)) { + earliest_time = qos_timestamp + 2 * qos_diff + frame_duration; + } else { + earliest_time = qos_timestamp + qos_diff; + } + } else { + earliest_time = GST_CLOCK_TIME_NONE; + } + + /* compare earliest_time to running-time of next buffer */ + if (earliest_time > timestamp) + goto drop_buffer; + + [...] +]]> + </programlisting> + </sect2> + + <sect2 id="section-handling-long"> + <title>Long term correction</title> + <para> + Long term corrections are a bit more difficult to perform. They + rely on the value of the propertion in the QOS event. Elements should + reduce the amount of resources they comsume by the proportion + field in the QoS message. + </para> + <para> + Here are some possible strategies to achieve this: + </para> + <itemizedlist> + <listitem> + <para> + Permanently dropping frames or reducing the CPU or bandwidth + requirements of the element. Some decoders might be able to + skip decoding of B frames. + </para> + </listitem> + <listitem> + <para> + Switch to lower quality processing or reduce the algorithmic + complexity. Care should be taken that this doesn't introduce + disturbing visual or audible glitches. + </para> + </listitem> + <listitem> + <para> + Switch to a lower quality source to reduce network bandwidth. + </para> + </listitem> + <listitem> + <para> + Assign more CPU cycles to critical parts of the pipeline. This + could, for example, be done by increasing the thread priority. + </para> + </listitem> + </itemizedlist> + <para> + In all cases, elements should be prepared to go back to their normal + processing rate when the proportion member in the QOS event approaches + the ideal proportion of 1.0 again. + </para> + </sect2> + </sect1> + + <sect1 id="section-throttle"> + <title>Throttling</title> + <para> + Elements synchronizing to the clock should expose a property to configure + them in throttle mode. In throttle mode, the time distance between buffers + is kept to a configurable throttle interval. This means that effectively + the buffer rate is limited to 1 buffer per throttle interval. This can be + used to limit the framerate, for example. + </para> + <para> + When an element is configured in throttling mode (this is usually only + implemented on sinks) it should produce QoS events upstream with the jitter + field set to the throttle interval. This should instruct upstream elements to + skip or drop the remaining buffers in the configured throttle interval. + </para> + <para> + The proportion field is set to the desired slowdown needed to get the + desired throttle interval. Implementations can use the QoS Throttle type, + the proportion and the jitter member to tune their implementations. + </para> + <para> + The default sink base class, has the <quote>throttle-time</quote> + property for this feature. You can test this with: + <command>gst-launch-1.0 videotestsrc ! + xvimagesink throttle-time=500000000</command> + </para> + </sect1> + + <sect1 id="section-messages"> + <title>QoS Messages</title> + <para> + In addition to the QOS events that are sent between elements in the + pipeline, there are also QOS messages posted on the pipeline bus to + inform the application of QoS decisions. The QOS message contains + the timestamps of when something was dropped along with the amount + of dropped vs processed items. Elements must post a QOS + message under these conditions: + </para> + <itemizedlist> + <listitem> + <para> + The element dropped a buffer because of QoS reasons. + </para> + </listitem> + <listitem> + <para> + An element changes its processing strategy because of QoS reasons + (quality). This could include a decoder that decides to drop every + B frame to increase its processing speed or an effect element + switching to a lower quality algorithm. + </para> + </listitem> + </itemizedlist> + </sect1> + +</chapter> |