xref: /third_party/EGL/extensions/EXT/EGL_EXT_yuv_surface.txt

Name

    EXT_yuv_surface

Name Strings

    EGL_EXT_yuv_surface

Contributors

    Matt Trusten
    Jeff Vigil
    Arshad Bebal
    Mohan Maiya
    Amit Bansal
    Tom Longo
    Eric Engestrom

Contacts

    Jeff Vigil, Qualcomm (jvigil 'at' qualcomm.com)

Notice

    Copyright (c) 2014 Qualcomm Technologies, Inc. All Rights Reserved.
    Qualcomm Technologies Proprietary and Confidential

Status

    Complete

Version

    Version 9, May 4th, 2017

Number

    EGL Extension #86

Dependencies

    EGL 1.4 is required.

    This extension is written against the wording of the EGL 1.4
    Specification.

Overview

    This extension defines a set of new EGL configuration attributes and values
    which allows EGL to create and use YUV surfaces.

    YUV formats can be described using a few simple parameters, and every format
    can be given with these six parameters. These parameters include the color
    order, the number of planes, subsample, plane depth, color conversion and
    depth range.

    This extension describes how EGL will handle YUV surfaces, but requires that
    the client API describe how to fill such a surface. An example of such an
    extension would be GL_EXT_yuv_target.

New Types

    None

New Procedures and Functions

    None

New Tokens

    A new set of EGLConfig attributes are available:
        EGL_YUV_ORDER_EXT                               0x3301
        EGL_YUV_NUMBER_OF_PLANES_EXT                    0x3311
        EGL_YUV_SUBSAMPLE_EXT                           0x3312
        EGL_YUV_DEPTH_RANGE_EXT                         0x3317
        EGL_YUV_CSC_STANDARD_EXT                        0x330A
        EGL_YUV_PLANE_BPP_EXT                           0x331A

    Accepted as a new value for the EGL_COLOR_BUFFER_TYPE attribute:
        EGL_YUV_BUFFER_EXT                              0x3300

    Accepted values for the EGL_YUV_ORDER_EXT attribute:
        EGL_YUV_ORDER_YUV_EXT                           0x3302
        EGL_YUV_ORDER_YVU_EXT                           0x3303
        EGL_YUV_ORDER_YUYV_EXT                          0x3304
        EGL_YUV_ORDER_UYVY_EXT                          0x3305
        EGL_YUV_ORDER_YVYU_EXT                          0x3306
        EGL_YUV_ORDER_VYUY_EXT                          0x3307
        EGL_YUV_ORDER_AYUV_EXT                          0x3308

    Accepted values for the EGL_YUV_SUBSAMPLE_EXT attribute:
        EGL_YUV_SUBSAMPLE_4_2_0_EXT                     0x3313
        EGL_YUV_SUBSAMPLE_4_2_2_EXT                     0x3314
        EGL_YUV_SUBSAMPLE_4_4_4_EXT                     0x3315

    Accepted values for the EGL_YUV_DEPTH_RANGE_EXT attribute:
        EGL_YUV_DEPTH_RANGE_LIMITED_EXT                 0x3318
        EGL_YUV_DEPTH_RANGE_FULL_EXT                    0x3319

    Accepted values for the EGL_YUV_CSC_STANDARD_EXT attribute:
        EGL_YUV_CSC_STANDARD_601_EXT                    0x330B
        EGL_YUV_CSC_STANDARD_709_EXT                    0x330C
        EGL_YUV_CSC_STANDARD_2020_EXT                   0x330D

    Accepted values for the EGL_YUV_PLANE_BPP_EXT attribute:
        EGL_YUV_PLANE_BPP_0_EXT                         0x331B
        EGL_YUV_PLANE_BPP_8_EXT                         0x331C
        EGL_YUV_PLANE_BPP_10_EXT                        0x331D

Additions to Chapter 2 of the EGL 1.4 Specification (EGL Operation)

Change the options of paragraph 6 from section 2.2: Rendering Contexts and
Drawing Surfaces

    A context can be used with any EGLSurface that it is compatible with
    (subject to the restrictions discussed in the section on address space). A
    surface and context are compatible if
        * They support the same type of color buffer (RGB or luminance)

To the following:

    A context can be used with any EGLSurface that it is compatible with
    (subject to the restrictions discussed in the section on address space). A
    surface and context are compatible if
        * They support the same type of color buffer (RGB, YUV, or luminance)

Additions to Chapter 3 of the EGL 1.4 Specification (EGL Functions and Errors)

Change the section marked from Buffer Descriptions and Attributes in Section 3.4
to:

    The Color Buffer

        The color buffer contains pixel color values, and is shared by all
    client APIs rendering to a surface.
        EGL_COLOR_BUFFER_TYPE indicates the color buffer type, and must be
    either EGL_RGB_BUFFER for an RGB color buffer, EGL_LUMINANCE_BUFFER for a
    luminance color buffer, or EGL_YUV_BUFFER_EXT for a YUV color buffer.  For
    an RGB buffer, EGL_RED_SIZE, EGL_GREEN_SIZE, EGL_BLUE_SIZE must be non-zero,
    and EGL_LUMINANCE_SIZE must be zero.  For a luminance buffer, EGL_RED_SIZE,
    EGL_GREEN_SIZE, EGL_BLUE_SIZE must be zero, and EGL_LUMINANCE_SIZE must be
    non-zero. For both RGB and luminance color buffers, EGL_ALPHA_SIZE may be
    zero or non-zero (the latter indicates the existence of a destination alpha
    buffer).  For a YUV buffer, all of EGL_RED_SIZE, EGL_GREEN_SIZE, EGL_BLUE_-
    SIZE, EGL_LUMINANCE_SIZE, and EGL_ALPHA_SIZE must be zero. Instead, the
    depth in bits per pixel for YUV is described using EGL_YUV_PLANE_BPP_EXT,
    which describes the size of a single pixel in the Y plane, and the other
    plane depths are derived from this.
        EGL_YUV_ORDER_EXT is used to describe the plane order.
        EGL_YUV_NUMBER_OF_PLANES_EXT describes the number of planes that will
    be used for this surface. The allowed values for EGL_YUV_NUMBER_OF_PLANES_-
    EXT must be greater than zero and not more than three.
        EGL_YUV_SUBSAMPLE_EXT allows the user to decide how the surface will be
    subsampled.

    Valid combinations are:

        EGL_YUV_SUBSAMPLE_EXT         EGL_YUV_NUMBER_OF_PLANES_EXT   EGL_YUV_ORDER_EXT          EGL_YUV_PLANE_BPP_EXT
        -------------------------     -----------------------------  ------------------         ------------------------
        EGL_YUV_SUBSAMPLE_4_2_0_EXT                2 or 3            EGL_YUV_ORDER_YUV_EXT or   EGL_YUV_PLANE_BPP_8_EXT or
                                                                     EGL_YUV_ORDER_YVU_EXT      EGL_YUV_PLANE_BPP_10_EXT

        EGL_YUV_SUBSAMPLE_4_2_2_EXT                  1               EGL_YUV_ORDER_YUYV_EXT or  EGL_YUV_PLANE_BPP_8_EXT or
                                                                     EGL_YUV_ORDER_YVYU_EXT or  EGL_YUV_PLANE_BPP_10_EXT
                                                                     EGL_YUV_ORDER_UYVY_EXT or
                                                                     EGL_YUV_ORDER_VYUY_EXT

        EGL_YUV_SUBSAMPLE_4_2_2_EXT               2 or 3             EGL_YUV_ORDER_YUV_EXT or   EGL_YUV_PLANE_BPP_8_EXT or
                                                                     EGL_YUV_ORDER_YVU_EXT      EGL_YUV_PLANE_BPP_10_EXT

        EGL_YUV_SUBSAMPLE_4_4_4_EXT                  1               EGL_YUV_ORDER_AYUV_EXT     EGL_YUV_PLANE_BPP_8_EXT or
                                                                                                EGL_YUV_PLANE_BPP_10_EXT

    Usage examples:

        Format                      Combination
        ------------------          --------------------------------------------------------------
        NV12                        EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_0_EXT
         - YUV420                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 2
         - 2 Planes : Y + UV        EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YUV_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_8_EXT

        NV21                        EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_0_EXT
         - YUV420                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 2
         - 2 Planes : Y + VU        EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YVU_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_8_EXT

        YV12                        EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_0_EXT
         - YUV420                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 3
         - 3 Planes : Y + V + U     EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YVU_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_8_EXT

        YUY2                        EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_2_EXT
         - YUV422                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 1
         - 1 Plane : Y + U + Y + V  EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YUYV_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_8_EXT

        AYUV                        EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_4_4_EXT
         - YUV444                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 1
         - 1 Plane : A + Y + U + V  EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_AYUV_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_8_EXT

        YUV420 Semi-Planar 10 Bit   EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_0_EXT
         - YUV420                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 2
         - 2 Planes : Y + UV        EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YUV_EXT
         - 10 bit depth             EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_10_EXT

        YUV422 Interleaved 10 Bit   EGL_YUV_SUBSAMPLE_EXT          = EGL_YUV_SUBSAMPLE_4_2_2_EXT
         - YUV422                   EGL_YUV_NUMBER_OF_PLANES_EXT   = 1
         - 1 Plane : Y + U + Y + V  EGL_YUV_ORDER_EXT              = EGL_YUV_ORDER_YUYV_EXT
         - 8 bit depth              EGL_YUV_PLANE_BPP_EXT          = EGL_YUV_PLANE_BPP_10_EXT

        EGL_YUV_PLANE_BPP_EXT describes the bit depth for the different
    planes of a YUV surface. The available options are EGL_YUV_PLANE_BPP_0_-
    EXT, EGL_YUV_PLANE_BPP_8_EXT and EGL_YUV_PLANE_BPP_10_EXT. If EGL_YUV_-
    PLANE_BPP_0_EXT is specified, no color buffer will be created for the
    surface.
        EGL_YUV_DEPTH_RANGE_EXT describes the range of the pixel value and is
    dependent on the EGL_YUV_PLANE_BPP_EXT setting:

        EGL_YUV_PLANE_BPP_EXT        EGL_YUV_DEPTH_RANGE_LIMITED_EXT   EGL_YUV_DEPTH_RANGE_FULL_EXT
                                     (Inclusive)                       (Inclusive)
        -------------------------    --------------------------------  -----------------------------
        EGL_YUV_PLANE_BPP_8_EXT or   Y: 16 to 235, UV: 16 to 240       Y: 0 to 255,  UV: 0 to 255
        EGL_YUV_PLANE_BPP_10_EXT     Y: 64 to 940, UV: 64 to 960       Y: 0 to 1023, UV: 0 to 1023

        If OpenGL or OpenGL ES rendering is supported for a luminance color
    buffer (as described by the value of the EGL_RENDERABLE_TYPE attribute,
    described below), it is treated as RGB rendering with the value of
    GL_RED_BITS equal to EGL_LUMINANCE_SIZE and the values of GL_GREEN_BITS and
    GL_BLUE_BITS equal to zero. The red component of fragments is written to the
    luminance channel of the color buffer, the green and blue components are
    discarded, and the alpha component is written to the alpha channel of the
    color buffer (if present).
        When rendering to the YUV surface, if EGL_YUV_ORDER_EXT is not EGL_-
    YUV_ORDER_AYUV_EXT, the alpha channel will always be fully opaque.
    Otherwise it is defined by the value provided to the alpha sample.
        EGL_BUFFER_SIZE gives the total of the color component bits of the color
    buffer for EGL_RGB_BUFFER or for EGL_LUMINANCE_BUFFER. For an RGB color
    buffer, the total is the sum of EGL_RED_SIZE, EGL_GREEN_SIZE, EGL_BLUE_SIZE,
    and EGL_ALPHA_SIZE. For a luminance color buffer, the total is the sum of
    EGL_LUMINANCE_SIZE and EGL_ALPHA_SIZE. When EGL_COLOR_BUFFER_TYPE is of type
    EGL_YUV_BUFFER_EXT, this will reflect the enumeration provided
    as an integer) for EGL_YUV_PLANE_BPP_EXT, giving a value of 0, 8 or 10.

   Other EGLConfig Attribute Descriptions

        EGL_YUV_CSC_STANDARD_EXT can be set to either EGL_YUV_CSC_STANDARD_-
    601_EXT, EGL_YUV_CSC_STANDARD_709_EXT, or EGL_YUV_CSC_2020_EXT. If the
    standard chosen is EGL_YUV_CSC_STANDARD_709_EXT, then the color conversion
    follows the ITU-R BT.709 standard. If EGL_YUV_CSC_STANDARD_EXT is set to
    EGL_YUV_CSC_2020_EXT, then the color conversion will be processed based on
    ITU-R BT.2020.


Change option 2 in the section marked as 3.4.1.2 Sorting of EGLConfigs to:

    2. Special: by EGL_COLOR_BUFFER_TYPE where the precendence is EGL_RGB_-
    BUFFER, EGL_LUMINANCE_BUFFER, EGL_YUV_BUFFER_EXT.

Change option 3 in the section marked as 3.4.1.2 Sorting of EGLConfigs to:

    3. Special: by larger total number of color bits (for an RGB color buffer,
    this is the sum of EGL_RED_SIZE, EGL_GREEN_SIZE, EGL_BLUE_SIZE, and
    EGL_ALPHA_SIZE; for a luminance color buffer, the sum of EGL_LUMINANCE_SIZE
    and EGL_ALPHA_SIZE; for YUV color buffers, this returns the integer value
    with respect to the enumeration provided for EGL_YUV_PLANE_BPP_EXT) If the
    requrested number of bits in attrib_list for a particular color component
    is 0 or EGL_DONT_CARE, then the number of bits for the component is not
    considered. Due to this, YUV color buffers are always last based on this
    rule.

The following options should be added between options 9 and 10 in section
3.4.1.2 (EGL_ALPHA_MASK_SIZE and EGL_NATIVE_VISUAL_TYPE):

    10. Special: EGL_YUV_ORDER_EXT will be sorted in the following order:
    EGL_NONE, EGL_YUV_ORDER_YUV_EXT, EGL_YUV_ORDER_YVU_EXT, EGL_YUV_ORDER_-
    YUYV_EXT, EGL_YUV_ORDER_YVYU_EXT, EGL_YUV_ORDER_UYVY_EXT, EGL_YUV_ORDER_-
    VYUY_EXT, and EGL_YUV_ORDER_AYUV_EXT.

Change option 10 in section 3.4.1.2 (EGL_NATIVE_VISUAL_TYPE) to:

    11. Special: by EGL_NATIVE_VISUAL_TYPE (the actual sort order is
    implementation-defined, depending on the meaning of native visual types).

New State

    Add to table 3.1 (EGLConfig Attributes):

    Attribute                       Type      Notes
    -----------------------------   ----      --------------------------------------------
    EGL_YUV_ORDER_EXT               enum      The order in which the samples will be found
                                              inside the surface
    EGL_YUV_NUMBER_OF_PLANES_EXT    integer   Number of planes for the surface, in the range of [1,3]
    EGL_YUV_SUBSAMPLE_EXT           enum      Describes the sampling rate of the different planes.
    EGL_YUV_DEPTH_RANGE_EXT         enum      Luma plane range. limited is [16,240] and
                                              full range is [0,255]
    EGL_YUV_CSC_STANDARD_EXT        enum      The standard used for color conversion.
    EGL_YUV_PLANE_BPP_EXT           enum      How many bits are used for each plane of
                                              the YUV surface

    Add to table 3.4 (Default values and match criteria for EGLConfig
    attributes):

    Attribute                       Default                           Selection  Sort     Sort
                                                                      Criteria   Order    Priority
    -------------------------       --------------------------------  ---------  -------  --------
    EGL_YUV_ORDER_EXT               EGL_DONT_CARE                     Exact      Special  10
    EGL_YUV_NUMBER_OF_PLANES_EXT    0                                 At least   None
    EGL_YUV_SUBSAMPLE_EXT           EGL_DONT_CARE                     Exact      None
    EGL_YUV_DEPTH_RANGE_EXT         EGL_DONT_CARE                     Exact      None
    EGL_YUV_CSC_STANDARD_EXT        EGL_DONT_CARE                     Exact      None
    EGL_YUV_PLANE_BPP_EXT           EGL_DONT_CARE                     Exact      None

Issues
    1.  How will a EGL surface created with a YUV config properly detect that
        valid values were used when filling pixel data?

        This problem occurs in situations when a specification which defines
        rendering to the YUV surface cannot guarantee values that conform to the
        configuration attributes. One possible situation for this is when a
        the GL client is using GL_EXT_yuv_target and the EGL YUV surface is the
        draw buffer. The shader can transform the values inside the shader, and
        there is no guarantee that the values will fall into the EGL_YUV_DEPTH_-
        RANGE_EXT value chosen. Other client specifications could have similar
        issues and it may not be performant for some hardware to detect this
        discrepancy. This includes interactions with extensions such as EGL_-
        lock_surface, or any others that provide a mechanism to change the
        contents of the YUV surface.

        One of the problems that can occur in situations like this will be
        compound mathematical error should operations need to take place on the
        buffer data.

        Proposal: Due to the performance problems or inability in some hardware
        to deal with this issue, the pixel contents will be undefined and any
        further operations to the pixel contents will also be undefined.
        Therefore it is up to the application to guarantee the correct values
        are stored within the surface. The EGL implementation need not attempt
        any form of verification on the pixel data of the YUV surface, nor does
        it have to guarantee the pixel data, even with communicating the surface
        to other modules through a post or any other operation EGL performs.

    2.  Should an EGL_COLOR_BUFFER_TYPE = EGL_DONT_CARE request enumerate all
        YUV configs?

        Resolved: Revision #9 changes the default values and selection criteria
        to allow for full enumeration of all YUV configs. The default value of
        EGL_COLOR_BUFFER_TYPE remains EGL_RGB_BUFFER, so this change only
        affects explicit EGL_DONT_CARE requests.

Example Configuration for NV12:

    const EGLint config_attribs[] =
    {
        EGL_SURFACE_TYPE,              EGL_WINDOW_BIT,
        EGL_RENDERABLE_TYPE,           EGL_OPENGL_ES2_BIT,
        EGL_COLOR_BUFFER_TYPE,         EGL_YUV_BUFFER_EXT,
        EGL_YUV_ORDER_EXT,             EGL_ORDER_YUV_EXT,
        EGL_YUV_NUMBER_OF_PLANES_EXT,  2,
        EGL_YUV_SUBSAMPLE_EXT,         EGL_YUV_SUBSAMPLE_4_2_0_EXT,
        EGL_YUV_DEPTH_RANGE_EXT,       EGL_YUV_DEPTH_RANGE_LIMITED_EXT,
        EGL_YUV_CSC_STANDARD_EXT,      EGL_YUV_CSC_STANDARD_601_EXT,
        EGL_YUV_PLANE_BPP_EXT,         EGL_YUV_PLANE_BPP_8_EXT,
        EGL_NONE
    };

Revision History
#1  March    12th, 2014    Initial Draft.

#2  March    25th, 2014    Reformated and added more detail.

#3  March    26th, 2014    Added usage examples.
                           Added memory layout definitions.

#4  March    26th, 2014    Fixed some typos and formatting.

#5  May      21st, 2014    Fixed some inconsistencies.
                           Added EGL_YUV_PLANE_DEPTH_0_QCOM.

#6  May      27th, 2014    Relaxed some requirements.
                           Changed EGL_YUV_PLANE_DEPTH_<0,8,10>_QCOM to
                           EGL_YUV_PLANE_BPP_<0,8,10>_QCOM.
                           Added more detail.

#7  October   7th, 2014    Issue (1) added.
                           Minor updates made (EXT_yuv_target)

#8  October  24th, 2014    Updated for EXT and token values.

#9  April    26th, 2017    Changed attributes default values and selection
                           criteria (see issue #2).
                           Changed status from Draft to Complete.