xref: /third_party/EGL/extensions/NV/EGL_NV_stream_remote.txt

Name

    NV_stream_remote
    NV_stream_cross_object
    NV_stream_cross_display
    NV_stream_cross_process
    NV_stream_cross_partition
    NV_stream_cross_system

Name Strings

    EGL_NV_stream_remote

Contributors

    Daniel Kartch
    Bogdan Naodovic
    James Jones
    Zander Clucas

Contacts

    Daniel Kartch, NVIDIA (dkartch 'at' nvidia.com)

Status

    Draft

Version

    Version 4 - June 01, 2016

Number

    EGL Extension #114

Extension Type

    EGL display extension

Dependencies

    Requires EGL_KHR_stream

    Interacts with EGL_KHR_stream_cross_process_fd

    All EGL_NV_stream_cross_* extensions require EGL_NV_stream_remote,
    but are independent of each other.

Overview

    This extension provides a generic framework for establishing stream
    connections when the producer and consumer endpoints are not
    associated with the same EGLStream object. The cross-process streams
    allowed by the EGL_KHR_stream_cross_process_fd extension are one
    example of such a connection. Other potential examples include
    streams between objects associated with different EGLDisplays in a
    single process, streams between processes residing in separate
    virtual machine partitions on a single system, or streams between
    entirely separate systems connected via the internet.

    The approach used by EGL_KHR_stream_cross_process_fd has several
    drawbacks:
      - It creates a new pair of stream creation and query functions
        which are specific to both the type of stream and the method
        used for communicating the stream's identity. Extending this
        methodology would require new pairs of functions for every new
        type of stream connection.
      - It allows creation of the connected stream objects without any
        indication of which side will be the consumer and which will be
        the producer. It even allows, although this was probably not
        intended, a cross-process stream to be established, but have
        both producer and consumer exist in the same process, with the
        other process acting as a voyeur that can observe state and
        perhaps modify attributes, but not operate on frames.
      - The use of file descriptors generated by EGL in one process
        and accessed by EGL in another to establish communication has
        potential security flaws, and may not be available at all on
        some real-time operating systems.

    Rather than implement new connection types using this model, we
    propose a more generic approach that does not require a
    proliferation of new interfaces and avoids any ambiguities in how
    the stream will be used. These extensions can coexist with
    EGL_KHR_stream_cross_process_fd, but allow for more control of
    stream communication by the applications.

    These extensions provide the framework with which arbitrary stream
    object pairs can be established, but by themselves are insufficient
    to create such pairs. In addition, a separate extension must be
    used to determine the protocol by which the two objects will
    communicate. See for example the EGL_NV_stream_socket extension.

    Several optional types of separate stream objects are defined,
    with successive levels of remoteness. It should be assumed that
    a more remote type will be less efficient in transfering images,
    and therefore applications should choose the least remote type
    available that is compatible with their use cases.

New Types

    None

New Functions

    None

New Tokens

    Returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when
    attribute is EGL_STREAM_STATE_KHR:

        EGL_STREAM_STATE_INITIALIZING_NV           0x3240

    Accepted as attribute names in eglCreateStreamKHR,
    eglCreateStreamAttribKHR, eglQueryStreamKHR, and
    eglQueryStreamAttribKHR:

        EGL_STREAM_TYPE_NV                         0x3241
        EGL_STREAM_PROTOCOL_NV                     0x3242
        EGL_STREAM_ENDPOINT_NV                     0x3243

    Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and
    returned by eglQueryStreamKHR and eglQueryStreamAttribKHR, as value
    when attribute is EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV or
    EGL_STREAM_ENDPOINT_NV:

        EGL_STREAM_LOCAL_NV                        0x3244

    When EGL_NV_stream_cross_object is present, accepted by
    eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
    eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
    attribute EGL_STREAM_TYPE_NV:

        EGL_STREAM_CROSS_OBJECT_NV                 0x334D

    When EGL_NV_stream_cross_display is present, accepted by
    eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
    eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
    attribute EGL_STREAM_TYPE_NV:

        EGL_STREAM_CROSS_DISPLAY_NV                0x334E

    When EGL_NV_stream_cross_process is present, accepted as
    value for attribute EGL_STREAM_TYPE_NV by eglCreateStreamKHR
    and eglCreateStreamAttribKHR. When either
    EGL_NV_stream_cross_process or EGL_KHR_stream_cross_process_fd
    is present, returned by eglQueryStreamKHR and
    eglQueryStreamAttribKHR for attribute EGL_STREAM_TYPE_NV:

        EGL_STREAM_CROSS_PROCESS_NV                0x3245

    When EGL_NV_stream_cross_partition is present, accepted by
    eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
    eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
    attribute EGL_STREAM_TYPE_NV:

        EGL_STREAM_CROSS_PARTITION_NV              0x323F

    When EGL_NV_stream_cross_system is present, accepted by
    eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by
    eglQueryStreamKHR and eglQueryStreamAttribKHR as value for
    attribute EGL_STREAM_TYPE_NV:

        EGL_STREAM_CROSS_SYSTEM_NV                 0x334F

    If EGL_KHR_stream_cross_process_fd is present, returned by
    eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is
    EGL_STREAM_PROTOCOL_NV:

        EGL_STREAM_PROTOCOL_FD_NV                  0x3246

    Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and
    returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when
    attribute is EGL_STREAM_ENDPOINT_NV:

        EGL_STREAM_PRODUCER_NV                     0x3247
        EGL_STREAM_CONSUMER_NV                     0x3248

Add to "Glossary" of EGL_KHR_stream:

    Local stream
    A stream implemented with both consumer and producer attached to a
    single EGLStream object.

    Remote stream
    A stream implemented with two EGLStream objects that communicate
    with each other, with the consumer attached to one object and the
    producer attached to the other.

Add to section "3.10.1 Creating an EGLStream" in EGL_KHR_stream:

    If a remote EGLStream is created, as described in section "3.10.5.x
    Remote streams", and is paired with an EGLStream object which has
    conflicting values for some attributes, creation will succeed, but
    the stream will enter state EGL_STREAM_STATE_DISCONNECTED_KHR once
    communication is established.

Add to list of failures in section "3.10.1 Creating an EGLStream" in
EGL_KHR stream:

        - EGL_BAD_MATCH is generated if the value of any one of the
          EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV, or
          EGL_STREAM_ENDPOINT_NV attributes is EGL_STREAM_LOCAL_NV and
          any other of these attributes is neither EGL_STREAM_LOCAL_NV
          nor EGL_DONT_CARE.

Add to section "3.10.2 Connecting an EGLStream to a consumer" in
EGL_KHR_stream:

    Any function which connects a consumer to an EGLStream will fail and
    generate an EGL_BAD_ACCESS error if the value of the EGLStream's
    EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_PRODUCER_NV.

Add to section "3.10.3 Connecting an EGLStream to a producer" in
EGL_KHR_stream:

    Any function which connects a producer to an EGLStream will fail and
    generate an EGL_BAD_ACCESS error if the value of the EGLStream's
    EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_CONSUMER_NV.

Add to "Table 3.10.4.4 EGLStream Attributes" in EGL_KHR_stream:

        Attribute                   Read/Write   Type        Section
        --------------------------  ----------   ------      ----------
        EGL_STREAM_TYPE_NV             io       EGLint      3.10.4.x
        EGL_STREAM_PROTOCOL_NV         io       EGLint      3.10.4.x+1
        EGL_STREAM_ENDPOINT_NV         io       EGLint      3.10.4.x+2

Add to beginning of state list in section "3.10.4.3 EGL_STREAM_STATE_KHR
Attribute" in EGL_KHR_stream:

        - EGL_STREAM_STATE_INITIALIZING_NV - The EGLStream object
          represents one endpoint of a remote stream and has not yet
          established communication with the opposite endpoint.

Replace the first entry in the state transition table in section
"3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in EGL_KHR_stream:

    -> EGL_STREAM_STATE_INITIALIZING_NV
    A new remote EGLStream is created in this state.

    EGL_STREAM_STATE_INITIALIZING_NV ->
    EGL_STREAM_STATE_CREATED_KHR
    Occurs when a remote EGLStream object establishes communication with
    the opposite endpoint.

    -> EGL_STREAM_STATE_CREATED_KHR
    A new local EGLStream or EGLStream of undetermined type is created
    in this state.

Add new subsections to the end of section "3.10.4 EGLStream Attributes"
in EGL_KHR_stream:

    3.10.4.x EGL_STREAM_TYPE_NV Attribute

    The EGL_STREAM_TYPE_NV attribute may be set when the stream
    is created, and indicates the type of connection between the
    EGLStream objects to which the consumer and producer are attached.
    Legal values are EGL_DONT_CARE and EGL_STREAM_LOCAL_NV. The
    default value is EGL_DONT_CARE.

    A value of EGL_STREAM_LOCAL_NV indicates that this is a local
    stream with both consumer and producer connected to a single
    EGLStream object.

    If EGL_DONT_CARE is initially specified, the value of the attribute
    will automatically be changed to the appropriate value once both
    consumer and producer are attached, depending on the functions used.

    Other possible values for EGL_STREAM_TYPE_NV may be defined by
    additional extensions to indicate a remote stream. Where used,
    the EGL_STREAM_PROTOCOL_NV attribute must also be set to a
    communication protocol compatible with the stream type.

    3.10.4.x+1 EGL_STREAM_PROTOCOL_NV Attribute

    The EGL_STREAM_PROTOCOL_NV attribute may be set when the stream is
    created, and indicates the manner in which communication is
    established between the EGLStream objects to which the consumer
    and producer are attached. Legal values are EGL_DONT_CARE and
    EGL_STREAM_LOCAL_NV. The default value is EGL_DONT_CARE.

    A value of EGL_STREAM_LOCAL_NV indicates that this is a local
    stream with both consumer and producer connected to a single
    EGLStream object, so no communication protocol is required.

    If EGL_DONT_CARE is initially specified, the value of the attribute
    will automatically be changed to the appropriate value once both
    consumer and producer are attached, depending on the functions used.

    Other possible values for EGL_STREAM_PROTOCOL_NV may be defined by
    additional extensions to indicate the communication protocol to be
    used for a remote stream. Not all communication protocols are
    compatible with all stream types, and vice versa. If incompatible
    types and protocols are specified, an EGL_BAD_MATCH error will be
    generated.

    3.10.4.x+2 EGL_STREAM_ENDPOINT_NV Attribute

    The EGL_STREAM_ENDPOINT_NV attribute may be set when the stream is
    created, and indicates the endpoints which will be attached to the
    EGLStream object. Legal values are EGL_DONT_CARE,
    EGL_STREAM_LOCAL_NV, EGL_STREAM_CONSUMER_NV, and
    EGL_STREAM_PRODUCER_NV. The default value is EGL_DONT_CARE.

    A value of EGL_STREAM_LOCAL_NV indicates that this is a local
    stream with both consumer and producer connected to a single
    EGLStream object.

    A value of EGL_STREAM_CONSUMER_NV indicates that the EGLStream
    object represents the consumer side of a remote stream.

    A value of EGL_STREAM_PRODUCER_NV indicates that the EGLStream
    object represents the producer side of a remote stream.

    If EGL_DONT_CARE is initially specified, the value of the attribute
    will automatically be changed to the appropriate value once both
    consumer and producer are attached, depending on the functions used.

If EGL_NV_stream_cross_object is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_OBJECT_NV to the list of legal values, and insert

    A value of EGL_STREAM_CROSS_OBJECT_NV indicates that the stream
    object represents one endpoint of a remote stream whose other
    endpoint is obtained from the same EGLDisplay.

If EGL_NV_stream_cross_display is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_DISPLAY_NV to the list of legal values, and insert

    A value of EGL_STREAM_CROSS_DISPLAY_NV indicates that the stream
    object represents one endpoint of a remote stream whose other
    endpoint may be obtained from a different EGLDisplay in the same
    process.

If EGL_NV_stream_cross_process or EGL_NV_stream_cross_process_fd is
present, in section "3.10.4.x" above, add EGL_STREAM_CROSS_PROCESS_NV
to the list of legal values, and insert

    A value of EGL_STREAM_CROSS_PROCESS_NV indicates that the stream
    object represents one endpoint of a remote stream whose other
    endpoint may reside in a separate process.

If EGL_NV_stream_cross_partition is present, in section 3.10.4.x
above, add EGL_STREAM_CROSS_PARTITION_NV to the list of legal values,
and insert

    A value of EGL_STREAM_CROSS_PARTITION_NV indicates that the stream
    object represents one endpoint of a remote stream whose other
    endpoint may reside in a separate virtual machine partition on
    the same system. The partitions are not required to be using the
    same operating systems, but must support compatible communication
    protocols.

If EGL_NV_stream_cross_system is present, in section 3.10.4.x above,
add EGL_STREAM_CROSS_SYSTEM_NV to the list of legal values, and insert

    A value of EGL_STREAM_CROSS_SYSTEM_NV indicates that the stream
    object represents one endpoint of a remote stream whose other
    endpoint may reside on an independent hardware system with no
    directly sharable memory resources.

If EGL_KHR_stream_cross_process_fd is present, in section 3.10.4.x+1
above, add EGL_STREAM_PROTOCOL_FD_NV to the list of legal values, and
insert

    A value of EGL_STREAM_PROTOCOL_FD_NV indicates that the stream is
    a remote stream whose communication is established using a file
    descriptor. The details of what this file descriptor represents
    are implementation dependent. If the EGL_STREAM_PROTOCOL_NV
    attribute is initialized with this value, the EGL_STREAM_TYPE_NV
    attribute must specify a value of EGL_STREAM_CROSS_PROCESS_NV or
    an EGL_BAD_MATCH failure is generated.

    If an initial value of EGL_DONT_CARE is specified and a file
    descriptor is subsequently obtained with
    eglGetStreamFileDescriptorKHR, the value will be automatically
    changed to EGL_STREAM_PROTOCOL_FD_NV.

Add a new subsection to the end of "3.10.5 EGLStream operation":

    3.10.5.x Remote streams

    An EGLStream object may be created as the endpoint of a remote
    stream by specifying EGL_STREAM_PRODUCER_NV or
    EGL_STREAM_CONSUMER_NV as the value for its EGL_STREAM_ENDPOINT_NV
    attribute. Valid and compatible EGL_STREAM_TYPE_NV and
    EGL_STREAM_PROTOCOL_NV values other than EGL_DONT_CARE or
    EGL_STREAM_LOCAL_NV must also be specified.

    If a value for EGL_STREAM_ENDPOINT_NV is not specified, the stream
    is assumed to be local, but other extensions (see for example
    EGL_KHR_stream_cross_process_fd) may allow it to be converted to a
    remote stream before the producer has been attached.

    When an EGLStream object is created as remote, any unspecified
    attribute will be initially set to a value of EGL_DONT_CARE. Pairs
    of EGLStream objects representing opposite endpoints of a stream are
    not required to specify the same attribute lists, but their
    attributes must be compatible. When communication is established
    between the endpoints, they will exchange attribute settings:
        - If both endpoints have a value of EGL_DONT_CARE for an
          attribute, the default value will be assigned.
        - If one endpoint has a value of EGL_DONT_CARE for an attribute,
          it will be set to the other endpoint's value.
        - If neither endpoint has a value of EGL_DONT_CARE for an
          attribute, their values must agree. In most cases, this means
          the values must be identical, but this may not be true for all
          attributes. In particular, one endpoint must specify an
          EGL_STREAM_ENDPOINT_NV value of EGL_STREAM_CONSUMER_NV, and
          the other must specify a value of EGL_STREAM_PRODUCER_NV.
    If the values for any attribute are not compatible, the stream will
    transition to the EGL_STREAM_STATE_DISCONNECTED_KHR state.

    Additionally, if the two EGLStream objects representing a remote
    stream are created for EGLDisplays which cannot share resources,
    the stream will transition to the EGL_STREAM_STATE_DISCONNECTED_KHR
    state.

    When using remote streams, there may be latency in communicating
    state changes between the EGLStream objects representing the two
    endpoints. For instance, when a new frame is inserted into the
    stream by the producer, the consumer endpoint may not immediately
    transition to EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. Similarly,
    when a frame is acquired by the consumer, the producer endpoint may
    not immediately transition to
    EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. Applications should rely
    only on the state of the endpoint on which they are operating to
    determine the expected results of that operation.

In section "3.10.1.1 Duplicating an EGLStream from a file descriptor" of
the EGL_KHR_stream_cross_process_fd extension, add to the failure list
for eglGetStreamFileDescriptor

        - EGL_BAD_ACCESS is generated if the EGL_STREAM_TYPE_NV
          attribute of <stream> is anything other than EGL_DONT_CARE or
          EGL_STREAM_CROSS_PROCESS_NV.

        - EGL_BAD_ACCESS is generated if the EGL_STREAM_PROTOCOL_NV
          attribute of <stream> is anything other than EGL_DONT_CARE or
          EGL_STREAM_PROTOCOL_FD_NV.

        - EGL_BAD_ACCESS is generated if the EGL_STREAM_ENDPOINT_NV
          attribute of <stream> is EGL_STREAM_LOCAL_NV.

Issues

    None

Revision History

    #4  (June 01, 2016)
        - Removed sync object definition. It will be provided by a
          separate extension.
        - Folded in cross-partition extension.
        - Added types for cross-object, cross-display, and
          cross-system streams.
        - General cleanup in preparation for publication.

    #3  (September 16, 2015) Zander Clucas
        - Removed dependence of the CROSS_PROCESS type on the
          EGL_NV_cross_process_fd extension
        - Added CROSS_PROCESS to the list of STREAM_TYPE legal values
        - Added CROSS_PROCESS requirement for cross_process_fd protocol

    #2  (December 11, 2014) Daniel Kartch
        - Rewrote as NV draft for earlier release.
        - Added section for creation of sync object.
        - Reserved enum values.

    #1  (October 10, 2014) Daniel Kartch
        - Initial EXT draft