1Name
2
3    KHR_stream_consumer_gltexture
4
5Name Strings
6
7    EGL_KHR_stream_consumer_gltexture
8
9Contributors
10
11    Acorn Pooley
12    Jamie Gennis
13    Marcus Lorentzon
14
15Contacts
16
17    Acorn Pooley, NVIDIA  (apooley 'at' nvidia.com)
18
19Notice
20
21    Copyright (c) 2011-2013 The Khronos Group Inc. Copyright terms at
22        http://www.khronos.org/registry/speccopyright.html
23
24Status
25
26    Complete.
27    Approved by the Khronos Board of Promoters on December 2, 2011.
28
29Version
30
31    Version 11, June 18, 2012
32
33Number
34
35    EGL Extension #33
36
37Dependencies
38
39    Requires EGL 1.2.
40    Requires OpenGL ES 1.1 or OpenGL ES 2.0.
41
42    Requires the EGL_KHR_stream extension.
43    Requires the GL_NV_EGL_stream_consumer_external extension.
44
45Overview
46
47    This extension allows an OpenGL(ES) texture to be connected to an
48    EGLStream as its consumer.  Image frames from the EGLStream can be
49    'latched' into the texture as the contents of the texture.  This
50    is equivalent to copying the image into the texture, but on most
51    implementations a copy is not needed so this is faster.
52
53New Procedures and Functions
54
55    EGLBoolean eglStreamConsumerGLTextureExternalKHR(
56                    EGLDisplay dpy,
57                    EGLStreamKHR stream)
58
59    EGLBoolean eglStreamConsumerAcquireKHR(
60                    EGLDisplay dpy,
61                    EGLStreamKHR stream);
62
63    EGLBoolean eglStreamConsumerReleaseKHR(
64                    EGLDisplay dpy,
65                    EGLStreamKHR stream);
66
67New Tokens
68
69    Accepted as an attribute in the <attrib_list> parameter of
70    eglCreateStreamKHR and as the <attribute> parameter of
71    eglStreamAttribKHR and eglQueryStreamKHR
72
73    EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR                0x321E
74
75Replace section "3.10.2.1 No way to connect consumer to EGLStream" in
76the EGL_KHR_stream extension with this:
77
78    3.10.2.1 GL Texture External consumer
79
80    Call
81
82        EGLBoolean eglStreamConsumerGLTextureExternalKHR(
83                    EGLDisplay dpy,
84                    EGLStreamKHR stream)
85
86    to connect the texture object currently bound to the active
87    texture unit's GL_TEXTURE_EXTERNAL_OES texture target in the
88    OpenGL or OpenGL ES context current to the calling thread as the
89    consumer of <stream>.
90
91    (Note: Before this can succeed a GL_TEXTURE_EXTERNAL_OES texture
92    must be bound to the active texture unit of the GL context current
93    to the calling thread.  To create a GL_TEXTURE_EXTERNAL_OES
94    texture and bind it to the current context, call glBindTexture()
95    with <target> set to GL_TEXTURE_EXTERNAL_OES and <texture> set to
96    the name of the GL_TEXTURE_EXTERNAL_OES (which may or may not have
97    previously been created).  This is described in the
98    GL_NV_EGL_stream_consumer_external extension.)
99
100    On failure EGL_FALSE is returned and an error is generated.
101
102        - EGL_BAD_STATE_KHR is generated if <stream> is not in state
103          EGL_STREAM_STATE_CREATED_KHR.
104
105        - EGL_BAD_ACCESS is generated if there is no GL context
106          current to the calling thread.
107
108        - EGL_BAD_ACCESS is generated unless a nonzero texture object
109          name is bound to the GL_TEXTURE_EXTERNAL_OES texture target
110          of the GL context current to the calling thread.
111
112        - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
113          EGLStreamKHR created for <dpy>.
114
115        - EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
116          initialized EGLDisplay.
117
118
119    On success the texture is connected to the <stream>, <stream> is
120    placed in the EGL_STREAM_STATE_CONNECTING_KHR state, and EGL_TRUE is
121    returned.
122
123    If the texture is later deleted, connected to a different
124    EGLStream, or connected to an EGLImage, then <stream> will be
125    placed into the EGL_STREAM_STATE_DISCONNECTED_KHR state.
126
127    If the <stream> is later destroyed then the texture will be
128    "incomplete" until it is connected to a new EGLStream, connected
129    to a new EGLImage, or deleted.
130
131
132    Call
133
134        EGLBoolean eglStreamConsumerAcquireKHR(
135                    EGLDisplay dpy,
136                    EGLStreamKHR stream);
137
138    to "latch" the most recent image frame from <stream> into the
139    texture that is the consumer of <stream>.  The GLES context
140    containing the texture must be bound to the current thread.  If
141    the GLES texture is also used in shared contexts current to other
142    threads then the texture must be re-bound in those contexts to
143    guarantee the new texture is used.
144
145    eglStreamConsumerAcquireKHR will block until either the timeout
146    specified by EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR expires, or the
147    value of EGL_BAD_STATE_KHR is neither EGL_STREAM_STATE_EMPTY_KHR nor
148    EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR (whichever comes first).
149
150    Blocking effectively waits until a new image frame (that has never
151    been consumed) is available in the EGLStream.  By default the
152    timeout is zero and the function does not block.
153
154    eglStreamConsumerAcquireKHR returns EGL_TRUE if an image frame was
155    successfully latched into the texture object.
156
157    If the producer has not inserted any new image frames since the
158    last call to eglStreamConsumerAcquireKHR then
159    eglStreamConsumerAcquireKHR will "latch" the same image frame it
160    latched last time eglStreamConsumerAcquireKHR was called.  If the
161    producer has inserted one new image frame since the last call to
162    eglStreamConsumerAcquireKHR then eglStreamConsumerAcquireKHR will
163    "latch" the newly inserted image frame.  If the producer has
164    inserted more than one new image frame since the last call to
165    eglStreamConsumerAcquireKHR then all but the most recently
166    inserted image frames are discarded and the
167    eglStreamConsumerAcquireKHR will "latch" the most recently
168    inserted image frame.
169
170    The application can use the value of EGL_CONSUMER_FRAME_KHR to
171    identify which image frame was actually latched.
172
173    On failure the texture becomes "incomplete", eglStreamConsumerAcquireKHR
174    returns EGL_FALSE, and an error is generated.
175
176        - EGL_BAD_STATE_KHR is generated if <stream> is not in state
177          EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or
178          EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR.
179
180        - EGL_BAD_ACCESS is generated if there is no GL context
181          current to the calling thread, or if the GL context current
182          to the calling thread does not contain a texture that is
183          connected as the consumer of the EGLStream.
184
185        - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
186          EGLStream created for <dpy>.
187
188        - EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
189          initialized EGLDisplay.
190
191
192    After using the texture call
193
194        EGLBoolean eglStreamConsumerReleaseKHR(
195                    EGLDisplay dpy,
196                    EGLStreamKHR stream);
197
198    to release the image frame back to the stream.
199    eglStreamConsumerReleaseKHR() will prevent the EGLStream and
200    producer from re-using and/or modifying the image frame until all
201    preceding GL commands that use the image frame as a texture have
202    completed.  If eglStreamConsumerAcquireKHR() is called twice on the
203    same EGLStream without an intervening call to
204    eglStreamConsumerReleaseKHR() then eglStreamConsumerReleaseKHR() is
205    implicitly called at the start of eglStreamConsumerAcquireKHR().
206
207    After successfully calling eglStreamConsumerReleaseKHR the texture
208    becomes "incomplete".
209
210    If eglStreamConsumerReleaseKHR is called twice without a successful
211    intervening call to eglStreamConsumerAcquireKHR, or called with no
212    previous call to eglStreamConsumerAcquireKHR, then the call does
213    nothing and the texture remains in "incomplete" state.  This is
214    not an error.
215
216    If eglStreamConsumerReleaseKHR fails EGL_FALSE is returned and an error is
217    generated.
218
219        - EGL_BAD_STATE_KHR is generated if <stream> is not in state
220          EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR or
221          EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR.
222
223        - EGL_BAD_ACCESS is generated if there is no GL context
224          current to the calling thread, or if the GL context current
225          to the calling thread does not contain the texture to which
226          the EGLStream is connected.
227
228        - EGL_BAD_STREAM_KHR is generated if <stream> is not a valid
229          EGLStream created for <dpy>.
230
231        - EGL_BAD_DISPLAY is generated if <dpy> is not a valid,
232          initialized EGLDisplay.
233
234
235    The application should estimate the time that will elapse from the
236    time a new frame becomes available (i.e. the state becomes
237    EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR) and the time the frame
238    is presented to the user.  The application should set this as the
239    value of the EGL_CONSUMER_LATENCY_USEC attribute by calling
240    eglStreamAttribKHR().  The value will depend on the complexity of
241    the scene being rendered and the platform that the app is running
242    on.  It may be difficult to estimate except by experimentation on
243    a specific platform.  The default value is implementation
244    dependent and may be a good enough estimate for some situations.
245    If the estimate changes over time the application may modify the
246    value of EGL_CONSUMER_LATENCY_USEC.
247
248    If the EGLStream is deleted while an image frame is acquired (i.e.
249    after calling eglStreamConsumerAcquireKHR and before calling
250    eglStreamConsumerReleaseKHR) then the EGLStream resources will not
251    be freed until the acquired image frame is released.  However it
252    is an error to call eglStreamConsumerReleaseKHR after deleting the
253    EGLStream because <stream> is no longer a valid handle.  In this
254    situation the image can be released (and the EGLStream resources
255    freed) by doing any one of
256        - deleting the GL_TEXTURE_EXTERNAL (call glDeleteTextures)
257        - connecting the GL_TEXTURE_EXTERNAL to another EGLStream
258            (call eglStreamConsumerGLTextureExternalKHR)
259        - connecting the GL_TEXTURE_EXTERNAL to an EGLImage (if the
260            GL_OES_EGL_image_external extension is supported, call
261            glEGLImageTargetTexture2DOES)
262
263Add a new subsection 3.10.4.6 at the end of section "3.10.4 EGLStream
264Attributes" in the EGL_KHR_stream extension spec:
265
266    3.10.4.6 EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR Attribute
267
268    This attribute is read/write.  The default value is 0.  It
269    indicates the maximum amount of time (in microseconds) that
270    eglStreamConsumerAcquireKHR should block.  If 0 (the default) it
271    will not block at all.  If negative it will block indefinitely.
272
273Issues
274    1.  How to notify the app when a new image is available
275          - callback?
276            - pro: easy to use
277            - con: introduces extra threads into EGL which does not define such
278              behavior now - would have to define a lot of semantics (e.g. what
279              can you call from the callback?)
280          - EGL_KHR_reusable_sync signaled?
281            - this is how EGL_KHR_stream_consumer_endpoint does it
282            - pro: simpler to specify
283            - pro: easy to use if that is all you are waiting for
284            - con: difficult to wait on this AND other events simultaneously?
285          - blocking call to eglStreamConsumerAcquireKHR?
286
287        RESOLVED: Use the EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_KHR to make
288        eglStreamConsumerAcquireKHR blocking if desired.  Additional
289        mechanisms can be added as layered extensions.
290
291    2.  What to call this extension?
292            EGL_NV_stream_consumer_gltexture
293            EGL_EXT_stream_consumer_gltexture
294            EGL_KHR_stream_consumer_gltexture
295            EGL_KHR_stream_consumer_gltexture_external
296
297        RESOLVED: EGL_KHR_stream_consumer_gltexture
298
299    3.  Should it be possible to connect an EGLStream to this consumer
300        (texture), and then later reconnect the same stream to a different
301        consumer?
302
303        RESOLVED: no
304
305        There may be reasons to allow this later, but for the time being
306        there is no use for this.  Adding this functionality can be
307        considered in the future with a layered extension.
308
309    4.  Do we need both this extension and
310        GL_NV_EGL_stream_consumer_external?  Should we just have one
311        extension that takes the place of both?  If so should it be an
312        EGL or a GL extension?
313
314        UNRESOLVED
315
316        SUGGESTION: need both
317
318        See issue 1 in GL_NV_EGL_stream_consumer_external.txt
319
320    5.  What happens if the EGLStream is deleted while the consumer
321        has an image acquired?
322
323        This case is a problem because after the EGLStream is deleted
324        the EGLStreamKHR handle is no longer valid, which means
325        eglStreamConsumerReleaseKHR cannot be called (because it would
326        return EGL_BAD_STREAM).
327
328        Possible resolutions:
329
330        A) Do not allow the EGLStream to be deleted while an image is
331        acquired.
332
333        B) Allow the EGLStream to be deleted.  Allow the EGLStreamKHR
334        handle to be used in a call to eglStreamConsumerReleaseKHR()
335        after it has been deleted.
336
337        C) Allow the EGLStream to be deleted.  It is an error to call
338        eglStreamConsumerReleaseKHR() after the stream is deleted.  To
339        release the image the app must
340              - delete the GL_TEXTURE_EXTERNAL texture object
341           or - connect another EGLStream to the GL_TEXTURE_EXTERNAL
342                  texture object
343           or - connect an EGLImage to the GL_TEXTURE_EXTERNAL
344                  texture object
345
346        D) Make the call to EGLStream implicitly call
347        eglStreamConsumerReleaseKHR if an image is acquired.  This
348        requires the GL context is current to the thread that deletes
349        the EGLStream.
350
351        E) Make the call to EGLStream implicitly call
352        eglStreamConsumerReleaseKHR if an image is acquired, and state
353        that this has to work even if the GL context is current to a
354        different thread or not current to any thread.
355
356        Pros/cons:
357        - B violates EGL object handle lifetime policies
358        - E is hard/impossible to implement on some systems
359        - D makes deletion fail for complicated reasons
360        - A makes deletion fail for less complicated reasons
361
362        RESOLVED: option C
363
364Revision History
365
366    #11 (June 18. 2012) Acorn Pooley
367        - Replace EGLStream with EGLStreamKHR in function prototypes.
368
369    #10 (October 12, 2011) Acorn Pooley
370        - Fix confusing error in eglStreamConsumerAcquireKHR description.
371
372    #9 (October 4, 2011) Acorn Pooley
373        - Convert from an NV extension to a KHR extension
374
375    #8 (September 30, 2011) Acorn Pooley
376        - Add issue 5 and clarify EGLStream deletion while image is
377          acquired.
378
379    #7 (September 27, 2011) Acorn Pooley
380        - Assign enum values (bug 8064)
381
382    #6 (Aug 3, 2011) Acorn Pooley
383        - rename GL_OES_EGL_stream_external to
384          GL_NV_EGL_stream_consumer_external
385
386    #5 (Aug 2, 2011) Acorn Pooley
387        - Add dependency on GL_OES_EGL_stream_external
388
389    #4 (Aug 2, 2011) Acorn Pooley
390        - Fix spelling and grammar
391
392    #3 (July 6, 2011) Acorn Pooley
393        - Rename EGL_KHR_image_stream to EGL_KHR_stream
394
395    #2  (June 29, 2011) Acorn Pooley
396        - change how texture is connected to stream to match
397          EGL_KHR_stream spec.
398        - Add EGL_CONSUMER_ACQUIRE_TIMEOUT_USEC_NV
399
400    #1  (April 20, 2011) Acorn Pooley
401        - initial draft
402# vim:ai:ts=4:sts=4:expandtab:textwidth=70
403