Is there support for raspberry pi windowed mode?

[seandepagnier]

I am pretty sure it is possible to support the raspberry pi bcm_host drivers in windowed mode, maybe using an environment variable or compile time flag.

I have played around with (works in 16 bit framebuffer)
https://github.com/PaulHaeberli/pi-eglonx

Which is a proof of concept, as well as (works in 32 bit framebuffer)
https://github.com/kika123/x11eglrpi

But unfortunately when I try to use the tweaked libEGL.so with the glshim libGL.so I have no success.

I know you are aware of these projects and wondered if you had already incorporated this into glshim somehow as I am still looking.

[lunixbochs]

Huh, this isn't really specific to the Raspberry Pi. They're just calling glReadPixels and dumping it into an X11 window. I could do this better even.

It might be pretty fast to do this alongside the X11 shm stuff used in TinyGLES.

This would give us windowed mode on a lot of framebuffer-only drivers, including the later OMAP stuff.

Anyway, on the latest glshim unstable you should be able to use the LIBGL_EGL environment variable to force a different EGL backend library.

[seandepagnier]

Right, it isn't specific to the raspberry, but it is needed for it.

It would be great to do this in a way that could support other boards
if needed, I know nothing about OMAP.

As far as using LIBGL_EGL, it isn't really sufficient for this because
I already tried this (although admittedly not using the environment
variable, but instead by replacing /usr/lib/libEGL.so) The libEGL.so
constructed was a bad hack and doesn't seem to work in conjunction
with glshim.

The only good solution I can see is to build support directly into
glshim for this. A lot of applications simply cannot work properly
without windowed mode as they still need to draw widgets, and I think
it's a perfect fit. I suggest by default disabled (or enabled by
compile flag) but also controlled by an environment variable.

As far as sticking glReadPixels and XPutImage into glxSwapBuffers.. is
sufficient at least for this to function but if it is possible to do
with shared memory (both of the hacks I found attempted and failed at
this) then it could eliminate one of two copies. Maybe using pixel
buffers or framebuffer objects in conjunction with x11shm it is
somehow possible to avoid any copies at all.

I'm sure of course in the case of tinyGLES you can make it avoid all
the copies as well which is great, but with drivers we don't modify,
one copy is probably required. I think the important thing is to
avoid color space conversion if at all possible, so the framebuffer
may have to be 32bits for best performance.

I am very impressed with the recent work on glshim, and I must study
the code a bit as it's greatly improved from last year.

[seandepagnier]

I got it working with this patch, what do you think?

diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt
index 3867a9c..7e0cb30 100644
--- a/src/CMakeLists.txt
+++ b/src/CMakeLists.txt
@@ -17,7 +17,7 @@ endif()

 add_library(GL SHARED ${GL_SOURCES})
 set_target_properties(GL PROPERTIES VERSION 1 SOVERSION 1.2.0)
-target_link_libraries(GL X11 m dl pthread)
+target_link_libraries(GL X11 m dl pthread Xext)

 add_library(GL_static STATIC EXCLUDE_FROM_ALL ${GL_SOURCES})
 set_target_properties(GL_static PROPERTIES OUTPUT_NAME GL)
diff --git a/src/glx/glx.c b/src/glx/glx.c
index aab18f7..1eb18df 100644
--- a/src/glx/glx.c
+++ b/src/glx/glx.c
@@ -22,6 +22,10 @@ bool eglInitialized = false;
 EGLSurface eglSurface;
 EGLConfig eglConfigs[1];

+#define X11HACK 1
+//#define X11HACK_TRUECOLOR 1
+#define X11HACK_SHM 1
+
 int8_t CheckEGLErrors() {
     LOAD_EGL(eglGetError);
     EGLenum error;
@@ -269,6 +273,13 @@ static void scan_env() {
     }
 }

+static Display *Xdsp;
+static Window Xwin;
+static XWindowAttributes Xgwa;
+static GC Xgc;
+static XImage *Ximage = 0;
+static int Ximage_width, Ximage_height;
+
 GLXContext glXCreateContext(Display *dpy, XVisualInfo *vis, GLXContext shareList, Bool direct) {
     scan_env();
     FORWARD_IF_REMOTE(glXCreateContext);
@@ -280,7 +291,7 @@ GLXContext glXCreateContext(Display *dpy, XVisualInfo *vis, GLXContext shareList
     LOAD_EGL(eglDestroySurface);
     LOAD_EGL(eglInitialize);
     LOAD_EGL(eglMakeCurrent);
-
+#if !X11HACK
     EGLint configAttribs[] = {
 #ifdef PANDORA
         EGL_RED_SIZE, 5,
@@ -297,6 +308,22 @@ GLXContext glXCreateContext(Display *dpy, XVisualInfo *vis, GLXContext shareList
 #endif
         EGL_NONE
     };
+#else
+    EGLint configAttribs[] = {
+#if X11HACK_TRUECOLOR
+        EGL_RED_SIZE, 8,
+        EGL_GREEN_SIZE, 8,
+        EGL_BLUE_SIZE, 8,
+   EGL_ALPHA_SIZE, 8,
+#endif
+        EGL_DEPTH_SIZE, 16,
+   EGL_BUFFER_SIZE, 8,
+        EGL_SURFACE_TYPE,
+   EGL_PIXMAP_BIT,
+        EGL_RENDERABLE_TYPE, EGL_OPENGL_ES_BIT,
+        EGL_NONE
+    };
+#endif

 #ifdef USE_ES2
     EGLint attrib_list[] = {
@@ -413,12 +441,32 @@ EGL_NO_SURFACE, or if draw or read are set to EGL_NO_SURFACE and context is
 not set to EGL_NO_CONTEXT.
 */

+#include <EGL/egl.h>
+#include <EGL/eglext_brcm.h>
+static char *readpixels_buf;
+
+#include <sys/ipc.h>
+#include <sys/shm.h>
+#include <X11/extensions/XShm.h>
+XShmSegmentInfo shminfo;
+
 Bool glXMakeCurrent(Display *dpy, GLXDrawable drawable, GLXContext ctx) {
     FORWARD_IF_REMOTE(glXMakeCurrent);
     PROXY_GLES(glXMakeCurrent);
     LOAD_EGL(eglCreateWindowSurface);
     LOAD_EGL(eglDestroySurface);
     LOAD_EGL(eglMakeCurrent);
+
+#if X11HACK
+    Xwin = (Window)drawable;
+    XGetWindowAttributes(dpy, Xwin, &Xgwa);
+    int width = Xgwa.width;
+    int height = Xgwa.height;
+
+    //    if(width == Ximage_width && height == Ximage_height)
+    //      return true;
+#endif
+
     EGLDisplay eglDisplay = get_egl_display(dpy);
     if (eglDisplay != NULL) {
         egl_eglMakeCurrent(eglDisplay, NULL, NULL, EGL_NO_CONTEXT);
@@ -430,9 +478,105 @@ Bool glXMakeCurrent(Display *dpy, GLXDrawable drawable, GLXContext ctx) {
     if (! ctx) {
         return true;
     }
+
+#if !X11HACK
     if (g_usefb)
         drawable = 0;
+
     eglSurface = egl_eglCreateWindowSurface(eglDisplay, eglConfigs[0], drawable, NULL);
+#else
+    //// create an X image for drawing to the screen
+    Xgc = DefaultGC(dpy, 0);
+
+    if(width != Ximage_width || height != Ximage_height) {
+#ifdef X11HACK_SHM
+      if(Ximage) {
+   XShmDetach(dpy, &shminfo);
+   XDestroyImage(Ximage);
+   shmdt(shminfo.shmaddr);
+   Ximage = NULL;
+      }
+#else
+      if(Ximage)
+   free(Ximage->data);
+      XFree(Ximage);
+#endif
+      free(readpixels_buf);
+      readpixels_buf = NULL;
+
+      width = (width-1)/2*2+2; // force even width
+#ifdef X11HACK_SHM
+      Ximage = XShmCreateImage(dpy, Xgwa.visual, Xgwa.depth,
+                  ZPixmap, 0, &shminfo, width, height);
+      /* Get the shared memory and check for errors */
+      shminfo.shmid = shmget(IPC_PRIVATE, width*Xgwa.depth*height,
+                IPC_CREAT | 0777 );
+      if(shminfo.shmid < 0) return false;
+
+      /* attach, and check for errrors */
+      shminfo.shmaddr = Ximage->data = (char *)shmat(shminfo.shmid, 0, 0);
+      if(shminfo.shmaddr == (char *) -1) return 1;
+
+      /* set as read/write, and attach to the display */
+      shminfo.readOnly = False;
+      XShmAttach(dpy, &shminfo);
+#else
+      char *buf = (char *)malloc(width*height*4);
+      Ximage = XCreateImage(dpy, Xgwa.visual, Xgwa.depth,
+               ZPixmap, 0, buf, width, height, 32, 0);
+#endif
+      Ximage_width = width;
+      Ximage_height = height;
+    }
+    
+
+      EGLint rt;
+#if X11HACK_TRUECOLOR
+      EGLint pixel_format = EGL_PIXEL_FORMAT_ARGB_8888_BRCM;
+#else
+      EGLint pixel_format = EGL_PIXEL_FORMAT_RGB_565_BRCM;
+#endif
+
+      LOAD_EGL(eglGetConfigAttrib);
+      //      LOAD_EGL(eglCreateGlobalImageBRCM);
+
+      void (*egl_eglCreateGlobalImageBRCM)() = dlsym(egl, "eglCreateGlobalImageBRCM");
+      LOAD_EGL(eglCreatePixmapSurface);
+         
+      egl_eglGetConfigAttrib(eglDisplay, eglConfigs[0], EGL_RENDERABLE_TYPE, &rt);
+    CheckEGLErrors();
+      if (rt & EGL_OPENGL_ES_BIT) {
+        pixel_format |= EGL_PIXEL_FORMAT_RENDER_GLES_BRCM;
+        pixel_format |= EGL_PIXEL_FORMAT_GLES_TEXTURE_BRCM;
+      }
+      if (rt & EGL_OPENGL_ES2_BIT) {
+        pixel_format |= EGL_PIXEL_FORMAT_RENDER_GLES2_BRCM;
+        pixel_format |= EGL_PIXEL_FORMAT_GLES2_TEXTURE_BRCM;
+      }
+      if (rt & EGL_OPENVG_BIT) {
+        pixel_format |= EGL_PIXEL_FORMAT_RENDER_VG_BRCM;
+        pixel_format |= EGL_PIXEL_FORMAT_VG_IMAGE_BRCM;
+      }
+      if (rt & EGL_OPENGL_BIT) {
+        pixel_format |= EGL_PIXEL_FORMAT_RENDER_GL_BRCM;
+      }
+
+      EGLint pixmap[5];
+      pixmap[0] = 0;
+      pixmap[1] = 0;
+      pixmap[2] = width;
+      pixmap[3] = height;
+      pixmap[4] = pixel_format;
+    
+      egl_eglCreateGlobalImageBRCM(width, height, pixmap[4], 0,
+#ifdef X11HACK_TRUECOLOR
+                  width*4,
+#else
+                  width*2,
+#endif                
+                  pixmap);
+      eglSurface = egl_eglCreatePixmapSurface(eglDisplay, eglConfigs[0], pixmap, NULL);
+#endif
     CheckEGLErrors();

     EGLBoolean result = egl_eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext);
@@ -508,11 +652,82 @@ void glXSwapBuffers(Display *dpy, GLXDrawable drawable) {
         // this will just block otherwise.
         int arg = 0;
         for (int i = 0; i < swap_interval; i++) {
-            ioctl(fbdev, FBIO_WAITFORVSYNC, &arg);
+        ioctl(fbdev, FBIO_WAITFORVSYNC, &arg);
         }
     }
 #endif
+
+#if !X11HACK
     egl_eglSwapBuffers(get_egl_display(dpy), eglSurface);
+#else
+    Xwin = (Window)drawable;
+    XGetWindowAttributes(dpy, Xwin, &Xgwa);
+
+    int width = Xgwa.width;
+    int height = Xgwa.height;
+
+    if(width != Ximage_width || height != Ximage_height) {
+      glXMakeCurrent(dpy, drawable, eglContext);
+      return;
+    }
+
+    glFinish();
+
+    if(!readpixels_buf)
+   readpixels_buf = malloc(width*height*4);
+
+    if(Xgwa.depth != 16) {
+      //  glReadPixels(0, 0, width, height,
+      //      GL_RGBA, GL_UNSIGNED_BYTE, Ximage->data);
+
+      glReadPixels(0, 0, width, height,
+          GL_RGBA, GL_UNSIGNED_BYTE, readpixels_buf);
+
+      int x, y;
+      for(y=0; y<height; y++) {
+        int srcy = height-1-y;     // flip y for X windows
+   int *pixptr = (int*)readpixels_buf + y*width;
+   int *dest = ((int*)(&(Ximage->data[0])))+(srcy*width);
+
+   memcpy(dest, pixptr, width*4);
+      }
+    } else {
+#if 1
+      glReadPixels(0, 0, width, height,
+          GL_RGBA, GL_UNSIGNED_BYTE, readpixels_buf);
+
+      unsigned int *pixptr = (unsigned int*)readpixels_buf;
+      int count, x, y;
+      for(y=0; y<height; y++) {
+        int srcy = height-1-y;     // flip y for X windows
+        unsigned int *dest = ((unsigned int*)(&(Ximage->data[0])))+(srcy*(width/2));
+        count = width/2;
+        while(count--) {
+     unsigned int src0 = pixptr[0];
+     unsigned int src1 = pixptr[1];
+     pixptr += 2;
+
+     *dest++ = ((src1 & 0xf8)      <<24) |
+       ((src1 & (0xfc<< 8))<<11) |
+       ((src1 & (0xf8<<16))>> 3) |
+       ((src0 & 0xf8)      << 8) |
+       ((src0 & (0xfc<< 8))>> 5) |
+       ((src0 & (0xf8<<16))>>19);
+        }
+      }
+#else
+      glReadPixels(0, 0, width, height,
+              GL_RGB, GL_UNSIGNED_SHORT_5_6_5, Ximage->data);
+#endif
+    }
+
+#ifdef X11HACK_SHM
+    XShmPutImage(dpy, Xwin, Xgc, Ximage, 0, 0, 0, 0, width, height, False);
+#else
+    XPutImage(dpy, Xwin, Xgc, Ximage, 0, 0, 0, 0, width, height);
+#endif
+
+#endif
     CheckEGLErrors();
 }

[lunixbochs]

Cool, thanks for the example!

This is definitely very interesting for the Pandora and Pyra (because both don't have windowed GL modes on new drivers), and there's a chance glshim will be the primary GL frontend on the Pyra. I added the issue #138 so I can think about supporting this when pretending to be libGLES as well as when we pretend to be libGL.

I'm going to think about how I want it to be integrated. I really don't like relying on ifdef. Mainline glshim tries to support everything out of the box with default compile options. I'm even going to move X11 detection to runtime at some point so it will work better on non-X11 platforms.

[seandepagnier]

This is a temporary hack. I was excited to make it work and that's
it. If you want this to work for gles, so you mean egl, not only glx,
you are going to have to stuff this stuff inside an egl wrapper
instead of the glx->egl layer as I have it. I don't see why this
wouldn't work though.

As for removing the ifdef, I see no problem. It can simply detect
this as it currently does when it tries to use the framebuffer, then
depending on environment variable either uses the framebuffer as it
does now, or this.

On thing I really want to resolve is efficiency concerns but not sure
how. This method works and it's of course much faster than software
opengl for my application, but it does a lot of memory copying and
colorspace transformations. I would need to investigate the
underlying xorg driver and ensure it's actually using dma etc to
improve this. For example, I see no frame rate difference if I use a
32bit renderbuffer and 16bit framebuffer or 16bit renderbuffer and
32bit framebuffer or using the same depth for both. I doing
colorspace conversion manually in a for loop with shifts and ands, so
this is not the bottle neck, but something in how xorg handls
XShmPutImage is very slow, and also glReadPixels isn't very good
either.

On 9/27/15, Ryan Hileman [email protected] wrote:

Cool, thanks for the example!

This is definitely very interesting for the Pandora and Pyra (because both
don't have windowed GL modes on new drivers), and there's a chance glshim
will be the primary GL frontend on the Pyra. I added the issue
#138 so I can think about
supporting this when pretending to be libGLES as well as when we pretend to
be libGL.

I'm going to think about how I want it to be integrated. I really don't like
relying on ifdef. Mainline glshim tries to support everything out of the box
with default compile options. I'm even going to move X11 detection to
runtime at some point so it will work better on non-X11 platforms.

---

Reply to this email directly or view it on GitHub:
#136 (comment)