Developing Computer Vision Apps for Smart Phones or Tablets such as the iPhone, iPad, iPod or Android:

Last updated on 11th April, 2011 by Shervin Emami. Posted originally on 1st July, 2010.

NOTE: This page was written in 2010 and therefore a lot of it may be out-of-date.

For writing portable computer vision software that works on multiple platforms across desktop and mobile, you can use OpenCV in C/C++ for the internal processing (except the HighGUI functions), and for the GUI you can use:

Library:Desktop:Phone Availability:
OpenFrameworksWin, Linux, MaciPhone
LibNUIWin, Linux, MaciPhone
UnityWin, Linux, Mac(can pay for iPhone, Nintendo Wii, XBox 360, or Sony PS3 support)
OpenGL ES (directly)Win, Linux, MaciPhone, Palm Pre, Android, Nokia N900
SDL (directly)Win, Linux, MacPalm Pre, Nokia N900 and ported to iPhone and Android
EDGELIBWin, LinuxiPhone, Win Mobile, Symbian / Nokia
Airplay SDKWin, MaciPhone, Android, Win Mobile, Symbian, BREW, Maemo. (Develop on PC for iPhone + Android)
DragonFire SDKWiniPhone, iPad. (Develop on PC for iPhone without a Mac or Objective-C)

 

OpenGL Features Available on Smart Phones:

OpenGL is available to provide graphics hardware acceleration in some form on nearly all mobile smart phones. OpenGL 1.3 and OpenGL 2.0 have been available for desktop computers for many years and provide many features, but a smaller version "OpenGL ES" was created specifically for devices such as smart phones. Therefore, only some of the OpenGL features are available in OpenGL ES. So I have listed various differences below:

OpenGL 1.3 and higher (on desktop computers only):

+ Supports 32bit and 64bit floating point and many integer data types. + Supports rendering with immediate-mode primitives (glBegin...glEnd). + Supports rendering with Display Lists (compiled drawing functions). + Supports rendering with Vertex Arrays (indexed array numbers). + Supports rendering with Vertex Buffer Objects (dynamic combination of Display Lists and Vertex Arrays). + OpenGL 2.0 and higher also support rendering with Shading Languages.

OpenGL ES 1.0 (based on OpenGL 1.3):

(Read "http://www.khronos.org/registry/gles/specs/1.1/es_cm_spec_1.1.12.pdf") - Common Lite (CL) profile *only* supports fixed-point data types (16.16bit fixed, or 16bit shorts). + Common (CM) profile supports *both* fixed-point and 32bit floating-point data types. + Supports rendering with Vertex Arrays (DrawArrays & DrawElements, but not IndexPointer or InterleavedArrays or ArrayElement or DrawRangeElements or MultiDraw*). - Doesnt support rendering with others such as immediate-mode primitives (glBegin...glEnd). - Doesnt support push & pop attributes, accumulation buffers, or copy pixels. - Doesnt support QUADS and QUAD_STRIPS, bitmaps, glPolygonMode (such as wireframe mode), glTexCoord (use glMultuTexCoord4f instead), glColor3 (use glColor4f or glColor4ub). + Supports atleast 16 glPushMatrix and glPopMatrix, instead of 32, but not glPushAttrib or glPushClientAttrib. + Supports glViewport, glFrustrum, glOrtho, glEnable(RESCALE_NORMAL) and glEnable(NORMALIZE). - Doesnt support glTexGen* or glGetTexGen* or glEnable(TEXTURE_GEN_*). - Doesnt support glColorMaterial, but has the default as AMBIENT_AND_DIFFUSE. - Front & Back side materials are always the same, even though two-sided lighting is allowed. - Supports glCullFace() and glMaterialf(FRONT_AND_BACK,*) but not glMaterialf(FRONT,*). - Supports anti-aliasing points & lines, but not POLYGON_SMOOTH. - Supports glPixelStorei(GL_UNPACK_ALIGNMENT, 1) for textures, but not glDrawPixels. - Supports glTexImage2D & glTexSubImage2D & glCompressedTexImage2D, but not 1D or 3D textures. - Supports texture wrap modes REPEAT and CLAMP_TO_EDGE but not CLAMP or CLAMP_TO_BORDER or MIRRORED_REPEAT or TEXTURE_WRAP_R. - Supports glHint, but not glHint(POLYGON_SMOOTH_HINT) for polygon smoothing. + Used in Symbian OS, Android and Blackberry.

OpenGL ES 1.1 (upgrade of OpenGL ES 1.0):

+ Includes everything in OpenGL ES 1.0. + Also supports rendering with Vertex Buffer Object (BindBuffer,GenBuffers,BufferData,BufferSubData,DeleteBuffers, STATIC_DRAW, DYNAMIC_DRAW, but not MapBuffer or UnmapBuffer or STREAM_* or STATIC_COPY or DYNAMIC_COPY). + Also supports push & pop state attributes. + Used in iPhone (2G, 3G & 3GS are ES-CM 1.1), Nokia N900?, Palm WebOS (using Plug-in).

OpenGL ES 2.0 (based on OpenGL 2.0):

- Does not support many things from OpenGL ES 1.0 or OpenGL 1.3. + Supports Shading Language. + Used in iPhone 3GS (ES-CM 2.0), Nokia N900, Android NDKr3, Palm WebOS (using Plug-in), and WebGL for browsers.