Rendering multiple 2D images in OpenGL-ES 2.0

Question

I am new to OpenGL, and trying to learn ES 2.0.

To start with, I am working on a card game, where I need to render multiple card images. I followed this http://www.learnopengles.com/android-lesson-four-introducing-basic-texturing/

I have created a few classes to handle the data and actions.

• MySprite holds the texture information, including the location and scale factors.
• Batcher draws all the sprites in one go. It is rough implementation.
  
• ShaderHelper manages creation of shaders and linking them to a program.
• GLRenderer is where the rendering is handled (it implements `Renderer`.)

Q1

My program renders one image correctly. Problem is that when I render 2 images, first one is replaced by the later one in its place, hence second one is rendered twice.

I suspect it is something related to how I create textures in MySprite class. But I am not sure why. Can you help?

Q2

I read that if I have to render 2 images, I need to use GL_TEXTURE0 and GL_TEXTURE1, instead of just using GL_TEXTURE0.

_GLES20.glActiveTexture(GLES20.GL_TEXTURE0);

But since these constants are limited (0 to 31), is there a better way to render more than 32 small images without losing the images' uniqueness?

Please point me to the right direction.

---

The code

GLRenderer:

public class GLRenderer implements Renderer {
    ArrayList<MySprite> images = new ArrayList<MySprite>();
    Batcher batch;
    int x = 0;
...
    @Override
    public void onSurfaceCreated(GL10 gl, EGLConfig config) {
        batch = new Batcher();
        MySprite s = MySprite.createGLSprite(mContext.getAssets(), "menu/back.png");
        images.add(s);
        s.XScale = 2;
        s.YScale = 3;

        images.add(MySprite.createGLSprite(mContext.getAssets(), "menu/play.png"));

        // Set the clear color to black
        GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1);

        ShaderHelper.initGlProgram();
    }

    @Override
    public void onSurfaceChanged(GL10 gl, int width, int height) {
        mScreenWidth = width;
        mScreenHeight = height;

        // Redo the Viewport, making it fullscreen.
        GLES20.glViewport(0, 0, mScreenWidth, mScreenHeight);

        batch.setScreenDimension(width, height);

        // Set our shader programm
        GLES20.glUseProgram(ShaderHelper.programTexture);
    }

    @Override
    public void onDrawFrame(GL10 unused) {
        // clear Screen and Depth Buffer, we have set the clear color as black.
        GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);

        batch.begin();
        int y = 0;
        for (MySprite s : images) {
            s.X = x;
            s.Y = y;
            batch.draw(s);

            y += 200;
        }
        batch.end();

        x += 1;
    }
}

Batcher:

public class Batcher {
    // Store the model matrix. This matrix is used to move models from object space (where each model can be thought
    // of being located at the center of the universe) to world space.
    private final float[] mtrxModel = new float[16];
    // Store the projection matrix. This is used to project the scene onto a 2D viewport.
    private static final float[] mtrxProjection = new float[16];
    // Allocate storage for the final combined matrix. This will be passed into the shader program.
    private final float[] mtrxMVP = new float[16];

    // Create our UV coordinates.
    static float[] uvArray = new float[]{
            0.0f, 0.0f,
            0.0f, 1.0f,
            1.0f, 1.0f,
            1.0f, 0.0f
    };
    static FloatBuffer uvBuffer;
    static FloatBuffer vertexBuffer;
    static boolean staticInitialized = false;
    static short[] indices = new short[]{0, 1, 2, 0, 2, 3}; // The order of vertexrendering.
    static ShortBuffer indicesBuffer;

    ArrayList<MySprite> sprites = new ArrayList<MySprite>();

    public Batcher() {
        if (!staticInitialized) {
            // The texture buffer
            uvBuffer = ByteBuffer.allocateDirect(uvArray.length * 4)
                    .order(ByteOrder.nativeOrder())
                    .asFloatBuffer();
            uvBuffer.put(uvArray)
                    .position(0);

            // initialize byte buffer for the draw list
            indicesBuffer = ByteBuffer.allocateDirect(indices.length * 2)
                    .order(ByteOrder.nativeOrder())
                    .asShortBuffer();
            indicesBuffer.put(indices)
                    .position(0);

            float[] vertices = new float[] {
                    0, 0, 0,
                    0, 1, 0,
                    1, 1, 0,
                    1, 0, 0
            };

            // The vertex buffer.
            vertexBuffer = ByteBuffer.allocateDirect(vertices.length * 4)
                    .order(ByteOrder.nativeOrder())
                    .asFloatBuffer();
            vertexBuffer.put(vertices)
                    .position(0);

            staticInitialized = true;
        }
    }

    public void setScreenDimension(int screenWidth, int screenHeight) {
        Matrix.setIdentityM(mtrxProjection, 0);
        // (0,0)--->
        //   |
        //   v
        //I want it to be more natural like desktop screen
        Matrix.orthoM(mtrxProjection, 0,
                -1f, screenWidth,
                screenHeight, -1f,
                -1f, 1f);
    }

    public void begin() {
        sprites.clear();
    }

    public void draw(MySprite sprite) {
        sprites.add(sprite);
    }

    public void end() {
        // Get handle to shape's transformation matrix
        int u_MVPMatrix = GLES20.glGetUniformLocation(ShaderHelper.programTexture, "u_MVPMatrix");
        int a_Position = GLES20.glGetAttribLocation(ShaderHelper.programTexture, "a_Position");
        int a_texCoord = GLES20.glGetAttribLocation(ShaderHelper.programTexture, "a_texCoord");
        int u_texture = GLES20.glGetUniformLocation(ShaderHelper.programTexture, "u_texture");

        GLES20.glEnableVertexAttribArray(a_Position);
        GLES20.glEnableVertexAttribArray(a_texCoord);

        //loop all sprites
        for (int i = 0; i < sprites.size(); i++) {
            MySprite ms = sprites.get(i);

            // Matrix op - start
                Matrix.setIdentityM(mtrxMVP, 0);
                Matrix.setIdentityM(mtrxModel, 0);

                Matrix.translateM(mtrxModel, 0, ms.X, ms.Y, 0f);
                Matrix.scaleM(mtrxModel, 0, ms.getWidth() * ms.XScale, ms.getHeight() * ms.YScale, 0f);

                Matrix.multiplyMM(mtrxMVP, 0, mtrxModel, 0, mtrxMVP, 0);
                Matrix.multiplyMM(mtrxMVP, 0, mtrxProjection, 0, mtrxMVP, 0);
            // Matrix op - end

            // Pass the data to shaders - start
                // Prepare the triangle coordinate data
                GLES20.glVertexAttribPointer(a_Position, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer);

                // Prepare the texturecoordinates
                GLES20.glVertexAttribPointer(a_texCoord, 2, GLES20.GL_FLOAT, false, 0, uvBuffer);

                GLES20.glUniformMatrix4fv(u_MVPMatrix, 1, false, mtrxMVP, 0);

                // Set the sampler texture unit to where we have saved the texture.
                GLES20.glUniform1i(u_texture, ms.getTextureId());
            // Pass the data to shaders - end

            // Draw the triangles
            GLES20.glDrawElements(GLES20.GL_TRIANGLES, indices.length, GLES20.GL_UNSIGNED_SHORT, indicesBuffer);
        }
    }
}

ShaderHelper

public class ShaderHelper {
    static final String vs_Image =
        "uniform mat4 u_MVPMatrix;" +
        "attribute vec4 a_Position;" +
        "attribute vec2 a_texCoord;" +
        "varying vec2 v_texCoord;" +
        "void main() {" +
        "  gl_Position = u_MVPMatrix * a_Position;" +
        "  v_texCoord = a_texCoord;" +
        "}";

    static final String fs_Image =
        "precision mediump float;" +
        "uniform sampler2D u_texture;" +
        "varying vec2 v_texCoord;" +
        "void main() {" +
        "  gl_FragColor = texture2D(u_texture, v_texCoord);" +
        "}";

    // Program variables
    public static int programTexture;
    public static int vertexShaderImage, fragmentShaderImage;

    public static int loadShader(int type, String shaderCode){

        // create a vertex shader type (GLES20.GL_VERTEX_SHADER)
        // or a fragment shader type (GLES20.GL_FRAGMENT_SHADER)
        int shader = GLES20.glCreateShader(type);

        // add the source code to the shader and compile it
        GLES20.glShaderSource(shader, shaderCode);
        GLES20.glCompileShader(shader);

        // return the shader
        return shader;
    }

    public static void initGlProgram() {
        // Create the shaders, images
        vertexShaderImage = ShaderHelper.loadShader(GLES20.GL_VERTEX_SHADER, ShaderHelper.vs_Image);
        fragmentShaderImage = ShaderHelper.loadShader(GLES20.GL_FRAGMENT_SHADER, ShaderHelper.fs_Image);

        ShaderHelper.programTexture = GLES20.glCreateProgram();             // create empty OpenGL ES Program
        GLES20.glAttachShader(ShaderHelper.programTexture, vertexShaderImage);   // add the vertex shader to program
        GLES20.glAttachShader(ShaderHelper.programTexture, fragmentShaderImage); // add the fragment shader to program
        GLES20.glLinkProgram(ShaderHelper.programTexture);                  // creates OpenGL ES program executables
    }

    public static void dispose() {
        GLES20.glDetachShader(ShaderHelper.programTexture, ShaderHelper.vertexShaderImage);
        GLES20.glDetachShader(ShaderHelper.programTexture, ShaderHelper.fragmentShaderImage);

        GLES20.glDeleteShader(ShaderHelper.fragmentShaderImage);
        GLES20.glDeleteShader(ShaderHelper.vertexShaderImage);

        GLES20.glDeleteProgram(ShaderHelper.programTexture);
    }
}

MySprite

public class MySprite {
    public int X, Y;
    public float XScale, YScale;
    private int w, h;

    int textureId = -1;

    private MySprite(Bitmap bmp, int textureId) {
        this.w = bmp.getWidth();
        this.h = bmp.getHeight();
        this.textureId = textureId;
        this.XScale = this.YScale = 1f;
    }

    public static MySprite createGLSprite(final AssetManager assets, final String assetImagePath) {
        Bitmap bmp = TextureHelper.getBitmapFromAsset(assets, assetImagePath);
        if (bmp == null) return null;

        MySprite ms = new MySprite(bmp, createGlTexture());
        Log.d("G1", "image id = " + ms.getTextureId());

        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
        GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);

        // Load the bitmap into the bound texture.
        GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bmp, 0);

        bmp.recycle();
        return ms;
    }

    private static int createGlTexture() {
        // Generate Textures, if more needed, alter these numbers.
        final int[] textureHandles = new int[1];
        GLES20.glGenTextures(1, textureHandles, 0);

        if (textureHandles[0] != 0) {
            GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureHandles[0]);
            return textureHandles[0];
        } else {
            throw new RuntimeException("Error loading texture.");
        }
    }
...
}

Answer 1

Your code mixes up two concepts: texture ids (or, as they are called in the official OpenGL documentation, texture names), and texture units:

• A texture id is a unique id for each texture object, where a texture object owns the actual data, as well as sampling parameters. You can have a virtually unlimited number of texture objects, with the practical limit typically being the amount of memory on your machine.
• A texture unit is an entry in a table of textures that are currently bound, and available to be sampled by a shader. The maximum size of this table is an implementation dependent limit, which can be queried with glGetIntegerv(GL_MAX_TEXTURE_IMAGE_UNITS, ...). The guaranteed minimum for compliant ES 2.0 implementations is 8.

You're using texture ids correctly while creating your textures, by generating an id with glGenTextures(), binding it with glBindTexture(), and then setting up the texture.

The problem is where you set up the textures for drawing:

GLES20.glUniform1i(u_texture, ms.getTextureId());

The value of the sampler uniform is not a texture id, it is the index of a texture unit. You then need to bind the texture you want to use to the texture unit you specify.

Using texture unit 0, the correct code looks like this:

GLES20.glUniform1i(u_texture, 0);
GLES20.glActiveTexture(GL_TEXTURE0);
GLES20.glBindTexture(ms.getTextureId());

A few remarks on this code sequence:

• Note that the uniform value is the index of the texture unit (0), while the argument of glActiveTexture() is the corresponding enum (GL_TEXTURE0). That's because... it was defined that way. Unfortunate API design, IMHO, but you just need to be aware of it.
• glBindTexture() binds the texture to the currently active texture unit, so it needs to come after glActiveTexture().
• The glActiveTexture() call is not really needed if you only ever use one texture. GL_TEXTURE0 is the default value. I put it there to illustrate how the connection between texture unit and texture id is established.

Multiple texture units are used if you want to sample multiple textures in the same shader.

Answer 2

To begin I'll point out some general things about OpenGL:

Each texture is a large square image. Loading that image into the gpu's memory takes time, as in you can't actively swap images into gpu's texture memory and hope for a fast run time.

Q1: The reason only the second image is showing is because of this line in your sprite class:

GLUtils.texImage2D(GLES20.GL_TEXTURE_2D, 0, bmp, 0);

You call that twice, therefore texture0 is replaced by the second image, and only that image is called.

To combat this, developers load a single image that contains a lot of smaller images in it, aka a texture map. The size of the image that can be loaded largely depends on the gpu. Android devices range roughly from 1024^2 pixels to 4096^2 pixels.

To use a smaller part of the texture for a sprite, you have to manually define the uvArray that is in your batcher class.

Let's imagine our texture has 4 images divided as follows:

 (0.0, 0.0) top left   _____ (1.0, 0.0) top right
                      |__|__| middle of the square is (0.5, 0.5) middle
 (0.0, 1.0) bot left  |__|__|(1.0, 1.0) bot right

That means the uv values for the top left image are:

static float[] uvArray = new float[]{
        0.0f, 0.0f, //top left
        0.0f, 0.5f, //bot left
        0.5f, 0.5f, //bot right
        0.5f, 0.0f  //top right
};

This way you just quadrupled the amount of sprites you can have on a texture.

Because of this you will have to pass no only which texture the sprite is on, but also it's custom uvs that the batcher should use.