Shaders and Materials

What is a Shader

From wikipedia:

In the field of computer graphics, a shader is a computer program that is used to do shading: the production of appropriate levels of color within an image, or, in the modern era, also to produce special effects or do video post-processing. A definition in layman's terms might be given as "a program that tells a computer how to draw something in a specific and unique way".

In other words, it is a piece of code that runs on the GPU (not CPU) to draw the different Cocos2d-x Nodes.

Cocos2d-x uses the OpenGL ES Shading Language v1.0 for the shaders. But describing the GLSL language is outside the scope of this document. In order to learn more about the language, please refer to: OpenGL ES Shading Language v1.0 Spec.

In Cocos2d-x, all Node objects that are renderable use shaders. As an example Sprite uses optimized shaders for 2d sprites, Sprite3D uses optimized shaders for 3d objects, and so on.

Customizing Shaders

Users can change the predefined shaders from any Cocos2d-x Node by calling:

sprite->setProgramState(programState);
sprite3d->setProgramState(programState);

The ProgramState object contains two important things:

• A Program: Basically this is the shader. It contains a vertex and fragment shader.
• And the state, which basically are the uniforms of the shader.

In case you are not familiar with the term uniform and why it is needed, please refer to the OpenGL Shading Language Specification

Setting uniforms to a ProgramState is as easy as this:

auto mvpMatrixLocation = _programState->getUniformLocation("u_MVPMatrix");
const auto& projectionMat = Director::getInstance()->getMatrix(MATRIX_STACK_TYPE::MATRIX_STACK_PROJECTION);
_programState->setUniform(mvpMatrixLocation, projectionMat.m, sizeof(projectionMat.m)));

You can even set callbacks as a uniform value:

auto location = _programState->getUniformLocation("u_progress");
_programState->setCallbackUniform(location, [](backend::ProgramState *programState, backend::UniformLocation uniform)
{
    float random = CCRANDOM_0_1();
    programState->setUniform(uniform, &random, sizeof(random));
}
);

And although it is possible to set ProgramState objects manually, an easier way to do it is by using Material objects.

What is a Material

Assume that you want to draw a sphere like this one:

The first thing that you have to do is to define its geometry, something like this:

...and then define the brick texture, like:

• But what if you want to use a lower quality texture when the sphere is far away from the camera?
• or what if you want to apply a blur effect to the bricks?
• or what if you want to enable or disable lighting in the sphere ?

The answer is to use a Material instead of just a plain and simple texture. In fact, with Material you can have more than one texture, and much more features like multi-pass rendering.

Material objects are created from .material files, which contain the following information:

• Material can have one or more Technique objects
• each Technique can have one more Pass objects
• each Pass object has:
  • a RenderState object,
  • a Shader object including the uniforms

As an example, this is how a material file looks like:

// A "Material" file can contain one or more materials
material spaceship
{
    // A Material contains one or more Techniques.
    // In case more than one Technique is present, the first one will be the default one
    // A "Technique" describes how the material is going to be renderer
    // Techniques could:
    //  - define the render quality of the model: high quality, low quality, etc.
    //  - lit or unlit an object
    // etc...
    technique normal
    {
        // A technique can contain one or more passes
        // A "Pass" describes the "draws" that will be needed
        //   in order to achieve the desired technique
        // The 3 properties of the Passes are shader, renderState and sampler
        pass 0
        {
            // shader: responsible for the vertex and frag shaders, and its uniforms
            shader
            {
                vertexShader = Shaders3D/3d_position_tex.vert
                fragmentShader = Shaders3D/3d_color_tex.frag

                // uniforms, including samplers go here
                u_color = 0.9,0.8,0.7
                // sampler: the id is the uniform name
                sampler u_sampler0
                {
                    path = Sprite3DTest/boss.png
                    mipmap = true
                    wrapS = CLAMP
                    wrapT = CLAMP
                    minFilter = NEAREST_MIPMAP_LINEAR
                    magFilter = LINEAR
                }
            }
            // renderState: responsible for depth buffer, cullface, stencil, blending, etc.
            renderState
            {
                cullFace = true
                cullFaceSide = FRONT
                depthTest = true
            }
        }
    }
}

And this is how to set a Material to a Sprite3D:

Material* material = Material::createWithFilename("Materials/3d_effects.material");
sprite3d->setMaterial(material);

And if you want to change between different Techniques, you have to do:

material->setTechnique("normal");

Techniques

Since you can bind only one Material per Sprite3D, an additional feature is supported that's designed to make it quick and easy to change the way you render the parts at runtime. You can define multiple techniques by giving them different names. Each one can have a completely different rendering technique, and you can even change the technique being applied at runtime by using Material::setTechnique(const std::string& name). When a material is loaded, all the techniques are loaded ahead too. This is a practical way of handling different light combinations or having lower-quality rendering techniques, such as disabling bump mapping, when the object being rendered is far away from the camera.

Passes

A Technique can have one or more passes That is, multi-pass rendering. And each Pass has two main objects:

• RenderState: contains the GPU state information, like depthTest, cullFace, stencilTest, etc.
• GLProgramState: contains the shader (GLProgram) that is going to be used, including its uniforms.

Material file format in detail

Material uses a file format optimized to create Material files. This file format is very similar to other existing Material file formats, like GamePlay3D's and OGRE3D's.

Notes:

• Material file extensions do not matter. Although it is recommended to use .material as extension
• id is optional for material, technique and pass
• Materials can inherit values from another material by optionally setting a parent_material_id
• Vertex and fragment shader file extensions do not matter. The convention in Cocos2d-x is to use .vert and frag

// When the .material file contains one material
sprite3D->setMaterial("Materials/box.material");
// When the .material file contains multiple materials
sprite3D->setMaterial("Materials/circle.material#wood");

Enums:

Types:

• scalar is float, int or bool.
• vector is a comma separated list of floats.

Predefined macros

When running on metal framework, a specific macro definition #define METAL\n was insert at head of fragment shader. As for non-metal situation, such as running on android, the macro definition #version 100\n precision highp float;\n precision highp int;\n was inserted to the head of vertex shader and the macro definition precision highp float;\n precision highp int;\n was inserted to the head of fragment shader.

Predefined uniforms

Predefined uniforms were removed form v4.