GLSL

This page is a tutorial by Hitchhiker about GLSL in OpenArena and how to use it.
GLSL allows to use features of modern video cards to show "cool" graphic effects: a graphic card capable of Pixel Shader 2.0 features is required to use GLSL.

If you are a player, just reading Manual/Graphic options#GLSL effects could be enough for you... if you are a mapper/developer who wants to create or apply GLSL effects (usually, for shaders to be used in maps), you should read this page.

Trivia: OpenArena is mostly OpenGL (Wikipedia) 1.1 suff. GLSL is OpenGL 2.0 stuff.

Introduction[]

OpenGL Shading Language (in short, GLslang or GLSL; see Wikipedia article) is a programming language that permits us to program how the pixels are drawn over a polygon. As we can program this behavior - we can make some interesting effects that would otherwise be impossible to do (a.l.e. we can color the pixel drawn depending on its position inside the game world).

OA can use GLSL programs to either texture a polygon of a 3d object (in game) or postprocess the final image that OA outputs to the monitor.

OA Postprocessing works like this:

the image of a game scene is drawn to OpenGL buffer by drawing 3d objects' polygons
if postprocessing is enabled the following two steps are performed:
- the image on the screen is captured in a texture image
- a big polygon that covers the entire screen is drawn using a glsl program (this glsl program uses the texture captured and can modify each pixel color as needed)

To enable postprocessing you only need to tell OA which glsl program to use when re-drawing the big polygon. This is done using the /r_postprocess <program> command.

Example:
/r_postprocess BWfilter
the above command tells OA to use glsl program called 'BWfilter'

CVARS[]

In OpenArena 0.8.8, a new cvar has been added to enable/disable the GLSL rendering path (default: disabled):
r_ext_vertex_shader <0 or 1> - 0 disable, 1 enable

A new cvar has been added to specify the GLSL postprocessing program (default: "none")
r_postprocess ProgramName
ProgramName value by default is "none", meaning no postprocessing.
Specifying program name other than "none" causes OA to load vertex and fragment shaders of that program name and compile them to glsl program which is then used to postprocess the framebuffer at the end of each video frame.
Once program name specified, please make sure to have a valid files available inside the GLSL folder of the OA pack file. Each program needs vertex (suffixed with '_vp.glsl') and fragment (suffixed with '_fp.glsl') source files available.
I.e. /r_postprocess "BWFilter" command will make OA look for and compile GLSL\BWFilter_vp.glsl and GLSL\BWFilter_fp.glsl glsl source code files.

Currently, postprocessing program can be composed of only one vertex source code and one fragment source code file. This means that if you wish to do multiple postprocessing effects you will have to join them to a single vertex source code and single fragment source code.

Notes:

Postprocessing will only run if CVAR r_ext_vertex_shader set to 1.
Each video frame's opengl Color Buffer is available under u_Texture0 uniform Sampler2d GLSL identifier at postprocessing time. Each video frame's opengl Depth Buffer is available under u_Texture7 uniform Sampler2d GLSL identifier at postprocessing time.

GLSL programs location within OA[]

Actually the PAK file pak6-patch088.pk3 contains GLSL folder. This folder contains source code of the existing glsl programs. You will notice that each program has two source code components – a vertex and a fragment one. More on the two components later.

Note: Postprocessing will automatically add _vp.glsl and _fp.glsl to the postprocessing program name. Thus the two components that will be loaded in the above example are: Bwfilter_vp.glsl and Bwfilter_fp.glsl. If you are creating your own postprocessing glsl program, save the two source files with correct names (i.e. if you are creating 'cooleffect' postprocessing effect, the names of the two files should be cooleffect_vp.glsl and cooleffect_fp.glsl).

On to using GLSL programs within the Q3 shader files[]

In order to use GLSL for polygons you need to add few lines of code to the Q3 shader.

Example (pay particular attention to the "//GLSL" lines):

textures/gothic_block/blocks11b 
{ 
    { 
         map textures/gothic_block/blocks11b.tga 
         rgbGen identity
         //GLSL map2 textures/stonewall2/spec1norm.jpg
         //GLSL program bump 
         //GLSL vertexProgram glsl/bump_vp.glsl 
         //GLSL fragmentProgram glsl/bump_fp.glsl
    } 
    { 
         map $lightmap 
         blendfunc filter 
         rgbGen identity 
         tcGen lightmap 
    } 
    { 
         map gfx/fx/detail/d_rock.tga 
         blendfunc gl_dst_color gl_src_color 
         tcMod scale 16 16 
         detail 
    } 
}

What the above //GLSL commands do is specify additional texture to make available to the 'bump' glsl program (using map2 command), specify the program name (using program command) and the two source code components that will compile as 'bump' glsl program (using vertexProgram and fragmentProgram commands).

If you wish to only use the glsl program and skip the second and the third stage of the Q3 shader you would need to add //GLSL program skip to the second and third stage.

Example:

textures/gothic_block/blocks11b 
{ 
    { 
         map textures/gothic_block/blocks11b.tga 
         rgbGen identity 
         //GLSL map2 textures/stonewall2/spec1norm.jpg
         //GLSL program bump 
         //GLSL vertexProgram glsl/bump_vp.glsl 
         //GLSL fragmentProgram glsl/bump_fp.glsl
    } 
    { 
         map $lightmap 
         blendfunc filter 
         rgbGen identity 
         tcGen lightmap 
         //GLSL program skip
    } 
    { 
         map gfx/fx/detail/d_rock.tga 
         blendfunc gl_dst_color gl_src_color 
         tcMod scale 16 16 
         detail 
         //GLSL program skip
    } 
}

Now when the engine goes on to rendering polygons that are marked as 'textures/gothic_block/blocks11b' it will use the above shader and at the first pass it will render the polygon using the 'bump' glsl program. 'bump' program can do anything it wants with the pixels that are part of the polygon – paint them using the texture specified in map2 command, invert the color values, etc..

The execution of the glsl program is done at polygon drawing time and any information that the engine sends to the glsl program is defined in the Q3 shader and the glsl program itself. This means that you can use the same commands with more than one glsl shader and change the parameters (like which image to use).
The list of the parameters that are actually used is specific to the glsl program – i.e. bump program will always use only 1 additional texture and bump2 program (if you choose to write it) can use 5 additional textures.

As the glsl programs are stored as simple text files and are compiled at runtime, you do not need to recompile OA in order to make changes to your glsl program. You do need to restart OA in order to see how the glsl program renders once in game.

There are few applications on the internet that permit you to create glsl programs without even using OA. Once your glsl programs are made using these applications they can easily be copied and used in game (possibly with just changing a name of a variable to reflect the variable available in OA). Please see http://advsys.net/ken/download.htm for one such application.

Getting a bit more techinical[]

Q3 shader commands and glsl program variables available in OA[]

Shaders (Quake 3)[]

This will be specifically interesting for developers. To enable rendering of a surface with a GLSL program, the following keywords have been introduced:

Program <name>

Name used to reference the program. To be used in combination with 'vertexProgram' and 'fragmentProgram'.
NOTE: There is a "magic program" called "skip" which will cause a shader's stage to be ignored when rendering.

VertexProgram <path1> ... <path8>

One to eight files with glsl code, making up the vertex stage of the program (remember that exactly one file needs a main() function as an entry point). To be used in combination with 'vertexProgram' and 'fragmentProgram'.
NOTE: for simplicity reasons I would recommend using only one source file for vertex program.

fragmentProgram <path1> ... <path8>

One to eight files with glsl code, making up the fragment stage of the program (remember that exactly one file needs a main() function as an entry point). To be used in combination with 'vertexProgram' and 'fragmentProgram'. NOTE: for simplicity reasons I would recommend using only one source file for fragment program.

(anim|clamp|clamanim|video)map[2-7] <path>

This will load any texture supplied in <path> and map it to texture unit [2-7].

Some notes[]

Notes:

Map <path> in the first stage of a shader equals texture unit 0.
Map <path> in the second stage of a shader equals texture unit 1 whenever OA would switch to multi-texturing.
If vertex shaders are disabled/unsupported all related keywords are ignored and rendering will work as expected from OA.
For compatibility reasons new shader keywords (above) need to be prefixed with //GLSL keyword. This keyword starting with '//' is recognized as a comment by older builds of OA, thus the entire line ignored. Newer builds will skip-over the "//GLSL" keyword and process the shader line as if the "//GLSL" keyword was not there.

Example of how new builds of OA 'see' the lines beginning with "//GLSL" keyword (shader stage):

    {
    //GLSL program <name> 
    }

becomes:

    {
    program <name> 
    }

Note: it's strongly recommended to always specify the '//GLSL' keyword.

Now that your Q3 shader is ready, you can start programming the glsl program itself.

Variables (GLSL)[]

So now you know how to activate your program. However you may need access to variables which are not automatically forwarded to GLSL programs (so unlike gl_* variables). For this purpose I have defined a list of keywords. Whenever one of these keywords is found to be a variable name (of the proper type) within your GLSL shader the engine will automatically supply you with useful data.

Recognized Keywords[]

uniform int u_AlphaGen;
uniform vec3 u_AmbientLight;
uniform int u_ColorGen;
uniform vec4 u_ConstantColor;
uniform vec3 u_DirectedLight;
uniform vec4 u_EntityColor;
uniform vec4 u_FogColor;
uniform int u_Greyscale;
uniform float u_IdentityLight;
uniform vec3 u_LightDirection;
uniform mat4 u_ModelViewMatrix;
uniform mat4 u_ModelViewProjectionMatrix;
uniform mat4 u_ProjectionMatrix;
uniform int u_TCGen0;
uniform int u_TCGen1;
uniform int u_TexEnv;
uniform sampler2D u_Texture0;
uniform sampler2D u_Texture1;
uniform sampler2D u_Texture2;
uniform sampler2D u_Texture3;
uniform sampler2D u_Texture4;
uniform sampler2D u_Texture5;
uniform sampler2D u_Texture6;
uniform sampler2D u_Texture7;
uniform float u_Time;
uniform vec3 u_ViewOrigin;

GLSL programs basics (example of a simple glsl program)[]

Each glsl program is made of two components: vertex shader and fragment shader.

Vertex shader[]

You probably know that in 3d graphics, each polygon is composed of vertexes that define the corners of that polygon.

Vertex shader is concerned with modifying the vertex information and calculating values that might be of interest to us. The vertex part of the glsl program is executed once for each vertex of the polygon. With the vertex shader part you can i.e. move the vertex around or use the vertex information to calculate different values.

Fragment shader[]

Fragment shader is where the actual color and depth value of the pixel drawn are calculated.

So let's try to write a simple GLSL program[]

We will create a glsl program that mixes two textures together depending on how high the pixel is in the game world. We will use Z value of the pixel for this. We will also use two textures.

Vertex program[]

Vertex program (a simple example – save sa zmix_vp.glsl in the GLSL folder within the PAK file):

varying float z;  // use varying type in vertex program to pass vertex info to the fragment program 
void main ()
{
    z=gl_Vertex.z;
    gl_Position=ftransform();
    gl_TexCoord[0]=gl_MultiTexCoord0;
}

Fragment program[]

Fragment program (a simple mix example – save sa zmix_fp.glsl in the GLSL folder within the PAK file):

uniform sampler2D u_Texture0;  //  accepting texture0 from OA engine / Q3 shader – map cmd
uniform sampler2D u_Texture2;  //  accepting texture2 from OA engine / Q3 shader – map2 cmd
varying float z;  	// accepting z variable from the vertex program 
void main ()
{
    gl_FragColor=mix(   texture2D(u_Texture0,gl_TexCoord[0].st),
                        texture2D(u_Texture2,gl_TexCoord[0].st),
                        clamp((z/200),0,1)); // we scale z value otherwise the transition between the two textures would be too sharp – we also clamp the result to range of 0..1			
}

Preparing Q3 shader[]

Once you have the above two source files saved, modify the Q3 shader to use the program. In example we will modify the shader in detailtest.shader file of the pak6-patch088.pk3. The shader name is textures/e7/e7bricks01 and is used in ce1m7 map. Make it look like this:

textures/e7/e7bricks01 
{ 
    {
         map textures/e7/e7bricks01.tga 
         map2 textures/acc_dm5/stntiles_moss2.jpg 
         //GLSL program  zmix 
         //GLSL vertexProgram glsl\zmix_vp.glsl 
         //GLSL fragmentProgram glsl\zmix_fp.glsl 
         rgbGen identity 
    } 
    { 
         map textures/detail/d_rock.tga 
         blendfunc gl_dst_color gl_src_color 
         tcMod scale 8 8 
         detail 
    } 
    { 
         map $lightmap 
         blendfunc filter 
         rgbGen identity 
         tcGen lightmap 
    } 
  
}

You will notice that the lightmap is moved to the last position (as opposed to the original Q3 shader). This is not really due to glsl but to the way that OA will try to compact the rendering passes – something I do not understand fully. Also, the lightmap pass could very well be integrated in the glsl program but I do not have an example of that yet?

Checking result[]

OK. Now the Q3 shader was updated, glsl program written and included in the PAK file. You just need to enable glsl in OA and load up ce1m7 map. You should see something like this:

Notice the transition between two wall textures - the wall is drawn using a single Q3 shader but the texture at lower part of the wall is different from the one at the top. You could use lava or, on some other map, a moss texture for the lower part of the wall.

You could also add map3 texture to be a black/white texture and multiply the clamped value by the value of the texture.

Sort of changing the fragment shader to be:

uniform sampler2D u_Texture0;  //  accepting texture0 from OA engine / Q3 shader – map cmd
uniform sampler2D u_Texture2;  //  accepting texture2 from OA engine / Q3 shader – map2 cmd
uniform sampler2D u_Texture3;  //  NOTICE THIS LINE - accepting texture3 from OA engine / Q3 shader – map3 cmd
varying float z;      // accepting z variable from the vertex program 
void main ()
{
gl_FragColor=mix(  texture2D(u_Texture0,gl_TexCoord[0].st),
                   texture2D(u_Texture2,gl_TexCoord[0].st),
                   texture2D(u_Texture3,gl_TexCoord[0].st).r*clamp((z/200),0,1)); // NOTICE THIS LINE - we scale z value otherwise the transition between the two textures would be too sharp – we also clamp the result to range of 0..1			
}

You can also scale the texture by multiplying the gl_TexCoord[0].st with vec2(2,4)

About OpenArena GLSL support[]

GLSL support developed by:

Copyright (C) 2009 Jens Loehr <jens.loehr@gmx.de> under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

Postprocessing, Depthbuffer access and OpenArena implemetation of Jens Loehr's GLSL support added by Hitchhiker.

GLSL support introduced in OpenArena 0.8.8.

External links[]

GLSL thread on OA forum