Conceptual Overview
Immediate Mode Graphics
ST3D makes use of the OpenGL application programming interface (API) to render low-level graphic primitives like lines, points and polygons. OpenGL is considered an "immediate mode" interface since it generally doesn’t manage or retain any information about the graphic primitives it draws. It simply draws a primitive when instructed to do so. For example, the following OpenGL commands, invoked from within Smalltalk, draw a green triangle on a black background (To try this code out, open the "CC OpenGL Lab" example in submenu/subfolder Tools->ST3D->ST3D Sample Applications).
Opengl
clearWithColor: CcColor black;
color: CcColor green;
begin: GL_TRIANGLES;
vertex: -0.5 @ 0 @ 0;
vertex: 0.5 @ 0 @ 0;
vertex: 0 @ 0.5 @ 0;
end.
If the view in which the triangle is rendered were subsequently invalidated for some reason (e.g., covered with another window and then uncovered), the triangle would be erased and it would be necessary for the application to re-execute the OpenGL commands to redraw the triangle.
Although OpenGL is a stable, well-documented and relatively easy to use API, building an application like a game or modeler directly on top of an immediate mode interface can be a complex and time consuming activity. To simplify and accelerate the development process, ST3D provides a number of object-oriented services, frameworks and tools that, in many cases, make it unnecessary to learn OpenGL or use it directly. For developers who would still prefer to work at the OpenGL level anyway, ST3D provides "raw" access to dozens of OpenGL functions and related structures. Also, even if not used for application development, ST3D’s higher level services (which are, themselves, built on OpenGL) provide a wealth of OpenGL programming examples.
Additional details on ST3D’s OpenGL subsystem can be found in Appendix A. For additional sources of information on OpenGL itself, see the section entitled "OpenGL" in Appendix F .
Retained Mode Graphics and Scene Graphs
ST3D provides a "retained mode" graphics system built on top of OpenGL’s immediate mode graphics. The retained mode layer supports the creation of graphic objects with associated state like position, orientation, color and texture (e.g., a spaceship located at position 500 @ 100 @ 0). These graphic objects are referred to as "nodes" and are held in a structure called a "scene graph".
A scene graph represents a complete three-dimensional world. It consists of a hierarchy of CcSceneGraphNode subinstances with an instance of CcRootNode at its root (See Figure 1). A node has exactly one parent (except for the root node, which does not have a parent) and can have one or more children.
Figure 1. Example of an ST3D scene graph
Attributes such as color and textures can be assigned to nodes to change their appearance. Each node also holds an instance of a CcTransform that affects its location, size and orientation.
A brief overview of the CcSceneGraphNode class hierarchy is presented below. Additional detail can be found in class and method comments.
- CcSceneGraphNode – Abstract class
which maintains node attributes and transformations. Instances know their parent node.
- CcGroupNode – Abstract class which
maintains child nodes and provides services for child node manipulation.
- CcRootNode – Instances or
subinstances are always at the root of a scene graph.
- CcDefaultWorld – Provides behavior for initializing a scene graph with a default set of nodes (e.g., a default camera and light).
- CcCamera – Abstract class. Subinstances determine the viewpoint from which a scene is rendered.
- CcLight – Provide lighting for a scene.
- CcSwitchNode – A node which allows only one of its children to be active at a time (its "selection"). When rendered, a CcSwitchNode simply renders the selected child.
- CcPart – The superclass for all
visible nodes. Adds behavior for bounding boxes calculations and convenience methods for
accessing attributes and transforms.
- CcShape – A kind of CcPart which
holds a single subinstance of CcGeometry.
- CcVertexShape – Holds a single instance of CcVertexGeometry.
- CcQuadricShape – Holds a single subinstance of CcQuadricGeometry.
- CcMeshShape - Holds a single subinstance of CcMeshGeometry.
- CcShape – A kind of CcPart which
holds a single subinstance of CcGeometry.
- CcRootNode – Instances or
subinstances are always at the root of a scene graph.
- CcGroupNode – Abstract class which
maintains child nodes and provides services for child node manipulation.
Shapes and Geometry
CcShape is an abstract subclass of CcPart that holds a single subinstance of CcGeometry. An instance of CcGeometry defines the raw, three-dimensional form of a shape (i.e., its appearance as a cube, sphere, or pool table). CcGeometry subclasses CcVertexGeometry and CcQuadricGeometry implement this spatial definition in two distinct ways.
CcVertexGeometry uses an explicit list of vertices (CcVertex) and faces (CcFace) to define its form. For example, an instance of CcVertexGeometry representing a cube would contain eight vertices and six faces with each face referencing the four vertices that define it. CcVertexGeometry instances are created, modified and saved to disk as .geo files using the ST3D Geometry Editor.
CcQuadricGeometry and its subclasses make use of OpenGL "quadric" objects to define shapes. OpenGL quadric objects automatically create the appropriate set of faces and vertices based on their type. Unlike instances of CcVertexGeometry, the faces and vertices of a CcQuadricGeometry are managed internally by OpenGL and are not accessible from Smalltalk. This means that quadric objects are generally used "as is" and are not defined and saved to disk using an editor. CcQuadricGeometry and its subclasses do allow some degree of configuration, however, related to level of detail (e.g., number of slices and stacks in a sphere).
OpenGL provides several types of quadric objects. These types are supported by subclasses of CcQuadricGeometry (e.g., CcSphereGeometry, CcCylinderGeometry).
Transforms
CcSceneGraphNodes hold an instance of CcTransform. A transform determines a node’s position, orientation and scale with respect to its parent node’s coordinate system. As a simple example, if the position of node A is 5 @ 0 @ 0 and it has a child, B, with a position of 2 @ 0 @ 0, child B’s "overall" or "world" position is its parent’s position plus its own position, or 7 @ 0 @ 0.
The position, orientation and scale specified by a node’s transform are considered its "local" values (again, with respect to its parent). When a node’s local transform values are translated into the coordinate system of the scene graph’s root node, they are referred to as "world" values.
Services are available on CcSceneGraphNode for accessing a node’s local position, orientation and scale and for mapping these local values to world values (and visa versa).
CcShape uses a second instance of CcTranform to control its dimension and origin. This transform, held in instance variable ‘geometryTransform’, does not affect the size or position of a shape’s children.
Node Attributes
Subinstances of CcAttribute can be assigned to a scene graph node to affect various aspects of the node’s appearance. These aspects include color, shading, shine and face culling. The attributes of a node are inherited by its descendents. If a node is assigned the color "red", for example, all of its descendents will be red.
CcShape also allows textures to be associated with faces and colors to be associated with either individual faces or vertices. These component level attributes are not inherited by the shape’s children.
Rendering a Scene Graph
It is often necessary to be able to provide several different "on screen" renditions of the same scene graph node. For example, in addition to its "normal" rendition, a node might have a highlighted rendition and a rendition used by the hit detection (picking) process.
Because of the variety of ways in which it might be rendered, a node is usually not responsible for rendering itself. Instead, ST3D uses a render manager (CcRenderManager) and a collection of renderers (subinstances of CcNodeRenderer) to render a scene. The render manager maintains a mapping between node classes and instances of node renderers. As the render manager traverses a scene graph, starting with the root node, it delegates the bulk of the rendering work to the appropriate renderer as each node is encountered. A node renderer is responsible for translating the high-level information contained in a scene graph node (e.g., red box at position 2 @ 3 @ 4) into the proper sequence of OpenGL drawing commands.
User Interface Support
Services for rendering scene graphs in a view and receiving mouse and keyboard input are provided by CcOpenglPresenter and CcOpenglView. The model of the presenter/view pair is normally set to the root node of the scene graph that is to be rendered in the view. The presenter also holds a subinstance of CcCamera (See Figure 2). The location and orientation of the camera determines the viewpoint from which the scene is rendered.
When asked to render the scene graph, the presenter first sets its camera as the active camera in the scene graph’s switch node (See Figure 2). It then trigger’s an event (#renderRequired) which provides clients of the presenter (e.g., an application shell) an opportunity to render the scene. The application usually delegates the actual rendering work to one or more instances of CcRenderManager. For example, CcEditor contains a single CcOpenglPresenter but uses two render managers: one for default object rendering and one for highlighting selected objects. When the #renderRequired event is triggered, the editor’s handler for the event tells its default render manager to render the scene graph, starting with the scene graph root.
An alternative to registering an event handler is to have the CcOpenglPresenter initialize a default render manager. This is done using CcOpenglPresenter>>initDefaultRenderer. In this case, only the default rendition of the nodes is displayed (For an example of this, see CcSimpleSceneExample).
It is important not to modify the scene graph in any way while it is being rendered. For example, it would be unwise to fork a separate process that updates the scene graph independently of the process that renders the scene. Changes to the scene graph and scene rendering should always be serialized (See CcAnimationExample1).
Figure 2. Connection of user interface components to scene graph
High-performance Rendering Using Display Lists and Vertex Arrays
OpenGL allows a sequence of commands to be compiled and stored in a structure known as a "display list". Once created, a display list can be rendered with a single OpenGL function call. This single call typically takes much less time to execute than the series of discrete calls used to build the list.
Any part, shape, or geometry object can be "display listed" by sending the object #enableDisplayList. It should be noted, however, that changes to an object’s attributes (e.g., color) will not be reflected in its appearance if its display list has already been initialized. If an attribute is changed, the display list should be reset using #resetDisplayList. Nesting of display lists is not supported.
Another OpenGL feature exploited by ST3D is vertex arrays. Vertex
arrays, like display lists, allow an entire shape to be rendered with only a few calls to
OpenGL. Unlike display lists, however, the use of vertex arrays is not enabled by sending
the shape a message. Instead, the node renderer responsible for rendering the
shape examines the shape’s properties to see if vertex arrays can be used.
In the case of a CcMeshShape, the renderer is always able to make use of vertex
arrays since mesh shapes store their vertex and face data in a vertex array
"friendly" format. CcVertexShapes, on the other
hand, must contain either all triangles or all quads, be smoothly shaded
and have no component-level attributes in order for the
renderer to make use of vertex arrays.
Part Persistence
Instances of CcPart are saved to disk using the Dolphin binary filer (extension .STB). The instance of CcVertexGeometry that is held by a CcVertexShape is not saved to the .STB file along with the rest of the shape, however. Instead, a reference to the geometry instance (the geometry’s filename) is saved.
CcVertexGeometry instances are saved to disk in an ST3D specific format using CcGeometryWriter. To allow flexibility in organizing files within the file system, the geometry filename saved with the vertex shape does not include any path information. When materializing a CcVertexShape instance from it’s STB file, its associated geometry is materialized from the first geometry file found in the ST3D\Geometry folder hierarchy that matches the specified file name. It follows that all geometry files within the Geometry folder hierarchy should have unique names.
The above discussion also applies to instances of CcTexture held by CcVertexShape. Instead of saving the texture along with the shape, a reference is saved which points to the filename of the texture’s .BMP or .TXR file (located in the ST3D "Images" folder hierarchy).
Unlike, CcVertexGeometry, an instance of CcMeshGeometry is saved along with its containing CcMeshShape when the mesh shape is saved.
Object Caching
ST3D caches instances of CcVertexGeometry and CcTexture since they tend to be large and time consuming to load. Instances of these classes are sharable across multiple subinstances of CcShape and are uniquely identified by their filename. Cache entries are lazy initialized when a specified filename is requested. Geometry and texture caches can be cleared by sending the appropriate class #resetCaches.
Modeler Support
A "modeler" is an application that allows a 3D model to be edited in a WYSIWYG environment and saved to disk (See Figure 3). A 3D model is nothing more than a scene graph containing one or more instances of CcVertexShape but it can represent anything from a spaceship to a landscape to the interior of a building.
Like other 3D applications, a modeler renders its scene graph in a CcOpenglView with the assistance of a CcOpenglPresenter, a CcRenderManager and various subinstances of CcNodeRenderer. Unlike the average game or simulation, however, a modeler must allow specific shapes and components (e.g., vertices) in the model to be selected, highlighted and manipulated. In some situations, the most natural way of editing the model is via direct manipulation using the mouse (i.e., drag/drop). In other cases, more precise methods, like forms-based data entry, are needed.
ST3D provides services for hit detection (a.k.a., "picking"), selection management and state management (i.e., undo/redo support). A framework for the development of interactive modeler tools (e.g., "rotate part" tool or "scale texture" tool) is also available. To further simplify modeler development, a subclass of CompositePresenter, CcEditor, is provided that pre-configures many of these services. See Appendix B, "Inside the Part Editor", for more information on the services and frameworks available for building modelers.
Working With Input Devices
ST3D supports two distinct mechanisms for obtaining data from input devices: the standard event-based system provided by Dolphin Smalltalk and a system based on Microsoft’s DirectX DirectInput.
The event-based system allows an application to respond to an event generated by the keyboard or mouse, as the event occurs. This system is familiar to most Smalltalk developers and is somewhat easier to use than the DirectInput approach.
The DirectInput system allows the state of the keyboard or joystick to be read (or polled) at a point in time that is convenient for the application. The polling-based approach is preferable for applications that involve animation since use of the event-based mechanism can adversely affect the frame rate of an animated scene. (Note: A given CcOpenglPresenter can only use one of these mechanisms at a time to access keyboard data. For example, if an application initializes the DirectInput-based system to read keyboard data, no keyboard events will be generated or received).
See Appendix C for more information on working with input devices.
Sound Effects and Music
ST3D allows wave files (.WAV) to be played using DirectX’s DirectSound subsystem. It also allows MIDI music files (.MID) and MP3 files to be played via the Windows Multimedia Command Interface (MCI). The "CoolPool" and "Lander" applications demonstrate the use of DirectSound within Smalltalk.
Sound effects
The key DirectSound-related classes are CcDxs and CcDxsSoundBuffer. CcDxs represents the DirectSound subsystem and provides services for playing wave files and for accessing cached instances of CcDxsSoundBuffer. In order for an application to use DirectSound, it must associate its application window with class CcDxs. CcDxs can only be associated with a single application window at a time (See methods #onViewOpened and #onViewClosed in CcGameShell).
An instance of CcDxsSoundBuffer is created for each wave file played. CcDxsSoundBuffer provides services for playing, copying and setting the volume and frequency of a sound. Sounds can either be played in "one shot" mode or "looped" so that the sound is played over and over. Multiple sounds can be playing at the same time.
Like parts, sound buffers are cached and accessed via a unique file name. Wave files must be located in the ST3D\WAV subdirectory tree.
Music
CcMediaPlayer provides services for playing a MIDI or MP3 file. Once the desired selection (i.e., file name) has been specified, a media player can be told to either play the selection once or play it repeatedly in a loop. DirectSound generated sound effects and MIDI/MP3 music can be playing at the same time and each subsystem’s volume level can be varied independently of the other. It is important, however, that the application initialize the DirectSound subsystem prior to initializing the media player if the systems are to be used concurrently.