<div dir="ltr">Here are my notes from the Procedural Textures Croquet/OpenGL/GLSL project. I had this in a personal wiki - it loses something as plain text. I haven't come across the code anywhere, and I fear it was lost a couple of years ago when I had a hard-drive failure. Time to redo it in WebGL anyway!<div>
<br></div><div>Cheers,</div><div>Dave</div><div><br><div><br></div><div><div>ProceduralTextures</div><div>DaveFaught, 10 August 2009 (created 3 May 2006)</div><div>tags:</div><div>Croquet</div><div>This is an open community project to procedurally build some cool textures for use in the Croquet environment. Plans for this project are still fluid. There are other active projects in the Croquet rendering area that may at some point present code conflicts with this project. We will attempt to make use of the best alternatives available.</div>
<div><br></div><div>The inspiration for the actual procedural textures is largely due to Jerzy Karczmarczuk's Clastic tutorial paper, and so this project is intended to be a library of general functions that can be combined in various ways, rather than a finished set of textures.</div>
<div><br></div><div>The Plan</div><div>The current Croquet SDK code base will be used for the OpenGL support and Tweak 2D UI support. The framework will put the project strictly in the local processing arena, outside of Croquet's replication and collaboration model, at least for now.</div>
<div><br></div><div>Current Status</div><div>The current code for this project is in the PublicContribs repository at: <a href="http://hedgehog.software.umn.edu:8888/PublicContribs">http://hedgehog.software.umn.edu:8888/PublicContribs</a> in the Tweak-OpenGL and ((OpenGL-OpenGL packages. It is necessary to manually do "OGLExtManager initialize" after loading the OpenGL-OpenGL package. Andreas also has an updated OpenGL package in the Hedgehog repository that needs to be merged in.)) <- I believe that this can be all replaced by UMN's new OpenGL packages in the Jabberwocky repository. This also depends on a fix for Mantis #6553, which can temporarily be a pretty trivial do-it-yourself kind of thing. Just change VectorColor#asVector4 so that it returns a Vector4 instance instead of a Vector3.</div>
<div><br></div><div>The current Tweak-OpenGL package contains:</div><div>Two, yes count them, two(2) different classes for a basic OpenGL display in Tweak. One is a subclass of CPlayer and is based on a hack of Bert Freudenberg's Croquet Jasmine-based COpenGLPlayer, and the other is a subclass of CPanel and was donated by Andreas Raab. The CPanel version can be launched directly from Morphic, and the CPlayer version needs to be opened in an already-open Tweak environment. There are a few other differences.</div>
<div>Two examples of the translated NeHe Tutorial #5 (the first one that has 3D objects), one for each of the basic Tweak OpenGL classes.</div><div>The beginnings of a GPUinfo query class to display the level of OpenGL support that the video card and drivers supply. Currently this is primarily used to verify GLSL and FramebufferObject support as required by the Wrinkle application.</div>
<div>Two versions of the Wrinkle application, the main product of this project. The first version, Wrinkle1, uses a single fragment(pixel) shader program at a time and displays the resulting texture dynamically. The second version, Wrinkle2, is more advanced through combining two dynamic procedural textures and allows saving a snapshot of the result to a .png or .jpg file by use of an OpenGL FramebufferObject.</div>
<div><br></div><div>There are short video clips at <a href="http://www.youtube.com/watch?v=Eo22Gdi_b9M">http://www.youtube.com/watch?v=Eo22Gdi_b9M</a> and <a href="http://www.youtube.com/watch?v=qBHbMxplM5c">http://www.youtube.com/watch?v=qBHbMxplM5c</a> </div>
<div><br></div><div>It has taken about 2 years to get the simple method working (calendar time, not working time :-) A similar page to this one is on the Consortium wiki at <a href="http://croquetconsortium.org/index.php/Procedural_Texturing">http://croquetconsortium.org/index.php/Procedural_Texturing</a> </div>
<div><br></div><div>Procedural Code</div><div>The procedural code is targetted to run on the Graphics Processor Unit (GPU) of the 3D graphics video card, using the OpenGL Shading Language (GLSL or glslang). Tobias Germer supplied the GLSL support for Croquet Jasmine, Joshua Gargus supplied the NullTerminatedStringArray class, and Dave Faught supplied the FrameBufferObject support.</div>
<div><br></div><div>The former simple method, used by Wrinkle1, is:</div><div>- Compile the shaders that will generate the texture.</div><div>- Enable the shaders.</div><div>- Render an OpenGL geometry with proper texture coordinates to dynamically display the generated texture.</div>
<div>- Interactively pass parameters to the running shaders from the Tweak UI.</div><div><br></div><div>The current method, used by Wrinkle2, is:</div><div>- Create an FBO with an attached texture.</div><div>- Compile the shaders that will generate the texture.</div>
<div>- Every render cycle:</div><div>— Enable the shaders.</div><div>— Interactively pass parameters to the running shaders from the Tweak UI.</div><div>— Render the shaders into the FBO with texture coordinates.</div><div>
— Disable the shaders and FBO.</div><div>— Render an OpenGL geometry to the display with texture coordinates mapped to the FBO's texture in the normal manner.</div><div>— Allow a snapshot of the FBO texture to be saved to a bitmap file, such as png. To get the texture back from the GPU. This step uses glReadPixels(), which is syncronous.</div>
<div><br></div><div>The input x and y values to the procedure are supplied by the OpenGL geometry texture coordinates, and so are each in the interval 0.0 to 1.0. This is different than many other implementations where the resulting bitmap pixel coordinates, for example between 0 and 127, are used. The results of the functions are assumed to be in the interval -1.0 to 1.0, and may be forced to that by using the fract() function. The function results are then generally mapped to an RGB color space, where each of the components are in the interval 0.0 to 1.0.</div>
<div><br></div><div>Some of this process is purposely vague in order to promote experimentation, but there are limitations imposed on the process by the GPU hardware.</div><div><br></div><div>One general procedure might therefore look like this:</div>
<div><br></div><div>/* convert the input x and y from a 0.0 : 1.0 interval to a -1.0 : 1.0 interval */</div><div>myX = (x * 2.0) - 1.0;</div><div>myY = (y * 2.0) - 1.0;</div><div><br></div><div>/* do some interesting function or combination of functions */</div>
<div>myZ = myX * myY;</div><div>...</div><div><br></div><div>/* ensure that the result in in the -1.0 : 1.0 interval */</div><div>myZ = fract(myZ);</div><div><br></div><div>/* convert the result to a 0.0 : 1.0 interval */</div>
<div>myZ = (myZ + 1.0) / 2.0;</div><div><br></div><div>/* map the result into an RGB color space */</div><div>color = mix (cAqua, cGold, myZ);</div><div><br></div><div>UI Design</div><div>There is some dynamic interaction between the Tweak UI (sliders and buttons) and the compiled shader programs, although the UI is not expected to ever get to the level that is achieved by the ArtOfIllusion Texture Editor. There will have to be a tradeoff between the code restrictions of smaller GPUs and the desire for procedural function flexibility. The desire to accomodate smaller GPUs pretty much rules out the AOI TexEd toolkit approach, so a more fixed stack with sliders for each stage would be appropriate, and by definition more limited. How much of a stack can be accommodated or is desirable?</div>
<div><br></div><div>There are a few different classes of functions that may feed this, for example ones that take a single parameter z=f(x) and those that take two parameters z=f(x,y). The fact that they (all?) produce a single output means that they cannot simply be linked in series.</div>
<div><br></div><div>Tweak Considerations (orig. from the mailing list)</div><div>Yes, I too have had problems when switching back out of a Tweak Project to Morphic, and I have for the moment elected not to deep dive into assisting with the debugging in this area. The result is that I generally don't use a Tweak Project at all.</div>
<div><br></div><div>Note that there is an environmental difference in opening a "Tweak Project" and opening a "Tweak Project Window" from the open menu. I have not had the same problems with a Tweak Project Window, and in fact it generally seems to cooperate with Morphic more cleanly. It is</div>
<div>actually much easier to develop and test code using a Tweak Project Window because of this cleaner interaction with Morphic.</div><div><br></div><div>So the way I go about developing with the OpenGL player in Tweak is to make my code changes with the Morphic class browser and other nice Morphic code tools, and test out the code with a Tweak Project Window (which can remain open or minimized while going back to the Morphic browser for more changes). When I think the code is relatively clean, then I save the image.</div>
<div><br></div><div>Andreas Raab has provided an alternative OpenGL Tweak panel which can be launched directly from Squeak. This panel may be a better alternative for some projects.</div><div><br></div><div>Background</div>
<div>Here are a few resources for general procedural texturing:</div><div><a href="http://users.info.unicaen.fr/~karczma/Work/Clastic_distr/clastic.html">http://users.info.unicaen.fr/~karczma/Work/Clastic_distr/clastic.html</a></div>
<div><a href="http://mrl.nyu.edu/~perlin/">http://mrl.nyu.edu/~perlin/</a></div><div><a href="http://graphics.cs.uiuc.edu/~jch/papers/">http://graphics.cs.uiuc.edu/~jch/papers/</a></div><div><a href="http://conal.net/Pan/">http://conal.net/Pan/</a></div>
<div>"Texturing and Modeling, A Procedural Approach" by Ebert, et al. is an excellent book on this subject.</div><div><br></div><div>Also, check out the procedural texture editor in the Open Source 3D modeler Art Of Illusion. You can use the VRML export feature of AOI and select "Create Image Files for Textures" to export 2D procedural textures that can be imported to Croquet with the VRMLImporter. Take a look at the AOI tutorial, Procedural Texture and Material Editor, Chapter 9, "Example Textures" for an idea what the GUI looks like. Only AOI's textures can be exported to VRML, and end up being both textures and materials in Croquet. In AOI, materials are more like OpenGL shader programs, and cannot be expressed in VRML.</div>
<div><a href="http://www.artofillusion.org">http://www.artofillusion.org</a></div><div><a href="http://www.artofillusion.org/docs/texedtut/chapter9">http://www.artofillusion.org/docs/texedtut/chapter9</a></div><div><br></div>
<div>Besides Clastic and the ArtOfIllusion Texture Editor, a couple of other free tools for creating procedural textures are .werkkzeug 3 TE, and mental mill Artist Edition that comes with the NVidia's FX Composer 2 beta 4.</div>
<div><a href="http://www.werkkzeug.com/">http://www.werkkzeug.com/</a></div><div><a href="http://developer.nvidia.com/object/fx_composer_home.html">http://developer.nvidia.com/object/fx_composer_home.html</a></div><div><a href="http://www.mentalimages.com/2_4_mentalmill/index.html">http://www.mentalimages.com/2_4_mentalmill/index.html</a></div>
<div><br></div><div>Michael Kleiber did some nice extensions to Croquet Jasmine to use OGL shader programs for his thesis.</div><div><a href="http://www.kleibi.com/index?page=univ">http://www.kleibi.com/index?page=univ</a></div>
<div><br></div><div>Other shader languages that might be considered at some point are Stanford's BrookGPU or UW's Sh.</div><div><a href="http://www.cs.lth.se/home/Calle_Lejdfors/pygpu/">http://www.cs.lth.se/home/Calle_Lejdfors/pygpu/</a></div>
<div><a href="http://www.graphics.stanford.edu/~hanrahan/talks/gp2/walk001.html">http://www.graphics.stanford.edu/~hanrahan/talks/gp2/walk001.html</a></div><div><a href="http://graphics.stanford.edu/projects/brookgpu/">http://graphics.stanford.edu/projects/brookgpu/</a></div>
<div><a href="http://merrimac.stanford.edu/">http://merrimac.stanford.edu/</a></div><div><a href="http://merrimac.stanford.edu/brook/">http://merrimac.stanford.edu/brook/</a></div><div><a href="http://libsh.org/">http://libsh.org/</a></div>
<div><a href="http://www.shadertech.com">http://www.shadertech.com</a></div><div><a href="http://developer.3dlabs.com">http://developer.3dlabs.com</a></div><div><a href="http://conal.net/Vertigo/">http://conal.net/Vertigo/</a></div>
<div><br></div><div>In order to have the GPU run the code for doing procedural textures, a higher level of GPU functionality is required than that needed to run normal Croquet. A recommended minimum GPU chipset is the NVidia GeforceFX series, or the ATI Radeon 9500.</div>
<div><a href="http://developer.nvidia.com/object/General_FAQ.html#s2">http://developer.nvidia.com/object/General_FAQ.html#s2</a></div><div><a href="http://www.ati.com/developer/atiopengl.pdf">http://www.ati.com/developer/atiopengl.pdf</a></div>
<div><br></div><div>More recent NVidia (Forceware 60 or newer) and ATI (Catalyst 4.5 or newer) drivers include the GLSL compiler within the drivers for Windows and Linux. Apple takes a much larger role in writing drivers for OS X (the Windows and Linux drivers are substantially the same code base, at least for NVIDIA), but finally did finish their glslang support.</div>
<div><a href="http://www.opengl.org/documentation/oglsl.html">http://www.opengl.org/documentation/oglsl.html</a></div><div><a href="http://www.3dshaders.com">http://www.3dshaders.com</a></div><div><a href="http://www.lighthouse3d.com/opengl/glsl/">http://www.lighthouse3d.com/opengl/glsl/</a></div>
<div><br></div><div>The new OpenGL FrameBuffer object is a new extension that is platform-independent and is more flexible than PBuffers, but just became generally available in the NVidia Forceware 75 drivers released in June 2005. Not sure exactly which release of ATI drivers contains FrameBuffer object support, but there is support in the October 2005 ATI Radeon SDK.</div>
<div><a href="http://www.opengl.org/documentation/extensions/EXT_framebuffer_object.txt">http://www.opengl.org/documentation/extensions/EXT_framebuffer_object.txt</a></div><div><a href="http://download.nvidia.com/developer/presentations/2005/GDC/OpenGL_Day/OpenGL_FrameBuffer_Object.pdf">http://download.nvidia.com/developer/presentations/2005/GDC/OpenGL_Day/OpenGL_FrameBuffer_Object.pdf</a></div>
<div><a href="http://www.nvnews.net/vbulletin/showthread.php?t=49157">http://www.nvnews.net/vbulletin/showthread.php?t=49157</a></div><div><a href="http://www.ati.com/developer/">http://www.ati.com/developer/</a></div><div>
<br></div><div>The class comment of OGLExtManager explains what needs to be done to add new OpenGL extensions. OGLExtManager does not support platform-specific extensions.</div><div><br></div><div>Some related code</div><div>
Here is some simple Croquet/Squeak code to generate a one-shot texture and display it. Based on example 1 in Clastic (see above). Does not require any special graphics support beyond running Squeak. This gives an idea of where we're headed, but this is not the method we will use. </div>
<div><br></div><div>Define the class AFirstTry as a subclass of Form, and then enter this method in AFirstTry: </div><div><br></div><div>plot1</div><div>"plot a function"</div><div>" (AFirstTry extent: 128@128 depth: 32) plot1 asMorph openInWorld "</div>
<div>| xmin xmax xincr ymin ymax yincr x y z scaled |</div><div>xmin := -1.</div><div>xmax := 1.</div><div>xincr := (xmax - xmin) / (self width).</div><div>ymin := -1.</div><div>ymax := 1.</div><div>yincr := (ymax - ymin) / (self height).</div>
<div>scaled := 0.1.</div><div><br></div><div>1 to: self width do: [ :xpixel |</div><div>x := (xmin + (xincr * xpixel)) / scaled.</div><div>1 to: self height do: [ :ypixel |</div><div>y := (ymin + (yincr * ypixel)) / scaled.</div>
<div><br></div><div>" the main mapping function "</div><div>z := ((x sin + y) sin + x) sin - (((y sin + x) sin + y) sin).</div><div>self colorAt: xpixel@ypixel put: (Color r: z g: 0 b: (1-z)).</div><div><br></div>
<div>].</div><div>].</div><div><br></div><div>"xfun1 (V2 x y) = sin(x+sin(y+sin x)) - sin(y+sin(x+sin y))</div><div>xfun2 (V2 x y) = floor(sin(x+sin(y+sin x))) - </div><div>floor(sin(y+sin(x+sin y)))</div><div>"</div>
<div><br></div><div>To execute the code, highlight the code in the comment and DoIt. </div><div><br></div><div>NVidia GPU Shader Capabilities</div><div>The table uses DirectX nomenclature (i.e., Vertex Shader and Pixel Shader version numbers). For GeForce FX and newer GPUs that are capable of flow control, the static instruction count represents the number of instructions in a program as it is compiled. The dynamic instruction count represents the number of instructions actually executed. In practice, the dynamic count can be vastly higher than the static count due to looping and subroutine calls.</div>
<div><br></div><div>GPU<span class="" style="white-space:pre"> </span>DX<span class="" style="white-space:pre"> </span>Vertex Shading Capability<span class="" style="white-space:pre"> </span>Pixel Shading Capability</div>
<div>GeForce FX<span class="" style="white-space:pre"> </span>2.0+<span class="" style="white-space:pre"> </span>256 static instructions<span class="" style="white-space:pre"> </span>512 static instructions</div><div>65,535 dynamic instructions<span class="" style="white-space:pre"> </span>512 dynamic instructions</div>
<div>GeForce 6 Series<span class="" style="white-space:pre"> </span>3.0<span class="" style="white-space:pre"> </span>512 static instructions<span class="" style="white-space:pre"> </span>2,048 static instructions</div><div>
65,535 dynamic instructions<span class="" style="white-space:pre"> </span>65,535 dynamic instructions</div><div><br></div><div>A personal note from daf:</div><div>Using the GPU to do the processing for procedural textures through vertex and pixel/fragment shaders is a current and exciting area of experimentation for many developers inside and outside of the Croquet community. While I am very interested in exploring this area, I am also a bit disappointed that I have to code the actual shader programs in a variant of C, instead of Smalltalk. Maybe someday this can be remedied by an industrious Croquet developer as an embedded language extension. Look at PyGPU, Vertigo or UW's Sh in the links above for ideas.</div>
</div><div><br></div></div></div>