The goal of this most recent dive of mine into the CS-pit of how to do little in 1001 ways and a lot of lines of cpp&xml¹ was stated in #eulora:

diana_coman:hmm, not sure things are clear there somehow; let me see from the other end: do you suggest I dig in CS and figure out how to directly get from it what I want it to do, ditching all those shaders xmls and whatnots?

As a result of the above, I went first through another re-reading of the CS manual with pen and paper at hand - it's basically the map but certainly not quite the territory. At any rate, thus fortified hardened of sorts, I turned to the self-doubting CS code² itself to figure out some - any! - way of cutting this knot and getting to the core of the engine - aka the *useful* part, ffs! - so that I can finally DO the graphics properly and on the fly, including icons, textures, materials, shaders, effects, terrain, lights, meshes, animations, everything! For the impatient reader, here's the update coming out of this only yesterday:

diana_coman: mircea_popescu: looking at the "cut straight to doing it in CS" here seems to even come with 2 layers really: first there's the added cruft on CS-core itself; then there is the layers-upon-layers of cruft added by PS/client.
diana_coman: and there is always that compromise between "supposedly reusing existing code saves time" and the bitter experience of "yes, but reusing THIS code costs time".

A large amount of cruft in CS comes from the very unfortunate idea of chasing a theoretical notion of flexibility & reusability while ignoring the practical result of the wrong sort of complexity being baked in. To quote from the CS manual itself:

diana_coman: lemme cite from CS manual for the ages: "Crystal Space is a highly modular framework. As such it is more complex then many other game engines or renderers you might encounter. The advantage of this complexity if that it offers you a lot of power in what you do and how you use it."
diana_coman: or such was the dream there.
diana_coman: to add from only 2 paras down: "the components and libraries are *more or less* independent of each other" (emphasis is mine).

Indeed, CS has as a result of the unfortunate design decision above, a lot of components ("plugins") and libraries that are quite dependent on one another and in such a tangled way that one is even advised to NOT attempt to load a plugin directly since that will just fail if dependencies are not already loaded! And here it starts, because that requires then a dedicated loader that supposedly knows the dependencies but also works only with values from a config file so nope, not exactly on the fly anyway. Do you see yet how much and what sort of power you gained in return for all this complexity?

If the above is not quite enough, add to it the fact that the pretense at "independent of each other" comes at the very steep price of the horror called SCF (Shared Class Facility) - put simply, this is a way for code to use methods based on known interfaces, supposedly unaware of which/what plugin might happen to implement those methods in any given run. In practice though this simply means that the plugins get bogged down with a wad of code just to "register" their implementations, they are still nevertheless not exactly independent because how can such a thing be in a graphics engine where everything is about the same thing really and finally, the code using plugins has to wrap even the simplest of stuff in nightmarish calls to the SCF so that if you want to change one tiny bit, you still have to write at least 10 lines and remember several levels of layered abstractions that have nothing to do really with the task at hand.

Oh, and lest we forget, all this festival of abstracting things away from your grasp in the name of your own "power of use" further means that trying to find the code that actually gets executed (ie trying to figure out just WHERE in the darned thing shaders for instance are really implemented as such) is an exercise in frustration: for one thing it's quite often scattered about in several components (there are loaders - hence parsers and tokenizers and whatnots - of xml too in this, for added fun!), for another thing it ends up 25-40 layers deep at most time and for the cherry on the top, since at each point you get only the "abstractions", tough luck to actually piece it all back together again before you go insane. How, just HOW exactly can anyone hallucinate this to be "power of use"?³

Still, since I'm not going to rip away the whole SCF madness out of CS⁴ - or not just yet, I'm an optimist at heart - it follows that I'm stuck as usual with the task of finding some practical way forward anyway. A closer study of those shaders reveals that they are - like most in graphics... - just one way of doing some things. Specifically, CS shaders are meant to be (ignoring for a bit the added xml nonsense) parametrised pieces of code that describe a type of surface. The description of a surface - hence a "shader" - can get quite complex since one can literally change each vertex of the surface and do so repeatedly, for instance by applying/combining/mixing different textures and techniques over several "passes". Essentially "shader" is just a name for "programmatically but still laboriously⁵ painting pixels of a surface just-so, going around some of the limitations the simpler texture+material surface definition might have". So on one hand shaders are yet again just an alternative to a simpler option but on the other hand they are supposedly the more powerful alternative - the question being of course: is this the same sort of power as discussed just above, namely the power to push the boulder uphill rather than the power of building anything?

Leaving aside for a bit the shader-power-doubts, the dig in CS revealed that the terrain system anyway quite expects to make use of shaders going also through all the dances of shader variables and contexts⁶. So at this point it seems that there aren't all that many options really and at least *some* sort of shaders will have to be supported and therefore sanely extracted/plugged directly into CS and then - possibly - the terrain system either beaten into shape too or at the very least made to work with those. Such joy.

Before ripping into CS code directly though, there's the added bonus of the client code itself. The original PS code added its own wrong sort of complexity on top of that pushed through by using CS. In a similar ill-fated attempt at "power if only in theory", there's a whole system of widgets that are meant to be independent but cannot be so and moreover there are a lot of "managers" that supposedly are independent too but rely on all sorts and in general just about ~everything has two significant dependencies: the "netmanager" aka direct communication with the server⁷ and the fact that everything has been already loaded because the whole world is in fact fully known before you even get to push the Start button.

Given that both the above dependencies of PS code are torn to shreds for Eulora's client, it quite follows that the minimal cut here is along the lines of ditching those parts entirely - while this WILL mean changing a lot of code, I still think it makes for a smaller cut than attempting to work around it. Basically it's again that frontal assault vs outflanking option: the whole mess is so wide and spread that it's not worth going around, even if there will be quite a bloody job to cut through it.

Considering all the above, there is some work to be done further on both CS and client code. However, since the client is anyway the final goal and the most effective "tester" of anything of interest, I'll first take some steps on cutting out the useless but very burdensome tangles in there: going from the starting point (and keeping to a minimum of changes still but don't count on this to mean "code will not be touched), the client should simply focus on what is does, namely user interface and nothing more. This means that it will display to the user whatever information is *available and relevant*, it will accept input from the user (and make it available for further processing if it involves anything other than UI/display changes) and it will further expose ways for *everything* to be reloaded/changed on the fly, since this is the hard requirement anyway.

Once the client gets to the stage where I can use and test more directly (aka loading on the fly instead of going through 1001 useless other steps just to get to the one I want) a shader with it, I'll switch again to CS and cut through so that a shader can also be specified and loaded on the fly without any xml involved. As this all adds up to a huge ball of tangled unknowns, there's very little sanity to any overall estimate at this stage but there will be for sure updates on the progress, as usual. At any rate, the cuts will be strictly on an as-needed basis and done step by step so that there will be gradual improvement while maintaining at all times a running client overall.

It's worth noting also that the above approach of cutting through the tangle in the client too is the only one that I can currently see to move forwards. Keeping otherwise around all the dead weight just about got to the point where it would stall almost everything trying to push further so it needs to go and good riddance.

1. To be fair, PS is the champ here and by far! While CS may be doing little with many lines of cpp&xml, PS does way less and with 10 times more lines of the same cpp&xml. ↩

2. Literally, as I illustrated in #eulora: diana_coman: in CS questions-from-the-code: "ctxt->iview = other.ctxt->iview; // @@@ Is this right?" ↩

3. I get it, one can on occasion feel very powerful - Popeye muscle-style! - just for getting this to work at all. As in, it took a LOT of pushing this uphill for it to go smoothly down the other side! Is this the sort of power you are after? ↩

4. Mainly because that is ~same with re-writing the engine really, could just as well start from there if it gets to it. ↩

5. Computer graphics is this precise achievement of programmatic-but-still-laborious, it's a wonder. ↩

6. The same variables can be and indeed are overwritten in CS at various levels so that you can go properly insane; for instance, you might have set the texture as opaque in the material but then you failed to set it in whatever shader got pulled in somewhere and so fuck you, the whole thing will be totally shiny instead. Did I mention how much I like computer graphics? ↩

7. You may wonder why would every widget and every class need to directly communicate with the server anyway, sure; there are plenty of answers to such wonder and none of them any good. ↩

Category: Coding, Eulora

Comments feed: RSS 2.0