Having previously acquired more limbs than they knew what to do with, my Euloran hopefuls discovered this week the wonders of moving bits and bobs about but they are really so excited with it all that they tend to have a hard time keeping it all - as in all parts of own self - together! Given that this week's disentangling of the poorly documented ball of mismatched formats, silent fails, different conventions and unstated expectations started with exploding hopefuls to the 4 corners of the world and sending on occasion various limbs to orbit the moon too, the current careful-movement-while-keeping-it-all-reasonably-together is marked nevertheless as a glorious achievement and amazing feetfeat! And to document it all for future reference (and current unloading of its weight¹ ), here's how it all went, reading it again from this week's notes:

Everything You Thought You Knew about Skeletons Turns Out to Be Wrong

My earlier reservations regarding cal3d's "skeleton" format proved to be even underdeveloped and overly optimistic! Simply adding some weights to the meshes (hence, triggering the *use* of the skeleton file for something other than just ensuring-that-cal3d-doesn't-crash² ) resulted in my carefully constructed hopeful being literally blown to bits all over Eulora's blue skies, fair plains and even dubious undergrounds.

At first I just sighed and read again the scant documentation of the format but that didn't reveal anything new. Then I checked and re-checked and otherwise redone with pen and paper in hand all those calculations involved, with quaternions and transformations and assorted "relative" this or "local that". Out of all this I got practice with quaternions for sure but otherwise not a clue as to the problem, nor any help with solving it: the format says "rotation/translation to bring a vertex from model space into bone space" and so my file has, for each and every damned bone but the result is still a bunch of scattered hopes of any hopeful; the format says "relative translation/rotation to parent bone" and that's exactly what my script calculates and spits into that .xsf file very carefully with all attending tags and quotes and whatnots but the result is STILL a scattered mess that looks more like a hopeless than like a hopeful. Then I read the Cal3d code where the skeleton is used and being rather unfamiliar otherwise with the beast facing me this time, it all seemed to match the description: yes, it transformed indeed as stated, the vertices from model space to bone space and the bones relative to their parents so then why and how come and what in the name of all that's sane in the world (little, I know) makes it end up with such a mess? So I started experimenting with the simplest possible skeleton and positions to make absolutely sure that it wasn't something wrong in my calculations just throwing everything about - and indeed, it wasn't that the calculations were wrong but rather that Cal3D clearly had a different interpretation or use for them than what I expected.

Having quite exhausted above all the code-docs-and-empirical solutions at hand (or at least those not requiring more coding than worth doing just for testing), there wasn't anything left at this stage than to read more on the wider topic, since the docs of Cal3d held nothing else and I was still clearly missing some crucial bit of information. Not that there was any clue as to *what* to read or where to even start with it but then again, if there were clues, it wouldn't be such a problem so where would the mess even be in that anymore? So I read again some more on all those "stages" that computer graphics afficionados are so very fond of and yes, what do you know - the trouble turns out to be that idiotic compartmentalizing approach that results in each stage of the graphics pipeline having its *own* conventions and notions as to what is what, without explicitly stating them either, since it's everyone within their own happily recreated village-bubble in there and therefore "everyone knows what it's supposed to mean", of course.

I'll spare you the details of all that reading and go straight to the solution: for the problem at hand, the crucial missing bit was that the "skeleton" at animation time is *not* about "where and how to position meshes to make this creature" but rather about how to *deform* the *already-positioned meshes* when a part (or the whole) moves. And so those translations and rotations (both relative to parent AND local) are NOT at all meant to describe the bone's *position* or orientation, no! They are meant to describe instead *deformations* that are chained on top of those of the parent, and nothing else! So in this sense there is really no bone at all, not even an articulation, just a bunch of hierarchical deformations (and this explains just *why* indeed, any "bone" can affect ANY vertex in fact) fitted so that they are enough to describe the desired sequences of movements for all the relevant animations of that creature, that's it. The whole pretense of "bones" and "articulations" and "parent bone" whatever is just a way to refer to it so that it's "more intuitive", nevermind that this sort of half-baked analogy *also* triggers all sorts of unhelpful "intuitive" stupid and so it's worse than anything. Instead of being mentioned as a footnote explanation or sidelined and treated as the sort of crutch that it is, it's so entrenched as to not allow discussion in any other terms. And moreover, it also gets further "help" by means of all those blender-like tools that "support" then the trouble it's born out of and the trouble it creates down the line, of course³.

Having thus realised that my whole .xsf file was basically wrong, the next step was to figure out what then should the "right" file even be. And what do you know - since I failed to do "what everyone does", I also fail to have their problems! Because didn't you know it, the "rigging" aka "fitting a skeleton inside a model" is a complicated problem and a terrible headache. Well, so it is, but for once... not for me! If it weren't for Cal3d crashing without a skeleton file, I would not even *need* that file really and moreover one could directly define the fitting hierarchy of deformations for each animation since in the end it's quite conceivable this might work for the best. As it is though, I have to fill the skeleton file - form filling brought to code, aren't you happy to hear it? - pretty much with 0 everywhere since yes, when "not moving", there's ...no deformation required anywhere, what surprise⁴.

Sure, one could note here that a more useful approach would then be perhaps to *actually* use the skeleton to properly define the shape by deforming an overall blob in which the bones are embedded. While this sounds rather interesting to me, it also seems not exactly high priority right now nor all that quick to get to the point where it's useful so for now at least I'll stick to my existing meshes, skeletons and models, looking instead for the way to define working animations *for them*, as they are. At least step one is finally working as expected: make the .xsf file the space-waster that Cal3d expects with no rotations nor translations (basically simply the *hierarchy* of bones is all the information in there) and the hopeful is once again back to its expected shape, itching to move already.

How Do You Put on Weights?

The whole shtick of "skeletal animation" is that the above mentioned hierarchical set of deformations apply differently to *every vertex* of the darned model. Nevermind mesh and such high levels of looking at the stuff, here we are back again at vertex level and staring at the weirdest question: how do you put on weight again? I mean: feeding vertices in the hope that they figure out as a result the best weight to put on for themselves is unlikely to help, so a different approach has to be found. The first naive approach may very well be to consider simply that all the vertices of one mesh are stuck with the corresponding bone and therefore are simply influenced by that - even possibly to different degrees depending on the distance from the bone. This naivety is quickly shot down by the practicality of inflexible requirements: while each vertex may have any number of weights attached (aka: it can use ANY number of the deformations available anywhere), the sum of weights has to add up to 1 - under penalty from CS/Cal3d implementation which rules silently and undocumentedly but reliably in practice that failure to provide the full 1 means pulling on that vertex to the origin (presumably some additional translation into the origin, default values ftw). So a vertex can use only one deformation but use it fully or it can use a weighted combination of any specific set of deformations it picks. How to pick then?

One dubious advantage of the additional reading forced by the previous part of this adventure is that I got also all sorts of stuff noted down. Among such stuff, there were precisely some approaches to "skinning" as the whole dance of weighing vertices and other imponderables is called when you want to signal you are from that village as opposed to one of the other villages involved. And among those, I recalled a promising non-linear method that took into account the angles formed by a point on the surface of the mesh with the 2 ends of the bone in the mesh: supposedly this non-linear method is great at bending the mesh without creating known troubles such as collapsing joints and/or "candy wrapper" (when it gets twisted). Re-reading that paper with an idea to implement that bit revealed several mistakes (e.g. a formula has plus on some pages and minus on others though it's meant to be minus really, everywhere) but at least I had something to use so I went with it: for each vertex of a mesh, my script calculates the angles with the ends of the bone and then decides accordingly on which end has what weight for that specific vertex.

Even with the simplest approach that I could find, the whole thing was far from clear. The main lack of clarity is what to do when the "end" that supposedly influences the vertex has no bone starting from it since that means effectively no way to specify deformations for *that*. For now I took the cheap and easy way everywhere, since those are anyway both early days and things that can be done at any later point when there is a working client and all the rest: if there are children on that end, the weight gets split equally among all of them; if not, it gets piled on the other end since it can't be just "lost", what else can I do.

In Which Cal3D Silently Refuses to Load Animations (and the client misbehaves some more).

This unexpected cherry on top of everything else was quite the illustration of unhelpful error messages. Having finally gotten *some* way to define an animation and a way to feed the right skeleton and to weigh each vertex and so on, I wrote the script to spit out the whole thing into the .xml that was supposedly just what Cal3D wanted to read from. This part is always annoying in that the slightest oversight (a space in some places!) means silent "failed to read" the file and then good luck figuring out where or what or why is wrong. So yeah, Cal3D refuses to eat it and then I spend quite some time going with a fine comb through it all to find just what does it choke on - nothing was wrong, nothing to choke on, everything fine, welcome to a fresh round of making sense of nonsense.

For sanity's sake, I'll jump to the solution this time too - it turned out that despite its claims to happily eat either the xml or the binary formats for all its files, in fact Cal3D has a soft spot for binary animation files and will NOT eat their corresponding xml versions at all (while it does eat meshes and skeletons in xml or in binary, no problem). Why? What do animations have so special and moreover why can't it SAY so, plainly? Eh, such are the questions to which knowing the answers won't help you any. So forget about it and dig out instead that converter that at least happily works, get it into the full chain otherwise since it was meant to be used anyway in production⁵ and then watch the hopeful bobbing up and down, even doing a sort of (rather rigid) bowing of sorts and what not.

Before moving on to more interesting things, I'll still note for the record here that the client further misbehaves since it quite forces that "origin of model should be at its legs" - it's not just the rotation but the more important fact that the whole "on ground" and "falling" assumes that! So if the origin of the model is in its middle, as soon as you try to move it, it will... fall! I don't have the words in this language to fully describe this stupid but there it is and so once again, models are at least for now (since I don't quite want to spend the time to hunt this down throughout the whole client) again with the origin at the bottom. And this is still not enough since then ...they get some parts underground, sigh. This might possibly have to do with yet another sticky-tape-solution a la PS, the "leg length" invention but I haven't looked at it this week.

Keeping It All Together

Having finally sorted out a whole working animation meant that all there was left was the tricky bit: how to create animations cheaply, for all and any hopeful that may be blessed with moving ability? Cheaply turns out to be cheap really: just pick a duration, a number of frames and then some offsets for translations and rotations at each bone, calculate everything, write it down and be done with it. In the end this is anyway what happens really, if meanwhile I did do a tiny amount of tweaking, having started to notice what is "too much" and what sort of things movement may require. And so my whole set of scripts got a brand new step added, for generating animations - it even generates 2 of them for each model, one for "idle" and one for "walk" (worth noting here that one can easily *mix* any number of animations simply from cal3d/client otherwise). Nevertheless, the thing to be had *this cheaply* is indeed quite clunky and the trouble has at least 2 roots: one is indeed the weights attachment and the other is, arguably, the transformations itself.

From the previous reading around again, I even know that there are basically 2 main approaches to this, the so-called FK and IK, standing for direct kinematics and inverse kinematics, respectively. Essentially FK focuses on how to chain deformations so that the result down the line fits a movement at root and/or somewhere in the middle, while IK focuses on what should the deformations be in order to obtain in/at the end a plausible transition from an initial position to a final one. One can even conceivably go one step further back and change the "skeleton" for animation, since indeed, there's no need for it to be stuck to whatever was used to generate a model - arguably static pose is less demanding than movement and there's no reason why one couldn't add exactly as many deformations as one wants, after all. None of those has any very cheap solutions that I'm aware of so far though and I rather think the whole thing *can* wait really, since it's all a matter of what values to put in each of those files, but in the end it's still the same files and the same pipeline anyway.

Why Is Screen Recording So Difficult?

While I haven't really figured out a passable answer to that question, the question itself is a new acquisition for me - it came as I was trying to make a recording of a moving hopeful to illustrate all those words here and I found out that imagemagick apparently would do it only by taking screenshots and then stringing them together for horrible results, while other "recommended" linux "great options" either don't work on my setup, are not available or will do a shitty job of it anyway. At any rate, after the "easy" part of recording that took way, way more than I thought it can ever take, I finally got something to work and so have a look at 2 hopefuls from the Gen6 set, both in "idle" and in "walking" mode (basically the moves back/forth and the sharp/full turns are when I turn them with the controls, while the rest of "movement" comes from the animations - the walk animation for instance has a gentle, small angle rotation in it too, a bit like looking left and right of sorts). Should have added perhaps "pessima" to the title, looking now at those animations again but anyways, here are 2 recordings (.ogv format) of the first moving hopefuls to grace Euloran soil:

recording hopeful 1
recording hopeful 2

1. One of those days - and perhaps before I die, too - I'll figure out some way to cleanup messes-that-I-didn't-even-make-myself without all the bitterness that seeps through at times for it being such a repeated and seemingly neverending experience. So far I figured out only how to laugh at it (and that took a lot of years to learn anyway) - but I do start getting ideas as to what a solution to this might look like, indeed.  ↩

2. Yes, absent skeleton file or having an invalid skeleton file causes cal3d to crash taking with it the whole client too. The issue is "documented" as in there are comments throughout CS code about it. What, did you expect anything more than that? ↩

3. Note that this is not just one tangent I could easily go into from here. It's a whole bunch of spiky tangents that I'd much rather use as tenderizers already. ↩

4. I suppose one could perhaps attempt to introduce here some deformations to smooth for instance those strangulating connections between bones - the trouble though is that all those will feed in further down throughout any moves too, basically the effects cascade and it's unclear at the moment how simple or complex it would be to getting something better from it. And moreover, since those deformations are performed every time the model is rendered, any additions here carry a performance penalty. Not so sure why would incurring this penalty make more sense than just adjusting the vertices to the desired rest position from the start, presuming that the transformations are known/some method to calculate them is found. ↩

5. The binary files that the converter spits from the xml are half the size or even smaller. The reason why xml has been used throughout this time is that the binary format is not documented and so the only reasonably cheap way to know what should go where is to use & write the xml files, then run the converter if/when desired, then load them into the client/cal3d. ↩

Category: Coding, Computer Graphics

Comments feed: RSS 2.0