~Referencing, of course, the previous version of EuCore and even that very beginning of creating a useful structure to it all.~
As I've opened up a test environment for helpful people to give a go to the latest client, they are likely to benefit as well from an updated and more comprehensive overview of the full client code, as an initial map of sorts, at the very least. So this is what I'm aiming for with this article - to provide an up to date map of the current client and a place for questions and answers on it all, in the comments.
While the previous article focuses exclusively on the EuCore part, meanwhile I really had to deal as well with the GUI part, as there wasn't anybody else doing anything about it otherwise. So the GUI part got significantly trimmed, mostly overhauled and otherwise changed from inside out to provide the sort of flexible and reasonably structured functionality that I need as opposed to the assumptions-ladden and spaghetti-shaped sort of "service" that was previously available. As a result, out of all those half a million lines of code and countless files, there are now... 40308 lines of c/cpp code in total remaining. More than half of this (27584 lines according to a wc run currently) is in fact the only remaining sprawling mess that got now at least insulated in its own directory, namely the PAWS horror. In more practical terms though, there are 11 classes that are almost entirely re-written and make up the bulk working part of the GUI, 2 classes (one entirely new and one adapted) for providing the whole set of views, windows and overlays that Eulora2 needs and otherwise a bunch of the remaining "utility" classes that are quite often simply dragged in by various spidery includes of the PAWS variety. Here's the full remaining tree of the GUI (ie the C/CPP part of the client):
~/client20/src/
├── client
│ ├── err.cpp
│ ├── err.h
│ ├── euaction.cpp
│ ├── euaction.h
│ ├── euloader.cpp
│ ├── euloader.h
│ ├── eumovement.cpp
│ ├── eumovement.h
│ ├── globals.h
│ ├── meshattach.cpp
│ ├── meshattach.h
│ ├── paws
│ │ ├── gameviews.cpp
│ │ ├── gameviews.h
│ │ ├── pawsborder.cpp
│ │ ├── pawsborder.h
│ │ ├── pawsbutton.cpp
│ │ ├── pawsbutton.h
│ │ ├── pawscheckbox.cpp
│ │ ├── pawscheckbox.h
│ │ ├── pawscombo.cpp
│ │ ├── pawscombo.h
│ │ ├── pawscrollbar.cpp
│ │ ├── pawscrollbar.h
│ │ ├── pawsframedrawable.cpp
│ │ ├── pawsframedrawable.h
│ │ ├── pawsimagedrawable.cpp
│ │ ├── pawsimagedrawable.h
│ │ ├── pawslistbox.cpp
│ │ ├── pawslistbox.h
│ │ ├── pawsmainwidget.cpp
│ │ ├── pawsmainwidget.h
│ │ ├── pawsmanager.cpp
│ │ ├── pawsmanager.h
│ │ ├── pawsmenu.cpp
│ │ ├── pawsmenu.h
│ │ ├── pawsminimalslot.cpp
│ │ ├── pawsminimalslot.h
│ │ ├── pawsmouse.cpp
│ │ ├── pawsmouse.h
│ │ ├── pawsnumberpromptwindow.cpp
│ │ ├── pawsnumberpromptwindow.h
│ │ ├── pawsokbox.cpp
│ │ ├── pawsokbox.h
│ │ ├── pawsprefmanager.cpp
│ │ ├── pawsprefmanager.h
│ │ ├── pawsprogressbar.cpp
│ │ ├── pawsprogressbar.h
│ │ ├── pawspromptwindow.cpp
│ │ ├── pawspromptwindow.h
│ │ ├── pawsradio.cpp
│ │ ├── pawsradio.h
│ │ ├── pawsselector.cpp
│ │ ├── pawsselector.h
│ │ ├── pawsstringpromptwindow.cpp
│ │ ├── pawsstringpromptwindow.h
│ │ ├── pawstabwindow.cpp
│ │ ├── pawstabwindow.h
│ │ ├── pawstextbox.cpp
│ │ ├── pawstextbox.h
│ │ ├── pawstexturemanager.cpp
│ │ ├── pawstexturemanager.h
│ │ ├── pawstextwrap.cpp
│ │ ├── pawstitle.cpp
│ │ ├── pawstitle.h
│ │ ├── pawstree.cpp
│ │ ├── pawstree.h
│ │ ├── pawswidget.cpp
│ │ └── pawswidget.h
│ ├── pscamera.cpp
│ ├── pscamera.h
│ ├── pscelclient.cpp
│ ├── pscelclient.h
│ ├── pscharcontrol.cpp
│ ├── pscharcontrol.h
│ ├── psengine.cpp
│ ├── psengine.h
│ ├── psmainwidget.cpp
│ ├── psmainwidget.h
│ ├── smgdata.cpp
│ └── smgdata.h
├── common
│ ├── placeholder
│ └── util
│ ├── command.h
│ ├── consoleout.cpp
│ ├── consoleout.h
│ ├── fileutil.cpp
│ ├── fileutil.h
│ ├── heap.h
│ ├── log.cpp
│ ├── log.h
│ ├── psconst.h
│ ├── pserror.cpp
│ ├── pserror.h
│ ├── psstring.cpp
│ ├── psstring.h
│ ├── psutil.cpp
│ ├── psutil.h
│ ├── psxmlparser.cpp
│ ├── psxmlparser.h
│ ├── README
│ ├── singleton.h
│ ├── stringarray.h
│ ├── strutil.cpp
│ └── strutil.h
└── placeholder
4 directories, 104 files
The main method is in psengine.cpp1 and that starts and initializes everything needed, then enters the loop provided by the graphics engine and at the end cleans up whatever is left. The first initialised is the graphics engine, CrystalSpace and that comes with its own "object registry", configuration and initialisation dances that have been however trimmed to a minimum and corralled into a single method (and a single, shared configuration file, too!) that is hopefully reasonably easy to read. After that, the Ada part of the client, namely the EuCore library (see below for more on its role) is given the green light to start and provided all goes fine, the next steps are to simply initialise the GUI logic that is in psengine itself and then to enter the CrystalSpace loop that runs for as long as the game is running.
The GUI part has the following main components:
- psengine - the central part that provides access to everything else and otherwise registers with the graphics engine to receive regular callbacks on events of interest. Such events of interest include each frame drawing and all user input. The callbacks on each frame drawing act effectively as a sort of heartbeat of the client as a whole and thus coordinating the different parts of the GUI. User input is generally received and then passed on to the relevant action handler (see euAction below) or to further potential handlers where relevant.
- pscelclient - a class2 that is meant to keep track of and otherwise provide all logic and management related to game entities. This is one level above the graphics engine and acting as a sort of bridge between the data as obtained from EuCore and its translation into the graphical format that the user gets to see. So pscelclient uses what the graphics engine provides, asking it to create, move, change and delete any graphics in the game's world to match the information it gets from the data available from EuCore.
- pawsmanager - meant to keep track of and otherwise provide all logic and management related to "widgets" aka the various windows, views and overlays that make up the game's interface rather than its world as such. This relies especially on the various views and overlays that are specific to Eulora2 and are defined in gameviews.h/cpp. This has most of the original mess intact but mostly sidelined and otherwise the useful bits and parts wrote in. The trouble cleaning this up is that ~all the paws calls back home to this and quite often with hardcoded include and calls too so it really wasn't worth yet my time to clear it all up on top of everything else.
- pscamera - a class that provides and maintains the available ways to view the game's world as it is otherwise populated by pscelclient.
- controlsManager - a class holding all the mappings of keys and mouse buttons so that they work as set in the configuration file and otherwise allow the user to provide input and interact with the game.
- euaction - a hierarchy of classes that provide the required input, processing and output capabilities to enable the user to perform all relevant actions in the game. The central psengine initialises a full set of objects from this hierarchy and then simply passes on to the relevant one the request for execution with any parameters as provided by the user at that point.
- euloader - a static utility class providing ways to create on the fly any graphics entity of the required type and using the assets indicated simply by a path. This includes creation of very simple elements such as textures but also quite complex ones such as animated characters that require a rather long list of calls to fully make when starting with just a bunch of files. While the creation of such graphics is quite tedious, at least it's enough to have it packed this way once and then it serves endlessly, effectively providing useful defaults for all the "parameters" that aren't actually needed or even known at start. The main user of this class is pscelclient.
- smgdata - a static utility class that imports all the Ada methods made available by EuCore and wraps them up for transparent use by CPP code from anywhere else in the GUI. Basically all requests to EuCore are made through this class.
Despite the name of the file and of the class, there is very little indeed left in there from the original code. I suppose I could have changed it all to euengine or something more appropriate but given how it's one of those included from just about everywhere due to the original "design", I didn't really consider all the resulting noise worth the trouble. ↩
This is also mostly re-written with only very little indeed remaining of the original "gem" pain. It could still do with a haircut at least, I'd say. ↩
While the above GUI part focuses exclusively on the interaction with the user, it relies on the separate EuCore library to communicate with the server and to exchange and obtain all required data. The EuCore library handles everything in this respect, including creating a RSA-based identity and account, initiating and maintaining fully encrypted communication with the server, obtaining and updating all data related to the game's world as available at different times to the player, retrieving from the server through the same encrypted communication any missing files and managing existing local files including their use as temporary defaults when needed and where appropriate. The full EuCore weighs in at about 27000 lines including tests and the mpi part that has about 8000 lines of code by itself. Here's an updated full picture of all Ada package dependencies:
Compared to the previous version, there is a striking change at the top where EuInterface is now the "entry" package instead of EuCore. This reflects actually better the meaning of the two packages and by now the whole code matured enough so that the refactoring was quite obvious, too. Basically EuCore is the starting package for the whole library, while EuInterface literally provides an entry point for any C code that wants to use the library and this includes now the previously missing C-compatible wrappers for EuCore's Start/Stop methods themselves. Thus any Ada code that uses this library can entirely discard EuInterface and just start with EuCore, while any C code using this library goes entirely through EuInterface and doesn't need to see or know of anything else.
Further comparing with the previous version of EuCore, there are also a few new packages, most notably Mgm_Cache that comes now between Heartbeat and EuCache, as well as C_Buffer at the very bottom. The C_Buffer package is literally for interfacing with C code so that Ada code can take advantage of faster writing to files mostly. The Mgm_Cache package does what the name suggests, namely provides the higher level logic to oversee the data cache that is EuCache itself. In fact, the Heartbeat package serves as on overall coordination mechanism across the whole of EuCore and as such it effectively provides regular calls as required to various parts such as the communication link (Comm_Link), the data cache (EuCache) and the assets downloader (Torrents). As the communication link is both the most critical and the most demanding component of these, the Heartbeat really grew first of all as a manager of Comm_Link and only once that was fully working the rest could be developed as well. At the moment, the management of EuCache grew big enough to make sense to extract it in a separate package of its own but otherwise I don't see a lot of benefit in doing the same for Torrents for instance.
Note that all dependencies in the graph above remain hierarchical as they should always be. The automated layout of the graph tends to make it look as if there was some nonsense going on perhaps with that seemingly horizontal line going from OAEP to Keccak, but that's just an artefact of positioning. In reality, Keccak is on a lower level than OAEP, being simply a hashing utility, although one that is most useful indeed. At any rate, OAEP uses Keccak and there is no dependency going the wrong way, as Keccak itself uses only raw_io and raw_types.
Hopefully this gives enough of an overview to have some idea as to what is where and how the whole fits together. If you have though further questions on it, feel free to ask in the comments below and I'll gladly answer.
Finally, if you made it all the way here and would still rather see a less schematic picture as well, here's a fresh one of the client in action at a meeting of like-minded creatures of Eulora2, surprised as they were by all the screenshooting commotion:
Comments feed: RSS 2.0
By way of verifying, helping to digest that graph, and getting a first high-level glimpse of the Ada part as a whole, I wrote out the dependencies as gleaned from the .ads+adb files then sorted them into topological strata (bottom-up as the graph flows).
1:
C_Buffer () [base_layer]
Real_Time_App ()
2:
Raw_Types (C_Buffer) [base_layer]
3:
CRC32 (Raw_Types)
C_Conversions (Raw_Types) [base_layer]
Config (Raw_Types) [base_layer]
Eulora_Objects (Raw_Types)
Keccak (Raw_Types) [base_layer]
OAEP (Keccak Raw_Types) [base_layer]
Raw_IO (Raw_Types) [base_layer]
Serpent (Raw_Types) [base_layer]
UDP (Raw_Types)
4:
Keccak.Hashing (Keccak Raw_IO Raw_Types) [base_layer]
Log (Config Raw_Types)
RNG (Config Raw_Types) [base_layer]
Snd_Rcv (Raw_Types UDP)
5:
History (Raw_Types Config Log)
Keys_IO (C_Conversions Keccak Keccak.Hashing Config RNG Raw_Types) [base_layer]
Messages (RNG CRC32 Serpent Keccak Keccak.Hashing C_Conversions Log Raw_Types Eulora_Objects)
RSA_OAEP (OAEP RNG C_Conversions Raw_Types) [base_layer]
Torrents (Log Keccak Keccak.Hashing Config Raw_Types Eulora_Objects)
6:
EuCache (Log Torrents Config Raw_Types Eulora_Objects)
Packing (RSA_OAEP Serpent Log Raw_Types)
7:
Comm_Link (Log Config Packing RSA_OAEP Keys_IO Raw_Types Eulora_Objects Real_Time_App Snd_Rcv)
8:
Requests (Eulora_Objects EuCache Log Comm_Link Messages Raw_Types)
9:
EuInterface (History Log Messages Comm_Link EuCache Config Eulora_Objects Requests)
Mgm_Cache (Eulora_Objects EuCache Requests Log Real_Time_App)
10:
HeartBeat (Raw_Types Eulora_Objects Torrents Mgm_Cache Requests Messages Log RNG Comm_Link)
11:
EuCore (HeartBeat EuInterface Requests EuCache Messages RNG Torrents Log Comm_Link History Raw_Types Eulora_Objects)
The main difference I noticed was EuCore seemingly importing EuInterface, not the other way around, putting it on top as in the previous article.
This was a helpful clarification because I didn't guess from the name that it was EuCache management code, rather than I dunno, a cache of management data in its own right. Hey, paws at least has a lot of managers to keep track of, right?
Nice list you've extracted. For what it's worth, the graph itself is made from the imports, yes, it's just done with code, that's all. For the difference you found, it looks like you simply went with a before-last version of the client code. The refactoring to nicely have EuInterface importing EuCore rather than the other way around was literally done as the most recent change to the deployed client so if you looked at the first version you've got, it would still have the old import, indeed. Check in the latest client you are running and it should be that EuInterface imports EuCore.
Bwahaha, a cache of management data in its own right would be quite something. And hey, there were *way* more managers in the original CPP code, a whole pile of them and propping one another up while doing ~nothing. But ~everyone was a manager (I suppose they'd be ceo classes nowadays!). Recalling and in-linking from the published history, just a few instances: where it was quite obvious that the paws code simply couldn't do without managers, the clear case where they are not only in the way but also plain wrong, the time when I call them out and refuse to play along, the 3000 LOC reduction in one cut of managers and then an even more satisfying cut of 54000 LOC including managers, ofc and actually *fixing* previous problems, too through that very delete.
Meant to say also, there is quite a bit of information in the comments in the code (and they are current) especially for things related to the data hierarchy, which is quite crucial to understand how the whole model of "be able to figure out the world as it changes" as opposed to "fixed, fully known upfront world" works. It took a while to even get my head around how to do that in the first place, so I wouldn't be surprised if it's not all that easy to get coming at it fresh, either. As always though and on anything - feel free to ask questions to clarify it.
Re that first footnote: I'ma argue that claiming ownership of "the main method" is actually the better move, and all the way down, too --though I can't see why changing it is in any way urgent. It may be literally nothing more than a name (or I guess, a tangle of names), but symbolically it touches on a lot more. Out of curiosity/ignorance, why's changing the names more than a matter of some sed manipulation? Actually, I'll go try and figure out why myself, why not.
Meanwhile, if paws is widgetlandia, what's psmainwidget about?
Thanks for the nudge and you are right, indeed. At any rate, at some point the names have to be changed too, to reflect the reality and although it's true that leaving it for later does reduce the code touched, it's most likely quite reaching the point where there is very little further reduction so yes, I should make this little but visible change sooner rather than later.
The noise is more on the code versioning side but it's not a reason to never do that change, certainly - there will simply be one patch that is nothing but changing names and references, it's certainly not even the worst in the tree, at that. If anything, it makes more obvious (yet again...) the very real benefits that a higher-level approach on the code-versioning side or at least *visualising* patches side would bring. Not like there's any lack of worthy further developments waiting their turn to be pursued but still worth making at least an explicit note of it, certainly.
This is exactly what any sane person would naturally ask, isn't it? I even tried in the beginning to ditch that psmainwidget but so I found that what it does is to pretty much keep it all together so it refreshes as expected, it "finds" the "widget under the mouse" (to pass a click to), it creates and handles that fading message print if asked and generally remains reasonably working. Basically the widgets in paws are meant as building blocks, defining this and that sort of widget. The "psmain" is the outermost container or top-most "parent" of the whole GUI of the game and it also links with non-paws stuff that normally would rather be a higher level than itself, really (but well, ps's notion of calls is spaghettical, not hierarchical). Moreover, the way the whole paws-spaghetti is made and calls are forwarded, it didn't really work out to simply use those supposedly "independent" widgets as separate building blocks on their own. With some more cleaning, it's likely still possible but at least so far it didn't burn enough to make it to the front of the work queue in any sense.
Finally got around to do that rename refactoring, via trusty grep + sed indeed. As expected, the resulting patch does weigh in at 3k lines which is quite hefty to manually check and even just to read through. Other than that though, the change itself went through without any trouble.
On the bright side, the resulting pain-patch pushes perhaps in more plain sight what a great opportunity there is for a more discerning diff (and thus VaMP) to identify and filter such changes automatically. If anyone is looking for something interesting to do, this would be one thing and it's not even by far the only one waiting around for someone to pick it up.