A crucial requirement of Eulora's new client is to actively request from the server ANY data that it may find itself missing at any point in time. At first glance, this seemed to me simply a matter of providing request services1 from Eulora's new Ada-based core and then adjusting the existing C/CPP code of the legacy client to make use of those services. This rather optimistic idea is of course plain wrong: "adjusting the existing C/CPP code" in this context is similar to saying that one "adjusts" a sheep to use the library - while it can certainly be done for various definitions of "done" and the sheep may indeed use the library one way or another, it's at best a huge waste (of time, of resources, even possibly of steak) for everyone involved and no matter how one looks at it.
Even leaving aside for a moment the trouble with "adjusting" the legacy tangle in any direction, the more important issue here is that this requirement is not as much a functional requirement as a non-functional, quality of service requirement: whichever part of the client provides data services, it should better be dedicated to the task and do whatever it takes to get it done instead of "providing" it only fair-weather style - if it's easy, there you are and if it's not easy then it's your problem really. In other words and in marked contrast to the very democratic "best"-effort-you-can-do-anything-anytime existing C/CPP code2, the new code should have a clearly defined task and then either complete it or die trying over and over again, taking full responsibility for the process involved, not just for some specific detail conveniently chosen nor - as an excuse for not delivering - for the outcomes that are not fully under its control3.
Considering therefore "active and dedicated request" as a quality of data service on Eulora's client side, it follows that its place is rather close to the data cache mentioned previously and at any rate inside the new Client Core since it's certainly not some additional part of client logic nor some bit of user interface. However, I'm reluctant to make it the responsibility of the cache itself since the cache is a passive structure that focuses on *storing* data and *providing access* to it. Mixing passive data storage with active data acquisition doesn't make much sense to me and even seems ill-advised for Eulora's client given the competing requirements: on one hand passive, immediate-response local data storage and on the other hand active, possibly-delayed and world-facing (i.e. communicating with the server) data acquisition. So I'd rather avoid this passive-active construction and have instead the two as separate entities: a EuCache dedicated to storing and retrieving on demand *any* data; a Requester dedicated to acquiring *any* data that is demanded of it. Note that the definition of "acquiring" here has nothing to do with the means through which the Requester actually gets this data (specifically nothing to do with the exact messages sent/received/exchanged with the server). Acquiring some data means simply that the required piece of data becomes available in the local cache aka EuCache. So the Requester will keep requesting this data from the server through whatever means it knows until either the data arrives and becomes available from EuCache or otherwise the whole client kills itself for lack of server4 and therefore of any possibility of playing the game.
Specifically, the Requester will be implemented in EuCore (and therefore in Ada) as a protected object exposing only a few procedures that are essentially notifications: some are notifications of demands for some specific piece of data (either an Object or a File really since those are the only 2 overall types of game-data that one can request from the server); the others are notifications of data being received or of timeout interval having elapsed (in other words a notification of failure to receive data). Note that the demand for an "Object" effectively means a demand of its properties and those might be indeed anything at all but each and every one of them will be either directly a value or otherwise an ID of another Object or of a File. All notifications (including those demanding data) are always accepted by the Requester but not necessarily immediately acted upon. Clients of the Requester do NOT control the actual requests to the server or messages exchanged and are not even concerned with those at all - the production of actual requests, their content and their timing are entirely the job of the Requester and under its sole control. Implementation-wise, the Requester will simply keep queues of requested Objects/Files and will then proceed as soon as it can to pack a request for as many of them as possible; this request will then be posted to the server and the Requester will set a timer to the timeout value so that in the worst case it is at least notified that nothing happened; when/if any data is received or when this timer expires, the Requester will check in EuCache to see what items (if any) of those requested are now available; any items that have become available will be discarded from the watchlist of the Requester (i.e. the demands for them are considered completed) and a new request may be created and posted to the server for any items that are still in demand but not yet available. Note that even in the event of a timeout, a "repeated" request to the server may not be identical to the previous request since the list of demanded data possibly changed in the interval.
One potentially iffy point for the Requester is its need to be notified of any incoming data. At the moment I don't see any real way around this, short of making the Requester poll at set times the EuCache and checking if any data of interest has meanwhile arrived. I don't really like this polling approach here because it's rather wasteful without good reason: any incoming data is indeed received and processed by another unit that is also on the same level with the Requester, namely the Consumer (the part that processes messages from the inbound queue). So the Consumer will have to notify the Requester when new data is received. While several Consumers may be active at the same time (at least one for Serpent and one for RSA messages) this is not a problem at all since the Requester is anyway a protected object i.e. thread-safe. Note also that even if (some of) the consumers fail to notify the Requester of some incoming data, the whole thing will still work if only slower than it could: the timeout timer will wake up the Requester and the check of data will happen there at any rate. In other words, the Requester is capable of reacting to events if notified of them but not dependent on those notifications to do its job correctly.
Given its rather complex task, the Requester is currently on the top conceptual layer of EuCore, making use of quite a lot of other units from lower levels. Currently, the main relevant units on this top level are the following:
- Data Cache aka EuCache - this is a passive, thread-safe entity responsible for storing all and any data given to it and retrieve or delete it on demand. As such, it *owns* the specific format in which data is stored5 and it simply exposes functions and procedures for storing, retrieving, deleting and checking for data.
- Communication Link aka Comm_Link - this is a passive, thread-safe entity responsible for persistent storage and updating of communication details, most notably RSA and Serpent keys for inbound and outbound communications as well as a message counter. This is effectively a specialized cache - where EuCache is for game data, Comm_Link is for communication protocol data. The requirements (including use contexts) and specifics of the two seem to me sufficiently different to keep them separate at least for now.
- Consumers of incoming messages (RSA and Serpent) - those are separate, active tasks, responsible for processing incoming messages. Consumers own and get to define what "processing" means exactly but their role is to extract the data contained in the messages received and make it available for use by Eulora's client. In practice this means currently that Consumers will pass any data received on to EuCache for storage and will notify the Requester of any data receipt.
- Requester - this is an active, thread-safe entity responsible for acquiring data that is in demand. It owns the process of data acquisition from the server and it accepts any demands of data specified by some identifier. While it guarantees that all demands will be served at some point in time as long as the whole program is running, it does not (and can not) guarantee when or if the corresponding data becomes available. It can't even guarantee that there IS any corresponding data and so ALL it can do is to guarantee that it will try with the same dedication for each and every bit of data to acquire it. Anyone demanding data can of course pester the Requester with more demands or give up or decide for themselves for how long they can or will wait.
And now that the main idea and overall design of this whole thing is at least quite clear to me, there remains of course the small bit of actually implementing it in full (rather than the current skeleton I already made) and sorting out the various implementation-level troubles that will appear as they always do, in all sorts of details. Relatively easy work compared to what follows it inevitably, namely teaching that C/CPP sheep to use the EuCore library...
Mainly picking, packing and encrypting everything in the right format + sending it through to the correct place. ↩
Seriously, think of it: existing client code is this event-driven thing where *anyone* can subscribe to any event and then "do" anything at any time provided of course that the any and anything are in fact very narrowly defined and set in stone even to the level of names of various art files (it has to be a zoneinfo.xml inside this and that and put exactly there, sort of thing). If this narrowing of "do" was not a price high enough to pay for such code "liberty", there is also the added reality of a huge tangle that does precious little indeed since everyone ends up calling anything from anywhere and no piece of code is really and truly responsible for anything bigger than a few variables here and there. And at the end of the day how could any code even be responsible for anything since it can't *own* any process by itself (shared! event-driven!) and it has to be passive, mainly reacting to some events or at most... signalling through events of its own but never able to rely on anyone giving a fig about its signalling! So there it is, code - like anything you do - is more of a mirror than you might think. And "teaching people to code" has way more layers than "teach them Java" and certainly more issues than current "courses" even seem to be able to imagine. ↩
And now that it's clearly stated like this, tell me first just how many people you know who actually handle their own work like that? And just what bit of "programming language" you think one needs to teach so that you get programmers to actually design their code this way? ↩
This "lack of server" is defined in practice as a number of successive timeouts on requests sent to the server, where the specific threshold value is chosen by the user via a config file. ↩
Currently game objects are stored in memory in Hashmaps aka associative arrays while art/files are stored directly on disk. Note however that any of this can be changed without touching anything outside EuCache as it's nobody else's business. ↩
Comments feed: RSS 2.0
And after a rather long discussion it turned out that even the above was not responsible enough: there is to be no exposed "request this or that" option as requesting unknown stuff is a duty not an option. So the Requester will become instead more of a ChiefLibrarian and actively work on acquiring new knowledge for the EuCache. The rest of the code has only the option of using whatever there is in the EuCache at any given moment.
Word.
[...] that I have in mind, this will retrieve any information it needs about the sector from EuCore / EuCache (via a CPP-to-Ada layer, as needed) and then either retrieves the required sector from the engine [...]
[...] more reading, thinking, discussing and structuring than coding. The concrete result is a clear client strategy for knowledge acquisition and a very basic initial skeleton of its implementation. However, as the *structure* of the [...]