12 Responses to “RFC: Eulora's Communication Protocol (EuComms)”

1. apeloyee says:

   April 25, 2018 at 8:55 p.m.

   There's no mention about what symmetric cipher "mode" is to be used (surely you aren't proposing encrypting each block independently...) nor of cryptographic authentication of packets (why permit monkeys to inject garbage?)
2. Diana Coman says:

   April 26, 2018 at 7:33 a.m.

   Thank you, apeloyee. The symmetric encryption with Serpent is discussed in some more detail in #trilema logs, here: http://btcbase.org/log/2017-11-22#1742298

   Regarding the second part: how do you see exactly injection of garbage? Note that on one hand the server has a set of keys for each client (and if it doesn't then the client has first to obtain them via the initial RSA-based handshake) so it's not as if anyone sending some serpent-encrypted thing is even taken into consideration. Furthermore, if a client goes to the trouble of initial registration and all that just to spew garbage afterwards...so what exactly? They pay for the traffic for one thing and banning is not impossible if required, for the other.

   So I'd say that monkeys are pushed to become humans through negative feedback: you can inject garbage but you pay for the resources you take with it and it gets discarded anyway.

   Or do you see some specific hole in the above?
3. apeloyee says:

   April 26, 2018 at 12:39 p.m.

   Sorry, was unclear in the first comment. "Monkey" refers to someone sitting between server and client (e.g. wifi spot operator), or even someone sniffing traffic (for some purposes).
   The log you linked don't appear to discuss the mode in particular; if it's encryption of independent blocks ("ECB mode"; I assume that since Serpent part contains, as it seems, no code to encrypt a whole packet, only a block), one has decent chances to splice a meaningful packet from blocks of other ones.

   >how do you see exactly injection of garbage?

   For example, by replaying old packets, with mangled payload part. And those will be attributed (and charged!) to the user, (or perharps you haven't written about that in detail yet, so I'm making wrong assumptions).
4. Mircea Popescu says:

   April 26, 2018 at 8:11 p.m.

   Ideally we intend to have packets = blocks.

   There are two mechanisms to prevent replay attacks. One is the arbitrary expiration of keys, which is discussed (and the fact that keys are not ever communicated in plaintext -- can you extract the serpent key from merely looking at encrypted material ?) ; the other is a nonce, which isn't specifically discussed besides a vague "4.3.3. Octet 7 to 10 – four octets ID of the item being sent".

   Do you expect there's viable attacks against in that situation ?
5. Diana Coman says:

   April 26, 2018 at 8:15 p.m.

   Adding to Mircea Popescu's answer, in case it's not clear from the post itself:

   Serpent is used for *all* the packet, not just the payload. The 128 bits limitation of Serpent means blocks of 128 bits, each block encrypted, yes. Hence preferably each packet 128 bits so that it's basically one packet=one block.

   RSA is used *only* at initial connection - this is not even meant to be repeated really. So new client sends RSA , receives RSA with set of Serpent keys and then uses those from then on (asking for more Serpent keys does not require RSA in itself).
6. apeloyee says:

   May 2, 2018 at 10:02 p.m.

   As long as 1 packet = 1 block, containing a long-enough nonce which is checked by the receiver, I don't see an attack; if blocks in a multi-block packet are independently encrypted, it depends on the details, but being potentialy broken depending on the format of payload, the client/server behavior when receiving invalid data (which isn't specified here) and on how much the attacker is able to guess (like with some TLS attacks).

   The blocks can be permuted or duplicated inside a single packet; normally tampering this way with chat would be noticed (it usually produces gibberish), but if the speaker's a bot saying standard/expected phrases, the bot can perharps be forced to say something "interesting".

   Since key rotation is optional, nonce-less blocks may still be copied from packet to packet.

   Perharps the action of the client can be replaced with an old one by sending a packet consisting of the first block of the new action packet (6-bytes header, 8-byte subject is likely to be the player character id; if the object id is big-endian, its first two bytes which fit into the first block may be zeroes also, hence payload can be the same) and the an old action packet with the first block dropped. (packets are encrypted, but the attacker can delay the legitimate packet and try sending [1st block of legitimate packet, 2nd..last blocks of old packet] for as many old packets as he expects to be action packets, and observe if the server replies).
7. Mircea Popescu says:

   May 3, 2018 at 6:11 p.m.

   Well, I suppose the only approach here is to wait for a prototype to toy with, rather than trying to shoot in the general darkness. Unless of course you want to be writing one ?
8. phf says:

   May 4, 2018 at 6:21 p.m.

   (i'm using 2018-04-29 hardcopy of the spec, which i unfortunately haven't saved a copy of, so i can't diff and find out if anything has changed since i printed it. it would be helpful to have, if and when the spec is being updated, a changelog at the bottom.)

   I did a read through on the spec, my commentary is very jumbled because I'm not understanding so many things and I'm making a lot of assumptions. I would appreciate if you could answer the questions below and address any misconception that I introduce with "presumably" and similar.

   Annex A uses section names to identify types of messages, but there's no corresponding enumeration of values that go into 4.3.2. How does section names map to two octets?

   Does server only ever send messages as a response to client requests, or there's an ongoing stream of status updates? I assume it's the later, since there's no mechanism for querying for certain message types (e.g. chat). You mention that the client has control over the noisiness of the server for billing purposes, but then there's no corresponding "ignore" mechanism in the spec (ignoring a particular in game character's messages). Could you also identify the message types that the server could send out of bound?

   E.g. does a change in the placement of an item around the player results in an A.5.5 message from server? Does a forced displacement of player results in an A.5.5 with player id? This is relevant to e.g. portal mechanism: since there's no description of stitching mechanisms for sectors, I assume each sector/map is self contained, and the travel between them happens through portals. Presumably it can be implemented by interaction with special items. Does a server then send an updated A.5.5 for character? If this is not correct how does transition between sectors work? Am i wrong to assume that sectors are self contained and they are actually stiched?

   On the other hand A.2.1 file information messages could be used to identify stale content. There's a sequences of messages that a client can send to check if e.g. the map for current sector has been updated, but would a server send A.2.1 implicitly if some file has been updated? (I assume not, but my two examples would make A.5.5 something that server would send implicitly, but A.2.1 not)

   There's a mention of additional file formats, I assume they are whatever crystal space/planeshift is using, but new clients will need them documented. Specifically a text based client needs a binary format description of the maps, without it doesn't seem like it's possible to implement spatial movement (unless the spec is designed to be self contained without the maps, and i'm just not seeing it).

   If I wanted to get the map of my current location, would this be a correct sequence of communication (=> connects to the next message type, and -> indicates an attribute from the message that's being used. I'd like to use more of these in the future as a shortcut way to talk about sample interactions.): A.5.6 request own char => A.5.5 -> id of sector => A.5.4 request sector map => A.5.3 -> map filename => A.2.4 request file content => A.2.2 -> file content

   Generally file transfer request for file content is driven either by sector map or graphics profile messages. Those use filename for file identitification, under what circumstances would a keccak hash requests be used by the client? How does file request work, when there's no corresponding file on server? As far as server reporting issues with the messages sent, there's only A.1.5 (A.5.31 being an ingame error). Is there a mechanism for getting all the files in general, of certain type, or relevant to the current location?

   What will be a mechanism for future spec expansion? The spec mentions videos, but there's no messages that carry video filename as a field. Relatedly, is there hidden knowledge as far as filenames are concerned that the standard client has, that's not documented here, or the entirety of available filenames can be discovered through protocol messages?

   A.2.1 file transfer has a file type field, but there's no corresponding enumeration.

   In A.4 and further, does id of object identify a specific object, or a class of objects? A.4.3 (movement) and A.5.7 (item in container) have a quantity parameter, but A.5.5 (item in sector) doesn't. Does that mean that e.g. "spoon" id=1 is an individual item when placed on the ground, but when dropped into inventory automatically bundles up?

   In A.4.2 what is subject? Is that the id of the character that's moving the item, or the current location of the item? I.e. John moves Spoon into chest" or "i move Spoon from self to Chest"? (the name subject seems to imply the first case, but then you can't distinguish between "pick up the spoon from ground" or "grab the spoon from a chest") Does "lock/unlock" action take door for subject, key for object? How does "combine" action map to subject/object (with two action slots, you presumably can only combine two items together)? "Exchange" action seems to imply a complex chain of messages, could you provide a sample message sequence as a demonstration of how exchange might operate?

   What's the enumaration of A.4.3 slot ids? Presumably it's something to do with character equipment, which is fixed slot ids, but also compound items where you can place specific thing into slot to make it operational, but how does a client discover slot id's available on a target?

   The only way to request an item seems to be A.5.8, where you provide "id of item". Is that the id of the container or sector name? Or the id of class of items? I'm not quite sure how to perform e.g. "list my inventory" "look what's inside chest" "see what items are around me within my visible range" actions from this. Does those actions happen implicitly by proximity or as a result of actions (E.g. by doing B.2 on a chest). what's the typical use case for A.5.8?

   How do i request a list of item characteristics? (There's e.g. A.5.19 for "skills" and "character", but I'm not seeing "request item's list of characteristic")

   A.4.5 what's the enumeration or otherwise the source of requested activity ids? Relatedly in A.4.4 there's mention of unix time, is there a message to get current unix time as known by server? (There's A.5.21 to get game time, perhaps a similar messge to get unix time)

   How do i move a character around? There's A.5.29 which lets me set an explicit coordinate for something, but I assume this is more for "moving items around". Would a player have a list of movement types, with basic things like "jump" and "10 steps forward"? I assume A.5.23 is a displacement vector, is rotation in radians? E.g. rotate left will be something like x=0, y=1.57, z=0. Is there a canonical enumeration of names of movement types for e.g. character?

   What's the logic to graphics profile nesting? Does further subdivision add more shapes or replace parent shapes (LoD)?
9. Mircea Popescu says:

   May 4, 2018 at 8:50 p.m.

   > How does section names map to two octets?

   Well, once the thing is set in stone there's going to be a serialization published.

   > I assume it’s the later

   Indeed.

   > (e.g. chat)

   There's been some discussion re using irc (Freenode specifically) to implement chat in the game. This is what we intend to do re chat.

   > Could you also identify the message types that the server could send out of bound?

   The client has no option to ignore server messages -- it'll get what it gets. The way to control flow is by not putting itself in the situation where the server sends unwanted messages, not by purporting to tell the server how to behave.

   > A.5.5 message from server?

   Yes.

   > Does a forced displacement of player results in an A.5.5 with player id?

   Yes.

   > I assume each sector/map is self contained, and the travel between them happens through portals.

   Exactly. This is currently live in the game, if you care to inspect it, there's a house with a ghost in it etc.

   > the map for current sector has been updated

   Maps don't get updated ; rather objects are added to them.

   > some file has been updated

   There's no expectation of files being updated ; the mechanism exists principally so client can download files ~it does not have~. It will be impractical for the client to cache all possible content, for one thing we're contemplating 2^128 maps.

   > but new clients will need them documented.

   There's no intention to force current planeshitness into future clients.

   > (unless the spec is designed to be self contained without the maps, and i’m just not seeing it)

   In principle the text client can simply issue movement commands, which the server accepts or rejects. There's no specific need to know the map to execute movement that I can see ?

   > : A.5.6 request own char => A.5.5 -> id of sector => A.5.4 request sector map => A.5.3 -> map filename => A.2.4 request file content => A.2.2 -> file content

   Not really. Upon accepted login the server would ~tell~ where you are. If client lacks any items the server references (such as object data, say, or maps w/e) then client asks for them and server sends them ; but if the client either has the item from earlier, or simply does not care (say it just wants to use its own local datafiles, as in the case of modding for instance) then no further communication is required.

   > under what circumstances would a keccak hash requests be used by the client

   Under all circumstances the client lacks a resource.

   > How does file request work, when there’s no corresponding file on server?

   It's not possible for the server to reference a file it does not have ; and it is not acceptable for the client to request files that were not referenced.

   > Is there a mechanism for getting all the files in general, of certain type, or relevant to the current location?

   No.

   > The spec mentions videos, but there’s no messages that carry video filename as a field.

   The field is really irrelevant in this sense, if the payload is a clip or a drawing or a blob of text it's still just a resource the client spits out.

   > is there hidden knowledge [...] or the entirety of available filenames can be discovered through protocol messages?

   The expectation is that all object data required by the client is required by the client because the server mentioned it ; and all items the server mentioned can be delivered on request.

   If you'll excuse the barbarism, here's a human made, hand schematic of conversation to perhaps better illustrate what's going on (but illustrate only, yes, this is not verbatim).

   1. Client says hello.
   2. Server says "welcome client, I see you don't need more keys ; you are on map 398798798 x 32823746 inside location 7 ; here's a list of objects within the view range you specified : o1, o2, o3, o4.
   3. Client asks what is object o3 ?
   4. Server says "o3 is a X Y Z with P Q R data".
   5. Client asks what is o3.Q ?
   6. Server delivers o3.Q which happens to be the item's icon, or mesh or associated noise or w/e.
   7. Client says "i'm doing action so and so"
   8. Server responds with a diff off 2 above, and so on forever.

   Does that clarify ?

   > Does that mean that e.g. “spoon” id=1 is an individual item when placed on the ground, but when dropped into inventory automatically bundles up?

   Not necessarily. Yes items in inventory can be bundled according to some rules ; but then again the mesh representing a spoon on the ground stands for a bundle of one or however many dropped spoons. I suspect a large portion of these questions stem from your never having spent an hour in the game. Is it completely unfeasible you compile for any of the known-to-work platforms and give it a try ?

   > In A.4.2 what is subject?

   Something does the moving and something is being moved.

   > you can’t distinguish between “pick up the spoon from ground” or “grab the spoon from a chest”)

   "ground" is a chest holding all chests.

   > Does “lock/unlock” action take door for subject, key for object?

   I expect it takes playerid for subject, door for object.

   > “Exchange” action seems to imply a complex chain of messages, could you provide a sample message sequence as a demonstration of how exchange might operate?

   This is a good idea. Can has Diana ?

   > What’s the enumaration of A.4.3 slot ids?

   Basically this permits splitting of stacks and such activities.

   > but how does a client discover slot id’s available on a target?

   As part of the properties description of the object, "o3 is a X Y Z" above.

   > Is that the id of the container or sector name?

   All items have unique ids irrespective of where they are found.

   To best understand the abstraction here : on one hand you have the containers. each individual map is a container. each object on the map can be a contaienr too (such as a craft table, or a chest) or not. each item will find itself in a container at all times. containers may be painted differently, one painted to look like a riverside, another painted to look like a human being, yet another to look like a tun. Nevertheless, as far as the item economy is concerned, they're containers.

   Items, identified by unique ids, always find themselves in a container. The container in which they are found gives them their physical location in the world -- to move your scepter of might from "the drawer inside the room inside the castle inside Your Lands" to "under the bed of the second-hottest demoness in the castle of baleful submission in the third circle of hell" one changes the parent id of item #your-scepter-of-might from #the-drawer-etc to #under-the-etc. Once that update is applied, any request by any party to any of the respective items (such as someone looking under the bed) will present the new situation.

   > I’m not quite sure how to perform e.g. “list my inventory”

   "list items where parent is me."

   > “look what’s inside chest”

   "list items where parent is o3.X"

   > “see what items are around me within my visible range”

   You get this anyway.

   > what’s the typical use case for A.5.8?

   The aforementioned "3. Client asks what is object o3 ?"

   > A.4.5 what’s the enumeration or otherwise the source of requested activity ids?

   This will need filling in.

   > is there a message to get current unix time as known by server? (There’s A.5.21 to get game time, perhaps a similar messge to get unix time)

   What do you mean similar ? They're the same thing.

   > Would a player have a list of movement types, with basic things like “jump” and “10 steps forward”?

   There's directional steps, that's pretty much it as the client works rigth now. It can [rather, it used to be] readily be hacked though to produce "flight" ie, announcing to the server "my position is now X Y Z".

   > Does further subdivision add more shapes or replace parent shapes (LoD)?

   Overlay, afaik.
10. Diana Coman says:

    May 5, 2018 at 3:56 p.m.

    @phf Mircea's answer is spot-on. Adding to it:

    1. "How does “combine” action map to subject/object (with two action slots, you presumably can only combine two items together)?"
    The subject is the one doing the action (i.e. your character in most cases). The object is the one on which the action is done (i.e. a container for instance). Think of the combine action as "push the combine button of this machine" and you'll get the idea: you combine as many items as there are in the machine when you press the button, no limitation due to this message's fields.

    2. "The only way to request an item seems to be A.5.8, where you provide “id of item”. Is that the id of the container or sector name? Or the id of class of items?" -> it's the id of one item. *Everything* from the game's point of view is "an item": a character, a container, a spoon, a house, a sector, whatever.

    "look what's inside chest" -> you send a "view" action (A.4.1 with action id corresponding to B.2) with subject = your char's id; object = chest's id
    "see items around me" -> you send a "view" action (same A.4.1 with B.2) with subject = your char's id; object = world's id (i.e. you are looking at the world, right?)
    The server DOES send the items around as soon as you can perceive them but nothing stops you from re-requesting them as many times as you wish using the above message.
    "list my inventory" -> basically, you need to look at yourself; so same view with both subject and object = your char's id
    Note that this does NOT mean that everybody looking at the same thing will necessarily see the same thing(s) - the protocol allows you to look and the server to tell you what you see as a result, nothing more.

    3. "How do i request a list of item characteristics?" -> apparently this didn't make it somehow into the post. I'll double check what happened there and add it if it's indeed missing.

    4. " “Exchange” action seems to imply a complex chain of messages, could you provide a sample message sequence as a demonstration of how exchange might operate? "

    Step 1: Client sends A.4.1 (basic action): subject = own char's id; object = other char's id (can be an NPC trader for instance of a PC); action =B.8 ("exchange")

    Step 2: Server replies with A.5.31 (error) for instance if that NPC/PC is dead or can't trade for some reason (the text in A.5.31 will prolly give a clue there);

    OR
    Server replies with A.4.4 (activity): id = new unique id; duration = some amount; start time = when server considers this started; status = 1 (ongoing) for instance;

    Step 3: Client replies then with A.4.1 (basic action) with subject = char's id; object = activity's id (i.e. the one from A.4.4 from server, above); action id = B.10 (abort)

    Step 4: Server replies with A.4.4 again with same id but status = -1 (i.e. failed/aborted)

    Alternatively, at Step 3, Client sends A.4.2 (item movement) with subject = char's id; object = a spoon item's id; quantity = 10; destination object = activity's id.
    In this case, the client is attempting to offer 10 of those spoons as part of that trade.

    If those spoons can be offered, the Server will reply with a message updating the location of the spoons item (or creating a new item if there were 12 spoons so only some are moved) so that their "container" is now the activity.

    All the while, either party to the exchange can send of course chat messages or any other sort really - the server will check the current situation, decide on outcome and communicate back.

    5. The only update on the protocol so far was on the 17th of April, touching only the hello message, as indicated in the post itself. If there are any other updates I'll make a log in the post itself I guess.

    Does the above cover all your questions or are there still unclear parts?
11. Diana Coman says:

    May 8, 2018 at 1:42 p.m.

    At 3 above (requesting the list of characteristics for an object): there isn't a specific data type for it because the list of characteristics are naturally sent as part of the server's response to a "view" action for that object. Basically the server responds to a "view" action on an object with its text description (5.9) and its list of characteristics (5.11).

    The "request" data types are essentially meant to cover the cases where there isn't a direct game action that effectively serves as a request.
12. Eulora's Communication Protocol, restated. on Trilema - A blog by Mircea Popescu. says:

    May 26, 2018 at 5:27 p.m.

    […] is a continuation/refinement of the original version proposed on Diana's blog a month […]