Ossasepia

December 6, 2018

SMG Comms Chapter 11: Arbitrary Size of Public Exponent

Filed under: Coding, EuCrypt, SMG_Comms — Diana Coman @ 10:59 a.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

The change from fixed-size to arbitrary size of the public exponent "e" of RSA keys1 turned out to be of course more than just introducing a parameter for e's size - all I can say about it is that I was rather ready for this development by now, given the known mess that is the MPI lib. So the only surprise here was the exact way in which MPI fails rather than the fact that it does fail. Let's take it slowly and in detail since it is - like everything else related to encryption - rather important to have it as clear as one can make it.

On the SMG Comms side, a shorter "e" is really no trouble at all: basically there is no such thing as "shorter" since it can perfectly be the same size as it always was, only starting with whatever number of 0s it needs to make up to the expected length, big deal. And this is in fact already handled and handled well in the wrappers I wrote previously for the RSA and MPI C code, since it was already clear that yes, any set of octets might start at any time with 0s but that doesn't make them fewer octets or anything of the sort. So at first look, there really isn't any need to change anything, since the "change" required is neatly and natively handled as what it is - just a specific case of the more general operation that is implemented. Still, for clarity and further use downstream, I decided to add a constant and a new subtype to Raw_Types simply for the purpose of providing an explicit way of using the exactly-8-octets-long e (neither the new constant nor the new subtype are put to use so far by any of the message pack/unpack or read/write methods):

    -- RSA public exponent (e) size in octets
    -- NB: this should normally match the E_LENGTH_OCTETS in smg_rsa.h
    -- NOT imported here for the same reason given at RSA_KEY_OCTETS above
  E_LENGTH_OCTETS : constant Positive := 8;

  subtype RSA_e is Octets( 1 .. E_LENGTH_OCTETS);

On the RSA side, the same constant "length of e in octets" goes into include/smg_rsa.h since it is not exactly a knob of the whole thing but rather a parameter for RSA key generation:

/**
 * This is the length of the public exponent e, given in octets.
 * TMSR standard e has KEY_LENGTH_OCTETS / 2 octets.
 * Eulora's communication protocol uses however e with 8 octets length.
 * New keypairs generated will have e precisely this length.
 * Change this to your preferred size of e for generating new keys with that size of e.
 * NB: this impacts key generation ONLY! (i.e. NOT encrypt/decrypt).
 */
static const int E_LENGTH_OCTETS = 8;

As the comments above stress, the "length of e" should normally be a concern in the code only when generating a new key pair; at all other times (encrypt/decrypt), the e that is provided will be used, whatever length it might be. Looking at the key generation code, the change to make is minimal since the code is sane - simply replace a local variable that specified the length of the required prime with the new global constant that now specifies the user's choice of length for e (in rsa/rsa.c, function gen_keypair):

	/* choose random prime e, public exponent, with 3 < e < phi */
	/* because e is prime, gcd(e, phi) is always 1 so no need to check it */
	do {
		gen_random_prime( E_LENGTH_OCTETS, sk->e);
	} while ( (mpi_cmp_ui(sk->e, 3) < 0) || (mpi_cmp(sk->e, phi) > 0));

Following the changes above, a re-read of all my rsa and smg_comms code confirmed that no, there is nothing else to change - it is after all just a matter of exposing a constant to the user, not any change of the underlying algorithm, so that's surely all, right? Well, no, of course it's not, because at the lower level, all those 0-led smaller "e" go into the MPI lib of gnarly entrails. And as it turns out, a quick test whipped out to see the whole thing in action got...stuck, going on for ever somewhere in the MPI code. Where? Well, the stack trace goes 8 levels deep into the MPI code and it looks (at times, as it rather depends on where one stops the neverending run...) like this:

#0  0x000000000040b492 in mpihelp_addmul_1 ()
#1  0x0000000000407cd4 in mpih_sqr_n_basecase ()
#2  0x0000000000407e68 in mpih_sqr_n ()
#3  0x0000000000408098 in mpih_sqr_n ()
#4  0x0000000000407d6b in mpih_sqr_n ()
#5  0x0000000000408098 in mpih_sqr_n ()
#6  0x0000000000407d6b in mpih_sqr_n ()
#7  0x0000000000405693 in mpi_powm ()

Claiming that one fully knows what goes on in 8 piled levels of MPI calls is rather disingenous at best so I won't even go there. However, a closer look at all that code, starting with mpi_powm and following the code seems to suggest that the issue at hand is that MPI simply can't handle correctly 0-led numbers. To which one should add of course "in some cases" so that one can't just say fine, wtf is it doing permitting any 0-led numbers then?? No, that would be too easy so the reality is that it permits and it probably even *requires* in places 0-led numbers but *in other places* it gets stuck2 on them. Aren't you happy to have followed this mess so far to such amazing conclusion? At any rate, going through the MPI code yields some more fun of course, such as this fragment in mpi-pow.c:

    /* Normalize MOD (i.e. make its most significant bit set) as required by
     * mpn_divrem.  This will make the intermediate values in the calculation
     * slightly larger, but the correct result is obtained after a final
     * reduction using the original MOD value.  */
    mp = mp_marker = mpi_alloc_limb_space(msize, msec);
    count_leading_zeros( mod_shift_cnt, mod->d[msize-1] );
    if( mod_shift_cnt )
  mpihelp_lshift( mp, mod->d, msize, mod_shift_cnt );
    else
  MPN_COPY( mp, mod->d, msize );

The obvious clue there is that at least one culprit of "can't handle 0-led numbers" is that divrem function that is indeed called as part of the exponentiation. But the added joke that's for insiders only is that the normalization there is done ad-hoc although there exists a function precisely for...normalizing aka trimming the leading 0s from an mpi! Eh, so what if it exists - by the time something gets as big and tangled as MPI, chances are nobody remembers everything there is but that's not a problem at all, right? But wait, just ~10 lines further down, there is another normalization and at *that* place, the author somehow remembered that there is even a macro defined for this purpose! And oh, another ~10 lines further, there is yet another way in which normalization is done on the spot (shifting the bits directly!). So what is it already that makes one write code like this, some misplaced purple-prose inclination, let's not repeat the same expression or what exactly? Frankly the only logical answer is that it's done on purpose - anything and everything to increase the number of lines of code. Increase productivity3 !!

Moving further, it turns out that this very same function actually *does* trim the leading 0s off the exponent at some point! Which of course begs now the question of just how and why is then a problem to give it a 0-led exponent? Essentially it trims it but too late/not fully/not for everything and not everywhere that it should do it, that's the best I can say about it. And overall, the fact of the matter is simply that MPI just doesn't correctly handle 0-led MPI values4, end of story. To quote from MPI code and comments themselves, the author's explanation:

/****************
 * Sometimes we have MSL (most significant limbs) which are 0;
 * this is for some reasons not good, so this function removes them.
 */

So it is "for some reasons not good", mmkay? It reminds me of the other display of great expertise in "reasons". Without wasting even more time on the MPI code of wonders5, the solution for SMG Comms is essentially a work around: the C wrappers get another job, namely to ensure that the values passed on to MPI are normalized. Note that the symmetrical opposite of this, namely adding missing leading 0s is already implemmented where needed (in the Ada code that actually deals perfectly fine with 0-led values since they are not oh-so-special, really). Thankfully, this is a very simple thing to do: instead of using directly the mpi_set_buffer method to set the value of an mpi number, define an mpi_set_normalized method that calls mpi_set_buffer + mpi_normalize:

void mpi_set_normalized(MPI m, const char *buffer,
                        unsigned int noctets, int sign) {
  mpi_set_buffer( m, buffer, noctets, sign );
  mpi_normalize( m );
}

Using the above code, all the mpi_set_buffer calls in c_wrappers are replaced by mpi_set_normalized calls and so there are no more 0-led mpi values passed on to the C code when calling rsa from Ada (since this is the purpose of those c_wrappers: to provide a sane interface for Ada to work with the insanity of C for RSA needs). Obviously, if you insist on calling the C rsa encrypt/decrypt methods directly, it's up to you to make sure you don't pass them 0-led values. While I could change the encrypt/decrypt methods themselves to normalize all the keys' components before doing anything, I think that's a very ugly and ultimately incorrect thing to do: the encrypt/decrypt should use precisely what they are given, not go about tampering with the inputs, regardless of "reasons". Yes, it is ugly and incorrect that MPI forces this normalization nonsense but that's not a justification for messing the encrypt/decrypt functions to cover up for it.

Note also that I specifically chose NOT to include the normalization in the existing method mpi_set_buffer because on one hand it's not the job of mpi_set_buffer to trim its inputs and on the other hand there is a need for mpi_set_buffer precisely as it is: there is code in there relying on being able to set the buffer of an mpi to anything, including 0-led vectors (even if at times, that doesn't remain 0-led for long). So no, modifying mpi_set_buffer is not a good option, even without considering the fact that MPI is better thrown away6 than changed.

The rest of the .vpatch for this chapter of SMG Comms contains simply the 2 additional tests (and changes needed for them in the test environment) that I wrote: one for the RSA c code, to flag the issue and one for the Ada code to ensure that there is at least one test with an exponent of 8 octets. I've used first the rsa code with the length of e set to 8 to generate a pair of RSA keys that are used then for the new test. So there is now a new file, "8_keys.txt" containing this new set of keys and the Ada test is simply another call with different parameters to read its input from this file as opposed to another.

Given that the arbitrary size of e touches essentially EuCrypt code, I also packed those minimal changes to smg_rsa.h and to key generation, together with the new test using a shorter e, into a .vpatch for EuCrypt. I've also added in there stern warnings at the encrypt/decrypt regarding the 0-led issue since it is the responsibility of the caller to either make sure they don't provide 0-led values or otherwise deal with a potentially-blocking call. Both .vpatch files and their corresponding signatures are on my Reference Code Shelf as well as linked here for your convenience:


  1. In practice this is more like 8-octets size for Eulora and otherwise 256 octets for TMSR rather than all sorts of sizes. Nevertheless, the big change is from one single size to ~anything more than one size since that means that the size can't be relied on to be anything specific at any given time. 

  2. Or as good as stuck: I did not have the patience to wait on it for more than 3 minutes, given that the operation in question should have barely taken a couple of seconds at maximum. 

  3. And probably surpass the 5-years plan too, since it's hardly distinguishable already. 

  4. Or, apparently, only values with "too many" 0 at the front aka at least one limb. 

  5. Yes, I did actually dig even deeper into the MPI code following this because I don't really have a choice not to as long as MPI remains part of SMG Comms and EuCrypt. The conclusion however is still the same, so I'm jumping here to the conclusion and you are free to tell me in detail in the comments section the results of your investigation on this. 

  6. Hopefully there will soon be a replacement for it and so it WILL be thrown away, I can't wait for that! 

November 30, 2018

SMG Comms Chapter 10: Actions and RSA Keys

Filed under: Coding, SMG_Comms — Diana Coman @ 1:57 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

Eulora's communication protocol uses RSA keys only for new players who don't yet have a set of Serpent keys agreed on for communication with the server. The main reason for not using RSA for all client-server communications is simply that RSA is essentially too expensive for that. As it happens, it turns out that republican RSA with its fixed-size 256 octets (2048 bits) public exponent is anyway too expensive even for this reduced role - communicating all those octets to the server inside a RSA package takes quite a lot of space. As a result, Eulora will use a smaller e, on only 8 octets (64 bits) that fit neatly into the message structure for requesting a new account in the game (5.1 RSA key set). This means of course that I'll also have to patch EuCrypt to allow arbitrary size of the public exponent in order to have a way to actually generate such RSA key pairs but this will have to be the next step and another post on its own. For now, at the level of read/write from/to SMG Comms messages, there's no direct concern with the crypto lib itself: the e will simply be 8 octets long at its specified place in the message and that is that.

Since the RSA Key Set message includes also some client information (protocol version and subversion, client hash, preferred padding), I've first defined a new data structure (in data_structs.ads) to hold all this in one place:

  type Player_RSA is
    record
      -- communication protocol Version number
      Proto_V    : Interfaces.Unsigned_8;

      -- communication protocol Subversion number
      Proto_Subv : Interfaces.Unsigned_16;

      -- Keccak hash of client binary
      Client_Hash: Raw_Types.Octets_8;

      -- public exponent (e) of RSA key (64 bits precisely)
      -- nb: this is protocol-specific e, aka shorter than TMSR e...
      e          : Raw_Types.Octets_8;

      -- public modulus (n) of RSA key (490 bits precisely)
      n          : Raw_Types.RSA_len;

      -- preferred padding; magic value 0x13370000 means random padding
      Padding    : Raw_Types.Octets_8;

    end record;

The choice to have the new structure shown above comes mainly from the fact that all the information in there is on one hand related (as it belongs to and describes one specific player at any given time) and on the other hand of no direct concern to this part of code. In other words, this part of the code reads and writes that information together but it has no idea regarding its use (nor should it have). It's for this same reason also that I preferred to keep e and n simply as members like any others of the Player_RSA record rather than having them stored already inside a RSA_pkey structure. For one thing there's no need for the read/write part to even know about the RSA_pkey structure (which is defined in rsa_oaep.ads where it belongs). And for another thing, having e and n as members of the record just like any others keeps the code both clear and easy to change in principle at a later time. Basically the read/write do as little as they can get away with - there is even no attempt to interpret e for instance as a number although its reduced size makes that possible here. Note that the protocol version and subversion are however interpreted as integers but in their case there's no point to keep them as raw octets. On the other hand, the choice of padding is kept as raw octets precisely because this is how it will be needed and used anyway.

Choosing the correct place for storing the padding option also gave me a bit to think about because it's not fully clear to me at this stage exactly where the padding belongs. Strictly speaking, padding is entirely the job of this level so there shouldn't normally be any leaking outside/upwards of anything to do with it. However, having the ability to choose types of padding means that the protocol itself effectively pushes this particular aspect upwards since it's the user ultimately who makes this choice. As a result, I decided to keep the mechanics of padding local (i.e. actual padding of messages + the magic value for requesting random padding + the interpretation of a padding parameter) while providing this Padding value in the Player_RSA record and otherwise refactoring all the Write procedures to require a Padding parameter indicating the desired choice of padding for that write. Moreover, to have this padding stuff in one single place, I also extracted the writing of counter+padding into its own procedure and then refactored all the Write procedures to call this one (since ALL messages always have at the end precisely a counter + padding). The main benefit to this is that it reduces the chances of making an error in one of the multiple places where otherwise one has to write the counter and then check the requested padding and then pad (if needed) accordingly. Other than this benefit, there isn't necessarily a big reduction in number of code lines nor really much an increase in clarity of the code since there is another procedure call to follow in there. Nevertheless, the alternative is worse: having copy-pasted same stuff in every write procedure and having to change all of it if anything changes. So here's the new Write_End procedure which is private to the Messages package since this is just a helper for all the other Write procedures:

  -- Writes Counter and padding (rng or otherwise) into Msg starting from Pos.
  procedure Write_End( Msg     : in out Raw_Types.Octets;
                       Pos     : in out Natural;
                       Counter : in Interfaces.Unsigned_16;
                       Padding : in Raw_Types.Octets_8) is
  begin
    -- check that there is space for Counter at the very least
    if Pos > Msg'Last - 1 then
      raise Invalid_Msg;
    end if;

    -- write counter
    Write_U16( Msg, Pos, Counter );

    -- pad to the end of the message
    if Pos <= Msg'Last then
      if Padding = RNG_PAD then
        RNG.Get_Octets( Msg( Pos..Msg'Last ) );
      else
        -- repeat the Padding value itself
        for I in Pos..Msg'Last loop
          Msg(I) := Padding( Padding'First + (I - Pos) mod Padding'Length );
        end loop;
      end if;
      -- either rng or fixed, update Pos though
      Pos := Msg'Last + 1;
    end if;
  end Write_End;

After the above changes, the read/write procedures for RSA key set from/to RSA messages are quite straightforward to write:

  procedure Write_RKeys_RMsg( K       : in Player_RSA;
                              Counter : in Interfaces.Unsigned_16;
                              Pad     : in Raw_Types.Octets_8;
                              Msg     : out Raw_Types.RSA_Msg) is
    Pos : Natural := Msg'First + 1;
  begin
    -- write correct message type
    Msg( Msg'First ) := RKeys_R_Type;

    -- write protocol version and subversion
    Msg( Pos ) := K.Proto_V;
    Pos := Pos + 1;
    Write_U16( Msg, Pos, K.Proto_Subv );

    -- write keccak hash of client binary
    Msg( Pos..Pos + K.Client_Hash'Length-1 ) := K.Client_Hash;
    Pos := Pos + K.Client_Hash'Length;

    -- write e of RSA key
    Msg( Pos..Pos + K.e'Length - 1 ) := K.e;
    Pos := Pos + K.e'Length;

    -- write n of RSA key
    Msg( Pos..Pos + K.n'Length - 1 ) := K.n;
    Pos := Pos + K.n'Length;

    -- write preferred padding
    Msg( Pos..Pos + K.Padding'Length - 1 ) := K.Padding;
    Pos := Pos + K.Padding'Length;

    -- write counter + padding
    Write_End( Msg, Pos, Counter, Pad );

  end Write_RKeys_RMsg;

  -- Reads a RSA Keyset (Player_RSA structures) from the given RSA Message.
  -- Opposite of Write_RKeys_RMsg above
  procedure Read_RKeys_RMsg( Msg      : in Raw_Types.RSA_Msg;
                             Counter  : out Interfaces.Unsigned_16;
                             K        : out Player_RSA) is
    Pos : Natural := Msg'First + 1;
  begin
    -- check type id and raise exception if incorrect
    if Msg(Msg'First) /= RKeys_R_Type then
      raise Invalid_Msg;
    end if;

    -- read protocol version and subversion
    K.Proto_V := Msg( Pos );
    Pos := Pos + 1;
    Read_U16( Msg, Pos, K.Proto_Subv );

    -- read Keccak hash of client binary
    K.Client_Hash := Msg( Pos..Pos+K.Client_Hash'Length - 1 );
    Pos := Pos + K.Client_Hash'Length;

    -- read e
    K.e := Msg( Pos .. Pos + K.e'Length - 1 );
    Pos := Pos + K.e'Length;

    -- read n
    K.n := Msg( Pos .. Pos + K.n'Length - 1 );
    Pos := Pos + K.n'Length;

    -- read choice of padding
    K.Padding := Msg( Pos .. Pos+K.Padding'Length - 1 );
    Pos := Pos + K.Padding'Length;

    -- read message counter
    Read_U16( Msg, Pos, Counter );

    -- the rest is message padding, so ignore it

  end Read_RKeys_RMsg;

As usual, I also wrote the tests for all the new procedures, including the private Write_End. However, the testing package as it was could not directly call this private procedure from Messages. My solution to this is to change the declaration of the testing package so that it is effectively derived from Messages - at the end of the day it makes sense that the tester simply needs to get to all the private bits and pieces. This change makes however for a lot of noise in the .vpatch but that's how it is. The new test procedure for the counter+padding is - quite as usual - longer than the code it tests1 :

  procedure Test_Padding is
    Msg     : Raw_Types.Serpent_Msg := (others => 12);
    Old     : Raw_Types.Serpent_Msg := Msg;
    Pos     : Natural := 16;
    NewPos  : Natural := Pos;
    Counter : Interfaces.Unsigned_16;
    U16     : Interfaces.Unsigned_16;
    O2      : Raw_Types.Octets_2;
    Pad     : Raw_Types.Octets_8;
    Pass    : Boolean;
  begin
    -- get random counter
    RNG.Get_Octets( O2 );
    Counter := Raw_Types.Cast( O2 );

    -- test with random padding
    Pad := RNG_PAD;
    Write_End( Msg, NewPos, Counter, Pad );
    -- check NewPos and counter
    Pass := True;
    if NewPos /= Msg'Last + 1 then
      Put_Line("FAIL: incorrect Pos value after Write_End with rng.");
      Pass := False;
    end if;
    Read_U16(Msg, Pos, U16);
    if U16 /= Counter then
      Put_Line("FAIL: incorrect Counter by Write_End with rng.");
      Pass := False;
    end if;
    -- check that the padding is at least different...
    if Msg(Pos..Msg'Last) = Old(Pos..Old'Last) or
       Msg(Pos..Pos+Pad'Length-1) = Pad then
      Put_Line("FAIL: no padding written by Write_End with rng.");
      Pass := False;
    end if;
    if Pass then
      Put_Line("PASS: Write_End with rng.");
    end if;

    -- prepare for the next test
    Pass   := True;
    Pos    := Pos - 2;
    NewPos := Pos;
    Msg    := Old;

    -- get random padding
    RNG.Get_Octets( Pad );

    -- write with fixed padding and check
    Write_End( Msg, NewPos, Counter, Pad );
    Pass := True;

    if NewPos = Msg'Last + 1 then
      -- check counter + padding
      Read_U16( Msg, Pos, U16 );
      if U16 /= Counter then
        Put_Line("FAIL: Counter was not written by Write_End.");
        Pass := False;
      end if;
      for I in Pos..Msg'Last loop
        if Msg( I ) /= Pad( Pad'First + (I - Pos) mod Pad'Length ) then
          Put_Line("FAIL: Msg(" & Natural'Image(I) & ")=" &
                    Unsigned_8'Image(Msg(I)) & " /= Pad(" &
                    Natural'Image(Pad'First+(I-Pos) mod Pad'Length) &
                    ") which is " &
                    Unsigned_8'Image(Pad(Pad'First+(I-Pos) mod Pad'Length)));
          Pass := False;
        end if;
      end loop;
    else
      Put_Line("FAIL: Pos is wrong after call to Write_End.");
      Pass := False;
    end if;
    if Pass then
      Put_Line("PASS: test for Write_End with fixed padding.");
    end if;
  end Test_Padding;

With the above read/write of a RSA key set, all the RSA messages specified in the protocol are provided. Of the Serpent messages, those not implemented are the Client Action, World Bulletin, Object Request and Object Info. All of those still require some details to be filled in but for the moment I went ahead and implemented read/write for Client Action based on a text representation of the action itself (i.e. precisely as specified in the protocol for 4.5 although the action can be/is in principle a fully specified structure by itself as described in section 7 of the specification). At this stage I'm not yet sure whether to provide another layer of read/write for that action text or whether to attempt to read/write directly the Action structures. So this will have to wait and as details are becoming clearer, the code will get changed /added to, no big deal. Anyway, the Write_Action and Read_Action for now:

  -- writes the action (octets+length) into the specified Serpent message
  procedure Write_Action( A       : in Raw_Types.Text_Octets;
                          Counter : in Interfaces.Unsigned_16;
                          Pad     : in Raw_Types.Octets_8;
                          Msg     : out Raw_Types.Serpent_Msg) is
    Pos    : Natural := Msg'First + 1;
    MaxPos : Natural := Msg'Last - 1; --2 octets reserved for counter at end
    U16    : Interfaces.Unsigned_16;
  begin
    -- check whether given action FITS into a Serpent message
    if Pos + 2 + A.Len > MaxPos then
      raise Invalid_Msg;
    end if;

    -- write correct type ID
    Msg( Msg'First ) := Client_Action_S_Type;

    -- write action's TOTAL length
    U16 := Interfaces.Unsigned_16(A.Len + 2);
    Write_U16( Msg, Pos, U16 );

    -- write the action itself
    Msg( Pos..Pos+A.Len-1 ) := A.Content;
    Pos := Pos + A.Len;

    -- write counter + padding
    Write_End( Msg, Pos, Counter, Pad );

  end Write_Action;

  -- reads a client action as octets+length from the given Serpent message
  procedure Read_Action( Msg      : in Raw_Types.Serpent_Msg;
                         Counter  : out Interfaces.Unsigned_16;
                         A        : out Raw_Types.Text_Octets) is
    Pos : Natural := Msg'First + 1;
    U16 : Interfaces.Unsigned_16;
  begin
    -- read and check message type ID
    if Msg( Msg'First ) /= Client_Action_S_Type then
      raise Invalid_Msg;
    end if;

    -- read size of action (content+ 2 octets the size itself)
    Read_U16( Msg, Pos, U16 );

    -- check size
    if U16 < 3 or Pos + Natural(U16) - 2 > Msg'Last - 1 then
      raise Invalid_Msg;
    else
      U16 := U16 - 2;  --size of content only
    end if;

    -- create action, read it from message + assign to output variable
    declare
      Act : Raw_Types.Text_Octets( Raw_Types.Text_Len( U16 ) );
    begin
      Act.Content := Msg( Pos..Pos+Act.Len-1 );
      Pos := Pos + Act.Len;
      A := Act;
    end;

    -- read counter
    Read_U16( Msg, Pos, Counter );

  end Read_Action;

As previously with the components of a RSA key, I chose to keep the "action" as raw octets rather than "text" aka String. This can be easily changed later if needed but for now I fail to see any concrete benefit in doing the conversion to and from String. The new Text_Octets type is defined in Raw_Types and I moved there the definition of Text_Len (previously in Messages) as well since it's a better place for it2:

  -- length of a text field (i.e. 16 bits, strictly > 0)
  subtype Text_Len is Positive range 1..2**16-1;

  -- "text" type has a 2-byte header with total length
  -- Len here is length of actual content ONLY (i.e. it needs + 2 for total)
  type Text_Octets( Len: Text_Len := 1 ) is
    record
      -- actual octets making up the "text"
      Content: Octets( 1..Len ) := (others => 0);
    end record;

There is of course new testing code for the read/write action procedures as well:

  procedure Serialize_Action is
    O2 : Raw_Types.Octets_2;
    U16: Interfaces.Unsigned_16;
    Len: Raw_Types.Text_Len;
    Counter: Interfaces.Unsigned_16;
  begin
    Put_Line("Generating a random action for testing.");
    -- generate random counter
    RNG.Get_Octets( O2 );
    Counter := Raw_Types.Cast( O2 );

    -- generate action length
    RNG.Get_Octets( O2 );
    U16 := Raw_Types.Cast( O2 );
    if U16 < 1 then
      U16 := 1;
    else
      if U16 + 5 > Raw_Types.Serpent_Msg'Length then
        U16 := Raw_Types.Serpent_Msg'Length - 5;
      end if;
    end if;
    Len := Raw_Types.Text_Len( U16 );

    declare
      A: Raw_Types.Text_Octets( Len );
      B: Raw_Types.Text_Octets;
      Msg: Raw_Types.Serpent_Msg;
      ReadC : Interfaces.Unsigned_16;
    begin
      RNG.Get_Octets( A.Content );
      begin
        Write_Action( A, Counter, RNG_PAD, Msg );
        Read_Action( Msg, ReadC, B );
        if B /= A then
          Put_Line("FAIL: read/write of Action.");
        else
          Put_Line("PASS: read/write of Action.");
        end if;
      exception
        when Invalid_Msg =>
          if Len + 5 > Raw_Types.Serpent_Msg'Length then
            Put_Line("PASS: exception correctly raised for Action too long");
          else
            Put_Line("FAIL: exception INCORRECTLY raised at action r/w!");
          end if;
      end;
    end;
  end Serialize_Action;

The (rather lengthy) .vpatch for all the above and my signature for it can be found on my Reference Code Shelf as usual or through those links:

The next step now is to patch the rsa/oaep part of SMG Comms to use the 8-octets public exponent and then to get back to EuCrypt and patch it to allow arbitrary size public exponent - so much for fixed size. In other words, it's a very good opportunity to re-read and review EuCrypt!


  1. It is also true that I spend waaaay less time on the tests than on the main code. In writing code like in any other writing, the result of less time spent on it is... longer writing rather than shorter, who'd have thought it, right? I mean who other than Samuel Clemens and pretty much everyone else who actually thought at all. 

  2. This is even more refactoring and therefore noise in the .vpatch, yes! It does say work in progress on the whole thing and in every post on this, right at the top. What did you think that meant? 

November 24, 2018

Proposed Change to W_Borrow (FFA)

Filed under: Coding — Diana Coman @ 8:32 p.m.

After more than half a year since last time I really looked at it, FFA (Finite Field Arithmetic) finally made its way back up on my list of tasks. Given the rather large break I took on this and the regrind of the original vpatches1, I've carved time out to re-start from the very beginning, as if I had never seen it before. So far, I went in detail through Chapter 1 and Chapter 2 and I am satisfied that I know them to the extent that I could re-write them (I actually did, even though by bits and pieces as I went). So I've updated the files on my Reference Code Shelf with the new .vpatches (using Keccak checksums) and my signatures for them. I'll link them here as well, for easy reference:

The break I took from FFA turns out to have been for the better in at least one way - it is actually easier for me to read the code now, mainly because of all the stuff I've been doing during this "break", of course2. Anyway, with Ada itself a bit more in the background rather than foreground for me, I had more time to actually explore those sort of things that popped out to me this time like the last time under the heading "I've checked it and it's correct so I can sign it but it still seems to be perhaps a bit less clear than it could be." Specifically, I'm talking of the expression introduced in Chapter 1 in the W_Borrow function for obtaining the borrow bit based on the two operands and the borrow bit from a previous operation (see word_ops.adb):

   -- Find the Borrow, from a subtraction where it is known that A - B == D:
   function W_Borrow(A : in Word; B : in Word; D : in Word)
                    return WBool is
   begin
      return WBool(Shift_Right( ( (not A) and B) or ( ( (not A) or B) and D),
                               Bitness - 1) );
   end W_Borrow;
   pragma Inline_Always(W_Borrow);

For comparison, have a look at the W_Carry function in the same word_ops.adb:

   -- Find the Carry, from an addition where it is known that A + B == S:
   function W_Carry(A : in Word; B : in Word; S : in Word)
                   return WBool is
   begin
      return WBool(Shift_Right( (A and B) or ( (A or B) and (not S) ),
                               Bitness - 1) );
   end W_Carry;

I don't know about you, but I can actually follow the boolean expression in W_Carry while at the same time I find that the W_Borrow thing pokes me in the eye repeatedly and I have a hell of a time to picture exactly wtf it says. Making the truth table for it and following the thing revealed that the result is indeed the intended one so I'm satisfied that the expression is correct. However, I still think that there is actually a simpler way to write the same thing, in a manner that even looks similar to the one in W_Carry, namely:

   -- Find the Borrow, from a subtraction where it is known that A - B == D:
   function W_Borrow(A : in Word; B : in Word; D : in Word)
                    return WBool is
   begin
      return WBool(Shift_Right( (B and D) or ( (B or D) and (not A) ),
                               Bitness - 1) );
   end W_Borrow;

To make sure that the new expression does indeed the same thing as the original one, I did both the truth tables and the actual transformation from one to another via Boolean algebra. I'll leave the truth tables as exercise for the interested reader but here's my working for transforming the original expression into mine:

  ( (not A) and B) or ( ( (not A) or B) and D) =

= ( (not A) and B) or ( (not A) and D) or (B and D)

= ( (not A) and (B or D) ) or (B and D)

= (B and D) or ( (not A) and (B or D) )

Given the above, I'm quite satisfied that the two expressions are equivalent and as a result I guess it is a matter of preference whether one chooses my version or Stanislav's. Unless there is some other reason that escapes me for using the original expression there, I find mine easier to understand and so preferable. Note that mine makes sense simply read as it is: there will be a borrow bit if either there are already 2 bits to take away from A (i.e. B and D) or otherwise if there is only one but A is 0. At any rate, since the W_Borrow function is present in the latest Chapter of FFA (12B) precisely as it was introduced in Chapter 1, I made a .vpatch on top of Chapter 12B with my proposed change to W_Borrow:

Note that I've added the corresponding line in the MANIFEST file but I followed the format described by Trinque in the V Manifest specification i.e. including the author name while Stanislav seems to have either forgotten that bit or preferred to not include it.

As a result of Stanislav's blog missing any sort of working comments currently, I wasn't able to simply add all this as a comment to his own Chapter 1 post, where I think it actually belongs. So I'm publishing it here as a post instead and I guess I'll host any further discussion on it too so feel free to leave a comment below.


  1. The regrinds are due to changing the checksums from SHA to Keccak; the content stays the same. However, one needs to read the new vpatches again before one can sign them in any meaningful way. 

  2. All the additional experience I gained with Ada shows when reading code and the difference is apparently large enough to be quite obvious - even hard to ignore basically. 

November 22, 2018

SMG Comms Chapter 9: Files (Transfers and Requests)

Filed under: Coding, SMG_Comms — Diana Coman @ 9:42 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

One significant goal of Eulora's communication protocol is to allow the client to request and obtain from the server absolutely *any* file that it might be missing so that there is effectively no need for an "update" of the client anymore. Two of the protocol's current messages target precisely this goal: File Transfer and File Request. The File Transfer message carries chunks of a file while the File Request carries a request from the client for files specified by their names (separated with ';'). And that means of course that one needs to do at least some sort of parsing to extract file names from that list while also handling at any time a set of unknown size containing strings of unknown length. In short: a mess.

My initial stab at the previously-mentioned mess was a rather ugly parametrized record (number of filenames) containing an array of ...parametrized records (each filename having its own length as a parameter). As you might imagine, this is not exactly the sort of thing that "fits in head" - at least not without a bigger hammer. And unsurprisingly, working with such a structure ended up in errors and trouble quite quickly (not to mention it felt about as pleasant as stabbing one's toe at every step). So I threw that first attempt to the bin and decided to store instead the filenames as one single long string (i.e. glued together into one string), alongside an array containing the starting positions of each separate filename in this long string. Basically instead of having separators in the string itself and mixed with the content, there is the content in one place and then neatly to the side the way to access directly any desired filename from the set. This keeps the protocol formatting (separators and the like) here in smg.comms where it belongs rather than pushing it higher up and at the same time it reduces the number of parameters to precisely 2: the number of filenames in there (hence the length of the array of start positions) and the total length of the resulting string (i.e. all filenames lengths added up together). The result is certainly an improvement over the first attempt but I can't say I am terribly fond of it as it is so if you have a better solution to this, go ahead and describe it in the comments below, including why and how it really is the better option - I'll gladly read it.

Using this approach of single string + start positions solved the issue of multiple variable lengths for strings. However, the File Request message still has all sorts of potential troublesome cases, including the case when the given set of filenames ends up longer than one message can carry (at write time) or there are multiple consecutive separators (at read time). My overall approach for such troubles with messages is to check whenever possible and effectively reject a message as invalid if a check fails. Anyway, working with an unknown bunch of unknown strings still remains at all times a sort of ugly spot, no matter what, so those read/write methods still look to me hairier than others so far and there isn't at the moment something specific that I can see to drastically improve them (short of fixing the size of filenames perhaps but I think that's more likely to just push the issue somewhere else as filenames will get padded/trimmed to size). Anyway, the new data structures for this chapter are defined in data_structs.ads:

  -- length of a text field (i.e. 16 bits, strictly > 0)
  subtype Text_Len is Positive range 1..2**16-1;

  -- A set of file names glued into a single string
  -- NB: there IS at least ONE filename and one character
  -- Upper limit for Text_Len is due to protocol's spec of text basic type
  type U16_Array is array (Text_Len range <> ) of Interfaces.Unsigned_16;
  type Filenames( F_No: Text_Len := 1; Sz: Text_Len := 1 ) is
    record
      -- filenames glued together into 1 single string
      S : String( 1 .. Sz ) := (others => '0');
      -- indices in S, at which each filename starts
      Starts: U16_Array( 1 .. F_No ) := (others => 1);
    end record;  

  -- A chunk of a file (for file transfer)
  type File_Chunk( Len     : Text_Len := 1;
                   Count   : Interfaces.Unsigned_16 := 1;
                   Name_Len: Text_len := 1) is
    record
      Filename: String(1..Name_Len);
      Content : Raw_Types.Octets(1..Len);
    end record;

In addition to the data structures above, this chapter adds the following:

  • Read/Write from/to Serpent Message for the File_Chunk structure (i.e. File Transfer message).
  • Read/Write from/to Serpent Message for the Filenames structure (i.e. File Request message).
  • Conversion methods from String to Octets and back. Those are private methods in the Messages packages and are meant for internal use only since messages use raw octets, while filenames and the like are meant as text/strings.
  • Read/Write from/to Octets for a 16 bits unsigned value. Similar to the conversion methods above, these are private methods in the Messages package and meant for internal use only. The reason for them to exist in the first place is that 16 bits values are relatively frequent in the protocol (counters and sizes) and their read/write requires an additional step to address potential endianness issues. Since this step is always the same and otherwise easy to forget + hard to debug if/when forgotten, it makes much more sense to have it packed together in one single procedure that can be called wherever needed.
  • Refactoring to replace all read/writes of 16-bit values by the new Read_U16/Write_U16 methods.
  • Tests for read/write of File Request and File Transfer messages.
  • Tests for converters from/to string/octets.
  • Small change to the test of pack/unpack RSA messages to ensure that "mangled" message is indeed always different from a valid package1.

The new read/write methods for File Transfer and File Request, described in messages.ads:

  ----------------- File Transfer ----------------------

  -- Writes the given File Chunk to a File Transfer type of message
  -- Chunk - chunk of file to write; contains counter, filename, content
  procedure Write_File_Transfer( Chunk   : in File_Chunk;
                                 Msg     : out Raw_Types.Serpent_Msg);

  -- The opposite of Write_File_Transfer method above.
  -- Chunk will contain the counter, filename and content
  procedure Read_File_Transfer( Msg     : in Raw_Types.Serpent_Msg;
                                Chunk   : out File_Chunk);

  ----------------- File Request  ----------------------
  -- Writes a message to request the files specified through their names
  -- Written parameter will hold the number of filenames actually written
  -- NB: this can be less than the number of filenames provided!
  -- When Written < FR.F_No, the FIRST Written filenames were written; the rest
  -- did not fit into the message and it's up to caller to decide what to do.
  procedure Write_File_Request( FR      : in Filenames;
                                Counter : in Interfaces.Unsigned_16;
                                Msg     : out Raw_Types.Serpent_Msg;
                                Written : out Natural);

  -- Reads a request for files; the opposite of Write_File_Request above
  -- Raises Invalid_Msg exception if the provided message fails checks.
  procedure Read_File_Request( Msg      : in Raw_Types.Serpent_Msg;
                               Counter  : out Interfaces.Unsigned_16;
                               FR       : out Filenames);

The implementation of those methods in messages.adb:

  ------ File Transfer ------
  procedure Write_File_Transfer( Chunk   : in File_Chunk;
                                 Msg     : out Raw_Types.Serpent_Msg) is
    Pos: Integer := Msg'First;
    U16: Interfaces.Unsigned_16;
  begin
    -- write type ID
    Msg(Pos) := File_Transfer_S_Type;
    Pos := Pos + 1;

    -- write filename as text field (size+2, text)
    -- check against overflows
    if Chunk.Name_Len > Text_Len'Last - 2 or
       Pos + Integer(Chunk.Name_Len) + 2 > Msg'Last then
      raise Invalid_Msg;
    end if;

    -- write total size: filename size + 2
    U16 := Interfaces.Unsigned_16( Chunk.Name_Len + 2 );
    Write_U16( Msg, Pos, U16 );

    -- write filename
    String_To_Octets( Chunk.Filename,
                      Msg(Pos..Pos+Integer(Chunk.Name_Len)-1) );
    Pos := Pos + Integer(Chunk.Name_Len);

    --write content
    -- check against overflow, including the 2 octets for counter at the end
    if Chunk.Len > Text_Len'Last - 2 or
       Pos + Integer(Chunk.Len) + 4 > Msg'Last then
      raise Invalid_Msg;
    end if;

    -- write total size for this text field
    U16 := Interfaces.Unsigned_16( Chunk.Len + 2 );
    Write_U16( Msg, Pos, U16 );

    -- write actual content
    Msg(Pos..Pos+Chunk.Content'Length-1) := Chunk.Content;
    Pos := Pos + Chunk.Content'Length;

    -- write counter
    Write_U16( Msg, Pos, Chunk.Count );

    -- write padding if needed
    if Pos <= Msg'Last then
      RNG.Get_Octets( Msg(Pos..Msg'Last) );
    end if;

  end Write_File_Transfer;

  -- The opposite of Write_File_Transfer method above.
  -- Counter will contain the message counter
  -- Chunk will contain the chunk counter, filename and content
  procedure Read_File_Transfer( Msg     : in Raw_Types.Serpent_Msg;
                                Chunk   : out File_Chunk) is
    Pos: Integer := Msg'First;
    U16: Interfaces.Unsigned_16;
    S_Name, E_Name: Integer; --start/end for filename in Msg
    S_Len: Text_Len; -- length of filename (needed as Text_Len anyway)
    S_Content, E_Content: Integer; --start/end for content in Msg
    Content_Len: text_Len; -- length of content (needed as Text_Len anyway)
  begin
    -- read and check type ID
    if Msg(Pos) /= File_Transfer_S_Type then
      raise Invalid_Msg;
    end if;
    Pos := Pos + 1;

    -- read filename size
    Read_U16( Msg, Pos, U16 );

    -- check for overflow and underflow; filename size >= 1
    if Pos + Integer(U16) - 2 > Msg'Last or
       U16 < 3 then
      raise Invalid_Msg;
    end if;
    U16 := U16 - 2;
    S_Len := Text_Len(U16);

    -- set start + end for reading filename later, when ready
    S_Name := Pos;
    E_Name := Pos + Integer(U16)-1;
    Pos := Pos + S_Len;

    -- read size of content
    Read_U16( Msg, Pos, U16 );
    -- check for overflow and underflow; content >=1; counter =2 octets
    if Pos + Integer(U16) - 1 > Msg'Last or
       U16 < 3 then
      raise Invalid_msg;
    end if;
    U16 := U16 - 2;
    Content_Len := Text_Len(U16);
    -- set start and end for reading content later, when ready
    S_Content := Pos;
    E_Content := Pos + Integer(U16) - 1;
    Pos := Pos + Content_Len;

    -- read counter
    Read_U16( Msg, Pos, U16 );
    -- check chunking validity i.e. if counter>0 then no padding
    if U16 /= 0 and Pos /= Msg'Last then
      raise Invalid_Msg;
    end if;

    -- create File_Chunk structure and fill it with data from Msg
    declare
      FC : File_Chunk( Len      => Content_Len,
                       Count    => U16,
                       Name_Len => S_Len);
    begin
      -- read from Msg
      FC.Content  := Msg( S_Content..E_Content );
      Octets_To_String( Msg( S_Name..E_Name ), FC.Filename);
      -- copy to output var
      Chunk := FC;
    end;

  end Read_File_Transfer;

  ---- File Requests ----
  procedure Write_File_Request( FR      : in Filenames;
                                Counter : in Interfaces.Unsigned_16;
                                Msg     : out Raw_Types.Serpent_Msg;
                                Written : out Natural) is
    Pos    : Integer := Msg'First;
    Max_Pos: Integer := Msg'Last - 2; -- 2 octets at end for counter
    Text_Sz: Integer;
    Max_Sz : Integer;
  begin
    -- write ID for File Request type
    Msg( Pos ) := File_Req_S_Type;
    Pos := Pos + 1;

    -- write Text size: filenames + separators
    -- consider fewer filenames if they don't ALL fit
    -- 2 octets are taken by size itself
    Max_Sz := Max_Pos - Pos - 1;
    Text_Sz := FR.Sz + FR.F_No - 1;
    if Text_Sz > Max_Sz then
      -- walk the array of filenames backwards and stop when they fit
      Written := FR.F_No - 1;
      -- calculate actual size written based on start of first discarded
        -- filename and (Written -1) octets for needed separators
      Text_Sz := Integer(FR.Starts(Written+1)) - FR.Starts'First +
                   (Written - 1);

      -- loop until either fits or nothing left
      while Written > 0 and Text_Sz > Max_Sz loop
        Written := Written - 1;
        Text_Sz := Integer(FR.Starts(Written+1))- FR.Starts'First +
                     (Written - 1);
      end loop;
      -- check that there is what to write, since nothing -> invalid message
      if Written = 0 then
        raise Invalid_Msg;
      end if;

    else --from if Text_Sz > Max_Sz
      -- ALL are written
      Written := FR.F_No;
    end if;

    -- write Text_Sz + 2 (i.e. TOTAL size)
    if Text_Sz + 2 > Integer(Interfaces.Unsigned_16'Last) then
      raise Invalid_Msg;
    end if;

    Write_U16( Msg, Pos, Interfaces.Unsigned_16(Text_Sz+2) );

    -- write filenames separated by Sep
    for I in 1..Written loop
      declare
        Start_Pos : Positive;
        End_Pos   : Positive;
        Len       : Positive;
      begin
        -- current start pos in FR.S
        Start_Pos := Positive( FR.Starts( FR.Starts'First + I - 1));

        -- calculate end based on start of next name or last
        if I < FR.F_No then
          End_Pos := Positive( FR.Starts( FR.Starts'First + I)) - 1;
        else
          End_Pos := FR.S'Last;
        end if;

        -- NB: this WILL fail if starting positions are not in order!
        Len := End_Pos - Start_Pos + 1;
        if Len <= 0 then
          raise Invalid_Msg;
        end if;

        --write the actual filename
        String_To_Octets( FR.S( Start_Pos..End_Pos ), Msg(Pos..Pos+Len-1) );
        Pos := Pos + Len;

        --if it's not the last one, write a separator
        if I < Written then
          Msg(Pos) := Sep;
          Pos := Pos + 1;
        end if;
      end;
    end loop;

    -- write the message counter in little endian at all times
    Write_U16( Msg, Pos, Counter );

    -- write padding if needed
    if Pos <= Msg'Last then
      Rng.Get_Octets( Msg(Pos..Msg'Last) );
    end if;
  end Write_File_Request;

  -- Reads a request for files; the opposite of Write_File_Request above
  procedure Read_File_Request( Msg      : in Raw_Types.Serpent_Msg;
                               Counter  : out Interfaces.Unsigned_16;
                               FR       : out Filenames) is
    Pos       : Integer := Msg'First;
    Max_Pos   : Integer := Msg'Last - 2; --at least 2 reserved for counter
    Text_Sz   : Integer;
    Max_Sz    : Integer := Max_Pos - Pos - 1; --text only i.e. w.o. size itself
    F_No      : Integer;
    U16       : Interfaces.Unsigned_16;
  begin
    -- read type ID and check
    if Msg(Pos) /= File_Req_S_Type then
      raise Invalid_Msg;
    end if;
    Pos := Pos + 1;

    -- read total size of filenames+separators
    Read_U16( Msg, Pos, U16 );
    Text_Sz := Integer(U16);
    -- take away the 2 octets for size itself
    Text_Sz := Text_Sz - 2;

    -- check that Text_Sz is not overflowing/underflowing
    if Text_Sz < 1 or Text_Sz > Max_Sz then
      raise Invalid_Msg;
    end if;

    -- count first the separators to know how many filenames
    -- NB: there is always at least 1 filename as Text_Sz > 0
    F_No := 1;
    for I in Pos .. Pos + Text_Sz - 1 loop
      if Msg(I) = Sep then
        F_No := F_No + 1;
      end if;
    end loop;

    -- create the output structure and discard separators
    -- text without separators should be Text_Sz - F_No + 1
    -- (because ONLY one separator between 2 filenames allowed)
    -- if it's not that => Invalid_Msg
    -- F_No and Text_Sz are not overflow (earlier check + calc)
    declare
      F     : Filenames(Text_Len(F_No), Text_Len(Text_Sz-F_No+1));
      S_Pos : Positive;
      Index : Positive;
    begin
      S_Pos := F.S'First;
      Index := F.Starts'First;
      F.Starts(Index) := Interfaces.Unsigned_16(S_Pos);

      for I in Pos .. Pos + Text_Sz - 1 loop
        -- copy over to F.S anything that is not separator
        if Msg(I) /= Sep then
          F.S( S_Pos ) := Character'Val(Msg(I));
          S_Pos := S_Pos + 1;
        else
          -- if it's separator, check and if ok, add next as start
          if I = Pos + Text_Sz or -- separator as last character is error
               Msg(I+1) = Sep or  -- 2 consecutive separators is error
               Index >= F.Starts'Last then -- too many separators is error
            raise Invalid_Msg;
          else
            Index := Index + 1;
            F.Starts( Index ) := Interfaces.Unsigned_16(S_Pos);
          end if;
        end if;
      end loop;

      -- copy the whole structure to output variable
      FR := F;
    end;

    -- read message counter now
    Pos := Pos + Text_Sz;
    Read_U16( Msg, Pos, Counter );

  end Read_File_Request;

The converters between String and Octets (messages.adb):

  -- String to Octets conversion
  procedure String_To_Octets(Str: in String; O: out Raw_Types.Octets) is
  begin
    Assert( Str'Length = O'Length );
    for I in 1..Str'Length loop
      O( O'First+I-1 ) := Character'Pos(Str(Str'First + I - 1 ));
    end loop;
  end String_To_Octets;

  -- Octets to string conversion
  -- NB: Str'Length has to be EQUAL to Octets'Length!
  procedure Octets_To_String(O: in Raw_Types.Octets; Str: out String) is
  begin
    Assert( O'Length = Str'Length );
    for I in 1..O'Length loop
      Str( Str'First+I-1 ) := Character'Val(O(O'First + I - 1 ));
    end loop;
  end Octets_To_String;

The read/write utilities for values on 16 bits (messages.adb):

  -- Write a 16 bits value to Octets at Pos; Pos increases by 2.
  procedure Write_U16( Msg: in out Raw_Types.Octets;
                       Pos: in out Natural;
                       U16: in Interfaces.Unsigned_16) is
  begin
    Msg(Pos..Pos+1) := Raw_Types.Cast(U16);
    Cast_LE(Msg(Pos..Pos+1));
    Pos := Pos + 2;
  end Write_U16;

  -- Read a 16-bits values from Octets from Pos; Pos increases by 2.
  procedure Read_U16( Msg: in Raw_Types.Octets;
                      Pos: in out Natural;
                      U16: out Interfaces.Unsigned_16) is
    O2  : Raw_Types.Octets_2;
  begin
    O2  := Msg(Pos..Pos+1);
    Cast_LE(O2);
    U16 := Raw_Types.Cast(O2);
    Pos := Pos + 2;
  end Read_U16;

The .vpatch and my signature for it can be found on my Reference Code Shelf as usual or through those links:


  1. Yes, I managed to fall during one of various testing sessions upon the one case where it was not, such are some of my talents, what can I tell you. 

November 17, 2018

SMG Comms Chapter 8: Keys Management

Filed under: Coding, SMG_Comms — Diana Coman @ 1:47 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

This chapter makes a few changes to previous code and adds a new part to handle read/write of messages concerning keys management (4.2 and 5.3 in the current protocol specification):

  • (Changed): I've refactored the code for read/write of messages transporting Serpent keys (i.e. the code from Chapter 7 of the SMG Comms series). At the time I wrote those for Chapter 7 I wasn't even sure that I'll continue with that approach to serializing the data structure so I did not bother too much/too early with other considerents. As it turned out that the approach implemented was indeed the most appropriate for the task at hand, the next focus was on making the best use of it given that both Serpent and RSA messages that transport keys effectively have the same structure and differ only in their ID (first octet) and length of padding (due to different sizes). So there clearly was no point in repeating the actual serialization code in 2 separate places, one for Serpent messages and one for RSA messages. Instead, the common core is now done by methods that are private to the Messages package and are called with the correct parameters by the read/write methods for Serpent and RSA messages. Having previously defined RSA and Serpent message types to reflect precisely what they are, namely subtypes of the generic, raw, "Octets" type helped significantly, of course: the private read/write simply take/provide Octets and ID, focusing on the common core and ignoring the rest (which is handled by the caller since that's the point at which Serpent/RSA is a concern). As a result, the public methods are now very short since the actual work is done in the private methods:
    package body Messages is
    
      ----------------------
      -- Serpent Messages --
      ----------------------
    
      procedure Write_SKeys_SMsg( Keyset  : in Serpent_Keyset;
                                  Counter : in Interfaces.Unsigned_16;
                                  Msg     : out Raw_Types.Serpent_Msg) is
      begin
        -- call internal write on Octets with correct type id
        Write_SKeys( Keyset, Counter, SKeys_S_Type, Msg );
      end Write_SKeys_SMsg;
    
      -- Reads a Serpent keyset from given Serpent Message
      procedure Read_SKeys_SMsg( Msg     : in Raw_Types.Serpent_Msg;
                                 Counter : out Interfaces.Unsigned_16;
                                 Keyset  : out Serpent_Keyset) is
      begin
        -- check type id and call internal Read_SKeys if correct
        if Msg(Msg'First) /= SKeys_S_Type then
          raise Invalid_Msg;
        else
          Read_SKeys( Msg, Counter, Keyset );
        end if;
      end Read_SKeys_SMsg;
    
      -- writes given key mgm structure into a Serpent message
      procedure Write_KMgm_SMsg( KMgm    : in Keys_Mgm;
                                 Counter : in Interfaces.Unsigned_16;
                                 Msg     : out Raw_Types.Serpent_Msg) is
      begin
        -- call internal write of key mgm with correct type ID
        Write_KMgm( KMgm, Counter, Key_Mgm_S_Type, Msg );
      end Write_KMgm_SMsg;
    
      -- reads a key mgm structure from the given Serpent message
      procedure Read_KMgm_SMsg( Msg     : in Raw_Types.Serpent_Msg;
                                Counter : out Interfaces.Unsigned_16;
                                KMgm    : out Keys_Mgm) is
      begin
        -- check type id and call internal Read_KMgm if correct
        if Msg(Msg'First) /= Key_Mgm_S_Type then
          raise Invalid_Msg;
        else
          Read_KMgm( Msg, Counter, KMgm );
        end if;
      end Read_KMgm_SMsg;
    
      ------------------
      -- RSA Messages --
      ------------------
    
      procedure Write_SKeys_RMsg( Keyset  : in Serpent_Keyset;
                                  Counter : in Interfaces.Unsigned_16;
                                  Msg     : out Raw_Types.RSA_Msg) is
      begin
        -- call internal write of Serpent keys with correct type ID
        Write_SKeys( Keyset, Counter, SKeys_R_Type, Msg );
      end Write_SKeys_RMsg;
    
      procedure Read_SKeys_RMsg( Msg     : in Raw_Types.RSA_Msg;
                                 Counter : out Interfaces.Unsigned_16;
                                 Keyset  : out Serpent_Keyset) is
      begin
        -- check type id and call internal Read_SKeys if correct
        if Msg(Msg'First) /= SKeys_R_Type then
          raise Invalid_Msg;
        else
          Read_SKeys( Msg, Counter, Keyset );
        end if;
      end Read_SKeys_RMsg;
    
      procedure Write_KMgm_RMsg( KMgm    : in Keys_Mgm;
                                 Counter : in Interfaces.Unsigned_16;
                                 Msg     : out Raw_Types.RSA_Msg) is
      begin
        -- call internal write of key mgm with correct type ID
        Write_KMgm( KMgm, Counter, Key_Mgm_R_Type, Msg );
      end Write_KMgm_RMsg;
    
      procedure Read_KMgm_RMsg( Msg     : in Raw_Types.RSA_Msg;
                                Counter : out Interfaces.Unsigned_16;
                                KMgm    : out Keys_Mgm) is
      begin
        -- check type id and call internal Read_KMgm if correct
        if Msg(Msg'First) /= Key_Mgm_R_Type then
          raise Invalid_Msg;
        else
          Read_KMgm( Msg, Counter, KMgm );
        end if;
      end Read_KMgm_RMsg;
    
      ------------------
      -- private part --
      ------------------
      procedure Cast_LE( LE: in out Raw_Types.Octets ) is
      begin
        -- flip octets ONLY if native is big endian.
        if System.Default_Bit_Order = System.High_Order_First then
          declare
            BE: constant Raw_Types.Octets := LE;
          begin
            for I in 1..LE'Length loop
              LE(LE'First+I-1) := BE(BE'Last-I+1);
            end loop;
          end;
        end if;
        -- NOTHING to do for native little endian
      end Cast_LE;
    
      procedure Write_SKeys( Keyset  : in Serpent_Keyset;
                             Counter : in Interfaces.Unsigned_16;
                             Type_ID : in Interfaces.Unsigned_8;
                             Msg     : out Raw_Types.Octets) is
        Pos   : Integer := Msg'First;
        Check : CRC32.CRC32;
        PadLen: Integer;
        K     : Serpent.Key;
      begin
        -- write Type ID
        Msg(Pos) := Type_ID;
        Pos := Pos + 1;
    
        -- write count of keys (NB: this IS 8 bits by definition)
        Msg(Pos) := Keyset.Keys'Length;
        Pos := Pos + 1;
    
        -- write keys
        for I in Keyset.Keys'Range loop
          -- retrieve Key to write
          K := Keyset.Keys( I );
    
          -- write key itself
          Msg(Pos..Pos+K'Length-1) := K;
          -- ensure little endian order in message
          Cast_LE(Msg(Pos..Pos+K'Length-1));
          Pos := Pos + K'Length;
    
          -- write CRC of key
          Check := CRC32.CRC( K );
          Msg(Pos..Pos+3) := Raw_Types.Cast(Check);
          Cast_LE(Msg(Pos..Pos+3));
          Pos := Pos + 4;
        end loop;
    
        -- write flag
        Msg(Pos) := Keyset.Flag;
        Pos := Pos + 1;
    
        -- write message counter
        Msg(Pos..Pos+1) := Raw_Types.Cast(Counter);
        Cast_LE(Msg(Pos..Pos+1));
        Pos := Pos + 2;
    
        -- write padding as needed; endianness is irrelevant here
        PadLen := Msg'Last - Pos + 1;
        if PadLen > 0 then
          declare
            Pad : Raw_Types.Octets(1..PadLen);
          begin
            RNG.Get_Octets( Pad );
            Msg(Pos..Pos+PadLen-1) := Pad;
          end;
        end if;
    
      end Write_SKeys;
    
      procedure Read_SKeys( Msg     : in Raw_Types.Octets;
                            Counter : out Interfaces.Unsigned_16;
                            Keyset  : out Serpent_Keyset) is
        Pos: Integer := Msg'First;
      begin
        -- read type and check
        if Msg(Pos) = SKeys_S_Type or
           Msg(Pos) = SKeys_R_Type then
          Pos := Pos + 1;
        else
          raise Invalid_Msg;
        end if;
    
        -- read count of keys and check
        if Msg(Pos) in Keys_Count'Range then
          declare
            N     : Keys_Count := Keys_Count(Msg(Pos));
            KS    : Serpent_Keyset(N);
            K     : Serpent.Key;
            Check : CRC32.CRC32;
            O4    : Raw_Types.Octets_4;
            O2    : Raw_Types.Octets_2;
          begin
            Pos := Pos + 1;
            --read keys and check crc for each
            for I in 1 .. N loop
              -- read key and advance pos
              K := Msg(Pos..Pos+K'Length-1);
              Cast_LE(K);
              Pos := Pos + K'Length;
              -- read crc and compare to crc32(key)
              O4 := Msg(Pos..Pos+3);
              Cast_LE(O4);
              Check   := Raw_Types.Cast(O4);
              Pos := Pos + 4;
              if Check /= CRC32.CRC(K) then
                raise Invalid_Msg;
              end if;
              -- if it got here, key is fine so add to set
              KS.Keys(KS.Keys'First + I -1) := K;
            end loop;
            -- read and set flag
            KS.Flag := Msg(Pos);
            Pos := Pos + 1;
            -- read and set message counter
            O2 := Msg(Pos..Pos+1);
            Cast_LE(O2);
            Counter := Raw_Types.Cast(O2);
            -- rest of message is padding so it's ignored
            -- copy keyset to output variable
            Keyset := KS;
          end;
        else
          raise Invalid_Msg;
        end if;
      end Read_SKeys;
    
      -- writes given key management structure to the given octets array
      procedure Write_KMgm( KMgm    : in Keys_Mgm;
                            Counter : in Interfaces.Unsigned_16;
                            Type_ID : in Interfaces.Unsigned_8;
                            Msg     : out Raw_Types.Octets) is
        Pos   : Integer := Msg'First;
      begin
        -- write given type id
        Msg(Pos) := Type_ID;
        Pos := Pos + 1;
    
        -- write count of server keys requested
        Msg(Pos) := KMgm.N_Server;
        Pos := Pos + 1;
    
        -- write count of client keys requested
        Msg(Pos) := KMgm.N_Client;
        Pos := Pos + 1;
    
        -- write id of key preferred for further inbound Serpent messages
        Msg(Pos) := KMgm.Key_ID;
        Pos := Pos + 1;
    
        -- write count of burnt keys in this message
        Msg(Pos..Pos) := Cast( KMgm.N_Burnt );
        Pos := Pos + 1;
    
        -- if there are any burnt keys, write their ids
        if KMgm.N_Burnt > 0 then
          Msg( Pos .. Pos + KMgm.Burnt'Length - 1 ) := KMgm.Burnt;
          Pos := Pos + KMgm.Burnt'Length;
        end if;
    
        -- write the message count
        Msg(Pos..Pos+1) := Raw_Types.Cast( Counter );
        Cast_LE( Msg(Pos..Pos+1) );
        Pos := Pos + 2;
    
        -- pad with random octets until the end of Msg
        RNG.Get_Octets( Msg(Pos..Msg'Last) );
    
      end Write_KMgm;
    
      -- attempts to read from the given array of octets a key management structure
      procedure Read_KMgm( Msg     : in Raw_Types.Octets;
                           Counter : out Interfaces.Unsigned_16;
                           KMgm    : out Keys_Mgm) is
        Pos       : Integer := Msg'First;
        Burnt_Pos : Integer := Msg'First + 4;
      begin
        -- read type and check
        if Msg(Pos) = Key_Mgm_S_Type or
           Msg(Pos) = Key_Mgm_R_Type then
          Pos := Pos + 1;
        else
          raise Invalid_Msg;
        end if;
    
        -- read the count of burnt keys and check
        -- NB: Burnt_Pos IS in range of Counter_8bits since it's an octet
        declare
          N_Burnt : Counter_8bits := Counter_8bits(Msg(Burnt_Pos));
          Mgm     : Keys_Mgm(N_Burnt);
          O2      : Raw_Types.Octets_2;
        begin
          -- read count of server keys requested
          Mgm.N_Server := Msg(Pos);
          Pos := Pos + 1;
    
          -- read count of client keys requested
          Mgm.N_Client := Msg(Pos);
          Pos := Pos + 1;
    
          -- read ID of Serpent key preferred for further inbound messages
          Mgm.Key_ID   := Msg(Pos);
          Pos := Pos + 2; --skip the count of burnt keys as it's read already
    
          -- read ids of burnt keys, if any
          if N_Burnt > 0 then
            Mgm.Burnt := Msg(Pos..Pos+N_Burnt-1);
            Pos := Pos + N_Burnt;
          end if;
    
          -- read and set message counter
          O2 := Msg(Pos..Pos+1);
          Cast_LE(O2);
          Counter := Raw_Types.Cast(O2);
          -- rest of message is padding so it's ignored
          -- copy the keys mgm structure to output param
          KMgm := Mgm;
        end;
      end Read_KMgm;
    
    end Messages;
    
  • (Changed): The Keys_Mgm record in Data_Structs.ads:
    • (Changed): The array of burnt keys was initially declared as holding the actual keys rather than just the ids. I've corrected this so that now the "Burnt" part of Keys_Mgm is simply an array of Unsigned_8 (i.e. Octets type)
    • (Changed): The parameter N_Burnt (the number of IDs of burnt keys in the structure) was initially declared as Unsigned_8. However, after changing the array to the correct Octets type, there was a bit of type-trouble: on one hand, the Octets type can have any length, so the type of its range can't be limited to Unsigned_8 and Unsigned_8 is NOT a subtype of Integer (it's a modular type instead); on the other hand, N_Burnt has to appear by itself in the definition of the record type since it is a discriminant so it couldn't be cast to an Integer subtype that Octets would accept. To address this, I defined the type Counter_8bit to match precisely the meaning of N_Burnt (or any other similar counter): values between 0 and 255.
    • (Changed): Since the N_Burns counter can be 0, it follows that the array of burnt ids should *not* always be present in the record (the only way to have an array of length 0 is effectively to not have it at all). Consequently, I further changed the Keys_Mgm record declaration to reflect this:
        ------------------------------
        -- Serpent Keys Management
        subtype Counter_8bits is Natural range 0..255;
        function Cast is new Ada.Unchecked_Conversion( Counter_8bits,
                                                       Raw_Types.Octets_1 );
        function Cast is new Ada.Unchecked_Conversion( Raw_Types.Octets_1,
                                                       Counter_8bits );
        type Keys_Mgm (N_Burnt: Counter_8bits := 0) is
          record
            -- count of server keys requested
            N_Server: Interfaces.Unsigned_8;
            -- count of client keys requested
            N_Client: Interfaces.Unsigned_8;
            -- ID of Serpent key preferred for further inbound Serpent msgs.
            Key_ID  : Interfaces.Unsigned_8;
            -- IDs of Serpent keys burnt by this message
            case N_Burnt is
              when 0 =>
                null;
              when others =>
                Burnt   : Raw_Types.Octets( 1..N_Burnt );
            end case;
          end record;
      
  • (Added): I've used the Keys_Mgm record from above and the same approach as for keys messages (common core serialization in private methods that are called by public read/write with correct IDs and Octets parameters) to implement the read/write from/to messages for keys management, whether Serpent or RSA messages.
  • (Added): New tests for the new read/write methods for key management messages.
  • (Changed): The existing tests for read/write of messages transporting keys are now updated to test also the read/write to/from RSA messages of the relevant type.

The .vpatch and my signature for it is on the Reference Code Shelf as well as linked here for your convenience:

November 13, 2018

V with VTools, Keccak Hashes and Its Own Tree

Filed under: Coding, TMSR — Diana Coman @ 10:49 p.m.

The republican versioning control system, V for Victory, is very much used, very much needed but nevertheless not yet versioned itself. As this has caused already way more talk in the logs than it's worth it, I promised I'll do a write-up of my own V setup and publish it, with a proper versioning for V itself included. So I've dug up old versions as well as my current setup and packaged everything in 2 different ways:

  1. A V-tree (using Keccak hashes) that captures the changes to v.pl code1 from the first version that I ever used, namely 999942. To use this, simply download the .vpatches, the .sig files and my signature, check them and then press the tree with either a different V that you might have or otherwise semi-manually with phf's vtools (vpatch more precisely). Note that you WILL need vtools (or equivalent) at any rate! Once you pressed V itself successfuly, you should have a v.pl that you can run - it will check its dependencies and complain if it doesn't find something (most notably the vtools parts namely vpatch and ksum).
  2. A signed .zip meant as a starter package for someone who hears of V for the first time in their life. Download starter_v.zip and starter_v.zip.diana_coman.sig. Check the signature! If and ONLY IF the check passes, unzip and then read the scripts in there. The build.sh script will simply build3 the vtools that are included in this starter pack and it will copy to the starter_v directory all the executables that are needed (v.pl renamed as vk.pl to make it clear it uses Keccak hashes, vdiff, vpatch and ksum). The included vtools are the code obtained from pressing current vtools tree up to and including the ksum .vpatch.

My changes to Mod6's v.pl simply replace older sha-based dependencies and calls with the vtools-based ones. Note that you'll need to have ksum and vpatch in your PATH or otherwise ready and accessible as v.pl will simply try to call them when it needs them.

For potential reference, here's my usual workflow to make a .vpatch:
mkdir a
mkdir b
cp -r old_stuff a/stuff
cp -r new_sutff b/stuff
vdiff a b > newpatch.vpatch
gpg --armor --output newpatch.vpatch.diana_coman.sig --detach-sig newpatch.vpatch

To check / press a V tree:
mkdir patches
mkdir .wot
mkdir .seals
cp some_patches patches/
cp corresponding_sig_files .seals
cp corresponding_trusted_pubkeys .wot
vk4 f
vk l
vk p v testdir chosen_patch.vpatch
cd testdir
read, compile, run etc

The .vpatches and .sig files:

The .zip file and corresponding .sig file:

For something to test your new shiny V on, head over to my Reference Code Shelf and take your pick. For trouble and questions, use the comments box below.


  1. Note that you are warmly invited to implement your own V! This version here is mod6's V implementation that was much discussed and iterated upon in the early days. 

  2. Note that V's versions DECREASE rather than increase, as per the explanation

  3. It requires GNAT. If you have no idea what that is, dig around, read the logs, ask humbly. 

  4. As I have all sorts of V implementations living side by side, I tend to give them different names - this is the vk for V-Keccak! 

November 10, 2018

SMG Comms Chapter 7: Read/Write Serpent Keysets to/from Serpent Messages

Filed under: Coding, SMG_Comms — Diana Coman @ 4:06 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

Over this past week I've been going deeper into Ada looking again and again -among other things- at record representation clauses and unions and serialization1 and basically anything and everything that could perhaps provide an elegant solution to serializing data structures to arrays of octets and back. And by "looking" I don't mean just reading but rather the usual combination of reading, writing and coding or at least attempting to code all sorts of ideas that seemed promising. What I have as a result is a much better grasp of Ada's neat representation clauses and parametrized records. What I don't have as a result is the very elegant and short solution I was hoping to find. Instead, I have a first pass at a solution together with the whole chain of reasoning that led me to it. So I'm publishing this as it is - a small step - and I'll gladly listen to better suggestions that are really fit for the job at hand.

The issue is in principle simple enough: write game data to the raw arrays of octets that are SMG messages ensuring the exact format specified by the communication protocol; conversely, read from those same raw arrays of octets that are SMG messages the actual game data. Stated like this, the solution suggesting itself is to define records matching exactly the format specified by the protocol, use representation clauses to control the exact place of all components and then simply convert the whole thing in one go. There are however a few things that I really don't like about this approach of using records matching exactly the full message structure as defined in Eulora's communication protocol:

  • The actual game data is of course only a part of each message. The rest includes padding or communication-level information that really has no business leaking in any way outside of the communication part itself. So I don't quite see the point in defining records for the full message since the read/write (that are supposedly called from outside the communication level itself) should NOT require/produce such records! Instead, the output should be *game data* for read and *array of octets* for write. As a result, I really don't see any benefit in defining message-matching records just for read/write to use *internally* - a sort of filling those component by component and doing the conversion as a whole instead of just doing the conversion component by component directly.
  • The internal structure of most messages is effectively parametrized: there is usually at least one component that represents the number of other following components. Ada handles this sort of thing very gracefully through parametrized records but having parametrized records inside a record doesn't seem to me like a very elegant solution nor one leading to a very clear representation of the whole. I admit I am still not 100% sure of the actual, exact representation of such a record containing itself parametrized records since my understanding is that Ada will allocated maximum space (i.e. space to fit potentially the largest structure) but then simply make available only as much as needed according to the actual value of the parameter.

Considering the above issues, my current solution aims to keep the communication-level parts of messages as an internal concern of the communication level and provide/require at read/write only the actual game data that is needed for each type of message. The conversion to/from octets is then done indeed component by component, as they are written/read and endianness is handled simply in one single place, at conversion: if native endianness is Big Endian then octets are flipped for any component that is represented on more than one octet; otherwise, on Little Endian iron, octets are kept as they are found (i.e. the values in messages are in Little Endian representation, yes).

To illustrate the approach above in practice, I've implemented already the read/write for Serpent Keysets into Serpent Messages (section 4.1 of the procotol specification). The communication-level data in this type of message consists in the type ID, the CRC information *for each key* and the message padding (variable-sized since it simply fills the message as needed). The message counter is kept separate since the server for instance will likely need to keep different counters for each client, so I'll leave for now the decision on this for a higher /different level. The keyset itself is the actual data really and it consists in the number of keys, their actual octets and the flag that indicates whether they are for client or for server use. So I define a parametrized record for representing a keyset only rather than the whole message. This is in the new file data_structs.ads that I see at the moment as the place for defining structures for game data. Hence, it contains for instance also the "SMG_Object" record (fixed size) that illustrates some use of representation clauses:

 -- Data structures for SMG Communication Protocol
 -- S.MG, 2018

with Interfaces; -- for fixed-size types
with Raw_Types;  -- for protocol raw types
with System;     -- for Bit_Order
with Serpent;    -- for Serpent.Key type

package Data_Structs is
  Pragma Pure(Data_Structs);

  -- an object in the world, with record layout fully specified
  -- a storage element is 8-bit aka 1 octet
  -- "at" gives number of storage element
  -- "range" gives bits inside storage element

  type SMG_Object is
    record
      -- ID of the object
      ID         : Interfaces.Unsigned_32;

      -- Position of the object in the world.
      -- For a world with map coordinates (MC) between -500 and +500,
      -- the relationship with given figure (GF) is:
      -- MC = GF / 65.535 - 500
      -- where 65.535 comes from the mapping 65535 / (500 - - 500)
      X, Y, Z    : Interfaces.Integer_16;

      -- Rotation of the object (RO) linked to GF by:
      -- RO = GF / 128*pi
      RX, RY, RZ : Interfaces.Unsigned_8;
    end record;
  for SMG_Object'Size use 104; -- in bits!
  for SMG_Object'Bit_Order use System.Low_Order_First;
  for SMG_Object use
    record
      ID at 0  range 0 .. 31;
      X  at 4  range 0 .. 15;
      Y  at 6  range 0 .. 15;
      Z  at 8  range 0 .. 15;
      RX at 10 range 0 .. 7;
      RY at 11 range 0 .. 7;
      RZ at 12 range 0 .. 7;
    end record;

  -------------------------
  -- A set of Serpent Keys
  -- parametrized record for a Serpent Keyset
  -- parameters:
  --   N is number of Serpent Keys contained
  -- this can be the content of messages 4.1 or 5.2 in protocol spec.

  -- an array of Serpent Keys
  -- MAXIMUM 40 keys allowed in a message, hence subtype for key count
  subtype Keys_Count is Interfaces.Unsigned_8 range 1..40;
  type SKeys_Array is array( Keys_Count range <>) of Serpent.Key;

  type Serpent_Keyset( N : Keys_Count := Keys_Count'Last) is
    record
      -- actual Serpent Keys
      Keys      : SKeys_Array( 1..N );
      -- whether for talking to client (LSB set) or talking to server (MSB set)
      Flag      : Interfaces.Unsigned_8;
    end record;

  ------------------------------
  -- Serpent Keys Management
  type Keys_Mgm (N_Burnt: Interfaces.Unsigned_8) is
    record
      -- count of server keys requested
      N_Server: Interfaces.Unsigned_8;
      -- count of client keys requested
      N_Client: Interfaces.Unsigned_8;
      -- ID of Serpent key preferred for further inbound Serpent msgs.
      Key_ID  : Interfaces.Unsigned_8;
      -- IDs of Serpent keys burnt by this message
      Burnt   : SKeys_Array( 1..N_Burnt );
    end record;

end Data_Structs;

The communication-level information such as CRC, type ids and padding are all handled by messages.ads/adb and are effectively private matters, of no concern to anything outside this messages package:

 -- Message reader & writers for SMG Communication Protocol
 -- This part effectively serializes/deserializes game data structures
 --   for transmission between client and server.
 -- Note that messages themeselves are simply arrays of octets.
 -- (see also raw_types.ads/adb and packing.ads/adb for related parts)
 -- NB: message ids and padding as per protocol spec are handled HERE ONLY.
 -- Relies on:
 --   RNG (for random padding)
 --   CRC32 (for CRC calculations)
 --   Raw_Types
 --   Data_Structs
 -- S.MG, 2018

with Raw_Types;
with RNG;
with CRC32;
with Data_Structs; use Data_Structs;
with Interfaces;

package Messages is
  -- exception raised when given message to read fails sanity checks
  Invalid_Msg: exception;

  ------------------------------------------------
  -- Writes a Serpent Keyset to the given Serpent Message
  --
  -- Keyset  - the set of keys to write to message
  -- Counter - the message count
  procedure Write_SKeys_SMsg( Keyset  : in Serpent_Keyset;
                              Counter : in Interfaces.Unsigned_16;
                              Msg     : out Raw_Types.Serpent_Msg);

  -- Reads a Serpent Keyset from the given Serpent Message
  -- The opposite of Write_SKeys_SMsg above
  -- Raises Invalid_Message exception if given message fails sanity checks
  procedure Read_SKeys_SMsg( Msg     : in Raw_Types.Serpent_Msg;
                             Counter : out Interfaces.Unsigned_16;
                             Keyset  : out Serpent_Keyset);

private

  -- if native is little endian, does nothing;
  -- if native is big endian, it flips the input's octets.
  -- (strictly for arrays of octets)
  procedure Cast_LE( LE: in out Raw_Types.Octets );

  -- protocol message types IDs, fixed as per protocol specification
  -- Serpent messages end in "S_Type"
  -- RSA messages end in "R_Type"
  -- Character action types end in "A_Type"

  -- Serpent message types
  SKeys_S_Type         : constant Interfaces.Unsigned_8 := 1;
  Key_Mgm_S_Type       : constant Interfaces.Unsigned_8 := 2;
  File_Transfer_S_Type : constant Interfaces.Unsigned_8 := 3;
  File_Req_S_Type      : constant Interfaces.Unsigned_8 := 4;
  Client_Action_S_Type : constant Interfaces.Unsigned_8 := 5;
  World_Bulletin_S_Type: constant Interfaces.Unsigned_8 := 6;
  Obj_Request_S_Type   : constant Interfaces.Unsigned_8 := 7;
  Obj_Info_S_Type      : constant Interfaces.Unsigned_8 := 8;

  -- RSA message types
  RKeys_R_Type         : constant Interfaces.Unsigned_8 := 251;
  SKeys_R_Type         : constant Interfaces.Unsigned_8 := 157;
  Key_Mgm_R_Type       : constant Interfaces.Unsigned_8 := 233;

  -- Character action types
  Lock_A_Type          : constant Interfaces.Unsigned_8 := 0;
  Make_A_Type          : constant Interfaces.Unsigned_8 := 1;
  Explore_A_Type       : constant Interfaces.Unsigned_8 := 2;
  Exchange_A_Type      : constant Interfaces.Unsigned_8 := 3;
  Attack_A_Type        : constant Interfaces.Unsigned_8 := 4;
  Repair_A_Type        : constant Interfaces.Unsigned_8 := 5;
  Move_A_Type          : constant Interfaces.Unsigned_8 := 6;
  Train_A_Type         : constant Interfaces.Unsigned_8 := 7;

end Messages;

The implementation in messages.adb, including read/write for Serpent keysets to/from Serpent messages and handling endianness for any multi-octet entity:

 -- Message reader & writers for SMG Communication Protocol
 -- S.MG, 2018

with Interfaces; use Interfaces;
with Serpent;
with System; use System;

package body Messages is

  procedure Write_SKeys_SMsg( Keyset  : in Serpent_Keyset;
                              Counter : in Interfaces.Unsigned_16;
                              Msg     : out Raw_Types.Serpent_Msg) is
    Pos   : Integer := Msg'First;
    Check : CRC32.CRC32;
    PadLen: Integer;
    K     : Serpent.Key;
  begin
    -- write Type ID
    Msg(Pos) := SKeys_S_Type;
    Pos := Pos + 1;

    -- write count of keys (NB: this IS 8 bits by definition)
    Msg(Pos) := Keyset.Keys'Length;
    Pos := Pos + 1;

    -- write keys
    for I in Keyset.Keys'Range loop
      -- retrieve Key to write
      K := Keyset.Keys( I );

      -- write key itself
      Msg(Pos..Pos+K'Length-1) := K;
      -- ensure little endian order in message
      Cast_LE(Msg(Pos..Pos+K'Length-1));
      Pos := Pos + K'Length;

      -- write CRC of key
      Check := CRC32.CRC( K );
      Msg(Pos..Pos+3) := Raw_Types.Cast(Check);
      Cast_LE(Msg(Pos..Pos+3));
      Pos := Pos + 4;
    end loop;

    -- write flag
    Msg(Pos) := Keyset.Flag;
    Pos := Pos + 1;

    -- write message counter
    Msg(Pos..Pos+1) := Raw_Types.Cast(Counter);
    Cast_LE(Msg(Pos..Pos+1));
    Pos := Pos + 2;

    -- write padding as needed; endianness is irrelevant here
    PadLen := Msg'Last - Pos + 1;
    if PadLen > 0 then
      declare
        Pad : Raw_Types.Octets(1..PadLen);
      begin
        RNG.Get_Octets( Pad );
        Msg(Pos..Pos+PadLen-1) := Pad;
      end;
    end if;
  end Write_SKeys_SMsg;

  -- Reads a Serpent keyset from given Serpent Message
  procedure Read_SKeys_SMsg( Msg     : in Raw_Types.Serpent_Msg;
                             Counter : out Interfaces.Unsigned_16;
                             Keyset  : out Serpent_Keyset) is
    Pos: Integer := Msg'First;
  begin
    -- read type and check
    if Msg(Pos) = SKeys_S_Type then
      Pos := Pos + 1;
    else
      raise Invalid_Msg;
    end if;

    -- read count of keys and check
    if Msg(Pos) in Keys_Count'Range then
      declare
        N     : Keys_Count := Keys_Count(Msg(Pos));
        KS    : Serpent_Keyset(N);
        K     : Serpent.Key;
        Check : CRC32.CRC32;
        O4    : Raw_Types.Octets_4;
        O2    : Raw_Types.Octets_2;
      begin
        Pos := Pos + 1;
        --read keys and check crc for each
        for I in 1 .. N loop
          -- read key and advance pos
          K := Msg(Pos..Pos+K'Length-1);
          Cast_LE(K);
          Pos := Pos + K'Length;
          -- read crc and compare to crc32(key)
          O4 := Msg(Pos..Pos+3);
          Cast_LE(O4);
          Check   := Raw_Types.Cast(O4);
          Pos := Pos + 4;
          if Check /= CRC32.CRC(K) then
            raise Invalid_Msg;
          end if;
          -- if it got here, key is fine so add to set
          KS.Keys(KS.Keys'First + I -1) := K;
        end loop;
        -- read and set flag
        KS.Flag := Msg(Pos);
        Pos := Pos + 1;
        -- read and set message counter
        O2 := Msg(Pos..Pos+1);
        Cast_LE(O2);
        Counter := Raw_Types.Cast(O2);
        -- rest of message is padding so it's ignored
        -- copy keyset to output variable
        Keyset := KS;
      end;
    else
      raise Invalid_Msg;
    end if;
  end Read_SKeys_SMsg;

  -- private part
  procedure Cast_LE( LE: in out Raw_Types.Octets ) is
  begin
    -- flip octets ONLY if native is big endian.
    if System.Default_Bit_Order = System.High_Order_First then
      declare
        BE: constant Raw_Types.Octets := LE;
      begin
        for I in 1..LE'Length loop
          LE(LE'First+I-1) := BE(BE'Last-I+1);
        end loop;
      end;
    end if;
    -- NOTHING to do for native little endian
  end Cast_LE;

end Messages;

Note that Messages calls directly RNG to obtain padding as and when it needs it. Similarly, it calls directly CRC32 to calculate the checksums as and when needed. The CRC32 is the previously published table-lookup CRC32 implementation from EuCrypt, now integrated into SMG Comms and slightly adapted to use the Octets type in Raw_Types:


------------------------------------------------------------------------------
------------------------------------------------------------------------------
-- This file is part of 'CRC32'                                             --
--                                                                          --
-- You do not have, nor can you ever acquire the right to use, copy or      --
-- distribute this software ; Should you use this software for any purpose, --
-- or copy and distribute it to anyone or in any manner, you are breaking   --
-- the laws of whatever soi-disant jurisdiction, and you promise to         --
-- continue doing so for the indefinite future. In any case, please         --
-- always : read and understand any software ; verify any PGP signatures    --
-- that you use - for any purpose.                                          --
--                                                                          --
-- See also http://trilema.com/2015/a-new-software-licensing-paradigm .     --
------------------------------------------------------------------------------
------------------------------------------------------------------------------

  -- CRC32, lookup-based implementation
  -- S.MG, 2018
  --
  -- The CRC32 is a checksum calculated as the remainder of dividing
  -- the input data by the 0x04C11DB7 polynomial. Specifications:
  -- Name   : "CRC-32"
  -- Width  : 32        (number of bits)
  -- Poly   : 04C11DB7  (generator polynomial)
  -- Init   : FFFFFFFF
  -- RefIn  : True      (input is reflected)
  -- RefOut : True      (output is reflected)
  -- XorOut : FFFFFFFF
  -- Check  : CBF43926  (expected value for input "123456789")
  --
  -- This implementation is based on the CRC32 specification in:
  -- Sarwate, D.V. "Computation of Cyclic Redundancy Checks via Table Look-Up"
  -- in Communications of the ACM, Vol. 31 No. 8, pp.1008-1013, Aug. 1988

with Interfaces; use Interfaces;
with Raw_Types;

package CRC32 is
  -- local, shorthand version of Interfaces. types
  subtype CRC32 is Interfaces.Unsigned_32;
  subtype Octet is Interfaces.Unsigned_8;

  subtype Octet_Array is Raw_Types.Octets;

  -- interface for external callers
    -- calculate CRC32 for the given string
  function CRC( S: in String ) return CRC32;

    -- calculate CRC32 for the given array of octets
  function CRC( Data: in Octet_Array ) return CRC32;

  -- internal constants and helper methods
private
	function Shift_Right( Value  : CRC32;
                        Amount : Natural)
                        return CRC32;
  pragma Import(Intrinsic, Shift_Right); 

  Init_Value : constant CRC32 := 16#FFFF_FFFF#; -- Initial value
  Xor_Out    : constant CRC32 := 16#FFFF_FFFF#; -- For extracting result
  LSB_Mask   : constant CRC32 := 16#0000_00FF#; -- lsb mask for a CRC32 value

  -- lookup table with precomputed values for CRC32
  Lookup : constant array (CRC32 range 0 .. 255) of CRC32 :=
     (16#0000_0000#, 16#7707_3096#, 16#EE0E_612C#, 16#9909_51BA#,
      16#076D_C419#, 16#706A_F48F#, 16#E963_A535#, 16#9E64_95A3#,
      16#0EDB_8832#, 16#79DC_B8A4#, 16#E0D5_E91E#, 16#97D2_D988#,
      16#09B6_4C2B#, 16#7EB1_7CBD#, 16#E7B8_2D07#, 16#90BF_1D91#,
      16#1DB7_1064#, 16#6AB0_20F2#, 16#F3B9_7148#, 16#84BE_41DE#,
      16#1ADA_D47D#, 16#6DDD_E4EB#, 16#F4D4_B551#, 16#83D3_85C7#,
      16#136C_9856#, 16#646B_A8C0#, 16#FD62_F97A#, 16#8A65_C9EC#,
      16#1401_5C4F#, 16#6306_6CD9#, 16#FA0F_3D63#, 16#8D08_0DF5#,
      16#3B6E_20C8#, 16#4C69_105E#, 16#D560_41E4#, 16#A267_7172#,
      16#3C03_E4D1#, 16#4B04_D447#, 16#D20D_85FD#, 16#A50A_B56B#,
      16#35B5_A8FA#, 16#42B2_986C#, 16#DBBB_C9D6#, 16#ACBC_F940#,
      16#32D8_6CE3#, 16#45DF_5C75#, 16#DCD6_0DCF#, 16#ABD1_3D59#,
      16#26D9_30AC#, 16#51DE_003A#, 16#C8D7_5180#, 16#BFD0_6116#,
      16#21B4_F4B5#, 16#56B3_C423#, 16#CFBA_9599#, 16#B8BD_A50F#,
      16#2802_B89E#, 16#5F05_8808#, 16#C60C_D9B2#, 16#B10B_E924#,
      16#2F6F_7C87#, 16#5868_4C11#, 16#C161_1DAB#, 16#B666_2D3D#,
      16#76DC_4190#, 16#01DB_7106#, 16#98D2_20BC#, 16#EFD5_102A#,
      16#71B1_8589#, 16#06B6_B51F#, 16#9FBF_E4A5#, 16#E8B8_D433#,
      16#7807_C9A2#, 16#0F00_F934#, 16#9609_A88E#, 16#E10E_9818#,
      16#7F6A_0DBB#, 16#086D_3D2D#, 16#9164_6C97#, 16#E663_5C01#,
      16#6B6B_51F4#, 16#1C6C_6162#, 16#8565_30D8#, 16#F262_004E#,
      16#6C06_95ED#, 16#1B01_A57B#, 16#8208_F4C1#, 16#F50F_C457#,
      16#65B0_D9C6#, 16#12B7_E950#, 16#8BBE_B8EA#, 16#FCB9_887C#,
      16#62DD_1DDF#, 16#15DA_2D49#, 16#8CD3_7CF3#, 16#FBD4_4C65#,
      16#4DB2_6158#, 16#3AB5_51CE#, 16#A3BC_0074#, 16#D4BB_30E2#,
      16#4ADF_A541#, 16#3DD8_95D7#, 16#A4D1_C46D#, 16#D3D6_F4FB#,
      16#4369_E96A#, 16#346E_D9FC#, 16#AD67_8846#, 16#DA60_B8D0#,
      16#4404_2D73#, 16#3303_1DE5#, 16#AA0A_4C5F#, 16#DD0D_7CC9#,
      16#5005_713C#, 16#2702_41AA#, 16#BE0B_1010#, 16#C90C_2086#,
      16#5768_B525#, 16#206F_85B3#, 16#B966_D409#, 16#CE61_E49F#,
      16#5EDE_F90E#, 16#29D9_C998#, 16#B0D0_9822#, 16#C7D7_A8B4#,
      16#59B3_3D17#, 16#2EB4_0D81#, 16#B7BD_5C3B#, 16#C0BA_6CAD#,
      16#EDB8_8320#, 16#9ABF_B3B6#, 16#03B6_E20C#, 16#74B1_D29A#,
      16#EAD5_4739#, 16#9DD2_77AF#, 16#04DB_2615#, 16#73DC_1683#,
      16#E363_0B12#, 16#9464_3B84#, 16#0D6D_6A3E#, 16#7A6A_5AA8#,
      16#E40E_CF0B#, 16#9309_FF9D#, 16#0A00_AE27#, 16#7D07_9EB1#,
      16#F00F_9344#, 16#8708_A3D2#, 16#1E01_F268#, 16#6906_C2FE#,
      16#F762_575D#, 16#8065_67CB#, 16#196C_3671#, 16#6E6B_06E7#,
      16#FED4_1B76#, 16#89D3_2BE0#, 16#10DA_7A5A#, 16#67DD_4ACC#,
      16#F9B9_DF6F#, 16#8EBE_EFF9#, 16#17B7_BE43#, 16#60B0_8ED5#,
      16#D6D6_A3E8#, 16#A1D1_937E#, 16#38D8_C2C4#, 16#4FDF_F252#,
      16#D1BB_67F1#, 16#A6BC_5767#, 16#3FB5_06DD#, 16#48B2_364B#,
      16#D80D_2BDA#, 16#AF0A_1B4C#, 16#3603_4AF6#, 16#4104_7A60#,
      16#DF60_EFC3#, 16#A867_DF55#, 16#316E_8EEF#, 16#4669_BE79#,
      16#CB61_B38C#, 16#BC66_831A#, 16#256F_D2A0#, 16#5268_E236#,
      16#CC0C_7795#, 16#BB0B_4703#, 16#2202_16B9#, 16#5505_262F#,
      16#C5BA_3BBE#, 16#B2BD_0B28#, 16#2BB4_5A92#, 16#5CB3_6A04#,
      16#C2D7_FFA7#, 16#B5D0_CF31#, 16#2CD9_9E8B#, 16#5BDE_AE1D#,
      16#9B64_C2B0#, 16#EC63_F226#, 16#756A_A39C#, 16#026D_930A#,
      16#9C09_06A9#, 16#EB0E_363F#, 16#7207_6785#, 16#0500_5713#,
      16#95BF_4A82#, 16#E2B8_7A14#, 16#7BB1_2BAE#, 16#0CB6_1B38#,
      16#92D2_8E9B#, 16#E5D5_BE0D#, 16#7CDC_EFB7#, 16#0BDB_DF21#,
      16#86D3_D2D4#, 16#F1D4_E242#, 16#68DD_B3F8#, 16#1FDA_836E#,
      16#81BE_16CD#, 16#F6B9_265B#, 16#6FB0_77E1#, 16#18B7_4777#,
      16#8808_5AE6#, 16#FF0F_6A70#, 16#6606_3BCA#, 16#1101_0B5C#,
      16#8F65_9EFF#, 16#F862_AE69#, 16#616B_FFD3#, 16#166C_CF45#,
      16#A00A_E278#, 16#D70D_D2EE#, 16#4E04_8354#, 16#3903_B3C2#,
      16#A767_2661#, 16#D060_16F7#, 16#4969_474D#, 16#3E6E_77DB#,
      16#AED1_6A4A#, 16#D9D6_5ADC#, 16#40DF_0B66#, 16#37D8_3BF0#,
      16#A9BC_AE53#, 16#DEBB_9EC5#, 16#47B2_CF7F#, 16#30B5_FFE9#,
      16#BDBD_F21C#, 16#CABA_C28A#, 16#53B3_9330#, 16#24B4_A3A6#,
      16#BAD0_3605#, 16#CDD7_0693#, 16#54DE_5729#, 16#23D9_67BF#,
      16#B366_7A2E#, 16#C461_4AB8#, 16#5D68_1B02#, 16#2A6F_2B94#,
      16#B40B_BE37#, 16#C30C_8EA1#, 16#5A05_DF1B#, 16#2D02_EF8D#);

end CRC32;

The CRC32 implementation in crc32.adb:

------------------------------------------------------------------------------
------------------------------------------------------------------------------
-- This file is part of 'CRC32'                                             --
--                                                                          --
-- You do not have, nor can you ever acquire the right to use, copy or      --
-- distribute this software ; Should you use this software for any purpose, --
-- or copy and distribute it to anyone or in any manner, you are breaking   --
-- the laws of whatever soi-disant jurisdiction, and you promise to         --
-- continue doing so for the indefinite future. In any case, please         --
-- always : read and understand any software ; verify any PGP signatures    --
-- that you use - for any purpose.                                          --
--                                                                          --
-- See also http://trilema.com/2015/a-new-software-licensing-paradigm .     --
------------------------------------------------------------------------------
------------------------------------------------------------------------------

  -- CRC32 implementation
  -- S.MG, 2018

package body CRC32 is
  function CRC( S: in String ) return CRC32 is
    Result : CRC32 := Init_Value;
    Value  : CRC32;
  begin
    -- process input character by character
    for C of S loop
      Value  := CRC32( Character'Pos( C ) );
      Result := Shift_Right(Result, 8) xor
                  Lookup( Value xor (Result and LSB_MASK));
    end loop;
    -- reflect result
    Result := Result xor Xor_Out;

    return Result;
  end CRC;

  function CRC( Data: in Octet_Array ) return CRC32 is
    Result : CRC32 := Init_Value;
  begin
    -- process input octet by octet
    for C of Data loop
      Result := Shift_Right(Result, 8) xor
                  Lookup( CRC32(C) xor (Result and LSB_MASK));
    end loop;
    -- reflect result
    Result := Result xor Xor_Out;

    return Result;
  end CRC;

end CRC32;

Other than the above, the .vpatch for this chapter contains also some basic testing of the read/write methods for Serpent keysets:

  -- Tests for the serialization of data structures in SMG Protocol
 -- S.MG, 2018

with RNG;
with Data_Structs; use Data_Structs;
with Messages; use Messages;
with Interfaces; use Interfaces;
with System;
with System.Storage_Elements; use System.Storage_Elements;
with Ada.Text_IO; use Ada.Text_IO;

package body Test_Serializing is

  procedure Serialize_Keyset_SS is
    Msg  : Serpent_Msg;
    KSet : Serpent_Keyset(5);
    LSB : Interfaces.Unsigned_8 := 16#01#;
    MSB : Interfaces.Unsigned_8 := 16#80#;
    LMSB: Interfaces.Unsigned_8 := 16#81#;
    Counter : Interfaces.Unsigned_16 := 101;
    NewSet: Serpent_Keyset;
    NewCounter: Interfaces.Unsigned_16:=0;
  begin
    Put_Line("Generating the Serpent Keys...");
    -- fill a set of Serpent Keys
    for I in 1..KSet.N loop
      RNG.Get_Octets(KSet.Keys(Interfaces.Unsigned_8(I)));
    end loop;
    KSet.Flag := LSB;

    Put_Line("Writing the keys to message...");
    -- write keyset to serpent message
    Write_SKeys_SMsg( KSet, Counter, Msg );

    Put_Line("Reading keys back from message...");
    -- read keyset from message
    Read_SKeys_SMsg( Msg, Counter, NewSet );

    Put_Line("Comparing the keysets...");
    -- compare the two keysets
    if NewSet /= KSet then
      Put_Line("FAIL: keysets are different!");
    else
      Put_Line("PASS: keysets are the same!");
    end if;

    Put_Line("Attempting to read from mangled message");
    begin
      Msg(Msg'First) := Msg(Msg'First)+25;
      Read_SKeys_SMsg( Msg, Counter, NewSet);
      Put_Line("FAIL: read failed to raise invalid message exception!");
    exception
      when Invalid_Msg =>
        Put_Line("PASS: exception correctly raised for invalid message");
    end;
  end Serialize_Keyset_SS;

end Test_Serializing;

As I don't currently have any access to a Big Endian computer, I couldn't quite test the whole thing properly. If you have a Big Endian computer and get around to test this code, please let me know how it behaves.

The above method seems to me robust at least and reasonably easy to follow. Feel free however to loudly disagree and shoot it down in the comments below - I'll gladly read any better proposal. As mentioned in the very beginning of this post, this is not exactly what I hoped to obtain although it is the thing I have so far and therefore the thing I'm publishing - to be discussed and improved on, hopefully.

The .vpatch and my signature for it are as usual on my Reference Code Shelf and linked below for your convenience:


  1. The official solution for anything touching serialization seems to boil down to "Streams or gtfo" apparently. And for my specific case here I don't quite see the point of bringing in the whole Streams mammouth

November 4, 2018

SMG Comms Chapter 6: Packing and Unpacking RSA

Filed under: Coding, SMG_Comms — Diana Coman @ 1:38 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

Building on the new convenient way to simply call rsa+oaep encrypt/decrypt from Ada, this chapter provides the Pack/Unpack functions that convert between SMG Comms RSA packets (each of them 1470 octets long, effectively several RSA-encrypted chunks glued together) and SMG Comms messages aka the actual content of the packet from the game's point of view. To do this, the Pack/Unpack functions simply chop the input into fixed-size parts, apply rsa+oaep encrypt/decrypt as suitable and then glue back the results in order, providing the whole thing as final output. Note that from the point of view of Pack/Unpack, both packets and messages have fixed lengths, as specified! In other words: the padding that might be added to data structures to get the whole thing to the specified length of a RSA message is preserved - that is no concern at this layer at all1. In a nutshell: Unpack removes any padding that Pack added but preserves any other padding that was added *before* calling Pack. In turn, Pack expects as input a fixed-size array of octets aka a RSA message as defined in Raw_Types - and the size of this is simply a multiple of the maximum size that one single RSA-encrypted operation can handle.

While the above fixed-size helps considerably to keep the whole thing quite easy to follow and clear, things are still a bit more gnarly than with Serpent, there's no escaping that. In particular, the RSA Decrypt operation CAN fail visibly - when the decoded "length" of the actual content turns out to be larger than maximum/provided space. For this reason, the Unpack function has a "Success" flag that gets set to false when output is essentially undefined. Note that the Unpack will fail in this way as soon as one of the chunks fails the RSA+OAEP decrypt in any way - there is no point in even attempting to recover some "partial" thing. Even more importantly: a True value for the Success flag does NOT indicate in any way that the result obtained is valid - all it says is that the RSA+OAEP decrypt operations completed successfully and the result is the one returned but whether that's a valid data structure or not is not a concern at this point (nor could it be since Unpack can't check such a thing in any way).

The .vpatch for this chapter includes the following main changes/additions:

  • A new note is added next to the offending Keccak line, describing the proposed way2 to preserve the No_Implicit_Conditionals restriction if desired. As mentioned in the forum, I do not quite see the need for this change at this moment but perhaps it will be needed at a later time, so there it is, right where it'll come in handy.
  • There's a correction of the value for the constant RSA_MSG_OCTETS. The new, correct value matches the fact that this is meant to be the fixed-size length of a SMG Comms RSA message rather than (as it was before) the max length that can be encrypted with RSA in one operation. So the RSA_MSG_OCTETS is calculated based on other specification-defined constants and it is a multiple of the max length that can be encrypted with RSA in one operation.
  • As part of doing the above correction of RSA_MSG_OCTETS, I also moved the initialization of some of the OAEP constants into Raw_Types (from OAEP packet where they were before). More precisely, the OAEP packet still uses its own internal constants (e.g. the "reserved" octets spelling "TMSR-RSA") but it initialises those (at elaboration time) with the values that it reads from Raw_Types. This reflects the fact that the OAEP packet in this project is the one using the Raw_Types that the protocol defines, not the other way around (in other words: OAEP does not define/impose those values - it's SMG Comms that does!).
  • The Packing package lost its "Pure" status (i.e. the compiler can't rely on it having no internal state and can't therefore cache calls to its method). The reason for this is at root the need for random padding for RSA since yes, can't cache the call to a method that has to use different, random padding at every call. In any case, for anyone interested, the specific route to follow this is that the Pack method for RSA calls OAEP+RSA which in turn use the non-pure Sequential_IO (via the RNG package) to read random octets from a Fuckgoats.
  • The new Pack and Unpack methods for RSA messages <-> packets conversions are in packing.ads/adb.
  • Corresponding tests for the Pack/Unpack methods are added to test_packing.ads/adb and called from testall.adb.

The signatures of the new Pack/Unpack methods in packing.ads are the following:

  -- Packing a RSA message into RSA packet, using the given key
  function Pack( Msg : in Raw_Types.RSA_Msg;
                 K   : in RSA_OAEP.RSA_pkey)
                return Raw_Types.RSA_Pkt;

  -- Unpacking a RSA packet into contained message, using the given key
  function Unpack( Pkt     : in Raw_Types.RSA_Pkt;
                   K       : in RSA_OAEP.RSA_skey;
                   Success : out Boolean)
                  return Raw_Types.RSA_Msg;

And the corresponding code for the new Pack/Unpack in packing.adb:

  -- Packing a RSA message into RSA packet, using the given key
  function Pack( Msg : in Raw_Types.RSA_Msg;
                 K   : in RSA_OAEP.RSA_pkey)
                return Raw_Types.RSA_Pkt is

  -- a chunk that can be processed via rsa+oaep at any given time
    Chunk: Raw_Types.Octets(1..Raw_Types.OAEP_MAX_LEN) := (others => 0);

  -- number of chunks in the message to process
  -- NO incomplete chunks will be processed!
  -- NB: values are set so that there are no incomplete blocks here
    N   : constant Natural := Msg'Length / Chunk'Length;

  -- intermediate result, as obtained from rsa_oaep
    Encr: Raw_Types.RSA_len := (others => 0);

  -- final resulting RSA Packet
    Pkt : Raw_Types.RSA_Pkt := (others => 0);
  begin
    -- there should ALWAYS be precisely N chunks in Msg to feed to rsa_oaep
    -- process chunks of Msg one at a time
    for I in 1..N loop
      -- get current chunk
      Chunk := Msg(Msg'First + (I-1) * Chunk'Length ..
                   Msg'First + I * Chunk'Length - 1 );
      -- call rsa oaep encrypt on current chunk
      RSA_OAEP.Encrypt( Chunk, K, Encr );
      -- copy result to its place in final packet
      Pkt( Pkt'First + (I-1) * Encr'Length ..
           Pkt'First + I * Encr'Length - 1 ) := Encr;
    end loop;
    -- return final result
    return Pkt;
  end Pack;

  -- Unpacking a RSA packet into contained message, using the given key
  function Unpack( Pkt     : in Raw_Types.RSA_Pkt;
                   K       : in RSA_OAEP.RSA_skey;
                   Success : out Boolean)
                  return Raw_Types.RSA_Msg is
    -- a chunk - basically input for RSA_OAEP.Decrypt
    Chunk : Raw_Types.RSA_len := (others => 0);

    -- intermediate result of rsa_oaep decrypt
    Decr  : Raw_Types.Octets( 1..Raw_Types.OAEP_MAX_LEN ) := (others => 0);
    Len   : Natural;
    Flag  : Boolean;

    -- number of chunks in the packet
    -- NB: there should be only FULL chunks! otherwise -> fail
    N   : constant Natural := Pkt'Length / Chunk'Length;

    -- final resulting message content of the given RSA packet
    Msg : Raw_Types.RSA_Msg := (others => 0);
  begin
    -- initialize Success flag
    Success := True;

    -- process given packet, chunk by chunk
    for I in 1..N loop
      -- get current chunk
      Chunk := Pkt( Pkt'First + (I-1) * Chunk'Length ..
                    Pkt'First + I * Chunk'Length - 1 );
      -- decrypt it via rsa+oaep
      RSA_OAEP.Decrypt( Chunk, K, Decr, Len, Flag );
      -- check result and if ok then copy it to final result at its place
      -- NB: if returned length is EVER less than OAEP_MAX_LEN then -> fail!
      -- the reason for above: there will be undefined bits in the output!
      if Len /= Raw_Types.OAEP_MAX_LEN or (not Flag) then
        Success := False;
        return Msg;
      else
        Msg( Msg'First + (I-1) * Decr'Length ..
             Msg'First + I * Decr'Length - 1 ) := Decr;
      end if;
    end loop;

    -- return obtained message
    return Msg;
  end Unpack;

The new tests for the above, in test_packing.adb:

  procedure Test_Pack_Unpack_RSA is
    Msg       : RSA_Msg;
    Decr_Msg  : RSA_Msg;
    PKey      : RSA_pkey;
    SKey      : RSA_skey;
    Success   : Boolean;
    Pkt       : RSA_Pkt;
    n: String := "C6579F8646180EED0DC1F02E0DDD2B43EABB3F702D79D9928E2CDA5E1D42DF5D9ED7773F80B1F8D9B0DB7D4D00F55647640D70768F63D3CED56A39C681D08D6191F318BB79DC969B470A7364D53335C8318EF35E39D5DF706AB6F2393C6DD2128C142DBAB1806EB35E26C908F0A48419313D2D0F33DD430655DBFEC722899EC21C238E8DB7003430BBC39BAD990F9887F6B03E1344F537EC97389B78DBC656718ACD7B0FDC13DD24534F417BC7A18F077A0C4227354CEA19670331B6CAA3DFC17BBA7E70C14510D9EB3B63F3014994EC87BD23E868C0AE6E9EC55027577F62C0280B2D7DD1135001844923E5455C4566E066B3FDE968C6BC4DC672F229FCE366440403D7A4F4A8BFBA5679B7D0844BA1231277D13A77C9E2B5A1CB138C1B7AB5B4D4832448723A3DE70ED2E86D5FC5174F949A02DE8E404304BEB95F9BF40F3AA3CA15622D2776294BE7E19233406FF563CB8C25A1CB5AADBC1899DA3F2AE38533931FE032EE3232C2CD4F219FADF95B91635C0762A476A4DE5013F4384093F0FB715028D97F93B2E6F057B99EE344D83ADF2686FD5C9C793928BEF3182E568C4339C36C744C8E9CA7D4B9A16AA039CBF6F38CC97B12D87644E94C9DBD6BC93A93A03ED61ECC5874586E3A310E958F858735E30019D345C62E5127B80652C8A970A14B31F03B3A157CD5";
    e: String := "F74D78E382FC19B064411C6C20E0FDB2985F843007A54C7D8400BB459468624126E7D175F397E55C57AF25858EAE2D2952FB7998C119A6103606733EB5E1D27FCA1FACF14ADE94101D383D1B25DA511805569BC344EAD384EDBF3F3A541B34887FE199D99D7F62E6E9D516F88D6F5AD3E020DF04D402A02CC628A0064362FE8516CF7CD6040E9521407AB90EE6B5AFFF9EA9EBB16A7D3407CE81FD3844F519880556AB94AB349C1F3BBB6FDB4C4B377FE4C091EBDC2C3A1BD3AA56382D8D80E7742B5C751008FD6ECDD2EC3B2E3B6C566F698ED672000B403766DD63C3ACBDE16A14FB02E83A2EB6AA018BFC0020401E790DEE24E9";
    d: String := "698DA05DA25B230211EEF0CBA12083A1457B749A11937AC9993859F69A3BF38D575E5166AF2EC88D77F1DF04E68AEA358EACF7659FD4722A4F5A1C8BA7676DA97A9FBA75451152F8F68887D3451A9CCFFFE9EB80979786E37495B17687A6212F77FA616E4C0CD8A8EB7AEB88EA6CCABB7F3E854FB94B35394A09F95F0D6F997947E865CC0606F437C30FE8C48D96FBF5E2F52807BC9E9ED7BBEB23D5C45EDDCD16FE2BF410A9A1E5EF879E71C0D41FAE270C0C5D442860103F8C3944E802F33DB38432F11F763A7AF593656108E4A98A44A8549913CE5DCEC1A6500F280E3190991B2B938561CFACD8BC5183AAC9A4914BFE52C3BE39BB83688E1DE52479107EF8E087DCDB409432FC954C6349407E81DDFB11AE92BABB32A31868597958C9C76E0B4156F380955F0E09C1F3B98BB4CDD59E1B5C7D8CC2AA7491B0D319D219CF459A527CE1AA2729DEC53269653BF0ED3E0253F4451168437E3B069E48350CA4C3EC82134E87135624C768D1330B0D70C6E447FD9945BF06FCB91AA334C0FD8EEF1ADBC15928B3DB62077B537F7E9F468CC95CD5AAFEAE1F760A863B48D07B163F670E2E5B550BB3E960230BA9FDAED9903AE2E669A7F3C4D1F1E25B8E8EDB8CC6E6FD2164E66F4E64ED77BEF1EC9E6CEA5624FD84C0680248746DC1C8187145F3CD2411659DAEAD11D";
    p: String := "CDD6F7673A501FB24C44D56CA1D434F6CB3334E193E02F8E906241906BCB7412DD2159825B24C22002F373E647C2DA62A854F3841C00FD5985D03227CA9B54A69380BA9D63BE738BDF9E65C247E43E1220EEDD9281DCA78B32A4E1B786B7697ED0C3195D5AF2990881B11D6FC9EC9F940067B2DEA2A516FAA5F269C98F0B67628A6D2708515A4A58041AA17A93E4C4DD95C85BC38351DDA1DCF3DFD91C505B22383132649CF9F9233852C7207075BCF43C71038F043F1EC53E9787FB051B7927D020903233C16897B993C8089D8464451F086E756CF20E46CE6ED4A6AC5C327A0AAFBECBAAFD177969E7C952C76A4F4E7C85BF7F63";
    q: String := "F6ACF0790A250802C8D45DAC549CDBEF7806D5877A5DF0069136A458FAC4F0B0858060A873DA6355A965A064A0BC1BBB874872CD7ED89674AD407533041E74BCA317EC73597D335115523F61A05071E5ED81EE2A05331F65D4DC7A25AD7938B124CF03F49154B6693FB0B598B33ABDEF85C599A57A9B7347EAFF82638E1CBC28FCDFFF1FF04A18C2DBF3938395C2F8D1782B43D3A25EF7633B5DDAC89EFD3BAA64D976425A0891E00B876E9DE9FE4B6492B0EA8DFC7C8DEEC61721356EC816295B1BD9CD9DA3E30D2D90DC9CB3987F4BE042104900E036F3044A016749EF910CCFB9F377A90849B4CCCF4471A74E67EF6C814C9467";
    u: String := "854B89ED10F52258D00D6B3FA7F1FD22752804668F51FF7806DB82E22CB8B3AA8448D9B8E9DB14D31A36AEC2BCFA89E341B7334D494E97ED8051244136192233332C4612D963E7B6AF2535FDB7FE97E28DDFEBDFB3E1AFC29D05DBDF37106A817D3AB1864C7F7F247982897EDA6A92BED47D9C68305CD170C7301ACEB05F8A6382E73CC7614B2D8D758669B3A99AB64114809254B0BE21F40341A5B48B9B032603B14875B87EB5E16603FD16552E146A0FC6964958DFC25AA9FFCCD1ED1F4DEAF9FBAA0D7357F5FF0803FEB9BA78E74AC6B3070F417CEC6CFC7A3CF1E305FC7B76B7ED71893999AF797B2EBDE41FE90F076CCEDBFB";
  begin
    -- initialize RSA key with values previously obtained from EuCrypt
    Hex2Octets( n, SKey.n );
    Hex2Octets( e, SKey.e );
    Hex2Octets( d, SKey.d );
    Hex2Octets( p, SKey.p );
    Hex2Octets( q, SKey.q );
    Hex2Octets( u, SKey.u );
    -- copy n and e for public key
    PKey.n := SKey.n;
    PKey.e := SKey.e;
    -- get random data for "message"
    RNG.Get_Octets(Msg);

    -- pack the message into corresponding packet
    Pkt := Packing.Pack( Msg, PKey );
    -- unpack and check the result
    Decr_Msg := Packing.Unpack( Pkt, SKey, Success );
    if (not Success) or (Decr_Msg /= Msg) then
      Put_Line("FAIL: pack/unpack with RSA.");
    else
      Put_Line("PASS: pack/unpack with RSA.");
    end if;

    -- try to unpack a mangled package
    Pkt(Pkt'First + 3) := Pkt(Pkt'First + 3) and 16#AB#;
    Decr_Msg := (others => 0);
    Decr_Msg := Packing.Unpack( Pkt, SKey, Success );
    if Success then
      Put_Line("FAIL: pack/unpack with RSA on mangled package.");
    else
      Put_Line("PASS: pack/unpack with RSA on mangled package.");
    end if;

  end Test_Pack_Unpack_RSA;

The .vpatch with all the above is on my Reference Code Shelf and linked below for your convenience:

UPDATE (5 November 2018): I've changed the tests so that they read the RSA keypair from a file rather than using a hard-coded key; as discussed in the logs, this is the only way I can see to keep all lines in code files under 80 columns while also keeping RSA components as they are i.e. well longer than 80 characters in any sane format. The added procedure simply reads key components in a rather blunt way, character by character and demanding *exactly* as many characters as needed to fill the length of key it expects. This is NOT a generic "key reading" but simply a quick switch of input for the basic tests:

  procedure ReadRSAKey( Filename: in String; Key: out RSA_OAEP.RSA_skey ) is
    package Char_IO is new Ada.Sequential_IO(Character);
    use Char_IO;
    Full : String(1..RSA_len'Length*2) := (others => '0');
    Half : String(1..RSA_half'Length*2) := (others => '0');
    F    : Char_IO.File_Type;
    C    : Character;
  begin
    Open( File => F, Mode => In_File, Name => Filename );

    -- read n
    for I in Full'Range loop
      Read(F, Full(I));
    end loop;
    -- read new line character and convert to hex
    Read(F, C);
    Hex2Octets(Full, Key.n);

    -- read e
    for I in Half'Range loop
      Read(F, Half(I));
    end loop;
    -- read new line character and convert to hex
    Read(F, C);
    Hex2Octets(Half, Key.e);

    -- read d
    for I in Full'Range loop
      Read(F, Full(I));
    end loop;
    -- read new line character and convert to hex
    Read(F, C);
    Hex2Octets(Full, Key.d);

    -- read p
    for I in Half'Range loop
      Read(F, Half(I));
    end loop;
    -- read new line character and convert to hex
    Read(F, C);
    Hex2Octets(Half, Key.p);

    -- read q
    for I in Half'Range loop
      Read(F, Half(I));
    end loop;
    -- read new line character and convert to hex
    Read(F, C);
    Hex2Octets(Half, Key.q);

    -- read u
    for I in Half'Range loop
      Read(F, Half(I));
    end loop;
    Hex2Octets(Half, Key.u);

    -- Close file
    Close( F );

    exception
      when Char_IO.End_Error =>
        Put_Line("ReadRSAKey ERROR: Unexpected end of file in " & Filename);
      when others =>
        Put_Line("ReadRSAKey ERROR: can not open file " & Filename);

  end ReadRSAKey;

Since this change of inputs for tests has nothing to do with either protocol code per se or with advancing the protocol implementation, I'm considering it more of a fix of sorts to this chapter's code and so it'll be a .vpatch by itself, to be found both on the Reference Code Shelf and linked from here:

UPDATE (12 November 2018): I've refactored the tests to extract the reading of a RSA key in a package of its own and thus use it afterwards from everywhere it's needed (i.e. not only in testing oaep_rsa but also in testing the packing for instance). Since this happened after I published Chapter 7, it's a new .vpatch on top:


  1. It will be handled at a higher level, where data structures are populated. 

  2. Thank you, asciilifeform! 

October 31, 2018

SMG Comms Chapter 5: RSA with OAEP from Ada

Filed under: Coding, SMG_Comms — Diana Coman @ 5:35 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

This brings together RSA and OAEP as well as the raw representation as octets of all the things involved: true random padding, RSA key components, plain messages, encrypted outputs, OAEP blocks or Keccak hashes. The main goal was to have some clear and easy to use methods in Ada for doing the whole RSA+OAEP dance from start to finish and then back as needed. Due to the current situation re MPI and RSA, a significant part of this goal involves keeping the C swamp at bay: calling C methods only when there really is at the moment no alternative and in general as few times as possible, making sure that anything built on top of this part of smg_comms can effectively ignore that there is any C code involved at a lower level at all. The reason for insulating the C code away like this is of course that I hope to be able to get rid of it entirely at some point and have everything in Ada without having to touch anything higher up than this part right here. At some point, but not at this point here, not just yet.

At first glance, this chapter was meant to be quite easy and straightforward: glue the different parts together and that's it! As usual, the details were more gnarly and as a result there were a few things I had to change/re-implement in order to get this part into some sort of reasonable shape:

  • I've added a few new subtypes of Octets to Raw_Types: RSA_len and RSA_half to hold a component of precisely the length or, respectively, half of the length set for the modulus of a RSA key. Those are needed to store the RSA keys in raw format aka arrays of octets. While I could also use the generic Octets type, the clear specification of length reflects the expected lengths of each RSA component and brings clarity so it's well worth it in my books.
  • The random padding is now obtained directly from Ada code rather than via C_wrappers and truerandom.c. The new Ada code simply reads from a Fuckgoats that it expects to be available at a pre-set path, *already initialised*1 and ready to read from. This means that random octets do not require anymore a call to C and the corresponding conversions from array of octets to char * and back.
  • The comparison between an OAEP block and the RSA key's modulus is simplified: it's only the top octet of both that is compared. This was discussed previously in the logs but at any rate, the reason for this approach is that it greatly reduces the complexity of the code since it's really a one line compare of 2 8-bit values that are already available as such at the very spot where they are needed! So no more calling the C swamp, converting to char* and then to MPIs and then comparing the full MPIs and returning the result through to Ada. The potential additional OAEP encryption performed at times is considered a price well worth paying in order to avoid the whole Ada->C->MPI->ADA dance that needs to be performed otherwise *each and every time*.
  • The conversions from Ada's array of octets to C's char* are done in local code, octet by octet and strictly on arrays that are precisely the same size. I am aware that GNAT provides in principle methods for such conversions, namely To_Ada and To_C but they still fail to convince me that they provide a better service really. If you think they do though, feel free to state your case in the comments below.
  • The OAEP implementation is the one from EuCrypt but with different access/interface methods: instead of accepting and producing Strings like the EuCrypt version, the OAEP package in SMG COMMS accepts and produces arrays of octets (i.e. Raw_Types.Octets type) since this is effectively what is needed here. Not to mention that I'm not particularly sad to get rid of Strings even when they are fixed size. Obviously, this means that I also replaced the conversions String <-> Bitstream with Octets <-> Bitstream. The rest however stays the same (including the underlying Keccak for hashes).
  • Because of moving all the OAEP+RSA part to Ada and imposing only one-way calls from Ada to C, there was no need anymore for some of the wrappers that were provided in EuCrypt's OAEP and rsa_oaep code. Those were simply discarded so that the code you get by pressing this patch includes simply what is needed for SMG Comms and nothing more.
  • Whenever the Ada code (specifically the rsa+oaep code) retrieves an array of octets from C, it will basically reverse the "normalization" that the MPI lib performs so that one doesn't end up in the idiotic situation where the retrieved value is actually different than the one given to MPI just because of leading zeroes. This is part of the price paid for using the MPI lib and at any rate it should not be the concern of any of the calling code so it's kept well separated so that any callers of RSA_OAEP.Encrypt or RSA_OAEP.Decrypt don't have to deal with it directly: the encryption uses at most msg_max_len octets and produces a fixed-size array while the decrypt similarly requires the same fixed-size array as input and will produce as output the actual length of the plain message and the plain message itself.
  • The "No Implicit Conditionals" restriction for the whole smg comms had to be discarded because of the Keccak implementation - with the restriction applied, compilation of Keccak fails2. At the moment I don't consider this to be a big issue here but feel free to loudly disagree in the comments below - if you have a solid case, I can change my mind.
  • I've updated the relevant tests for RSA (including those from the C_Wrappers package previously published) to use RSA keys of the new size that SMG Comms uses (as opposed to the EuCrypt size). I've added also new tests for the new RSA+OAEP Ada code, of course.

The new Ada code in rng.ads for reading random octets directly from an FG that is already initialised:

 -- True Random octets generator reading from an FG and using Raw_Types.
 -- S.MG, 2018

with Ada.Sequential_IO;
with Interfaces;
with Raw_Types;

package RNG is

  -- for reading from the FG one Octet at a time
  package Octet_IO is new Ada.Sequential_IO(Element_Type =>
                                              Interfaces.Unsigned_8);

  -- path to the FG; NB: the FG has to be initialized prior to using SMG Comms!
  RNG_PATH: constant String := "/dev/ttyUSB0";

  -- exception raised when FG is not accessible / read fails
  FG_Failure : exception;

  -- reads O'Length octets from the FG at RNG_PATH
  procedure Get_Octets( O: out Raw_Types.Octets );
end RNG;

And the corresponding rng.adb, short and sweet:

 -- S.MG, 2018

with Ada.Exceptions; use Ada.Exceptions;

package body RNG is

  procedure Get_Octets( O: out Raw_Types.Octets ) is
    F : Octet_IO.File_Type;
  begin
    begin
      Octet_IO.Open( File => F, Mode => Octet_IO.In_File, Name => RNG_PATH );
      for I in O'Range loop
        Octet_IO.Read( F, O(I) );
      end loop;
      Octet_IO.Close(F);
    exception
      when others => Raise_Exception(FG_Failure'Identity,
                                     "Failed to access default RNG source!");
    end;
  end Get_Octets;

end RNG;

The code for doing oaep+rsa encrypt from Ada, in rsa_oaep.ads:

 -- Ada implementation of RSA with OAEP according to TMSR and Eulora spec
 -- Uses:
 --   - Eulora's raw types (Ada, raw_types.ads)
 --   - Keccak hashes (Ada, keccak.ads/adb)
 --   - OAEP schema (Ada, oaep.ads/adb)
 --   - RNG (Ada, rng.ads/adb) for true random padding
 --   - C wrappers lib (C, c_wrappers/) for:
 --       - MPI (C, mpi/)
 --       - RSA (C, rsa/rsa.c)
 --
 -- S.MG, 2018

with Raw_Types;
with Interfaces.C; use Interfaces.C;

package RSA_OAEP is
  -- exception for mismatched lengths when converting octets <-> char arrays
  Mismatched_Lengths_Error: exception;

  -- public RSA key with n,e stored as raw octets
  type RSA_pkey is
    record
      n : Raw_Types.RSA_len; --public modulus
      e : Raw_Types.RSA_half; --public exponent
    end record; --RSA_pkey

  -- private (secret) RSA key with components stored as raw octets
  type RSA_skey is
    record
      n : Raw_Types.RSA_len;  --public modulus
      e : Raw_Types.RSA_half; --public exponent
      d : Raw_Types.RSA_len;  --secret exponent e*d=1 mod phi; phi=(p-1)*(q-1)
      p : Raw_Types.RSA_half; --prime p
      q : Raw_Types.RSA_half; --prime q
      u : Raw_Types.RSA_half;  --inverse of p mod q; for faster calculations
    end record; --RSA_skey

  -- Encryption RSA+OAEP (i.e. using public key)
  -- NB: at most OAEP.MAX_LEN_MSG octets from Plain will be encrypted!
  -- Relies directly on:
  --   oaep.adb/ads
  --   c_wrappers.c ( RSA )
  --   rng.adb/ads ( random padding )
  procedure Encrypt( Plain: in Raw_Types.Octets;
                     Key  : in RSA_pkey;
                     Encr : out Raw_Types.RSA_len );

  -- Decryption RSA+OAEP (i.e. using private/secret key)
  -- The opposite of Encrypt above.
  -- NB: Plain has to have ENOUGH space for result!
  -- Result can be at most OAEP.MAX_LEN_MSG octets.
  -- Relies directly on:
  --   oaep.adb/ads
  --   c_wrappers.c (RSA)
  -- Plain_Len gives the length in OCTETS of the decrypted message.
  procedure Decrypt( Encr      : in Raw_Types.RSA_len;
                     Key       : in RSA_skey;
                     Plain     : out Raw_Types.Octets;
                     Plain_Len : out Natural;
                     Success   : out Boolean);

 --helper methods:
 -- mainly conversions to/from C's char* and imports from C_wrappers
  -- encrypt with public RSA key given as struct, Ada style
  -- NB: result is potentially 0-led (i.e. at end of Encr not at start!)
  procedure Public_RSA( Plain: in Raw_Types.Octets;
                        Key  : in RSA_pkey;
                        Encr : out Raw_Types.RSA_len);

  -- encrypt with public RSA key given as char arrays, via C_Wrappers
  -- this returns the length of result because C functions trim leading 0s!
  function Public_RSA_C( Encr      : out Interfaces.C.char_array;
                         Encr_Len  : in Integer;
                         Plain     : in Interfaces.C.char_array;
                         Plain_Len : in Integer;
                         RSA_N     : in Interfaces.C.char_array;
                         N_Len     : in Integer;
                         RSA_E     : in Interfaces.C.char_array;
                         E_Len     : in Integer)
                     return Integer;
   pragma Import(C, Public_RSA_C, "public_rsa_octets"); 

  -- decrypt with private RSA key given as struct, Ada style
  -- NB: Plain has to have ENOUGH space!
  -- NB: Result is potentially 0-led (i.e. at the end of Plain, not at start!)
  -- @return actual length of result
  procedure Private_RSA( Encr  : in Raw_Types.RSA_len;
                         Key   : in RSA_skey;
                         Plain : out Raw_Types.Octets);

  -- encrypt with private/secret RSA key given as char arrays (via C_wrappers)
  -- this returns length because C methods trim leading 0s
  function Private_RSA_C( Plain     : out Interfaces.C.char_array;
                          Plain_Len : in Integer;
                          Encr      : in Interfaces.C.char_array;
                          Encr_Len  : in Integer;
                          RSA_N     : in Interfaces.C.char_array;
                          N_Len     : in Integer;
                          RSA_E     : in Interfaces.C.char_array;
                          E_Len     : in Integer;
                          RSA_D     : in Interfaces.C.char_array;
                          D_Len     : in Integer;
                          RSA_P     : in Interfaces.C.char_array;
                          P_Len     : in Integer;
                          RSA_Q     : in Interfaces.C.char_array;
                          Q_Len     : in Integer;
                          RSA_U     : in Interfaces.C.char_array;
                          U_Len     : in Integer)
                      return Integer;
  pragma Import( C, Private_RSA_C, "private_rsa_octets" );

  -- convert from Ada's Octets (array of octets) to C's char* (char_array)

  -- This copies the octets from O to the beginning of A
  -- NB: there are NO checks or memory allocations here!
  -- Caller has to make sure that:
  --   A has allocated space for at least O'Length octets!
  procedure Octets_To_Char_Array( O   : in Raw_Types.Octets;
                                  A   : out Interfaces.C.char_array);

  -- This copies first O'Length characters from A to O
  -- NB: this does NOT allocate /check memory!
  -- Caller has to ensure that:
  --    A has space at least O'Length characters
  procedure Char_Array_To_Octets( A   : in Interfaces.C.char_array;
                                  O   : out Raw_Types.Octets);

end RSA_OAEP;

With the corresponding implementation in rsa_oaep.adb:

 -- S.MG, 2018

with Interfaces; use Interfaces;
with OAEP;
with RNG;

package body RSA_OAEP is

  -- OAEP + RSA with given public key.
  -- Steps:
  -- 1. repeat oaep.encrypt of Plain
  --      until the result has first octet < than first octet of Key's modulus.
  -- 2. RSA.encrypt with Key on the result from 1.
  procedure Encrypt( Plain: in Raw_Types.Octets;
                     Key  : in RSA_pkey;
                     Encr : out Raw_Types.RSA_len ) is
    -- block obtained through OAEP padding of given input Plain
    Blk     : OAEP.OAEP_Block := ( others => 0 );
    Entropy : OAEP.OAEP_Block;
    Len     : constant Natural := entropy'Length;
  begin
    -- loop cond: 1st octet of oaep block is < 1st octet of key's modulus
    loop
      -- get random bits via RNG (FG)
      RNG.Get_Octets( Entropy );

      -- oaep encrypt
      OAEP.OAEP_Encrypt( Plain, Entropy, Blk );

      -- check that the oaep block is suitable i.e. comparison of 1st octet
      if Blk(Blk'First) < Key.n(Key.n'First) then exit; end if;

    end loop; -- oaep block < modulus

    -- RSA encrypt
    Public_RSA( Blk, Key, Encr );
  end; --Encrypt (OAEP+RSA)

  -- RSA+OAEP Decrypt
  -- Steps:
  -- 1. RSA Decrypt (secret key)
  -- 2. OAEP Decrypt
  procedure Decrypt( Encr : in Raw_Types.RSA_len;
                     Key  : in RSA_skey;
                     Plain: out Raw_Types.Octets;
                     Plain_Len: out Natural;
                     Success: out Boolean) is
    Blk     : OAEP.OAEP_Block;
    Msg     : OAEP.OAEP_HALF;
    Msg_Len : Natural;
  begin
    -- RSA decrypt with provided secret key
    -- NB: result HAS TO BE AN OAEP BLOCK HERE! Even if - potentially - 0 led!
    Private_RSA( Encr, Key, Blk );

    -- OAEP decrypt
    OAEP.OAEP_Decrypt(Blk, Msg_Len, Msg, Success);
    -- switch to Length in OCTETS!
    Msg_Len := Msg_Len / 8;

    -- check that result FITS in given output array - otherwise, fail
    if Msg_Len > Plain'Length then
      Success := FALSE;
      Plain_Len := Msg_Len; --to give a clue to caller
    else
      Plain_Len := Msg_Len;
      Plain( Plain'First ..
             Plain'First + Plain_Len-1) := Msg( Msg'First ..
                                                Msg'First + Plain_Len -1 );
    end if;
  end Decrypt;

  -- helper methods
  -- encrypt with public RSA key given as struct, Ada style
  procedure Public_RSA( Plain: in Raw_Types.Octets;
                        Key  : in RSA_pkey;
                        Encr : out Raw_Types.RSA_len) is
    Encr_char  : char_array( size_t(Encr'First) .. size_t(Encr'Last));
    Plain_char : char_array( size_t(Plain'First) .. size_t(Plain'Last));
    N_char     : char_array( size_t(Key.n'First) .. size_t(Key.n'Last));
    E_char     : char_array( size_t(Key.e'First) .. size_t(Key.e'Last));
    out_len    : Integer;
  begin
    -- convert to char array
    Octets_To_Char_Array( Plain, Plain_char );
    Octets_To_Char_Array( Key.n, N_char );
    Octets_To_Char_Array( Key.e, E_char );
    -- call C imported function
    out_len := Public_RSA_C( Encr_char  , Encr'Length,
                             Plain_char , Plain'Length,
                             N_char     , Key.n'Length,
                             E_char     , Key.e'Length);
    -- convert back to octets
    Char_Array_To_Octets( Encr_char, Encr );
    -- C code trims leading 0s -> need to move octets if out_len0);
    end if;
    -- no need to return anything!
  end Public_RSA;

  procedure Private_RSA( Encr  : in Raw_Types.RSA_len;
                         Key   : in RSA_skey;
                         Plain : out Raw_Types.Octets) is
    Plain_Char : char_array( size_t(Plain'First) .. size_t(Plain'Last) );
    Plain_Len  : Integer;
    Encr_Char  : char_array( size_t(Encr'First)  .. size_t(Encr'Last) );
    N_Char     : char_array( size_t(Key.n'First) .. size_t(Key.n'Last) );
    E_Char     : char_array( size_t(Key.e'First) .. size_t(Key.e'Last) );
    D_Char     : char_array( size_t(Key.d'First) .. size_t(Key.d'Last) );
    P_Char     : char_array( size_t(Key.p'First) .. size_t(Key.p'Last) );
    Q_Char     : char_array( size_t(Key.q'First) .. size_t(Key.q'Last) );
    U_Char     : char_array( size_t(Key.u'First) .. size_t(Key.u'Last) );
  begin
    -- convert key and encrypted message to C char_arrays
    Octets_To_Char_Array( Encr, Encr_Char );
    Octets_To_Char_Array( Key.n, N_Char );
    Octets_To_Char_Array( Key.e, E_Char );
    Octets_To_Char_Array( Key.d, D_Char );
    Octets_To_Char_Array( Key.p, P_Char );
    Octets_To_Char_Array( Key.q, Q_Char );
    Octets_To_Char_Array( Key.u, U_Char );
    -- call RSA decrypt via C_Wrappers
    Plain_Len := Private_RSA_C( Plain_Char, Plain'Length,
                                Encr_Char , Encr'Length,
                                N_Char    , Key.n'Length,
                                E_Char    , Key.e'Length,
                                D_Char    , Key.d'Length,
                                P_Char    , Key.p'Length,
                                Q_Char    , Key.q'Length,
                                U_Char    , Key.u'Length);
    -- convert result back to Octets
    Char_Array_To_Octets( Plain_Char, Plain );
    -- if length < OAEP_Block'Length,it's 0-led and got trimmed, so move it
    if Plain_Len < Plain'Length then
      Plain(Plain'Last-Plain_Len+1..Plain'Last):= Plain(Plain'First ..
                                                   Plain'First + Plain_Len -1);
      Plain(Plain'First .. Plain'Last-Plain_Len) := (others => 0);
    end if;
    -- no need to return anything!
  end Private_RSA;

  procedure Octets_To_Char_Array( O   : in Raw_Types.Octets;
                                  A   : out Interfaces.C.char_array) is
  begin
    -- check that lengths ARE THE SAME!
    if A'Length /= O'Length then
      raise Mismatched_Lengths_Error;
    end if;
    -- copy values over octet by octet
    for I in 0 .. O'Length-1 loop
      A( A'First + Interfaces.C.size_t( I )) :=
                     Interfaces.C.Char( Character'Val(O(O'First + I)));
    end loop;
  end Octets_To_Char_Array;

  procedure Char_Array_To_Octets( A   : in Interfaces.C.char_array;
                                  O   : out Raw_Types.Octets) is
  begin
    -- check that lengths ARE THE SAME!
    if A'Length /= O'Length then
      raise Mismatched_Lengths_Error;
    end if;
    -- copy values over octet by octet
    for I in 0..O'Length -1 loop
      O( O'First + I ) := Character'Pos( Character(
                             A( A'First + Interfaces.C.size_t( I )) ));
    end loop;
  end Char_Array_To_Octets;

end RSA_OAEP;

The new oaep.ads using Raw_Types.Octets:

-- Implementation of TMSR's OAEP with Keccak as hash function
-- NB: this uses Eulora's protocol constants (esp. RSA key length) and types.
--
-- S.MG, 2018

with Keccak;    -- Keccak is used as hash function
with Raw_Types; -- Eulora's protocol raw types and constant values

with Interfaces; use Interfaces; -- for Unsigned_8 type and bit-level ops

package OAEP is
  pragma Pure( OAEP ); -- stateless, no side effects -> can cache calls

  -- constants for OAEP
  OAEP_LENGTH_OCTETS : constant := Raw_Types.RSA_KEY_OCTETS;
  OAEP_LENGTH_BITS   : constant := OAEP_LENGTH_OCTETS * 8;
  OAEP_HALF_OCTETS   : constant := OAEP_LENGTH_OCTETS / 2;
  TMSR_STR           : constant String := "TMSR-RSA";
    -- "TMSR-RSA" as unsigned_8 values:
  TMSR               : constant Raw_Types.Octets := (84,77,83,82,45,82,83,65);
  MAX_LEN_MSG        : constant Natural := OAEP_HALF_OCTETS -
                                           TMSR_STR'Length - 3;

  -- subtypes for OAEP encrypt/decrypt
  subtype OAEP_Block is Raw_Types.Octets( 1 .. OAEP_LENGTH_OCTETS );
  subtype OAEP_HALF is Raw_Types.Octets( 1 .. OAEP_HALF_OCTETS );

  -- padding & formatting of maximum MAX_LEN_MSG octets of the given input
  -- uses TMSR's OAEP schema:
  -- 1.format M00 as: [random octet][sz1][sz2]"TMSR-RSA"[random]*Message
  --    where sz1 and sz2 store the length of the message in bits
  --    the random octets before message are padding to make OAEP_LENGTH_OCTETS
  -- 2. R = OAEP_HALF_OCTETS random bits
  -- 3. X = M00 xor hash(R)
  -- 4. Y = R xor hash(X)
  -- 5. Result is X || Y
  -- NB: the Entropy parameter should be random octets from which this method
  -- will use as many as required for the OAEP encryption of given Msg
  -- NB: at MOST MAX_LEN_MSG octets of Msg! (Msg at most MAX_LEN_MSG*8 bits!)
  procedure OAEP_Encrypt( Msg      : in Raw_Types.Octets;
                          Entropy  : in OAEP_Block;
                          Output   : out OAEP_Block);

  -- This is the opposite of OAEP_Encrypt above.
  -- @param Encr - an OAEP block previously obtained from OAEP_Encrypt
  -- @param Len - this will hold the length of the obtained message (in bits!)
  -- @param Output - the first Len octets of this are the recovered message
  -- @param Success - set to TRUE if message was recovered, false otherwise
  -- NB: when Success is FALSE, both Len and Output have undefined values
  procedure OAEP_Decrypt( Encr    : in OAEP_Block;
                          Len     : out Natural;
                          Output  : out OAEP_HALF;
                          Success : out Boolean);

private
  -- gnat-specific methods for bit-level operations
	function Shift_Right( Value  : Unsigned_8;
                        Amount : Natural )
                        return Unsigned_8;
  pragma Import(Intrinsic, Shift_Right); 

	function Shift_Left( Value  : Unsigned_8;
                        Amount : Natural )
                        return Unsigned_8;
  pragma Import(Intrinsic, Shift_Left); 

  -- helper method: xor 2 half-oaep blocks
  function XOR_Octets(A : in OAEP_HALF;
                      B : in OAEP_HALF)
                     return OAEP_HALF;

  -- conversions between bitstream and string
  -- NB: caller has to ensure correct size of output parameter! no checks here.
  procedure ToOctets   ( B : in Keccak.Bitstream;
                         O : out Raw_Types.Octets );

  procedure ToBitstream( O : in Raw_Types.Octets;
                         B : out Keccak.Bitstream );

  -- wrapper for Sponge to use Octets for input/output
  procedure HashKeccak( Input     : in Raw_Types.Octets;
                        Output    : out Raw_Types.Octets;
                        Block_Len : in Keccak.Keccak_Rate :=
                                       Keccak.Default_Bitrate);

end OAEP;

And the implementation in oaep.adb:

-- S.MG, 2018

package body OAEP is

  -- padding & formatting of maximum MAX_LEN_MSG*8 bits of the given input
  -- uses TMSR's OAEP schema:
  -- 1.format M00 as: [random octet][sz1][sz2]"TMSR-RSA"[random]*Message
  --    where sz1 and sz2 store the length of the message in bits
  --    the random octets before message are padding to make OAEP_LENGTH_OCTETS
  -- 2. R = OAEP_HALF_OCTETS random bits
  -- 3. X = M00 xor hash(R)
  -- 4. Y = R xor hash(X)
  -- 5. Result is X || Y
  -- NB: the Entropy parameter should be random octets from which this method
  -- will use as many as required for the OAEP encryption of given Msg
  -- NB: at MOST MAX_LEN_MSG octets of Msg! (Msg at most 1960 bits)
  procedure OAEP_Encrypt( Msg     : in Raw_Types.Octets;
                          Entropy : in OAEP_Block;
                          Output  : out OAEP_Block) is
    M00    : OAEP_HALF;
    R      : OAEP_HALF;
    HashR  : OAEP_HALF;
    X      : OAEP_HALF;
    HashX  : OAEP_HALF;
    Y      : OAEP_HALF;
    MsgLen : Natural;
    PadLen : Natural;
  begin
    -- calculate maximum length of msg and needed amount of padding
    -- make sure also that only MAX_LEN_MSG octets at most are used from Msg
    MsgLen := Msg'Length;                          -- real msg length
    if MsgLen > MAX_LEN_MSG then
      MsgLen := MAX_LEN_MSG;  --only first MAX_LEN_MSG octets used
      PadLen := 0;            --no padding needed
    else
      PadLen := MAX_LEN_MSG - MsgLen; -- add padding as needed
    end if;

    -- step 1: header and format to obtain M00
      -- first octet is random bits
    M00( M00'First ) := Entropy( Entropy'First );

      -- next 2 octets hold the used length of Msg (number of octets)
    M00( M00'First + 2) := Unsigned_8( ( MsgLen * 8 ) mod 256 );
    M00( M00'First + 1) := Unsigned_8( ( (MsgLen * 8 ) / 256 ) mod 256 );

      -- next 8 octets are reserved for later use, currently "TMSR-RSA"
    M00( M00'First + 3 .. M00'First + 10 ) := TMSR;

      -- random bits for padding, if Msg is less than maximum length
    for I in 1 .. PadLen loop
      M00( M00'First + 10 + I ) := Entropy( Entropy'First + I );
    end loop;

      -- the message itself
    M00( M00'Last - MsgLen + 1 .. M00'Last ) :=
                               Msg( Msg'First .. Msg'First + MsgLen - 1 );

    -- step 2: R = Raw_Types.OAEP_HALF_OCTETS random octets
    -- can take LAST octets from given entropy as they are NOT used before
    -- (even if original message was empty, padding uses at most half - 10
    --   while entropy has full block length)
    R := Entropy( Entropy'Last - OAEP_HALF_OCTETS + 1 .. Entropy'Last );

    -- step 3: X = M00 xor hash(R)
    HashKeccak( R, HashR );
    X := XOR_Octets(M00, HashR);

    -- step 4: Y = R xor hash(X)
    HashKeccak( X, HashX );
    Y := XOR_Octets(R, HashX);

    -- step 5: Output is X || Y
    Output( Output'First .. Output'First + X'Length - 1 ) := X;
    Output( Output'Last - Y'Length + 1 .. Output'Last )   := Y;

  end OAEP_Encrypt;

  procedure OAEP_Decrypt( Encr    : in OAEP_Block;
                          Len     : out Natural;
                          Output  : out OAEP_HALF;
                          Success : out Boolean ) is
    X, Y, M, R   : OAEP_HALF;
    HashX, HashR : OAEP_HALF;
    LenOctets    : Natural;
  begin
    -- step 1: separate X and Y
    X := Encr( Encr'First .. Encr'First + X'Length - 1 );
    Y := Encr( Encr'Last - Y'Length + 1 .. Encr'Last );

    -- step 2: R = Y xor hash(X)
    HashKeccak( X, HashX );
    R := XOR_Octets(Y, HashX);

    -- step 3: M = X xor hash(R)
    HashKeccak( R, HashR );
    M := XOR_Octets(X, HashR);

    -- step 4: extract length and message
    Len := Natural(M( M'First + 1 )) * 256 +
           Natural(M( M'First + 2 ));
    LenOctets := Len / 8; 

    if LenOctets > MAX_LEN_MSG or LenOctets < 0 then
      Success := False;  -- error, failed to retrieve message
    else
      Success := True;
      Output( Output'First .. Output'First + LenOctets - 1 ) :=
        M( M'Last - LenOctets + 1 .. M'Last );
    end if;

  end OAEP_Decrypt;

-- private, helper methods
  procedure HashKeccak(Input     : in Raw_Types.Octets;
                       Output    : out Raw_Types.Octets;
                       Block_Len : in Keccak.Keccak_Rate :=
                                      Keccak.Default_Bitrate) is
    BIn  : Keccak.Bitstream( 0 .. Input'Length * 8 - 1 );
    BOut : Keccak.Bitstream( 0 .. Output'Length * 8 - 1 );
  begin
    ToBitstream( Input, BIn );
    Keccak.Sponge( BIn, BOut, Block_Len );
    ToOctets( BOut, Output );
  end HashKeccak;

  function XOR_Octets(A : in OAEP_HALF;
                      B : in OAEP_HALF)
                     return OAEP_HALF is
    R : OAEP_HALF;
  begin
    for I in R'Range loop
      R(I) := A(I) xor B(I);
    end loop;
    return R;
  end XOR_Octets;

  -- conversion between types
  procedure ToOctets(B: in Keccak.Bitstream; O: out Raw_Types.Octets ) is
    Pos : Natural;
  begin
    Pos := B'First;
    for I in O'Range loop
      O(I) := Unsigned_8( B( Pos     ) ) +
              Unsigned_8( B( Pos + 1 ) ) * 2 +
              Unsigned_8( B( Pos + 2 ) ) * 4 +
              Unsigned_8( B( Pos + 3 ) ) * 8 +
              Unsigned_8( B( Pos + 4 ) ) * 16 +
              Unsigned_8( B( Pos + 5 ) ) * 32 +
              Unsigned_8( B( Pos + 6 ) ) * 64 +
              Unsigned_8( B( Pos + 7 ) ) * 128;
      Pos := Pos + 8;
    end loop;
  end ToOctets;

  procedure ToBitstream(O: in Raw_Types.Octets; B: out Keccak.Bitstream ) is
    V   : Unsigned_8;
    Pos : Natural;
  begin
    Pos := B'First;
    for I in O'Range loop
      V := O( I );
      B( Pos     ) := Keccak.Bit( V and 1 );
      B( Pos + 1 ) := Keccak.Bit( Shift_Right( V, 1 ) and 1 );
      B( Pos + 2 ) := Keccak.Bit( Shift_Right( V, 2 ) and 1 );
      B( Pos + 3 ) := Keccak.Bit( Shift_Right( V, 3 ) and 1 );
      B( Pos + 4 ) := Keccak.Bit( Shift_Right( V, 4 ) and 1 );
      B( Pos + 5 ) := Keccak.Bit( Shift_Right( V, 5 ) and 1 );
      B( Pos + 6 ) := Keccak.Bit( Shift_Right( V, 6 ) and 1 );
      B( Pos + 7 ) := Keccak.Bit( Shift_Right( V, 7 ) and 1 );

      Pos := Pos + 8;
    end loop;
  end ToBitstream;

end OAEP;

The new tests added to tests/test_rsa_oaep.adb and called from tests/testall.adb, basically calling naked rsa from Ada, OAEP by itself and then rsa+oaep on a few different inputs:

 --S.MG, 2018

with Interfaces; use Interfaces;
with Interfaces.C; use Interfaces.C;
with RSA_OAEP; use RSA_OAEP;
with OAEP; use OAEP;
with Raw_Types; use Raw_Types;
with RNG; use RNG;
with Keccak; use Keccak;

with Ada.Text_IO; use Ada.Text_IO;

package body Test_RSA_OAEP is

  procedure test_char_array is
    S : String := OAEP.TMSR_STR;
    O : Octets := OAEP.TMSR;
    A : char_array(0..O'Length-1) := (others => '0');
    B : Octets(0..O'Length -1) := (others => 0);
    Fail : Boolean := FALSE;
  begin
    Octets_To_Char_Array(O, A);
    Char_Array_To_Octets(A, B);

    if B /= O then
      Put_Line("FAIL: char_array_to_octets");
    else
      Put_Line("PASS: char_array_to_octets");
    end if;

    for I in 0..S'Length-1 loop
      declare
        C : Character := Character(A(A'First + size_t(I)));
        E : Character := S(S'First + I);
      begin
        if C /= E then
          Fail := TRUE;
          Put("Error at pos " & Integer'Image(I) & ": ");
          Put(Integer'Image(Character'Pos(C)));
          Put_Line(" instead of " & Integer'Image(Character'Pos(E)));
        end if;
      end;
    end loop;
    if FAIL then
      Put_Line("FAIL: test octets_to_char_array");
    else
      Put_Line("PASS: test octets_to_char_array");
    end if;
  end test_char_array;

  -- test OAEP encrypt + decrypt
  procedure test_oaep is
    Plain: Octets(1..MAX_LEN_MSG);
    Short: Octets(0..10);
    Encr : OAEP_Block;
    Decr : OAEP_HALF;
    Len  : Natural;
    Entropy: OAEP_Block;
    Success : Boolean;
  begin
    RNG.Get_Octets(Plain);
    RNG.Get_Octets(Entropy);
    RNG.Get_Octets(Short);

    -- test full length message
    OAEP_Encrypt(Plain, Entropy, Encr);
    OAEP_Decrypt(Encr, Len, Decr, Success);

    if not Success or Len/8 /= Plain'Length then
      Put_Line("FAIL: oaep encrypt/decrypt on max len message.");
    else
      if Decr(Decr'First..Decr'First+Len/8-1) /=
              Plain(Plain'First..Plain'First+Len/8-1) then
        Put_Line("FAIL: oaep encrypt/decrypt on max len message - " &
                 "result different from expected.");
      else
        Put_Line("PASS: oaep encrypt/decrypt on max len message.");
      end if;
    end if;

    -- test short message
    OAEP_Encrypt(Short, Entropy, Encr);
    OAEP_Decrypt(Encr, Len, Decr, Success);
    if not Success or Len/8 /= Short'Length then
      Put_Line("FAIL: oaep encrypt/decrypt on short message.");
    else
      if Decr(Decr'First..Decr'First+Len/8-1) /=
             Short(Short'First..Short'First+Len/8-1) then
        Put_Line("FAIL: oaep encrypt/decrypt on short message - " &
                 "result different from expected.");
      else
        Put_Line("PASS: oaep encrypt/decrypt on short message.");
      end if;
    end if;

  end test_oaep;

  -- test JUST RSA (i.e. without oaep) with RSA key pair previously generated
  procedure test_rsa is
    n: String := "C6579F8646180EED0DC1F02E0DDD2B43EABB3F702D79D9928E2CDA5E1D42DF5D9ED7773F80B1F8D9B0DB7D4D00F55647640D70768F63D3CED56A39C681D08D6191F318BB79DC969B470A7364D53335C8318EF35E39D5DF706AB6F2393C6DD2128C142DBAB1806EB35E26C908F0A48419313D2D0F33DD430655DBFEC722899EC21C238E8DB7003430BBC39BAD990F9887F6B03E1344F537EC97389B78DBC656718ACD7B0FDC13DD24534F417BC7A18F077A0C4227354CEA19670331B6CAA3DFC17BBA7E70C14510D9EB3B63F3014994EC87BD23E868C0AE6E9EC55027577F62C0280B2D7DD1135001844923E5455C4566E066B3FDE968C6BC4DC672F229FCE366440403D7A4F4A8BFBA5679B7D0844BA1231277D13A77C9E2B5A1CB138C1B7AB5B4D4832448723A3DE70ED2E86D5FC5174F949A02DE8E404304BEB95F9BF40F3AA3CA15622D2776294BE7E19233406FF563CB8C25A1CB5AADBC1899DA3F2AE38533931FE032EE3232C2CD4F219FADF95B91635C0762A476A4DE5013F4384093F0FB715028D97F93B2E6F057B99EE344D83ADF2686FD5C9C793928BEF3182E568C4339C36C744C8E9CA7D4B9A16AA039CBF6F38CC97B12D87644E94C9DBD6BC93A93A03ED61ECC5874586E3A310E958F858735E30019D345C62E5127B80652C8A970A14B31F03B3A157CD5";
    e: String := "F74D78E382FC19B064411C6C20E0FDB2985F843007A54C7D8400BB459468624126E7D175F397E55C57AF25858EAE2D2952FB7998C119A6103606733EB5E1D27FCA1FACF14ADE94101D383D1B25DA511805569BC344EAD384EDBF3F3A541B34887FE199D99D7F62E6E9D516F88D6F5AD3E020DF04D402A02CC628A0064362FE8516CF7CD6040E9521407AB90EE6B5AFFF9EA9EBB16A7D3407CE81FD3844F519880556AB94AB349C1F3BBB6FDB4C4B377FE4C091EBDC2C3A1BD3AA56382D8D80E7742B5C751008FD6ECDD2EC3B2E3B6C566F698ED672000B403766DD63C3ACBDE16A14FB02E83A2EB6AA018BFC0020401E790DEE24E9";
    d: String := "698DA05DA25B230211EEF0CBA12083A1457B749A11937AC9993859F69A3BF38D575E5166AF2EC88D77F1DF04E68AEA358EACF7659FD4722A4F5A1C8BA7676DA97A9FBA75451152F8F68887D3451A9CCFFFE9EB80979786E37495B17687A6212F77FA616E4C0CD8A8EB7AEB88EA6CCABB7F3E854FB94B35394A09F95F0D6F997947E865CC0606F437C30FE8C48D96FBF5E2F52807BC9E9ED7BBEB23D5C45EDDCD16FE2BF410A9A1E5EF879E71C0D41FAE270C0C5D442860103F8C3944E802F33DB38432F11F763A7AF593656108E4A98A44A8549913CE5DCEC1A6500F280E3190991B2B938561CFACD8BC5183AAC9A4914BFE52C3BE39BB83688E1DE52479107EF8E087DCDB409432FC954C6349407E81DDFB11AE92BABB32A31868597958C9C76E0B4156F380955F0E09C1F3B98BB4CDD59E1B5C7D8CC2AA7491B0D319D219CF459A527CE1AA2729DEC53269653BF0ED3E0253F4451168437E3B069E48350CA4C3EC82134E87135624C768D1330B0D70C6E447FD9945BF06FCB91AA334C0FD8EEF1ADBC15928B3DB62077B537F7E9F468CC95CD5AAFEAE1F760A863B48D07B163F670E2E5B550BB3E960230BA9FDAED9903AE2E669A7F3C4D1F1E25B8E8EDB8CC6E6FD2164E66F4E64ED77BEF1EC9E6CEA5624FD84C0680248746DC1C8187145F3CD2411659DAEAD11D";
    p: String := "CDD6F7673A501FB24C44D56CA1D434F6CB3334E193E02F8E906241906BCB7412DD2159825B24C22002F373E647C2DA62A854F3841C00FD5985D03227CA9B54A69380BA9D63BE738BDF9E65C247E43E1220EEDD9281DCA78B32A4E1B786B7697ED0C3195D5AF2990881B11D6FC9EC9F940067B2DEA2A516FAA5F269C98F0B67628A6D2708515A4A58041AA17A93E4C4DD95C85BC38351DDA1DCF3DFD91C505B22383132649CF9F9233852C7207075BCF43C71038F043F1EC53E9787FB051B7927D020903233C16897B993C8089D8464451F086E756CF20E46CE6ED4A6AC5C327A0AAFBECBAAFD177969E7C952C76A4F4E7C85BF7F63";
    q: String := "F6ACF0790A250802C8D45DAC549CDBEF7806D5877A5DF0069136A458FAC4F0B0858060A873DA6355A965A064A0BC1BBB874872CD7ED89674AD407533041E74BCA317EC73597D335115523F61A05071E5ED81EE2A05331F65D4DC7A25AD7938B124CF03F49154B6693FB0B598B33ABDEF85C599A57A9B7347EAFF82638E1CBC28FCDFFF1FF04A18C2DBF3938395C2F8D1782B43D3A25EF7633B5DDAC89EFD3BAA64D976425A0891E00B876E9DE9FE4B6492B0EA8DFC7C8DEEC61721356EC816295B1BD9CD9DA3E30D2D90DC9CB3987F4BE042104900E036F3044A016749EF910CCFB9F377A90849B4CCCF4471A74E67EF6C814C9467";
    u: String := "854B89ED10F52258D00D6B3FA7F1FD22752804668F51FF7806DB82E22CB8B3AA8448D9B8E9DB14D31A36AEC2BCFA89E341B7334D494E97ED8051244136192233332C4612D963E7B6AF2535FDB7FE97E28DDFEBDFB3E1AFC29D05DBDF37106A817D3AB1864C7F7F247982897EDA6A92BED47D9C68305CD170C7301ACEB05F8A6382E73CC7614B2D8D758669B3A99AB64114809254B0BE21F40341A5B48B9B032603B14875B87EB5E16603FD16552E146A0FC6964958DFC25AA9FFCCD1ED1F4DEAF9FBAA0D7357F5FF0803FEB9BA78E74AC6B3070F417CEC6CFC7A3CF1E305FC7B76B7ED71893999AF797B2EBDE41FE90F076CCEDBFB";
    Plain: OAEP_Block := (others => 0);
    Decr : OAEP_Block := (others => 0);
    Encr : RSA_len;
    pkey: RSA_pkey;
    skey: RSA_skey;
  begin
    -- initialize with RSA pair previously generated
    Hex2Octets( n, skey.n );
    Hex2Octets( e, skey.e );
    Hex2Octets( d, skey.d );
    Hex2Octets( p, skey.p );
    Hex2Octets( q, skey.q );
    Hex2Octets( u, skey.u );
    -- copy n and e for public key
    pkey.n := skey.n;
    pkey.e := skey.e;
    -- get random data
    RNG.Get_Octets(Plain);
    -- make first octet < RSA key's modulus first octet
    Plain(Plain'First) := 16#00#;
    -- naked rsa encrypt/decrypt
Put_Line("Encrypting with RSA public key...");
    Public_RSA( Plain, pkey, Encr );
Put_Line("Decrypting with RSA private key...");
    Private_RSA( Encr, skey, Decr );
Put_Line("Checking...");

    -- check result
    if Decr /= Plain then
      Put_Line("FAIL: RSA encrypt/decrypt result doesn't match plain.");
    else
      Put_Line("PASS: RSA encrypt/decrypt");
    end if;
  end test_rsa;

  -- test rsa+oaep with RSA key pair previously generated
  procedure test_rsa_oaep is
    n: String := "C6579F8646180EED0DC1F02E0DDD2B43EABB3F702D79D9928E2CDA5E1D42DF5D9ED7773F80B1F8D9B0DB7D4D00F55647640D70768F63D3CED56A39C681D08D6191F318BB79DC969B470A7364D53335C8318EF35E39D5DF706AB6F2393C6DD2128C142DBAB1806EB35E26C908F0A48419313D2D0F33DD430655DBFEC722899EC21C238E8DB7003430BBC39BAD990F9887F6B03E1344F537EC97389B78DBC656718ACD7B0FDC13DD24534F417BC7A18F077A0C4227354CEA19670331B6CAA3DFC17BBA7E70C14510D9EB3B63F3014994EC87BD23E868C0AE6E9EC55027577F62C0280B2D7DD1135001844923E5455C4566E066B3FDE968C6BC4DC672F229FCE366440403D7A4F4A8BFBA5679B7D0844BA1231277D13A77C9E2B5A1CB138C1B7AB5B4D4832448723A3DE70ED2E86D5FC5174F949A02DE8E404304BEB95F9BF40F3AA3CA15622D2776294BE7E19233406FF563CB8C25A1CB5AADBC1899DA3F2AE38533931FE032EE3232C2CD4F219FADF95B91635C0762A476A4DE5013F4384093F0FB715028D97F93B2E6F057B99EE344D83ADF2686FD5C9C793928BEF3182E568C4339C36C744C8E9CA7D4B9A16AA039CBF6F38CC97B12D87644E94C9DBD6BC93A93A03ED61ECC5874586E3A310E958F858735E30019D345C62E5127B80652C8A970A14B31F03B3A157CD5";
    e: String := "F74D78E382FC19B064411C6C20E0FDB2985F843007A54C7D8400BB459468624126E7D175F397E55C57AF25858EAE2D2952FB7998C119A6103606733EB5E1D27FCA1FACF14ADE94101D383D1B25DA511805569BC344EAD384EDBF3F3A541B34887FE199D99D7F62E6E9D516F88D6F5AD3E020DF04D402A02CC628A0064362FE8516CF7CD6040E9521407AB90EE6B5AFFF9EA9EBB16A7D3407CE81FD3844F519880556AB94AB349C1F3BBB6FDB4C4B377FE4C091EBDC2C3A1BD3AA56382D8D80E7742B5C751008FD6ECDD2EC3B2E3B6C566F698ED672000B403766DD63C3ACBDE16A14FB02E83A2EB6AA018BFC0020401E790DEE24E9";
    d: String := "698DA05DA25B230211EEF0CBA12083A1457B749A11937AC9993859F69A3BF38D575E5166AF2EC88D77F1DF04E68AEA358EACF7659FD4722A4F5A1C8BA7676DA97A9FBA75451152F8F68887D3451A9CCFFFE9EB80979786E37495B17687A6212F77FA616E4C0CD8A8EB7AEB88EA6CCABB7F3E854FB94B35394A09F95F0D6F997947E865CC0606F437C30FE8C48D96FBF5E2F52807BC9E9ED7BBEB23D5C45EDDCD16FE2BF410A9A1E5EF879E71C0D41FAE270C0C5D442860103F8C3944E802F33DB38432F11F763A7AF593656108E4A98A44A8549913CE5DCEC1A6500F280E3190991B2B938561CFACD8BC5183AAC9A4914BFE52C3BE39BB83688E1DE52479107EF8E087DCDB409432FC954C6349407E81DDFB11AE92BABB32A31868597958C9C76E0B4156F380955F0E09C1F3B98BB4CDD59E1B5C7D8CC2AA7491B0D319D219CF459A527CE1AA2729DEC53269653BF0ED3E0253F4451168437E3B069E48350CA4C3EC82134E87135624C768D1330B0D70C6E447FD9945BF06FCB91AA334C0FD8EEF1ADBC15928B3DB62077B537F7E9F468CC95CD5AAFEAE1F760A863B48D07B163F670E2E5B550BB3E960230BA9FDAED9903AE2E669A7F3C4D1F1E25B8E8EDB8CC6E6FD2164E66F4E64ED77BEF1EC9E6CEA5624FD84C0680248746DC1C8187145F3CD2411659DAEAD11D";
    p: String := "CDD6F7673A501FB24C44D56CA1D434F6CB3334E193E02F8E906241906BCB7412DD2159825B24C22002F373E647C2DA62A854F3841C00FD5985D03227CA9B54A69380BA9D63BE738BDF9E65C247E43E1220EEDD9281DCA78B32A4E1B786B7697ED0C3195D5AF2990881B11D6FC9EC9F940067B2DEA2A516FAA5F269C98F0B67628A6D2708515A4A58041AA17A93E4C4DD95C85BC38351DDA1DCF3DFD91C505B22383132649CF9F9233852C7207075BCF43C71038F043F1EC53E9787FB051B7927D020903233C16897B993C8089D8464451F086E756CF20E46CE6ED4A6AC5C327A0AAFBECBAAFD177969E7C952C76A4F4E7C85BF7F63";
    q: String := "F6ACF0790A250802C8D45DAC549CDBEF7806D5877A5DF0069136A458FAC4F0B0858060A873DA6355A965A064A0BC1BBB874872CD7ED89674AD407533041E74BCA317EC73597D335115523F61A05071E5ED81EE2A05331F65D4DC7A25AD7938B124CF03F49154B6693FB0B598B33ABDEF85C599A57A9B7347EAFF82638E1CBC28FCDFFF1FF04A18C2DBF3938395C2F8D1782B43D3A25EF7633B5DDAC89EFD3BAA64D976425A0891E00B876E9DE9FE4B6492B0EA8DFC7C8DEEC61721356EC816295B1BD9CD9DA3E30D2D90DC9CB3987F4BE042104900E036F3044A016749EF910CCFB9F377A90849B4CCCF4471A74E67EF6C814C9467";
    u: String := "854B89ED10F52258D00D6B3FA7F1FD22752804668F51FF7806DB82E22CB8B3AA8448D9B8E9DB14D31A36AEC2BCFA89E341B7334D494E97ED8051244136192233332C4612D963E7B6AF2535FDB7FE97E28DDFEBDFB3E1AFC29D05DBDF37106A817D3AB1864C7F7F247982897EDA6A92BED47D9C68305CD170C7301ACEB05F8A6382E73CC7614B2D8D758669B3A99AB64114809254B0BE21F40341A5B48B9B032603B14875B87EB5E16603FD16552E146A0FC6964958DFC25AA9FFCCD1ED1F4DEAF9FBAA0D7357F5FF0803FEB9BA78E74AC6B3070F417CEC6CFC7A3CF1E305FC7B76B7ED71893999AF797B2EBDE41FE90F076CCEDBFB";
    Plain: Octets(1..MAX_LEN_MSG) := (others=>20);
    Short: Octets(1..10);
    Decr : RSA_len;
    Encr : RSA_len;
    pkey: RSA_pkey;
    skey: RSA_skey;
    Success: Boolean;
    Len : Natural;
  begin
    -- initialize with RSA pair previously generated
    Hex2Octets( n, skey.n );
    Hex2Octets( e, skey.e );
    Hex2Octets( d, skey.d );
    Hex2Octets( p, skey.p );
    Hex2Octets( q, skey.q );
    Hex2Octets( u, skey.u );
    -- copy n and e for public key
    pkey.n := skey.n;
    pkey.e := skey.e;

    -- test with 0 message of length Plain'Length
    RSA_OAEP.Encrypt(Plain, pkey, Encr);
    RSA_OAEP.Decrypt(Encr, skey, Decr, Len, Success);
    if (not Success) or Len /= Plain'Length
         or Plain /= Decr(Decr'First..Decr'First+Plain'Length-1) then
      Put_Line("FAIL: RSA_OAEP on max len message 20-filled.");
    else
      Put_Line("PASS: RSA_OAEP on max len message 20-filled.");
    end if;

    -- get random data for "plain" message
    RNG.Get_Octets(Plain);
    RSA_OAEP.Encrypt(Plain, pkey, Encr);
    RSA_OAEP.Decrypt(Encr, skey, Decr, Len, Success);
    if (not Success) or Len /= Plain'Length
         or Plain /= Decr(Decr'First..Decr'First+Plain'Length-1) then
      Put_Line("FAIL: RSA_OAEP on random data of max length.");
    else
      Put_Line("PASS: RSA_OAEP on random data of max length.");
    end if;

    -- get random data for "short" message
    RNG.Get_Octets(Short);
    RSA_OAEP.Encrypt(Short, pkey, Encr);
    RSA_OAEP.Decrypt(Encr, skey, Decr, Len, Success);
    if (not Success) or Len /= Short'Length
         or Short /= Decr(Decr'First..Decr'First+Short'Length-1) then
      Put_Line("FAIL: RSA_OAEP on random data of short length.");
    else
      Put_Line("PASS: RSA_OAEP on random data of short length.");
    end if; 

  end test_rsa_oaep;

-- helper methods
  procedure Hex2Octets( Hex: in String; O: out Raw_Types.Octets ) is
    S : String := "16#AA#";
    -- to make sure that input String has EVEN number of chars (ie full octets)
    H : String(1..Hex'Length+Hex'Length mod 2) := (others=>'0');
  begin
    -- first char is 0 if needed to cover full octet...
    H(H'Length-Hex'Length+1..H'Length) := Hex;
    O := (others => 0);
    for I in 0 .. H'Length/2-1 loop
      S := "16#" & H(H'First + I*2 .. H'First + I*2 + 1) & "#";
      O(O'Last - H'Length/2 + 1 + I) := Unsigned_8'Value(S);
    end loop;
  end Hex2Octets;

  procedure PrintOctets( O: in Raw_Types.Octets; Title: in String ) is
  begin
    Put_Line(Title);
    for V of O loop
      Put(Unsigned_8'Image(V) & " ");
    end loop;
    New_Line;
  end PrintOctets;

end Test_RSA_OAEP;

The full .vpatch for this chapter is quite long at 1714 lines but a significant part of that is essentially imported from EuCrypt (Keccak most notably but also the main part of OAEP). As usual, the .vpatch and my signature for it can be found on my Reference Code Shelf or directly from the links below:


  1. Yes, I adopted Stanislav's approach to reading from the FG. I also had a look at his code at www.loper-os.org/?p=2175  

  2. The exact line is this one: http://www.dianacoman.com/2018/01/18/eucrypt-chapter-6-keccak-transformations/#selection-159.735-159.812 and it's been pointed out already (in the logs, where else?) that there is an implicit check against div by 0 which could be avoided simply by rewriting the line as Output(X, Y) := Rotate_Left(Input(X,Y), ( (T+1)*(T+2)/2) and (Z_Length -1) );  

October 25, 2018

SMG Comms Chapter 4: C Wrappers for RSA and MPI

Filed under: Coding, SMG_Comms — Diana Coman @ 7:28 p.m.

~ This is a work in progress towards an Ada implementation of Eulora's communication protocol. Start with Chapter 1.~

The C part of SMG Comms is - unsurprisingly - a pain in the ass and a three-pointed thorn in the backside:
1. All the RSA operations are coded in C because of the heavy dependency on the MPI lib. This is not something I can change at the moment since I don't yet have an Ada implementation of long integer arithmetics. Sigh.
2. The OAEP padding schema has to be called repeatedly until the result is indeed (when considered as a very large number) a value smaller than the RSA key's modulus. So although the OAEP padding itself (including underlying Keccak are implemented in Ada), the comparison between obtained value and the modulus of the key goes through the MPI swamp so it forces a call to C. Nevertheless, the "implemented in Ada" and "repeat until the result CAN be encrypted with given RSA key" are both absolutely right and proper and as they should be so there's nothing to change here. Hooray.
3. As a result of 1 and 2 above, the code flow can easily get ugly because one ends up with calls from Ada to C (to do RSA operations for instance) but *also* from C to Ada (to do Keccak, if one aims to provide oaep+rsa from C directly). This is currently the sad state of the wrappers provided as first attempt in EuCrypt for rsa-oaep. Nevertheless, it's horribly ugly, difficult to follow and overall unsanitary so it has to go.

My current solution to the above issue with rsa-oaep is to isolate properly the C code and stick to calls in only one direction, namely from Ada to C. To do this however, I first need to write some more ...C code. Eurgh. The new C code is a set of wrappers for RSA and MPI methods that I need to call from Ada (essentially from what will be Ada implementation of the rsa-with-oaep part): those wrappers provide a sane interface that matches SMG Comms' raw types layer - basically those methods can be called with arrays of octets as input/output without forcing onto the rest of the code MPI structures, unpredictable memory allocation and other abominations.

The above approach has two very important benefits: the code is much easier to follow since there is no ugly back and forth between Ada and C; the C part (MPI especially but also the RSA layer) is properly separated from the Ada part so that it should be relatively easy to simply replace it with a sane Ada implementation whenever such a wonder becomes available. The main downside to this is that it adds conversions from arrays of octets to MPIs and back as part and parcel of using the wrappers but I still think it's the better option available at this moment.

The C wrappers that I need for SMG Comms include the comparison of 2 MPI values (so mpi_cmp from MPI lib), the encryption and decryption with given RSA key as well as a procedure to generate a RSA key (from rsa lib). In addition, I wrote an mpi_to_octets function that does as the name suggests: reads the raw octets stored in the MPI and returns them to the caller. The reason for this to feature as a separate function is the sad fact that MPI's own "get buffer" allocates memory and then expects the caller to de-allocate it. Rather than import this headache and propagate it all through the new code, I wrapped it up in here and dealt with it: callers of mpi_to_octets should allocate their own memory and de-allocate it accordingly. The signatures of those wrapper methods are in all their enumerated-arrays-of-octets glory (RSA key components are passed as array of octets just as well as anything else) in c_wrappers/c_wrappers.c:

//Wrapper methods for C implementations of RSA and MPI.
//To be used / called from Ada so that the C part remains well separated and
//  can therefore be swapped easily at a later stage for something sane.
//S.MG, 2018

#include "mpi.h"

//Comparison of 2 arrays of octets interpreted as MPIs.
//This method creates 2 MPIs out of the given arrays of octes and then
//  calls the mpi_cmp method from mpi/mpi-cmp.c, returning its result.
// ************************************************************************
// ***NB: Make SURE that a and b have indeed allocated at least len_a and
// *****    len_b octets respectively! NO CHECKS PERFORMED BY THIS METHOD!
// ************************************************************************
//@param a An array of octets representing the first MPI.
//@param len_a The length of the first array (number of octets).
//@param b An array of octets representing the second MPI.
//@param len_b The length of the second array (number of octets).
//@return 0 when a=b, -1 when ab
int mpi_cmp_octets(char *a, unsigned int len_a, char *b, unsigned int len_b);

//Encryption of given octets with public RSA key: n and e given as octets too.
// ************************************************************************
// ***Length of output is KEY_LENGTH_OCTETS.
// ***NB: Make SURE that out, input, n and e have enough space allocated!!
// ***NB: NO MEMORY ALLOCATED for its parameters by this method and
// *****    NO CHECKS PERFORMED BY THIS METHOD!
// ************************************************************************
//@param out Pointer to ALREADY ALLOCATED space for the encrypted data.
//@param len_out Length of the allocated space for out (in octets).
//@param input Pointer to the data to be encrypted.
//@param len_input Length of the allocated space for input (in octets).
//@param n Pointer to the public RSA modulus to use for encryption.
//@param len_n Length of n (in octets).
//@param e Pointer to the public RSA exponent to use for encryption.
//@param len_e Length of e (in octets).
//@return The actual length of the output i.e. number of chars written to out.
int public_rsa_octets( char *out  , unsigned int len_out,
                        char *input, unsigned int len_input,
                        char *n    , unsigned int len_n,
                        char *e    , unsigned int len_e);

//Encryption of given octets with *private* RSA key given as octets.
// ************************************************************************
// ***Length of output is KEY_LENGTH_OCTETS.
// ***NB: Make SURE that ALL pointers have enough space allocated!!
// ***NB: NO MEMORY ALLOCATED for its parameters by this method and
// *****    NO CHECKS PERFORMED BY THIS METHOD!
// ************************************************************************
//@param out Pointer to ALREADY ALLOCATED space for the encrypted data.
//@param len_out Length of the allocated space for out (in octets).
//@param input Pointer to the data to be encrypted.
//@param len_input Length of the allocated space for input (in octets).
//@param n Pointer to the public RSA modulus of the given key.
//@param len_n Length of n (in octets).
//@param e Pointer to the public RSA exponent of the given key.
//@param len_e Length of e (in octets).
//@param d Pointer to the private RSA exponent of the given key.
//@param len_d Length of d (in octets).
//@param p Pointer to the prime p of the given key.
//@param len_p Length of p (in octets).
//@param q Pointer to the prime q of the given key.
//@param len_q Length of q (in octets).
//@param u Pointer to the inverse of p mod q for the given key.
//@param len_u Length of u (in octets).
//@return The actual length of the output i.e. number of chars written to out.
int private_rsa_octets( char *out, unsigned int len_out,
                        char *input, unsigned int len_input,
                        char *n    , unsigned int len_n,
                        char *e    , unsigned int len_e,
                        char *d    , unsigned int len_d,
                        char *p    , unsigned int len_p,
                        char *q    , unsigned int len_q,
                        char *u    , unsigned int len_u);

//Generates a new RSA key and stores its components at the specified locations.
//@param n Pointer to the public RSA modulus of the given key.
//@param len_n Length of n (in octets).
//@param e Pointer to the public RSA exponent of the given key.
//@param len_e Length of e (in octets).
//@param d Pointer to the private RSA exponent of the given key.
//@param len_d Length of d (in octets).
//@param p Pointer to the prime p of the given key.
//@param len_p Length of p (in octets).
//@param q Pointer to the prime q of the given key.
//@param len_q Length of q (in octets).
//@param u Pointer to the inverse of p mod q for the given key.
//@param len_u Length of u (in octets).
void gen_rsa_octets( char *n, unsigned int *len_n,
                     char *e, unsigned int *len_e,
                     char *d, unsigned int *len_d,
                     char *p, unsigned int *len_p,
                     char *q, unsigned int *len_q,
                     char *u, unsigned int *len_u);

//Copies the buffer of m to the location to which out points.
//*****************************************************************
//*** This method does NOT allocate memory for out!
//*** NB: caller should allocate enough memory!
//*****************************************************************
//@param out pointer to allocated memory where to copy the MPI's octets
//@param len_out size of out; will be replaced by actual number of octets copied
//@param m The MPI whose octets are to be retrieved
void mpi_to_octets( char *out, unsigned int *len_out, MPI m);

Corresponding code for the wrapper methods:

//Wrapper methods for C implementations of RSA and MPI.
//To be used / called from Ada so that the C part remains well separated.
//S.MG, 2018

#include "c_wrappers.h"
#include "mpi.h"
#include "smg_rsa.h"
#include <assert.h>
#include <string.h> //for memmove...

//Wrapper for comparing the given arrays of octets as MPIs
//Steps:
//1. allocate space for 2 MPIs u and v
//2. set buffer of u to content of a, buffer of v to content of b
//3. call mpi_cmp(u,v) and store result
//4. de-allocate u and v
//5. return result
int mpi_cmp_octets(char *a, unsigned int len_a, char *b, unsigned int len_b) {
  //variable to hold the final result of comparing a to b as MPIs
  int result;

  //calculate how much space is needed for each MPI
	unsigned int nlimbs_a = mpi_nlimb_hint_from_nbytes( len_a );
	unsigned int nlimbs_b = mpi_nlimb_hint_from_nbytes( len_b );

  //allocate space for the 2 MPIs
	MPI u = mpi_alloc(nlimbs_a);
	MPI v = mpi_alloc(nlimbs_b);

  //set the given octets as the values of the 2 MPIs
  //the sign is set to 0 (last parameter).
  mpi_set_buffer(u, a, len_a, 0);
  mpi_set_buffer(v, b, len_b, 0);

  //compare the MPIs as numbers and store the result
  result = mpi_cmp(u, v);

  //tidy up: free the memory allocated for the 2 MPIs
  mpi_free(u);
  mpi_free(v);

  //return the result comparing a to b as MPIs
  return result;
}

//Encryption of given input with a public RSA key: n and e given as octets too.
//Steps:
//1. create (allocate memory for) MPIs for out and input;
//2. set the input as buffer for the corresponding MPI;
//3. create and set the public key structure with n,e as contents;
//4. call rsa/public_rsa and retrieve the result storing it in out;
//5. free allocated memory for all the MPIs.
//6. return the actual length of the encrypted result
int public_rsa_octets( char *out  , unsigned int len_out,
                        char *input, unsigned int len_input,
                        char *n    , unsigned int len_n,
                        char *e    , unsigned int len_e) {

  // precondition: output has enough memory allocated
  assert( len_out >= KEY_LENGTH_OCTETS );

  //allocate memory for input and output MPIs
	unsigned int nlimbs_in  = mpi_nlimb_hint_from_nbytes( len_input );
	unsigned int nlimbs_out = mpi_nlimb_hint_from_nbytes( len_out   );
	MPI in_mpi  = mpi_alloc(nlimbs_in);
	MPI out_mpi = mpi_alloc(nlimbs_out);

  //set input as buffer for in_mpi
  mpi_set_buffer(in_mpi, input, len_input, 0);

  //create public key structure and set its contents to given n, e
  RSA_public_key pk;
	unsigned int nlimbs_n = mpi_nlimb_hint_from_nbytes( len_n );
	unsigned int nlimbs_e = mpi_nlimb_hint_from_nbytes( len_e );
  pk.n = mpi_alloc(nlimbs_n);
  pk.e = mpi_alloc(nlimbs_e);
  mpi_set_buffer(pk.n, n, len_n, 0);
  mpi_set_buffer(pk.e, e, len_e, 0);

  //call rsa public_key encryption and retrieve the result, storing it in out
  public_rsa( out_mpi, in_mpi, &pk);
  int len = len_out;
  mpi_to_octets( out, &len, out_mpi );

  //tidy up: free allocated memory for ALL MPIs.
  mpi_free(in_mpi);
  mpi_free(out_mpi);
  mpi_free(pk.n);
  mpi_free(pk.e);

  //return actual length
  return len;
}

//Decryption of given input with the private key given through its components.
//Steps:
//1. create (allocate memory for) MPIs for out and input;
//2. set the input as buffer for the corresponding MPI;
//3. create and set the private key structure with n,e,d,p,q,u as contents;
//4. call rsa/private_rsa and retrieve the result storing it in out;
//5. free allocated memory for all the MPIs.
//6. return the actual length of the result
int private_rsa_octets( char *out, unsigned int len_out,
                        char *input, unsigned int len_input,
                        char *n    , unsigned int len_n,
                        char *e    , unsigned int len_e,
                        char *d    , unsigned int len_d,
                        char *p    , unsigned int len_p,
                        char *q    , unsigned int len_q,
                        char *u    , unsigned int len_u) {
  // precondition: output has enough memory allocated
  assert( len_out >= KEY_LENGTH_OCTETS );

  //allocate memory for input and output MPIs
	unsigned int nlimbs_in  = mpi_nlimb_hint_from_nbytes( len_input );
	unsigned int nlimbs_out = mpi_nlimb_hint_from_nbytes( len_out   );
	MPI in_mpi  = mpi_alloc(nlimbs_in);
	MPI out_mpi = mpi_alloc(nlimbs_out);

  //set input as buffer for in_mpi
  mpi_set_buffer(in_mpi, input, len_input, 0);

  //create private key structure and set its contents to given n,e,d,p,q,u
  RSA_secret_key sk;
	unsigned int nlimbs_n = mpi_nlimb_hint_from_nbytes( len_n );
	unsigned int nlimbs_e = mpi_nlimb_hint_from_nbytes( len_e );
	unsigned int nlimbs_d = mpi_nlimb_hint_from_nbytes( len_d );
	unsigned int nlimbs_p = mpi_nlimb_hint_from_nbytes( len_p );
	unsigned int nlimbs_q = mpi_nlimb_hint_from_nbytes( len_q );
	unsigned int nlimbs_u = mpi_nlimb_hint_from_nbytes( len_u );
  sk.n = mpi_alloc(nlimbs_n);
  sk.e = mpi_alloc(nlimbs_e);
  sk.d = mpi_alloc(nlimbs_d);
  sk.p = mpi_alloc(nlimbs_p);
  sk.q = mpi_alloc(nlimbs_q);
  sk.u = mpi_alloc(nlimbs_u);
  mpi_set_buffer(sk.n, n, len_n, 0);
  mpi_set_buffer(sk.e, e, len_e, 0);
  mpi_set_buffer(sk.d, d, len_d, 0);
  mpi_set_buffer(sk.p, p, len_p, 0);
  mpi_set_buffer(sk.q, q, len_q, 0);
  mpi_set_buffer(sk.u, u, len_u, 0);

  //call rsa secret_key encryption and retrieve the result, storing it in out
  secret_rsa( out_mpi, in_mpi, &sk );
  int len = len_out;
  mpi_to_octets( out, &len, out_mpi );

  //tidy up: free memory previously allocated for MPIs
  mpi_free(in_mpi);
  mpi_free(out_mpi);
  mpi_free(sk.n);
  mpi_free(sk.e);
  mpi_free(sk.d);
  mpi_free(sk.p);
  mpi_free(sk.q);
  mpi_free(sk.u);

  //return number of octets copied in out - real length of result
  return len;
}

//Generates a new RSA key and stores its components at the specified locations.
void gen_rsa_octets( char *n, unsigned int *len_n,
                     char *e, unsigned int *len_e,
                     char *d, unsigned int *len_d,
                     char *p, unsigned int *len_p,
                     char *q, unsigned int *len_q,
                     char *u, unsigned int *len_u) {
  // precondition: all pointers have enough memory allocated
  assert( *len_n >= KEY_LENGTH_OCTETS );
  assert( *len_e >= KEY_LENGTH_OCTETS );
  assert( *len_d >= KEY_LENGTH_OCTETS );
  assert( *len_p >= KEY_LENGTH_OCTETS / 2);
  assert( *len_q >= KEY_LENGTH_OCTETS / 2);
  assert( *len_u >= KEY_LENGTH_OCTETS / 2);

  //the secret key structure that will hold generated key components
  RSA_secret_key sk;
	int nlimbs = mpi_nlimb_hint_from_nbytes( KEY_LENGTH_OCTETS );
	int nlimbs_pq = mpi_nlimb_hint_from_nbytes( KEY_LENGTH_OCTETS / 2 );

	sk.n = mpi_alloc(nlimbs);
	sk.e = mpi_alloc(nlimbs);
	sk.d = mpi_alloc(nlimbs);
	sk.p = mpi_alloc(nlimbs_pq);
	sk.q = mpi_alloc(nlimbs_pq);
	sk.u = mpi_alloc(nlimbs_pq);

  //generate the rsa key pair - this may take a while!
  gen_keypair(&sk);

  //copy components to their place
  mpi_to_octets( n, len_n, sk.n );
  mpi_to_octets( e, len_e, sk.e );
  mpi_to_octets( d, len_d, sk.d );
  mpi_to_octets( p, len_p, sk.p );
  mpi_to_octets( q, len_q, sk.q );
  mpi_to_octets( u, len_u, sk.u );

  //tidy up: free ALL MPIs
  mpi_free(sk.n);
  mpi_free(sk.e);
  mpi_free(sk.d);
  mpi_free(sk.p);
  mpi_free(sk.q);
  mpi_free(sk.u);
}

void mpi_to_octets( char *out, unsigned int *len_out, MPI m) {
  //copy the components as raw octets to the given pointers
  int len = 0;
  int sign;
  unsigned char * buffer = mpi_get_buffer( m, &len, &sign );

  //check and don't copy MORE than there is allocated space in out!
  assert( len <= *len_out );
  memmove( out, buffer, len );
  *len_out = len;    //save actual length of the component

  xfree( buffer ); //free the buffer that was allocated by mpi_get_buffer
}

The basic tests for the new C wrappers are the following:

//S.MG, 2018

#include "mpi.h"
#include "smg_rsa.h"

#include "c_wrappers.h"

void test_mpi_cmp() {
  int result;
  int i;
  char a[KEY_LENGTH_OCTETS];
  char b[KEY_LENGTH_OCTETS];

  //initialize mpis
  for (i=0;i b
  a[240] = 241;
  result = mpi_cmp_octets(a, KEY_LENGTH_OCTETS, b, KEY_LENGTH_OCTETS);
  if (result == 1)
    printf("PASS: mpi_cmp_octets on a > b.\n");
  else {
    printf("FAIL: mpi_cmp_octets on a > b ");
    printf("returned %d instead of 1.\n", result);
  }

  //tidy up
}

void test_gen_rsa_octets() {
  RSA_secret_key sk;
  RSA_public_key pk;
	int nlimbs = mpi_nlimb_hint_from_nbytes( KEY_LENGTH_OCTETS );
	int nlimbs_pq = mpi_nlimb_hint_from_nbytes( KEY_LENGTH_OCTETS / 2 );

  //allocate memory
	sk.n = mpi_alloc(nlimbs);
	sk.e = mpi_alloc(nlimbs);
	sk.d = mpi_alloc(nlimbs);
	sk.p = mpi_alloc(nlimbs_pq);
	sk.q = mpi_alloc(nlimbs_pq);
	sk.u = mpi_alloc(nlimbs_pq);

	pk.n = mpi_alloc(nlimbs);
	pk.e = mpi_alloc(nlimbs);

  //generate key pair
  int len_n = KEY_LENGTH_OCTETS;
  int len_e = len_n;
  int len_d = len_n;
  int len_p = KEY_LENGTH_OCTETS / 2;
  int len_q = len_p;
  int len_u = len_p;
  char n[KEY_LENGTH_OCTETS];
  char e[KEY_LENGTH_OCTETS];
  char d[KEY_LENGTH_OCTETS];
  char p[KEY_LENGTH_OCTETS / 2];
  char q[KEY_LENGTH_OCTETS / 2];
  char u[KEY_LENGTH_OCTETS / 2];
  gen_rsa_octets(n, &len_n,
                 e, &len_e,
                 d, &len_d,
                 p, &len_p,
                 q, &len_q,
                 u, &len_u);

  //check encryption/decr works
  mpi_set_buffer(sk.n, n, len_n, 0);
  mpi_set_buffer(sk.e, e, len_e, 0);
  mpi_set_buffer(sk.d, d, len_d, 0);
  mpi_set_buffer(sk.p, p, len_p, 0);
  mpi_set_buffer(sk.q, q, len_q, 0);
  mpi_set_buffer(sk.u, u, len_u, 0);

  mpi_set_buffer(pk.n, n, len_n, 0);
  mpi_set_buffer(pk.e, e, len_e, 0);

  MPI encr = mpi_alloc(0);
	MPI plain = mpi_alloc(0);
  MPI out = mpi_alloc(0);
	mpi_fromstr(plain, "0x\
5B6A8A0ACF4F4DB3F82EAC2D20255E4DF3E4B7C799603210766F26EF87C8980E737579\
EC08E6505A51D19654C26D806BAF1B62F9C032E0B13D02AF99F7313BFCFD68DA46836E\
CA529D7360948550F982C6476C054A97FD01635AB44BFBDBE2A90BE06F7984AC8534C3\
28097EF92F6E78CAE0CB97");
  public_rsa(encr, plain, &pk);
  secret_rsa(out, encr, &sk);

  if (mpi_cmp(out, plain) != 0)
    printf("FAIL: test_gen_rsa encr/decr failed.\n");
  else
    printf("PASS: test_gen_rsa encr/decr passed.\n");

  //tidy up
  mpi_free(sk.n);
  mpi_free(sk.e);
  mpi_free(sk.d);
  mpi_free(sk.p);
  mpi_free(sk.q);
  mpi_free(sk.u);

  mpi_free(pk.n);
  mpi_free(pk.e);

  mpi_free(plain);
  mpi_free(encr);
  mpi_free(out);
}

void test_rsa_octets() {
  int noctets = 512;
	RSA_public_key pk;
	pk.n = mpi_alloc(0);
	pk.e = mpi_alloc(0);

  RSA_secret_key sk;
  sk.n = mpi_alloc(0);
  sk.e = mpi_alloc(0);
  sk.d = mpi_alloc(0);
  sk.p = mpi_alloc(0);
  sk.q = mpi_alloc(0);
  sk.u = mpi_alloc(0);

//key pair previously generated with EuCrypt
  mpi_fromstr(sk.n, "0x\
CD2C025323BEA46FFF2FA8D7A9D39817EA713421F4AE03FA8120641193892A70BFECF5\
83101635A432110D3DDE6339E3CC7ECC0AD91C026FCACE832DD3888A6FCA7BCE56C390\
5A5AC8C7BC921DA675E4B62489B254EB34659D547D71165BC998983A81937BD251AEE1\
2D985EC387D5376F5DCC5EF7EC530FBD6FD2AA7285EE1AF3335EA73163F0954F30402E\
D7B374EE84A97B1849B0674B0DA0A2050BD79B71ABB1559F3A9CFDB8557DED7BC90CF2\
09E8A847E9C226140845B7D03842162E7DA5DD16326CB1F71A248D841FE9076A09911F\
2F4F5E3EA44EA8DE40332BF00406990BCCF61C322A03C456EF3A98B341E0BDBC1088CE\
683E78510E76B72C2BCC1EE9AEDD80FFF18ABFC5923B2F36B581C25114AB2DF9F6C2B1\
9481703FD19E313DCD7ACE15FA11B27D25BCE5388C180A7E21167FB87750599E1ED7C7\
50F4A844E1DC2270C62D19671CF8F4C25B81E366B09FC850AE642136D204A9160AEECE\
575B57378AA439E9DD46DC990288CD54BAA35EEE1C02456CD39458A6F1CBF012DCEDF4\
27CCF3F3F53645658FC49C9C9D7F2856DB571D92B967AB5845514E0054DDB49099F5DD\
04A6F6F5C5CE642276834B932881AEB648D1F25E9223971F56E249EF40CF7D80F22621\
CDD0260E9E7D23746960ADB52CF2987584FB1DE95A69A39E5CB12B76E0F5C1A0529C0C\
065D2E35720810F7C7983180B9A9EA0E00C11B79DC3D");

  mpi_fromstr(sk.e, "0x\
DD4856B4EE3D099A8604AE392D8EFEC094CDF01546A28BE87CB484F999E8E75CDFCD01\
D04D455A6A9254C60BD28C0B03611FC3E751CC27EF768C0B401C4FD2B27C092834A6F2\
49A145C4EDC47A3B3D363EC352462C945334D160AF9AA72202862912493AC6190AA3A6\
149D4D8B9996BA7927D3D0D2AD00D30FD630CF464E6CAF9CF49355B9A70E05DB7AE915\
F9F602772F8D11E5FCDFC7709210F248052615967090CC1F43D410C83724AA5912B2F0\
52E6B39449A89A97C79C92DC8CB8DEEFCF248C1E1D2FC5BFE85165ECA31839CAA9CEB3\
3A92EBDC0EB3BAC0F810938BB173C7DA21DCBB2220D44CBA0FD40A2C868FC93AC5243E\
C137C27B0A76D65634EBB3");

  mpi_fromstr(sk.d, "0x\
7C8A6FA1199D99DCA45E9BDF567CA49D02B237340D7E999150BC4883AE29DEC5158521\
B338F35DC883792356BDDBB3C8B3030A6DD4C6522599A3254E751F9BA1CB1061C5633C\
81BBFACF6FCD64502614102DFED3F3FA284066C342D5E00953B415915331E30812E5FB\
CD6680ADCCDEE40B8376A3A225F2E160EA59C7566804526D73BB660A648A3EF9802313\
B2F841E8458B2AAACE7AACF31083E8F3F630298138393BC88BBD7D4AA4334949651D25\
365B10DBF4A4A08E20A6CC74BFDD37C1C38E2ADC2A283DF06590DF06B46F67F6ACA67F\
AC464C795261659A2F9558802D0BBAA05FD1E1AF2CDC70654723DF7EFAEA148B8CDBEB\
C89EA2320AB9BBB1BC4311475DF3D91446F02EF192368DFEBAC598CCFD4407DEC58FDC\
1A94CCDD6E5FBA9C52164ACEA8AEE633E557BCCEACB7A1AF656C379482D784A120A725\
32F9B2B35173D505F21D5AD4CB9511BC836DC923730B70291B70290A216CA3B21CFF79\
E895C35F4F7AF80E1BD9ED2773BD26919A76E4298D169160593E0335BE2A2A2D2E8516\
948F657E1B1260E18808A9D463C108535FB60B3B28F711C81E5DE24F40214134A53CE5\
9A952C8970A1D771EBEFFA2F4359DCF157995B3F1950DE3C6EC41B7FF837148F55F323\
372AF3F20CE8B8038E750C23D8F5041FA951327859B0E47483F0A47103EF808C72C251\
006FA526245291C8C84C12D2EF63FB2301EA3EEDA42B");

  mpi_fromstr(sk.p, "0x\
E236732452039C14EC1D3B8095BDDCFB7625CE27B1EA5394CF4ED09D3CEECAA4FC0BF6\
2F7CE975E0C8929CE84B0259D773EA038396479BF15DA065BA70E549B248D77B4B23ED\
A267308510DBEE2FD44E35D880EE7CFB81E0646AA8630165BD8988C3A8776D9E704C20\
AA25CA0A3C32F27F592D5FD363B04DD57D8C61FFDCDFCCC59E2913DE0EE47769180340\
E1EA5A803AA2301A010FF553A380F002601F0853FCACDB82D76FE2FACBCD6E5F294439\
0799EA5AE9D7880D4E1D4AE146DC1D4E8495B9DD30E57E883923C5FC26682B7142D35C\
D8A0FC561FE725A6CF419B15341F40FE0C31132CBD81DD8E50697BD1EBFFA16B522E16\
F5B49A03B707218C7DA60B");

  mpi_fromstr(sk.q, "0x\
E830482A3C4F5C3A7E59C10FF8BA760DB1C6D55880B796FFDA4A82E0B60E974E81D04B\
2A4AD417823EBFB4E8EFB13782943562B19B6C4A680E3BA0C8E37B5023470F4F1AC1F8\
A0B10672EF75CD58BCD45E6B14503B8A6A70AFE79F6201AF56E7364A1C742BE1453FD2\
24FDC9D66522EAF4466A084BCB9E46D455A2946E94CBF028770F38D0B741C2CC59308F\
71D8C2B4B9C928E0AE8D68DEB48A3E9EFD84A10301EBD55F8221CA32FC567B306B2A8E\
116350AFB995859FDF4378C5CFD06901494E8CFA5D8FAC564D6531FA8A2E4761F5EFBA\
F78750B6F4662BE9EA4C2FAD67AF73EEB36B41FC15CB678810C19A51DF23555695C4C1\
546F3FACA39CAA7BB8DBD7");

  mpi_fromstr(sk.u, "0x\
846232322775C1CD7D5569DC59E2F3E61A885AE2E9C4A4F8CB3ACBE8C3A5441E5FE348\
A2A8AC9C2998FBF282222BF508AA1ECF66A76AEDD2D9C97028BFD3F6CA0542E38A5312\
603C70B95650CE73F80FDD729988FBDB5595A5BF8A007EA34E54994A697906CE56354C\
E00DF10EB711DEC274A62494E3D350D88736CF67A477FB600AC9F1D6580727585092BF\
5EBC092CC4D6CF75769051033A1197103BE269942F372168A53771746FBA18ED6972D5\
0B935A9B1D6B5B3DD50CD89A27FE93C10924E9103FACF7B4C5724A046C3D3B50CC1C78\
5F5C8E00DBE1D6561F120F5294C170914BC10F978ED4356EED67A9F3A60D70AFE540FC\
5373CBAE3D0A7FD1C87273");

// copy the public key components
  pk.n = mpi_copy( sk.n );
  pk.e = mpi_copy( sk.e );

// some plain text message
	MPI plain = mpi_alloc(0);
	mpi_fromstr(plain, "0x\
5B6A8A0ACF4F4DB3F82EAC2D20255E4DF3E4B7C799603210766F26EF87C8980E737579\
EC08E6505A51D19654C26D806BAF1B62F9C032E0B13D02AF99F7313BFCFD68DA46836E\
CA529D7360948550F982C6476C054A97FD01635AB44BFBDBE2A90BE06F7984AC8534C3\
28097EF92F6E78CAE0CB97");

// expected encrypted MPI (via rsa.c directly)
  MPI encr = mpi_alloc(0);
  public_rsa( encr, plain, &pk);
  MPI decr = mpi_alloc(0);
  secret_rsa( decr, encr, &sk);
  if (mpi_cmp(decr, plain) != 0)
    printf("FAIL: decrypted != plain in test_rsa_octets (MPI call)\n");

//allocate the char arrays for _octets rsa
  int len_n = noctets;
  int len_e = len_n;
  int len_d = len_n;
  int len_p = noctets / 2;
  int len_q = len_p;
  int len_u = len_p;
  char n[noctets];
  char e[noctets];
  char d[noctets];
  char p[noctets / 2];
  char q[noctets / 2];
  char u[noctets / 2];

//copy the key components into char arrays
  mpi_to_octets(n, &len_n, sk.n);
  mpi_to_octets(e, &len_e, sk.e);
  mpi_to_octets(d, &len_d, sk.d);
  mpi_to_octets(p, &len_p, sk.p);
  mpi_to_octets(q, &len_q, sk.q);
  mpi_to_octets(u, &len_u, sk.u);

//call _octets rsa and check results
  int len_encr = noctets;
  int len_decr = noctets;
  int len_plain = noctets;
  char plain_o[noctets];
  char encr_o[noctets];
  char expected_encr_o[noctets];
  char decr_o[noctets];
  char expected_decr_o[noctets];

  mpi_to_octets(plain_o, &len_plain, plain);
  mpi_to_octets(expected_encr_o, &len_encr, encr);
  mpi_to_octets(expected_decr_o, &len_decr, decr);
  len_decr = noctets;

  int len;
  len = public_rsa_octets( encr_o, len_encr, plain_o, len_plain,
                           n, len_n, e, len_e);
  if (len != len_encr)
    printf("FAIL: actual len of encr is %d; expected %d\n", len, len_encr);
  else
    printf("PASS: actual len of encr matches expected: %d\n", len);
  int errors= 0;
  int i;
  for (i=0;i0)
    printf("FAIL: found %d errors in public_rsa_octets output\n", errors);
  else
    printf("PASS: no errors found in public_rsa_octets output\n");

  len_encr = len;
  len = private_rsa_octets( decr_o, len_decr, encr_o, len_encr,
                            n, len_n, e, len_e,
                            d, len_d, p, len_p,
                            q, len_q, u, len_u);
  if (len != len_plain)
    printf("FAIL: actual len of decr is %d; expected %d\n", len, len_plain);
  else
    printf("PASS: actual len of decr matches expected: %d\n", len);

  errors = 0;
  for (i=0;i0)
    printf("FAIL: found %d errors in private_rsa_octets output\n", errors);
  else printf("PASS: no errors found in private_rsa_octets_output\n");

  //tidy up
  mpi_free(sk.n);
  mpi_free(sk.e);
  mpi_free(sk.d);
  mpi_free(sk.p);
  mpi_free(sk.q);
  mpi_free(sk.u);

  mpi_free(pk.n);
  mpi_free(pk.e);

  mpi_free(plain);
  mpi_free(encr);
  mpi_free(decr);
}

int main(int ac, char **av) {
  if (ac < 2) {
    printf("Usage: %s testID\n", av[0]);
    return -1;
  }

  int id = atoi(av[1]);
  switch (id) {
    case 1:
      test_mpi_cmp();
      break;
    case 2:
      test_gen_rsa_octets();
      break;
    case 3:
      test_rsa_octets();
      break;
    default:
      printf("Current test ids:\n");
      printf("1 test of mpi_cmp_octets\n");
      printf("2 test of gen_rsa_octets (can be very SLOW!)\n");
      printf("3 test of rsa_octets (can take a few minutes)\n");
  }
  return 0;
}

The rest (and the longest part by far) of the .vpatch for this chapter includes the MPI lib from EuCrypt and the RSA lib from the same EuCrypt minus the oaep parts that are not going to be used by SMG Comms. Specifically, the changes to the RSA lib from EuCrypt are the following:

  • The constant KEY_LENGTH_OCTETS is changed to the value used by Eulora: 490. I've updated also all the references to the previously fixed value 4096 throughout the comments.
  • I discarded two methods from rsa.c/h: rsa_oaep_encrypt, rsa_oaep_decrypt, oaep_encrypt_c, oaep_decrypt_c.

In addition to the above, the .vpatch also makes a small change to the smg_comms.gpr file reflecting the fact that SMG Comms does not touch mpi and rsa directly but only through the new C wrappers so that the .gpr file uses "c_wrappers.gpr" and not rsa.gpr + mpi.gpr.

The .vpatch is as usual on my Reference Code Shelf and linked also below for your convenience:

Older Posts »

Theme and content by Diana Coman