Ossasepia

October 13, 2018

Results of Testing UDP - Take 2

Filed under: Coding, UDP — Diana Coman @ 8:32 p.m.

As the first test revealed that most UDP packets make it safely at least when sent 1 second apart, I decided that the next week-long test should run with a substantially smaller delay: 0.01 seconds between each 2 packets. Moreover, I've changed the sender to simply send 10000 packets at each run, alternating between 1472 and 1470 octets in size. The focus on those 2 sizes reflects the fact that ALL packets are either 14721 or 14702 octets long in the newest version of Eulora's communication protocol. Here's the summary of the data I collected:

UY UK
Sent: 21400003 2140000
Received: 2134569 (99.74%) 2139788 (99.99%)
Errors received: 0 0
1470-size sent: 1070000 1070000
1470-size received: 1067322 (99.75%) 1069901 (99.99%)
1472-size sent: 1070000 1070000
1472-size received: 1067247 (99.74%) 1069887 (99.99%)

From the above, it would seem that even 0.01 delay between any two packets being sent is not causing any problem at all - at least not for the sizes considered4. If anything, the % of received packets increased slightly compared to the previous test. Of the packets that made it to the other side, none were mangled, hence no errors received5. Unsurprising perhaps, the 2 octets difference in size between the 2 types of packets does not seem to have the slightest impact on % of packets that make it to the other side - the route itself is likely to be more important in this respect.

The timings remain iffier to investigate as I did not change the tester specifically for this. Applying the same correction as in the previous test (i.e. calculating first the difference between the logged local times and then simply adjusting so that the minimum difference is 0) yields again the same stats on both sides (if slightly different values than during the last test):

Data Min 1st Q Median Mean 3rd Q Max
UK -> UY 0 7 13 13.09 20 27
UY -> UK 0 7 13 13.19 20 27

As before, I would not suggest taking the above timings for anything really. The only thing that can perhaps be said is that packets don't seem to take all that much to arrive at their destination at least on those routes. This is of course no guarantee at all, just a limited observation of this particular data set.


  1. Serpent encrypted packets that are meant for most communications in Eulora. 

  2. RSA encrypted packets that are meant only for new accounts in Eulora. 

  3. I've ran this for almost 9 days so more than the previous 1 week for the first test. 

  4. Arguably larger sizes that would get fragmented on the way might have higher chances of becoming lost. 

  5. There was however one spam packet received by the node in UY from an entirely unrelated IP but I removed that before making the summary above since it has nothing to do with the test itself. Also, the fact that one will receive spam is not exactly news otherwise so there isn't much to say about it at this moment. 

October 4, 2018

Results of Testing UDP - Take 1

Filed under: Coding, UDP — Diana Coman @ 6:09 p.m.

For one full week, between 26 September 2018 and 3 October 2018, my UDP Tester ran on 2 computers, one in the UK and one in Uruguay (UY), sending and receiving UDP messages in both directions. On each side, the receiver ran continuously, logging all UDP messages that it received during the whole interval. By contrast, the sender ran on both sides hourly but at different times so that the communications did not overlap. I don’t expect it would have been any trouble even if they did overlap but this was meant to be a test of UDP under best conditions and for this reason I set the times so that the sender on one end always finished a full run before the one on the other end started. For the same reason, the messages were sent at a rate of at most 1 message per second1.

At each run, the sender sent exactly 2043 UDP messages with lengths between 6 and 2048, each message having a different length. The order of messages was pseudo-random, relying on the Mersenne Twister prng using as seed the local time at the start of the run (in unix format). The sender kept a log of all messages it sent, including destination IP and port, seed used for MT, local time when message was sent and size of message. The receiver also logged basic information about each message: source IP and port, local time and message size as observed by receiver as well as those contained in the message’s own header, number of observed incorrect bits in the message’s payload as well as the expected and actual values of incorrect octets.

A first look at the week-long data yielded a bit of a surprise in that the UY receiver had actually received *more* messages than were sent from the UK! At a closer look, it turned out that 4933 UDP messages arriving at the UY node were actually sent by its own local switch! And moreover, they were all, without exception, recorded as corrupted since neither size nor payload matched the expected values2. At the moment those switch-generated messages are a bit of a mystery - it’s unclear what they are exactly or why and how they appeared. Working hypotheses would be that they are either local dhcp messages (although the port number would be a weird choice for those) or stray frags of bigger UDP messages. My one single attempt to replicate this behaviour while simultaneously capturing everything with tcpdump has so far failed - there were no such unexpected messages at all over several hours of UK sender at work. I might perhaps try again at a later date after I’m done with the more pressing tests that I need for SMG comms or simply process the existing error log and reconstitute the already observed weird messages from there. Anyway, for now I put those anomalous messages to the side and focus instead on the rest of the messages (which were sent as expected either by the node in the UK or by the one in UY). Here’s a summary of the data thus cleaned:

UK node UY node
Total sent: 3459283 3452674
Total received: 3447175 3451836
% received: 99.84%7 99.78%8
Errors received9: 0 0

Arguably the lost messages are of most interest in all the above: can one say perhaps that the largest messages10 get lost more often? Not really or at least not based on this little set of data. Compare the summary stats for three groups of messages: all messages sent from the UK (reflecting as expected the sizes sent and the fact that the same number of messages of each size are sent), all messages lost at UY (i.e. did not make it on the way from UK to UY) and all messages lost at UK (i.e. did not make it on the way from UY to UK):

Data Min 1st Q Median Mean 3rd Q Max
All sent from UK 6 516 1027 1027 1538 2048
Lost at UY (UK->UY) 13 513 1049 1051 1602 2045
Lost at UK (UY->UK) 16 553.5 1072 1061 1576 2047

While the data set of lost messages is quite small (550 messages lost at UK and 745 at UY), note that this is mainly due to the fact that there are relatively few losses overall: less than 0.4% of messages sent got lost on the way. So it would seem that at least under the conditions and on the routes considered11, UDP is not all that unreliable anyway. In any case, those summaries above seem to me remarkably close to one another - meaning that there isn’t any visible evidence that some sizes would get lost more than others, at least not for the set of sizes considered. Arguably sizes of up to 2048 octets of message are quite fine for communications over UDP - or at any rate, just as fine as smaller sizes.

In terms of order of received messages, the UY node received ALL messages precisely in the order in which they were sent but the UK node reported 66 messages in total that arrived out of order. Although this is a tiny number, it is perhaps reasonable to assume that it might increase in worse conditions (e.g. significantly less than 1 second between sending messages).

The actual timings are a bit iffier to investigate since the precision of UDP Tester turns out to be less than what would be needed for such task. Moreover, there is something weird going on with the way I recorded the time because the difference between the two nodes should be of ~34 seconds (UY node local time = UK node local time + 34) but this doesn't quite square with all the data especially at the UY receiver end12. On the more positive side though, at least the measurement bias there is constant for all the data and it doesn't introduce any weird effects so I can still attempt to infer something considering also that observed behaviour suggests that most UDP messages really make it to the other end within 1 second. Consequently, I calculated the delta on both sides as TR - TS at first and then I added (on UK side) respectively subtracted (on UY side) the quantity needed to make the lowest delta 0. So at the UY receiver, delta = TR - TS - 11 while at the UK receiver, delta = TR - TS + 32. With this correction, the summary stats for the delta on both sides turn out to be remarkably similar:

Data Min 1st Q Median Mean 3rd Q Max
Deltas at UK node: 0 5 11 10.60 16 21
Deltas at UY node: 0 5 11 10.62 16 21

Note that I do *not* recommend taking the above delta values for anything really, as the tester's precision in recording time is just not enough for this.

You are of course warmly invited to run your own tests and to play with this dataset in any way you find fit. So here's the data from both nodes, including the additional 4933 messages that the UY node received from its own switch:

  • udp_test_take1_data.zip (~10MB)
  • SHA512SUM: 963b8a1467630eea35532122ab7c2d25cb8741001808841f7cf02b34abb6ad5300adcb1d667dd902b4278dd2b373dc46427b0b0bbc918ee52f326456535a4114 udp_test_take1_data.zip

Have fun!


  1. Specifically: the sender had a delay of 1 second between any two consecutive messages. 

  2. The UDP tester simply fills the message up to any length with values calculated as Pos mod 256 where Pos is the position of the respective octet in the full message. 

  3. This is precisely 345928/2043=169 runs. 

  4. This includes a partial 170th run since I stopped the whole test while the UY sender was running already its 170th run. 

  5. 550 messages lost in total. 

  6. 745 messages lost in total. 

  7. 344717 / 345267 * 100 

  8. 345183 / 345928 * 100 

  9. This refers to messages received but with payloads that don’t match the expected values. 

  10. Note that this test capped the messages at 2048 so “largest” here means strictly < 2049 octets. 

  11. The UK node is a “consumer” node i.e. behind a router and on a residential connection; the UY node is S.MG’s test server with Pizarro. 

  12. Considering TR as TimeReceived and TS as TimeSent, at UY receiver the delta should be calculated as TR - (TS + 34) = TR - TS - 34; however, there are entries with TR-TS as low as 11 so basically it would seem that messages arrived before they were even sent. 

September 27, 2018

Tester for UDP Communications

Filed under: Coding, UDP — Diana Coman @ 8:33 p.m.

This code builds on Stanislav Datskovskiy's minimal UDP lib to provide a convenient way of gathering data to evaluate UDP communications between any desired two nodes. The initial specification was to provide a reliable way to "send a soup of all packets lengths from 1 to 65536 bytes each hour back and forth" but this was refined further down the line1 to reduce the maximum length of the payload to 2048 and to include a Mersenne-Twister (MT) pseudorandom number generator (prng) for scrambling the various message sizes. To accomplish that, there are two main parts that I'm adding to the original UDP library:

  1. The MT lib that is simply an Ada implementation of the MT prng algorithm.
  2. The UDP_Tester package that uses a slightly adapted UDP lib and the MT lib above to provide a UDP sender and a corresponding UDP receiver that implement the testing specification and log the relevant data.

MT lib
The MT lib is a standalone Ada library implementing the well-known Mersenne Twister algorithm for pseudorandom number generation2. I am not aware of any other Ada implementation of MT and since I really don't want to add C code to my plate unless I absolutely have to, I simply ported to Ada the reference C implementation provided by the original authors of MT. As such, there isn't much to discuss about the implementation itself - I'll point however the changes I made with respect to the original implementation, namely:

  • In my Ada version of MT, there is no default seeding of the MT generator. The C version allows the caller to ask the generator for numbers without having seeded it - in this case, the C implementation seeds itself with a magic default value that is hardcoded at the checking spot and then just proceeds as if nothing was wrong. I find this approach abhorent because it effectively hides an error (calling the generator without seeding it first) rather than complaining about it and forcing its correction. Consequently, my MT lib will simply abort if the generator is asked for numbers without having been seeded first.3
  • Because Ada is most pointedly not a sort of C, there was no need for the various types of hacks added to the C version to ensure that it still worked correctly on 64-bit machines and not only on the 32-bit machines for which the code was initially written. In Ada it's enough to specify one's types correctly to be exactly and guaranteed 32-bits and then proceed to work with them as such, regardless of whether the processor running the code has 32-bit or 64-bit or x-bit registers.
  • My MT lib provides just one type of pseudorandom numbers, namely unsigned integers on 32 bits. The original C implementation had several wrappers around this main function to map the 32 bits pseudorandom numbers to the interval (0,1) for instance and similar. All the mappings were trivial and they can be easily done by any caller that requires them - precisely in the way they require them. At this stage at least I don't quite see the need for those as part of MT lib and so I left them out - there is only the core offering of 32 pseudorandom bits as it were and the caller can then use or interpret them as they see fit.
  • To check that my implementation results in precisely the same sequence of numbers as the original C MT, I added an automated test (still in Ada) that uses the reference seed and checks the output of MT lib (number by number) against the reference output from the original C MT, reporting any mismatches.

The code of MT lib is self-contained with declarations in mt.ads:

 -- Ada implementation of the Mersenne Twister Pseudo-Random number generator
 -- S.MG, 2018

with Interfaces; use Interfaces;

package MT is
  -- Interfaces.Unsigned_32 in GNAT is mod 2**32 and has bitwise shifts defined
  subtype U32 is Interfaces.Unsigned_32;

  -- period parameters
  N          : constant := 624;
  M          : constant := 397;
  MATRIX_MASK: constant U32 := 16#9908_b0df#;
  UPPER_MASK : constant U32 := 16#8000_0000#;
  LOWER_MASK : constant U32 := 16#7fff_ffff#;

  -- array type for storing the state vector of the generator
  type State_Type is Array( 0 .. N-1 ) of U32;

  -- array type for initialization by array - change key len here if needed
  KEY_LEN    : constant := 4;
  type Init_Array_Type is Array( 0 .. KEY_LEN - 1 ) of U32;

  -- exception raised by a call to generator before initializing it
  No_Init_Exception : exception;

  -- initialize the generator with a seed (number)
  procedure Init_Genrand(Seed : in U32);

  -- initialize the generator with array of 8-octets elements
  procedure Init_Genrand(Seed : in Init_Array_Type);

  -- generate the next pseudo-random 32 bits number in the sequence
  function Gen_U32 return U32;

  -- for testing
  function Get_State return State_Type;

  -- internals of the generator, NOT for direct access
private
  -- actual state of the generator
  State      : State_Type;

  -- flag for generator routine
  Mti_Flag : U32 := N + 1;  -- default value -> state(N) is not initialised

end MT;

And implementation in mt.adb:

 -- Ada implementation of the Mersenne Twister Pseudo-Random number generator
 -- S.MG, 2018

package body MT is

  procedure Init_Genrand(Seed : in U32) is
  begin
    State(0) := Seed;
    for I in State'First + 1 .. State'Last loop
      State(I) := U32(1812433253) *
                  ( State(I - 1) xor
                    ( Shift_Right(State(I - 1), 30) )
                  ) + U32(I) ;
    end loop;
    Mti_Flag := N;
  end Init_Genrand;

  procedure Init_Genrand(Seed : in Init_Array_Type) is
    Default_Seed: constant U32 := U32(19650218); -- magic value!
    I, J, K : Integer;
  begin
    Init_Genrand(Default_Seed);
    I := 1;
    J := 0;
    if N > Seed'Length then
      K := N;
    else
      K := Seed'Length;
    end if;

    while K > 0 loop
      State(I) := (State(I) xor
                  ( (State(I-1) xor
                     Shift_Right(State(I-1), 30)
                    ) * U32(1664525)
                  )) + Seed(J) + U32(J);
      I := I + 1;
      J := J + 1;
      if I >= N then
        State(0) := State(N-1);
        I := 1;
      end if;
      if J >= Seed'Length then
        J := 0;
      end if;
      K := K - 1;
    end loop;

    K := N -1;
    while K > 0 loop
      State(I) := (State(I) xor
                  ( (State(I-1) xor
                     Shift_Right(State(I-1), 30)
                    ) * U32(1566083941)
                  )) - U32(I);
      I := I + 1;
      if I >= N then
        State(0) := State(N-1);
        I := 1;
      end if;
      K := K - 1;
    end loop;
    State(0) := 16#8000_0000#; -- MSB is 1 to ensure non-zero initial state
  end Init_Genrand;

  function Gen_U32 return U32 is
    Y     : U32;
    MASK1 : constant U32 := U32(1);
    Mag01 : Array ( 0 .. 1 ) of U32;
  begin
    -- Mag01[x] is x * Matrix_A of the algorithm for x 0 or 1
    Mag01(0) := U32(0);
    Mag01(1) := MATRIX_MASK;

    -- if no numbers available, generate another set of N words
    if Mti_Flag >= N then

      -- check it's not a non-initialised generator
      if Mti_Flag = (N + 1) then
         -- Generator was NOT initialised!
         -- Original C code initialises with default seed 5489
         -- This code will simply raise exception and abort
         raise No_Init_Exception;
      end if;

      for K in 0 .. N - M - 1 loop
        Y := ( State(K)   and UPPER_MASK ) or
             ( State(K+1) and LOWER_MASK );
        State(K) := State(K+M) xor
                      Shift_Right(Y, 1) xor
                        Mag01(Integer(Y and MASK1));
      end loop;
      for K in N-M .. N - 2 loop
        Y := ( State(K)   and UPPER_MASK  ) or
             ( State(K+1) and LOWER_MASK);
        State(K) := State(K + M - N) xor
                      Shift_Right(Y, 1) xor
                        Mag01(Integer(Y and MASK1));
      end loop;
      Y := (State(N-1) and UPPER_MASK ) or
             (State(0) and LOWER_MASK );
      State(N - 1) := State(M-1) xor
                        Shift_Right(Y, 1) xor
                          Mag01(Integer(Y and MASK1));
      Mti_Flag := 0;
    end if;

    -- retrieve next available number
    Y        := State(Integer(Mti_Flag));
    Mti_Flag := Mti_Flag + 1;

    -- tempering
    Y := Y xor Shift_Right(Y, 11);
    Y := Y xor (Shift_Left(Y, 7) and 16#9d2c_5680#);
    Y := Y xor (Shift_Left(Y, 15) and 16#efc6_0000#);
    Y := Y xor Shift_Right(Y, 18);

    -- return tempered number
    return Y;
  end Gen_U32;

  function Get_State return State_Type is
  begin
    return State;
  end Get_State;

end MT;

The test for the above MT lib is in its own testmt/test_mt.adb, including the reference output (that really is the largest part of the whole file):

  --S.MG, 2018

with Ada.Text_IO; use Ada.Text_IO;
with Interfaces; use Interfaces;
with MT;

procedure Tests_MT is
  Seeds : MT.Init_Array_Type;
  X     : MT.U32;
  No    : constant Integer := 1000;
  Result: Array(0..No-1) of MT.U32 := (
    1067595299,  955945823,  477289528, 4107218783, 4228976476,
    3344332714, 3355579695,  227628506,  810200273, 2591290167,
    2560260675, 3242736208,  646746669, 1479517882, 4245472273,
    1143372638, 3863670494, 3221021970, 1773610557, 1138697238,
    1421897700, 1269916527, 2859934041, 1764463362, 3874892047,
    3965319921,   72549643, 2383988930, 2600218693, 3237492380,
    2792901476,  725331109,  605841842,  271258942,  715137098,
    3297999536, 1322965544, 4229579109, 1395091102, 3735697720,
    2101727825, 3730287744, 2950434330, 1661921839, 2895579582,
    2370511479, 1004092106, 2247096681, 2111242379, 3237345263,
    4082424759,  219785033, 2454039889, 3709582971,  835606218,
    2411949883, 2735205030,  756421180, 2175209704, 1873865952,
    2762534237, 4161807854, 3351099340,  181129879, 3269891896,
     776029799, 2218161979, 3001745796, 1866825872, 2133627728,
      34862734, 1191934573, 3102311354, 2916517763, 1012402762,
    2184831317, 4257399449, 2899497138, 3818095062, 3030756734,
    1282161629,  420003642, 2326421477, 2741455717, 1278020671,
    3744179621,  271777016, 2626330018, 2560563991, 3055977700,
    4233527566, 1228397661, 3595579322, 1077915006, 2395931898,
    1851927286, 3013683506, 1999971931, 3006888962, 1049781534,
    1488758959, 3491776230,  104418065, 2448267297, 3075614115,
    3872332600,  891912190, 3936547759, 2269180963, 2633455084,
    1047636807, 2604612377, 2709305729, 1952216715,  207593580,
    2849898034,  670771757, 2210471108,  467711165,  263046873,
    3569667915, 1042291111, 3863517079, 1464270005, 2758321352,
    3790799816, 2301278724, 3106281430,    7974801, 2792461636,
     555991332,  621766759, 1322453093,  853629228,  686962251,
    1455120532,  957753161, 1802033300, 1021534190, 3486047311,
    1902128914, 3701138056, 4176424663, 1795608698,  560858864,
    3737752754, 3141170998, 1553553385, 3367807274,  711546358,
    2475125503,  262969859,  251416325, 2980076994, 1806565895,
     969527843, 3529327173, 2736343040, 2987196734, 1649016367,
    2206175811, 3048174801, 3662503553, 3138851612, 2660143804,
    1663017612, 1816683231,  411916003, 3887461314, 2347044079,
    1015311755, 1203592432, 2170947766, 2569420716,  813872093,
    1105387678, 1431142475,  220570551, 4243632715, 4179591855,
    2607469131, 3090613241,  282341803, 1734241730, 1391822177,
    1001254810,  827927915, 1886687171, 3935097347, 2631788714,
    3905163266,  110554195, 2447955646, 3717202975, 3304793075,
    3739614479, 3059127468,  953919171, 2590123714, 1132511021,
    3795593679, 2788030429,  982155079, 3472349556,  859942552,
    2681007391, 2299624053,  647443547,  233600422,  608168955,
    3689327453, 1849778220, 1608438222, 3968158357, 2692977776,
    2851872572,  246750393, 3582818628, 3329652309, 4036366910,
    1012970930,  950780808, 3959768744, 2538550045,  191422718,
    2658142375, 3276369011, 2927737484, 1234200027, 1920815603,
    3536074689, 1535612501, 2184142071, 3276955054,  428488088,
    2378411984, 4059769550, 3913744741, 2732139246,   64369859,
    3755670074,  842839565, 2819894466, 2414718973, 1010060670,
    1839715346, 2410311136,  152774329, 3485009480, 4102101512,
    2852724304,  879944024, 1785007662, 2748284463, 1354768064,
    3267784736, 2269127717, 3001240761, 3179796763,  895723219,
     865924942, 4291570937,   89355264, 1471026971, 4114180745,
    3201939751, 2867476999, 2460866060, 3603874571, 2238880432,
    3308416168, 2072246611, 2755653839, 3773737248, 1709066580,
    4282731467, 2746170170, 2832568330,  433439009, 3175778732,
      26248366, 2551382801,  183214346, 3893339516, 1928168445,
    1337157619, 3429096554, 3275170900, 1782047316, 4264403756,
    1876594403, 4289659572, 3223834894, 1728705513, 4068244734,
    2867840287, 1147798696,  302879820, 1730407747, 1923824407,
    1180597908, 1569786639,  198796327,  560793173, 2107345620,
    2705990316, 3448772106, 3678374155,  758635715,  884524671,
     486356516, 1774865603, 3881226226, 2635213607, 1181121587,
    1508809820, 3178988241, 1594193633, 1235154121,  326117244,
    2304031425,  937054774, 2687415945, 3192389340, 2003740439,
    1823766188, 2759543402,   10067710, 1533252662, 4132494984,
      82378136,  420615890, 3467563163,  541562091, 3535949864,
    2277319197, 3330822853, 3215654174, 4113831979, 4204996991,
    2162248333, 3255093522, 2219088909, 2978279037,  255818579,
    2859348628, 3097280311, 2569721123, 1861951120, 2907080079,
    2719467166,  998319094, 2521935127, 2404125338,  259456032,
    2086860995, 1839848496, 1893547357, 2527997525, 1489393124,
    2860855349,   76448234, 2264934035,  744914583, 2586791259,
    1385380501,   66529922, 1819103258, 1899300332, 2098173828,
    1793831094,  276463159,  360132945, 4178212058,  595015228,
     177071838, 2800080290, 1573557746, 1548998935,  378454223,
    1460534296, 1116274283, 3112385063, 3709761796,  827999348,
    3580042847, 1913901014,  614021289, 4278528023, 1905177404,
      45407939, 3298183234, 1184848810, 3644926330, 3923635459,
    1627046213, 3677876759,  969772772, 1160524753, 1522441192,
     452369933, 1527502551,  832490847, 1003299676, 1071381111,
    2891255476,  973747308, 4086897108, 1847554542, 3895651598,
    2227820339, 1621250941, 2881344691, 3583565821, 3510404498,
     849362119,  862871471,  797858058, 2867774932, 2821282612,
    3272403146, 3997979905,  209178708, 1805135652,    6783381,
    2823361423,  792580494, 4263749770,  776439581, 3798193823,
    2853444094, 2729507474, 1071873341, 1329010206, 1289336450,
    3327680758, 2011491779,   80157208,  922428856, 1158943220,
    1667230961, 2461022820, 2608845159,  387516115, 3345351910,
    1495629111, 4098154157, 3156649613, 3525698599, 4134908037,
     446713264, 2137537399, 3617403512,  813966752, 1157943946,
    3734692965, 1680301658, 3180398473, 3509854711, 2228114612,
    1008102291,  486805123,  863791847, 3189125290, 1050308116,
    3777341526, 4291726501,  844061465, 1347461791, 2826481581,
     745465012, 2055805750, 4260209475, 2386693097, 2980646741,
     447229436, 2077782664, 1232942813, 4023002732, 1399011509,
    3140569849, 2579909222, 3794857471,  900758066, 2887199683,
    1720257997, 3367494931, 2668921229,  955539029, 3818726432,
    1105704962, 3889207255, 2277369307, 2746484505, 1761846513,
    2413916784, 2685127085, 4240257943, 1166726899, 4215215715,
    3082092067, 3960461946, 1663304043, 2087473241, 4162589986,
    2507310778, 1579665506,  767234210,  970676017,  492207530,
    1441679602, 1314785090, 3262202570, 3417091742, 1561989210,
    3011406780, 1146609202, 3262321040, 1374872171, 1634688712,
    1280458888, 2230023982,  419323804, 3262899800,   39783310,
    1641619040, 1700368658, 2207946628, 2571300939, 2424079766,
     780290914, 2715195096, 3390957695,  163151474, 2309534542,
    1860018424,  555755123,  280320104, 1604831083, 2713022383,
    1728987441, 3639955502,  623065489, 3828630947, 4275479050,
    3516347383, 2343951195, 2430677756,  635534992, 3868699749,
     808442435, 3070644069, 4282166003, 2093181383, 2023555632,
    1568662086, 3422372620, 4134522350, 3016979543, 3259320234,
    2888030729, 3185253876, 4258779643, 1267304371, 1022517473,
     815943045,  929020012, 2995251018, 3371283296, 3608029049,
    2018485115,  122123397, 2810669150, 1411365618, 1238391329,
    1186786476, 3155969091, 2242941310, 1765554882,  279121160,
    4279838515, 1641578514, 3796324015,   13351065,  103516986,
    1609694427,  551411743, 2493771609, 1316337047, 3932650856,
    4189700203,  463397996, 2937735066, 1855616529, 2626847990,
      55091862, 3823351211,  753448970, 4045045500, 1274127772,
    1124182256,   92039808, 2126345552,  425973257,  386287896,
    2589870191, 1987762798, 4084826973, 2172456685, 3366583455,
    3602966653, 2378803535, 2901764433, 3716929006, 3710159000,
    2653449155, 3469742630, 3096444476, 3932564653, 2595257433,
     318974657, 3146202484,  853571438,  144400272, 3768408841,
     782634401, 2161109003,  570039522, 1886241521,   14249488,
    2230804228, 1604941699, 3928713335, 3921942509, 2155806892,
     134366254,  430507376, 1924011722,  276713377,  196481886,
    3614810992, 1610021185, 1785757066,  851346168, 3761148643,
    2918835642, 3364422385, 3012284466, 3735958851, 2643153892,
    3778608231, 1164289832,  205853021, 2876112231, 3503398282,
    3078397001, 3472037921, 1748894853, 2740861475,  316056182,
    1660426908,  168885906,  956005527, 3984354789,  566521563,
    1001109523, 1216710575, 2952284757, 3834433081, 3842608301,
    2467352408, 3974441264, 3256601745, 1409353924, 1329904859,
    2307560293, 3125217879, 3622920184, 3832785684, 3882365951,
    2308537115, 2659155028, 1450441945, 3532257603, 3186324194,
    1225603425, 1124246549,  175808705, 3009142319, 2796710159,
    3651990107,  160762750, 1902254979, 1698648476, 1134980669,
     497144426, 3302689335, 4057485630, 3603530763, 4087252587,
     427812652,  286876201,  823134128, 1627554964, 3745564327,
    2589226092, 4202024494,   62878473, 3275585894, 3987124064,
    2791777159, 1916869511, 2585861905, 1375038919, 1403421920,
      60249114, 3811870450, 3021498009, 2612993202,  528933105,
    2757361321, 3341402964, 2621861700,  273128190, 4015252178,
    3094781002, 1621621288, 2337611177, 1796718448, 1258965619,
    4241913140, 2138560392, 3022190223, 4174180924,  450094611,
    3274724580,  617150026, 2704660665, 1469700689, 1341616587,
     356715071, 1188789960, 2278869135, 1766569160, 2795896635,
      57824704, 2893496380, 1235723989, 1630694347, 3927960522,
     428891364, 1814070806, 2287999787, 4125941184, 3968103889,
    3548724050, 1025597707, 1404281500, 2002212197,   92429143,
    2313943944, 2403086080, 3006180634, 3561981764, 1671860914,
    1768520622, 1803542985,  844848113, 3006139921, 1410888995,
    1157749833, 2125704913, 1789979528, 1799263423,  741157179,
    2405862309,  767040434, 2655241390, 3663420179, 2172009096,
    2511931187, 1680542666,  231857466, 1154981000,  157168255,
    1454112128, 3505872099, 1929775046, 2309422350, 2143329496,
    2960716902,  407610648, 2938108129, 2581749599,  538837155,
    2342628867,  430543915,  740188568, 1937713272, 3315215132,
    2085587024, 4030765687,  766054429, 3517641839,  689721775,
    1294158986, 1753287754, 4202601348, 1974852792,   33459103,
    3568087535, 3144677435, 1686130825, 4134943013, 3005738435,
    3599293386,  426570142,  754104406, 3660892564, 1964545167,
     829466833,  821587464, 1746693036, 1006492428, 1595312919,
    1256599985, 1024482560, 1897312280, 2902903201,  691790057,
    1037515867, 3176831208, 1968401055, 2173506824, 1089055278,
    1748401123, 2941380082,  968412354, 1818753861, 2973200866,
    3875951774, 1119354008, 3988604139, 1647155589, 2232450826,
    3486058011, 3655784043, 3759258462,  847163678, 1082052057,
     989516446, 2871541755, 3196311070, 3929963078,  658187585,
    3664944641, 2175149170, 2203709147, 2756014689, 2456473919,
    3890267390, 1293787864, 2830347984, 3059280931, 4158802520,
    1561677400, 2586570938,  783570352, 1355506163,   31495586,
    3789437343, 3340549429, 2092501630,  896419368,  671715824,
    3530450081, 3603554138, 1055991716, 3442308219, 1499434728,
    3130288473, 3639507000,   17769680, 2259741420,  487032199,
    4227143402, 3693771256, 1880482820, 3924810796,  381462353,
    4017855991, 2452034943, 2736680833, 2209866385, 2128986379,
     437874044,  595759426,  641721026, 1636065708, 3899136933,
     629879088, 3591174506,  351984326, 2638783544, 2348444281,
    2341604660, 2123933692,  143443325, 1525942256,  364660499,
     599149312,  939093251, 1523003209,  106601097,  376589484,
    1346282236, 1297387043,  764598052, 3741218111,  933457002,
    1886424424, 3219631016,  525405256, 3014235619,  323149677,
    2038881721, 4100129043, 2851715101, 2984028078, 1888574695,
    2014194741, 3515193880, 4180573530, 3461824363, 2641995497,
    3179230245, 2902294983, 2217320456, 4040852155, 1784656905,
    3311906931,   87498458, 2752971818, 2635474297, 2831215366,
    3682231106, 2920043893, 3772929704, 2816374944,  309949752,
    2383758854,  154870719,  385111597, 1191604312, 1840700563,
     872191186, 2925548701, 1310412747, 2102066999, 1504727249,
    3574298750, 1191230036, 3330575266, 3180292097, 3539347721,
     681369118, 3305125752, 3648233597,  950049240, 4173257693,
    1760124957,  512151405,  681175196,  580563018, 1169662867,
    4015033554, 2687781101,  699691603, 2673494188, 1137221356,
     123599888,  472658308, 1053598179, 1012713758, 3481064843,
    3759461013, 3981457956, 3830587662, 1877191791, 3650996736,
     988064871, 3515461600, 4089077232, 2225147448, 1249609188,
    2643151863, 3896204135, 2416995901, 1397735321, 3460025646);

begin
  Seeds(0) := MT.U32(16#123#);
  Seeds(1) := MT.U32(16#234#);
  Seeds(2) := MT.U32(16#345#);
  Seeds(3) := MT.U32(16#456#);
  MT.Init_Genrand(Seeds);

  Put_Line("Generating numbers...");
  for I in 0 .. No-1 loop
    X := MT.Gen_U32;
    Put(".");
    if X /= Result(I) then
      Put_Line("");
      Put_Line("ERROR at position " & Integer'Image(I) &
               "; expected " & MT.U32'Image(Result(I)) &
               " but result is " & MT.U32'Image(X));
    end if;
  end loop;
  Put_Line("");
end Tests_MT;

UDP_Tester
The UDP_Tester has two main components: the sender and the receiver. For convenience, there are simple wrappers for those components - udp_sender.adb and udp_receiver.adb respectively - so that a build of UDP_Tester will directly provide two executables - sender and receiver. The IP and port numbers are knobs that can be easily changed from those wrappers directly since they are passed as parameters to the UDP_tester methods themselves.
udp_sender.adb:

  -- S.MG, 2018

with UDP_Tester;

procedure UDP_Sender is
  -- Sender and Receiver addresses + ports
  Receiver_IP   : constant String   := "161.0.121.202"; -- smg.test machine
  Receiver_Port : constant          := 7000;

  Sender_Port   : constant          := 5000;

begin

  UDP_Tester.Sender( Receiver_IP, Receiver_Port, Sender_Port) ;

end UDP_Sender;

udp_receiver.adb:

  -- S.MG, 2018

with UDP_Tester;

procedure UDP_Receiver is
  -- Port to listen on
  Receiver_Port : constant          := 7000;
begin

  UDP_Tester.Receiver( Receiver_Port );

end UDP_Receiver;

The UDP_Tester.Receiver method simply runs an endless loop in which it listens for UDP packages on a given port number and logs basic information about each pacakge it receives. The UDP_Tester.Sender method assembles and sends in a pseudorandom order all packages with length between 6 (size of own header) and maximum length. Note that the sender will send each size of package *only once* and it will simply finish once it sent one package of each size. Consequently, for data collection purposes, the sender will have to be started repeatedly from outside - most conveniently through a cron task4. Because the workings of both sender and receiver are quite basic, I won't cover them here in more detail - the code and comments should make it clear enough and if that's not the case, you can always ask your questions here in the comments. However, it's worth noting instead a few important points about the tester as a whole and the changes it imposes on the UDP lib:

  • To enable sending of packets with various sizes, the UDP lib had to be made generic. In Ada, this means that a calling packet can then instantiate the UDP type with its desired parameters (in this case the size of the UDP package). Specifically, from the UDP_Tester.Sender code, this reads:
          declare
            K : constant Positive := Packet_Size;
            -- sender will have *current* size of UDP packet
            package UDP_Sdr is new UDP( Payload_Size => K );
    

    Essentially the sender runs a loop and at each iteration it selects one of the remaining packet sizes, it instantiates UDP_Sdr as a parametrized UDP package and only then proceeds to use it to send the actual data.

    Note that this change is *not* required in production - it's simply needed for testing purposes. It's for this reason that it's part of the .vpatch for UDP_Tester rather than a .vpatch in itself on the UDP lib directly.

  • The sender will wait 1 second between any 2 packages it sends. This is mainly because a first version without this delay proved to result in significant packet loss that was clearly due to the burst mode (i.e. more than 2000 packages sent in 1 second) rather than anything else. Since the point of the test was to gather some data on UDP communication in relaxed rather than stress conditions, the delay of 1 second was introduced. However, should you wish to stress-test for any reason, you can easily remove this delay - simply comment out the "delay 1.0" line in the sender and that's all.
  • The UDP_Tester currently logs some information on both sides. Both sender and receiver will first check for a fixed-name file and if it doesn't exist they will create it, writing the header to it as well. After that, all new data will simply be appended to the file. The sender logs the size, local time, destination IP, destination port and seed used for the MT prng - this seed is the local time (unix style) when the sender is started. The receiver logs the size (both as received and as claimed in the packet's own header), the time (local time when sent and local time when received), the source IP, the source port and the number of octets that are different from expected. In addition, the receiver will further log in a separate file the actual octets that are found to contain errors, if any (together with information to easily link them back to the exact package they were part of).
  • There is no attempt to match directlty the times at the sender and at the receiver - any difference between the two local times should be noted when the tests are run and then provided together with the data so that the analysis can be done in a meaningful way. Specifically, the local times used by sender and receiver are simply retrieved with Ada.Calendar.Clock and then converted to unix style by subtracting from them the "epoch" (0 on 1/1/1970).
  • The use of generic packages and of Ada.Calendar in UDP_Tester imposed a relaxation of restrictions upstream i.e. in the UDP and MT libs. Since the relaxation of those restrictions is again (just like the generics change) required for testing *only*, they are simply commented out as part of the changes included in the UDP_Tester .vpatch.

To obtain a working UDP_Tester, simply download the relevant set of .vpatch and .sig files and press with your favourite V5. The patches are available as usual on my Reference Shelf and linked from here as well, for your convenience:

At the moment I am running the above tester in both directions between a node in the UK and one in Uruguay so hopefully I'll soon have some data to look at!


  1. Follow the discussion in the logs as there is no better way to get the details than to actually read through the log. 

  2. M. Matsumoto and T. Nishimura, "Mersenne Twister: A 623-dimensionally equidistributed uniform pseudorandom number generator", ACM Transactions on Modelling and Computer Simulation Vol. 8, No. 1, January, pp.3-30, 1998. 

  3. If one really wants a default value, this could be perhaps added as a constant in the package and then used as such by the caller but I must say that I don't quite see the point of this. 

  4. It is of course possible to change the code so that the sender also runs in a loop and simply wakes up at given intervals and resends packages but I don't quite see any real benefit to doing it this way - on the contrary, I see benefit to *not* doing it this way, since cron tasks are quite perfect for this and even more robust (no idle time and guaranteed recovery even in more extreme cases such as computer reset). 

  5. NB: Using Keccak hashes, as per current republican standard for V-trees. 

Theme and content by Diana Coman