A Walk among the Trees of V

About two weeks ago, I took one hour to produce and publish a tiny vpatch on the mp-wp V-tree, with the whole experience triggering for me a need to look at V1 again, as noted in the logs:

diana_coman: http://logs.ossasepia.com/log/trilema/2019-11-29#1953885 - tbh I think we do; I've been mulling this for a while ie I'm not happy with the full V setup as it is - at least as it is for me currently - but so far it never made it to the front to push for some solution.
ossabot: Logged on 2019-11-29 17:10:06 mircea_popescu: do we have a problem there, is it ?
mircea_popescu: well, ideologically the thing stood at "let development be painful-er than entirely painless, it'll produce better code in the end".
mircea_popescu: then again, that was pushed through in a different context, maybe needs reexamination
diana_coman: mircea_popescu: trouble is that it's *not* development that is painful-er, not at all; it's...publishing that is painfuller and I doubt that has any merits.
diana_coman: should add that I think the problem is relatively superficial ie one of tools as they currently stand, not one of principles.
diana_coman: and then ofc something stares me in the face but I can't yet even fully scope it, gah.

Given the above, I took the time and trawled the logs and the blogs and the whole of TMSR really, on a V-quest of sorts, to recover for myself the clearer picture of what was initially proposed, what was further refined, what became/is available, what's current practice and what is indeed visibly missing or misshaped - if anything - in all of this. While I can't tell upfront whether or to what extent anyone else will agree with my findings or my views on this or anything else at all really, I consider my time on this well-spent anyway, as there was plenty that I didn't quite remember properly nor found otherwise referenced from a single place and so I'll start with a timeline of the main documents I found, mainly to save me a lot of digging for it all at next V-considering juncture:

  1. In August 2015, the V-genesis signed document is published on the Foundation's mailing list2
  2. On the very last day of the same August 2015, Mircea Popescu republishes the text in the signed message at 1 above and adds to it a Python implementation of V, as a NoSuchLabs product by asciilifeform3.
  3. In January 2016, Mircea Popescu publishes The V Manual Genesis, stating in clear V's fundamental principles.
  4. In February 2016, Benjamin Vulpes publishes a gentle introduction to V, which becomes over time - and still is, as a simple log-search will amply show - the canonical reference thrown at all newcomers and deemed an excellent introduction to the use of V4.
  5. In December 2017, I started my Reference Code Shelf and noted in its very introduction that code is the current literature and V-signatures enable meaningful reviewing and as a result publishing of the text known as "code".
  6. In January 2018, Spyked starts his own store of V-signed artefacts, introducing it with a discussion on intellectual ownership as enabled by V.
  7. In June 2018, Trinque publishes the specification of the relatively new addition to V - the manifest file.

Other than the above, there are also several implementations of V5, a few learning-V articles (such as that of 2018 by a meanwhile vanished Esthlos or a more recent one by Whaack who helpfully groups some links together as well) and a lot of discussion of all sorts6, scattered through the logs and never synthesised anywhere, as far as I can see. As it's even noted at some point:

mircea_popescu: but even if one could say "well,premature to write a v compendium, not yet even stable yet", the greater point remains. there's been a dispute of tabs and spaces in the logs. nowhere else. how would one know to look ? there's been a dispute of alphabets and trees and MANY Things that constitute the "Subtle" in the above "subtle chjanges each month".
mircea_popescu: v is esentially found today as an enchanted castle surrounded by 5000 rakes upon which the prince is welcome to step, and once he stepped he can come to us and by the shape in his forehead we can describe the rake he stepped on.
mircea_popescu: all nice and good, especially if you're not the prince...

I do not presume that I'm writing here any V compendium, mainly because I don't think it's yet all that well and clearly defined to start with. Nevertheless, for my own future reference and with my very practical & personal goal in mind of figuring out the current state of V and what exactly is bothering me about it all, here's a list of what I consider to be the most significant points of V, theory and practice, as extracted out of all the conversations, the few reference materials listed above and the observed use of V by those in my WoT:

  1. Code is text7 and V is the enabler of a literate approach to coding8 that includes as a minimum:
    1. The only possible source of code (and hence author of code) being the reader of code and never the writer9. Worth adding here that only a WoT-person can ever be a reader of code10 and only through their signing and publishing of the corresponding vpatch and seal.
    2. Code itself being - as the text that it is - deliberately structured with due consideration given not only to its meaning, but also to its source, and to its context.
    3. While each V-tree can only ever have one single genesis, the textuality and intertextuality inherent with code-as-text means that over time the different V-trees are nevertheless tightly inter-related as they provide context, reference and further meaning to one another. This is also known as building Yggdrasil as everything is meant to eventually merge - though never "automatically" - into a single tree.
    4. As each V-tree is itself the context for any and all code contained, it follows that moving code from one V-tree to another is effectively an act of translation and as such it can never be automated or a mechanical copy/paste. What is required each and every time is an author (hence a reader & signer).
  2. Writing code is a liability and as such, it should be a painful rather than painless activity, as to result in better code overall11.
  3. Any V implementation is only as powerful as the underlying differ that it relies on. This is still an unsolved/unclear problem and even the parts that are clear - a differ should detect moves as distinct from delete+add - are not yet implemented.

As the above main points are essentially the state of the V-theory rather than V-practice, I think it's worth looking as well at the current state of practice to see how well and in what places it actually fits the theory at all. My summary of existing V-practice is this12:

  1. Use:
    1. Seven Reference Code Shelves by any other name or indeed whether named at all or not: ave1, diana_coman, jfw, lobbes, phf, spyked, trinque.
    2. Most vpatches published are introduced by their initial (ie first-signers) authors through dedicated articles on the author's blog. It's worth noting that some authors (most notably hanbot, billymg and asciilifeform) do not provide a single place for all their signed vpatches. Each vpatch is simply linked and thus easily found only from the corresponding article that introduces it.
    3. Some additional signatures are provided by some authors without any discussion/article/own publishing of the signed vpatch that I could find. Those seem to be however mainly older (eg signatures on the trb original vpatches as stored on btcbase.org. Sometimes signatures are provided simply in the comments on the introduction article (as is the case for some of the FFA patches, it would seem, e.g. this sig by Benjamin Vulpes).
    4. Some items that are apparently in regular use - such as vtools or the perl implementation of V or the Ada implementation of keccak - fail to get signatures from all known users. What this means is entirely unclear to me.
  2. Implementation/tools:
    1. One incomplete differ as part of the vtools suite, signed by bvt and phf.
    2. Two implementations of Keccak hashing: one in Ada signed by diana_coman as part of EuCrypt and then translated to vtools & signed as part of that by bvt and phf; one in C signed by jfw.
    3. Three implementations of a V-presser: one in perl signed by diana_coman and spyked, one in Python signed by phf, one in Lisp signed by esthlos.

Looking at the current use of V through the lens of the theory's main points and considering also my original 1-hour-for-vpatch trigger for all this, it strikes me that there is essentially some incomplete/limited support for the technical V-press/vpatch production and only a tentative "emerging best practice" regarding the literate approach to code as text - maintain a code shelf + publish an introductory article if you are the first signer of a vpatch seems to be all of it really13. Overall, I also don't quite see much of Yggdrasil yet - perhaps it's just too early or perhaps it will really take a lot more reading and discussing of other people's code before that will happen.

From my own perspective and after mulling all the above for some time, I ended up with quite a list of issues that poke me in the eye:

  1. While I have made as much use of MP-WP for code discussion and presentation as I could, it rather hurts each and every time - hence my original note that it's not at all the code writing that is more difficult but the code publishing that hurts and I really think this hurt hurts in precisely the wrong place really. Thinking of this, my current inclination is to make myself a script to format the content of each vpatch properly for article-inclusion, having at the very least: lines numbered and with html hooks for reference, ALL the antecedent code shown (if there is an antecedent, ofc) and on it, each deletion/addition/change marked and clearly visible (colours/background, most probably). Adding footnotes to code directly might make sense too for that matter, though I can see it getting perhaps too heavy.
  2. Trying for the sake of this article to really cover *all* vpatches in my WoT and to at least not miss anybody's discussion of code and/or signature on anything was a huge pain. The sort of pain that pushes for a script to walk my WoT and check periodically people's code shelves (IF they have one...), pinging me on any new and/or unsigned-by-me vpatches. Basically a "feedbot" specifically for V, what! Let there be a way to listen on the vpatches conversation by itself.
  3. There's very little towards no discussion published of *other people's code*. Ave1 has some on FFA (e.g. on chapter 2) and I have some on FFA as well (e.g. reading notes and an overview) but I can't say I saw much else to read from others on any code other than their own. I have no idea what's the issue here and therefore no proposal for it either.
  4. The only translations of code that I'm aware of would be the keccak translated from EuCrypt to Vtools and otherwise my own translations of EuCrypt parts (mainly for Eulora's server code, hence not to be published as such).
  5. Grinding and regrinding of vpatches seems to be an extremely rare event so far. I think this is not ideal at all but on the other hand just thinking of all the time that I'd sink into a full regrind of EuCrypt for instance - which is not even signed by others so I don't even have *that* headache! - makes me lose instantly any desire to regrind it. Perhaps this is as it should be but I'm not so sure it's helping in any way towards any Yggdrasil. Still, this can wait presumably until a later time when regrinds are more actively needed and therefore this pain comes to the front - either to be accepted as a useful pain or to be otherwise reduced.

As a conclusion to this mammoth of an article14 I can now look happily at the fresh list of "to do" stuff that might - or might not - even help alleviate some of the pain I see in the above. On the bright side, it all served at least to unload from my head that niggling "gah" and therefore to be able to get back otherwise to all the rest of gahs to be addressed in other places. May all this be of some use to my future self at any rate.


  1. V is meant to be for Victory; one-letter names though make for SUCH a pain when searching for stuff. 

  2. Since meanwhile the link changed, the Foundation is dead (if not yet burried) and all my previous experience shows one has to mirror stuff if one wants to be able to find it again some years later, here's the text of this so-called "Ode to V" signed message:

    -----BEGIN PGP SIGNED MESSAGE-----
    Hash: SHA512

    The V-genesis patch is a most important milestone in the evolution of computer programming, from its troglodyte state prior to the involvement of The Most Serene Republic, towards its civilised state as you enjoy it today.

    For the first time in the century or so of history of this particular human endeavour, text was deliberately structured with due consideration given not only to its meaning, but also to its source, and to its context. Prior attempts at structuring software, at first consisting of a naive approach focused on meaning only, over time added a half-hearted consideration of context, very unequally and rather haphazardly. The change the V-genesis introduced is exactly like the move from understanding and controlling movement in terms of mass and somewhat velocity, such as it occurs in the mind of a monkey throwing rocks, to understanding and controlling movement in terms of mass, impulse and energy, such as it occurs in the launching of satellites. The proposition that there is still room for improvement in the endless march of human thought, on this topic as on any other, is absolutely indubitable. The proposition that the sum total of what came before can be compared with this particular advance is strictly ridiculous.

    Used together with specialised scripts, V-genesis allows an agent to reconstruct a complete Bitcoin tree, verify its correctness, and manage his investment of trust at all junctures so that he is never required to implicitly trust either an unknown code author, or a code snippet of unknown provenance.

    There remains after today no alternative manner to deploy Bitcoin software, or indeed any software that is not a toy intended to be used by children playing, outside of this paradigm. May the switchover be bloody and painful in all the right places.
    -----BEGIN PGP SIGNATURE-----
    Version: GnuPG v1.4.10 (GNU/Linux)

    iQIcBAEBCgAGBQJV16UhAAoJEIpzbw4vt7RSpiIP/j7y5WAmLlw2/K9JeGkrhxUy
    O9LYLfzfovApcm+BqI9ZKQ9RSmkfxSLtomuCrVqQJ5ZjAuL6ODhlQDPfNg6RxEDl
    gaYqx3flUG/YruF9xHodiyOL1X1RPKXqorfOVGsZBJ5xDPReTVE5OdhTJYXfq/cF
    Em7FAXP6RQk2vyAjiAjpAewL/dMr3la9ur6cMARFSvBPuck6cHiaEn6NCruW4+oW
    bbdDA/eMgrnh1p48C7CST+JZt7lAIDJBYAp37CzS7XCY+CXf6qnhAyVRf/FgTkI7
    jnc5u8TpfY8OpLrK3P2TL+4MmlvWY+q/j2Qaz3p5lsXiabEdo3qaZzu+q3dT5UX9
    SxmiAJP8XknLk5RY/RgbA+aVR8hqhKX8w8XA59Ois0y+s0sLGcF6V3XablCFQMnV
    LfIOe+rgf8j5CqXgMHBDW8KX0IFgUPw+90OX4mdz8kucfMSx7MlrjYxFXI5EHDHu
    aEehCrhYUrHQwi3i5mb9EOLwJOz4sfk+UM2lquDOGEGLtZaW2PS0xNgs/cWyNFlJ
    vTAkevd2Gnr3fN21ZyWFPzjSPrpPrBhvd7nLcQZ69snS5C5R7XO6SOWXOuYJZWL0
    gkV1KqusvuFXcOKFwaOmxqOc3ssX699dVb935L66/ze8WlL7FWOssH/PljGhmoDA
    YePXbkM4NBoFHkngSfnw
    =+YVp
    -----END PGP SIGNATURE-----

     

  3. Meanwhile NoSuchLabs closed down and asciilifeform can't work with MP

  4. As with the Ode to V item, I'm uneasy with NOT mirroring this, because meanwhile Benjamin Vulpes has lost his interest in either V or TMSR altogether. So here's the dump and let the footnotes grow forever longer, as they apparently do whether you want them to or not:

    V-tronics 101: A gentle introduction to The Most Serene Republic of Bitcoin's cryptographically-backed version control system
    Filed under: V, common lisp, software development, tmsr — Benjamin Vulpes @ 12:00 a.m.
    V-tronics 101: A gentle introduction to The Most Serene Republic of Bitcoin's cryptographically-backed version control system

    As befits a properly revolutionary agglomeration of individuals, The Most Serene Republic of Bitcoin is so willing to throw out staples of the "modern development workflow" that one might go so far as to call us hidebound reactionaries lusting for the bad old days of subversion and CVS. The inherent appeal of watching not just enemies but also random idiots passing by squirming on the stake aside, V marks a sea change in how thinking people work on software together. I am not the first to note this.
    What else has been written on the topic?

    Mircea Popescu's original Ode to V (technically an ode to the Genesis Patch)
    Mircea Popescu's The V Manual Genesis
    Stanislav's original V
    punkman's V
    Shane's muchly-iterated upon V (link updated to point at the Foundation website, since the files linked from the mailing list post 404 now)
    my own shitty and not-working-properly-or-maybe-even-at-all V
    phf's excellent patch flow visualizer

    The operators of any of these V implementations may be found and interrogated politely in #trilema.
    How does it work? What does it do?

    If you have:

    source code patches
    signatures of those patches
    public keys for those signatures

    …V will 'press' a source tree from those patches, filtered by those signatures, and further filtered by the public keys. In other words, V delivers a hard guarantee that you (or anyone) can produce a source tree comprised only of code that people you trust authored or reviewed. This stands in stark contrast to building the "master" branch of the Power Ranger Bitcoin Client, which contains a zillion patches from people you've never heard of, much less that you trust to write code that will in point of fact touch your money.

    V is a very simple beast. It loads (v)patches from one directory, detached signatures of those patches from a second directory, public keys from another directory, determines the correct order to apply those patches, and then does so.

    Stan's original V shipped with 8 tools:

    wot Lists out the keys V will use to filter the patchset. Distinct from the B,TMSR~ Web of Trust, the key dir represents the WoT that is trusted to produce a given reference implementation (RI) source tree.
    roots Implicitly a function of the whole patches/seals/wot setup (as is the whole system), identifies and prints to the terminal all "root" patches (literarily, the patches upon which the entire patchset depends but that do not themselves depend on anything else, and technically any patch with a false antecedent).
    antecedents Takes 1 argument, the patch for which to look up antecedents. Returns a list of patches upon which the given patch depends.
    descendants Mirror of antecedents, returns patches that depend on its argument.
    flow Prints the ordered list of all (valid, signed) patches that could be pressed.
    press Takes one argument, the final patch V should apply to the RI source tree. Has the side effect of applying all patches returned by flow up to and including the patch passed in as an argument.
    origin Takes one argument, the SHA512 hash of one of the files in the tree, and returns the patch that produced that file in that state.

    vpatches and vdiff

    The whole edifice is built upon a foundation of vpatches, which are produced by the hitherto unmentioned vdiff. I'll reproduce vdiff (another of Stan's productions) here, as it's brief enough to inline:

    #!/bin/bash

    diff -uNr $1 $2 | awk 'm = /^(---|\+\+\+)/{s="sha512sum \"" $2 "\" 2>/dev/null " | getline x; if (s) { split(x, a, " "); o = a[1]; } else {o = "false";} print $0 " " o} !m { print $0 }'

    Which, near enough as I can tell from reading the source, and as far as my experiments corroborate, mutates the output of the Unix diff tool to include hashes of each file as it is written to disk in directory a and as it is written to disk in directory b. The algo is as follows:

    diff the two directories
    Pipe the results of that diff into an awk command that:
    searches for either '—' or '+'
    runs sha512sum on the awk match immediately following
    Which you'll note is always the relative path to a specific file in the tree, and is guaranteed to be the filename because uncoerced awk splits on spaces
    assigns the first line from the output of sha512sum to the variable x
    if sha512sum returned a hash value for the file, replace the '—/+++ ' line with a '—/+++ ' line
    Write mutated Unix diff to standard out

    Perhaps a worked example will make this a little bit more clear:

    $ tree
    .
    -> a
    --> a.txt
    --> b.txt
    -> b
    --> a.txt
    --> b.txt

    2 directories, 4 files

    $ diff -uNr a b
    diff -uNr a/a.txt b/a.txt
    --- a/a.txt 2016-02-07 05:41:46.000000000 +0000
    +++ b/a.txt 2016-02-07 05:41:46.000000000 +0000
    @@ -1 +1 @@
    -foo
    \ No newline at end of file
    +bar
    \ No newline at end of file

    $ vdiff a b
    diff -uNr a/a.txt b/a.txt
    --- a/a.txt f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc6638326e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7
    +++ b/a.txt d82c4eb5261cb9c8aa9855edd67d1bd10482f41529858d925094d173fa662aa91ff39bc5b188615273484021dfb16fd8284cf684ccf0fc795be3aa2fc1e6c181
    @@ -1 +1 @@
    -foo
    \ No newline at end of file
    +bar
    \ No newline at end of file

    $ vdiff a b
    diff -uNr a/a.txt b/a.txt
    --- a/a.txt f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc6638326e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7
    +++ b/a.txt d82c4eb5261cb9c8aa9855edd67d1bd10482f41529858d925094d173fa662aa91ff39bc5b188615273484021dfb16fd8284cf684ccf0fc795be3aa2fc1e6c181
    @@ -1 +1 @@
    -foo
    \ No newline at end of file
    +bar
    \ No newline at end of file
    diff -uNr a/b.txt b/b.txt
    --- a/b.txt 6ac1a0cc10e1337e14eaa72e33c0f64fa4d401f9b284c59142b595b7f1c7bf4f72da82f81c903845b242b837b237feacf7b43481187cb35f73c77b936f498188
    +++ b/b.txt 67574bf4234031b94f660f9bfc5bf2b08e470d5c1fdc06a5cb67dc405c21c6be15a0a19df54fd81c50caf0a0650b5da0622f9a1108771329ebda8329d66ca5a6
    @@ -1 +1 @@
    -zanzibar
    +zanzibare

    $ tree
    .
    -> a
    --> a.txt
    --> b.txt
    -> b
    --> a.txt
    --> b.txt
    --> c.txt

    2 directories, 5 files

    $ vdiff a b
    diff -uNr a/a.txt b/a.txt
    --- a/a.txt f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc6638326e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7
    +++ b/a.txt d82c4eb5261cb9c8aa9855edd67d1bd10482f41529858d925094d173fa662aa91ff39bc5b188615273484021dfb16fd8284cf684ccf0fc795be3aa2fc1e6c181
    @@ -1 +1 @@
    -foo
    \ No newline at end of file
    +bar
    \ No newline at end of file
    diff -uNr a/b.txt b/b.txt
    --- a/b.txt 6ac1a0cc10e1337e14eaa72e33c0f64fa4d401f9b284c59142b595b7f1c7bf4f72da82f81c903845b242b837b237feacf7b43481187cb35f73c77b936f498188
    +++ b/b.txt 67574bf4234031b94f660f9bfc5bf2b08e470d5c1fdc06a5cb67dc405c21c6be15a0a19df54fd81c50caf0a0650b5da0622f9a1108771329ebda8329d66ca5a6
    @@ -1 +1 @@
    -zanzibar
    +zanzibare
    diff -uNr a/c.txt b/c.txt
    --- a/c.txt false
    +++ b/c.txt 5a7992b1a354c174e074b9a110561fca44d6e38f45e3f9dd80207f30139587ea00f2392ad285f2287f6033348b7190779d1f48c1628926c8da30e63b8eef1e5f
    @@ -0,0 +1 @@
    +trichome

    $ sha512sum a/a.txt
    f7fbba6e0636f890e56fbbf3283e524c6fa3204ae298382d624741d0dc6638326e282c41be5e4254d8820772c5518a2c5a8c0c7f7eda19594a7eb539453e1ed7 a/a.txt

    sha512sum b/a.txt
    d82c4eb5261cb9c8aa9855edd67d1bd10482f41529858d925094d173fa662aa91ff39bc5b188615273484021dfb16fd8284cf684ccf0fc795be3aa2fc1e6c181 b/a.txt

    Of note:

    identical files across trees are never mentioned
    files in tree B but not A show 'false' for their antecedent hash value
    vdiff output shows the 512-round SHA hash of a/a.txt and a/b.txt in the expected place in the diff

    Complexities

    There are 2 complex subsystems in a V implementation: the GPG integration, and the toposorting. Let's start with the simple one and move to the complex one.

    GPG, amusingly to me, is actually the simpler of the two. The sole complexity in working with GPG is that it really really really wants to be a stateful program, going through "key importation" gyrations, and writing those keys to a special place on disk – the "keyring". The approach I take with my V-tron is to create a brand-spanking-new GPG "homedir" on every single run, and pass the location of that homedir to all calls to GPG on that run. New run? New GPG homedir. If you go so far as to use `mktemp`, you may even choose to eschew trashing the GPG homedir at the end of the run. The most important thing to note about working with GPG in the context of a V implementation is that persistence of state past a given run is categorically forbidden, and doubly so whenever touching GPG.

    By far the most complex component of V is the toposort, and the program semantics supporting it. I'll include 2 implementations: one from Stan's original v.py and one from my own derpy lispy implementation.

    Original first:

    def toposort(unsorted):
    sorted = []
    unsorted = dict(unsorted)
    while unsorted:
    acyclic = False
    for node, edges in unsorted.items():
    for edge in edges:
    if edge in unsorted:
    break
    else:
    acyclic = True
    del unsorted[node]
    sorted.append( (node, edges) )
    if not acyclic:
    fatal("Cyclic graph!")
    return sorted

    Stan delivers a beautifully terse toposort implementation here. He leverages language features in both data structure and control flow, and the resulting code is compact and readable.

    It takes a single argument, the unsorted list of patches, and returns the sorted list of patches. In between, it coerces the unsorted list to a dictionary (leveraging a language feature), and starts looping over the list of patches, setting the cyclicity flag to false on each run. While iterating over each patch, it also iterates over each patch's descendents (eg, the patches that depend upon it). If it finds any of the patch's descendents in the list of yet-unsorted patches, it terminates that iteration at the patch level, and moves on to the next patch. Should it fail to find any descendents in the list of yet-unsorted patches, this indicates that the current patch is a leaf node on the patch graph, can be appended to the list of sorted patches, deleted from the list of unsorted patches, and the acyclicity flag set to true for this iteration over the unsorted list. Predictably, this leaves the root node (assuming there's only one) for last, which means the list must be reversed once it's been constructed. Once the list of unsorted patches has been exhausted, the sorted list is complete and returned to the call site.

    Should at any point the iteration over patches (nodes in the implementation) complete without setting the acyclicity flag, this indicates that the patchset is cyclic and invalid. Cyclic patch sets (eg patch A depending on B depending on C which in turn depends on A) will be impossible to press, and a V implementation must guard against it.

    Now, for an exercise in humility. Since the staff rarely let me program anymore, and only in environments in which I was moderately well-practiced before self-promoting to the appropriate level for my own incompetence (which is to say shell scripting and Linux sysadminnery-cum-automation, Python webapps, and Clojure whatever), I like to do my work for B,TMSR~ in Common Lisp wherever possible. Clojure having been a wonderful learning experience, and myriad people to whom I look up claiming that 99% of other languages "features" to have been implemented in CL before I was even born, I strive for competence in it. Strive being the key word. I am also slaving to wrap my head around CLOS, the Common Lisp Object System, so there's a fair bit of object-oriented programming in there.

    (defmethod toposort (unsorted)
    (let1
    (loop
    while unsorted
    with sorted = '() do
    (loop
    for node in unsorted
    for edges = (get-descendents node patches)
    with acyclic = nil
    do
    (block node
    (loop for e in edges do
    (if (find e unsorted)
    (return-from node) ) )
    (setf acyclic t
    unsorted (remove node unsorted)
    sorted (cons node sorted) ) )
    finally (assert (eq t acyclic) ) )
    finally
    (return sorted) ) ) )

    I strove for a clean port of the previously-cited implementation to CL, but there are obviously some differences. Instead of contrasting differences, I'll simply describe how mine works.

    The first thing done is to make a local copy of the unsorted patches – this makes referentially-transparent calls into the rest of the project significantly easier. Stan's program makes excellent and judicious use of global state, but I am nowhere near disciplined enough to do the same to good effect. After binding a copy of the unsorted list, the toposort implementation enters its main loop.

    The main loop repeats for as long as there are patches in the unsorted list. Iteration over the list of patches sets the acyclicity flag and manages the mapping of patch descendents to graph edges in Stan's semantic map. With the node, edges, and cyclicity flags assigned, the toposort function enters the 'node' block, looping over the edges and returning from the 'node' block if any patch descendants remain in the unsorted list. Should the node block fail to find descendents in the unsorted list, it performs the same actions as the equivalent Python function, setting the acyclic flag to `t`, removing the node (patch) in question from the unsorted list, and consing the node (patch) onto the sorted list.

    Yes, it conses furiously. No, I don't know why that's a bad thing, or even that it is a bad thing in this context. GAZE UPON MY IGNORANCE YE MIGHTY AND TREMBLE.

    Finally (heh), the toposort function returns the sorted list.
    Who should use it?

    People who:

    need a version control system built on a foundation of strong crypto
    In stark contrast to version control systems with strong crypto bolted on the side, as in the case of Git (and I assume DARCS, but this is probably just underinformed slander), a robust V implementation will perform all possible cryptographic operations before talking to the user, and should any such operation fail it will complain loudly and take no other actions.
    need to publish changes to a codebase independently of any version control system
    This is entirely possible with V! vpatches can be applied by the unix patch tool with no modifications, and in an utterly worst-case scenario it should be entirely possible for anyone to eventually cough up a toposort, if they need it badly enough.
    people who need guaranteed code evolution provenance
    While it's possible to dig through a Git history for the commit logs, what actual hard guarantee do I have that those logs have not been conjured forth from thin air to say whatever the conjurer wishes? What V brings the world is a system for collapsing signed patches into a source tree.

    Who should write their own?

    Anyone who intends to work on the Reference Implementation should reimplement V (and I suspect that most doing so already have). It is not a terrifically complex task, I knocked my first version out over the course of a weekend, and a weekend is a small price to pay for the knowledge so derived and to build the confidence of those you might hope some day to call your peers in your work.

    To paraphrase a frequent utterance of Mircea Popescu, the era of writing lots of software is over. The Republic is now far more interested in well-vetted software of provably known provenance, reimplementations of extant tooling by members of the WoT, and perserverance in the face of nobody giving a shit about your life's work, than we are in "more tools", "better tools" or random code delivered by random nobodies.
    Where do I get a copy?

    Mine's not ready for public consumption, so you can't. There's an old version kicking around on various hdd's, but it's really really not suited for consumption, so I shan't link it again. I linked Stan's original above in my sources section.

    Beyond that, I strongly suggest you start at The Foundation website. Search for "V:" and "vdiff.sh". Check signatures, and read the source. Then sit down, and write your own.
    Summa

    This is mostly a bridge between the ultra-technical and highly manual world of V operator tooling, and the rest of the world that might be capable of reading various V sources but can't be arsed for whatever reason. I'll leave the bombast and politics to those typically responsible for such.
    Footnotes:

    ascii_butugychag: btw i hope everybody understands that life with 'v' is always going to resemble dark age blood sports like cvs, etc. far more than modern greased poles (e.g., 'git')

    The man(?) may be one of the more succesfully anonymous contributors to B,TMSR~.

    The very same who wrote the original ERC/gribble integration! Of note to noobs is that Phil sent that to me months before he ever started talking regularly in #bitcoin-assets. This eased his entry into the fold dramatically. Contrast it with polarbeard's recent contributions: a 2,600-long patch containing both an epic rewrite of the log messages and a nuking of the rather stupid 'log rotation' feature Satoshi's bitcoin shipped with, that I had to bereate him into splitting the useful part out of so that I could review it in isolation.

    The PFV implies some amount of V implementation running behind the scenes and under the hood, but as of this date phf's declined to share the operating mechanics of the thing. His implementation calls out to a patched C GPG library instead of shelling out to the GPG binaries, which is particularly interesting.

    There are no users of V. One may operate V for results, but one may not simply use a V implementation and expect it to work. It is not a cell phone to be used by people who have no business touching computers, it is far more akin to the cockpit of a glider with analog dials and physical levers upon which to push and pull for the extremely-well-versed-in-how-it's-actually-supposed-to-work-and-in-which-environments.

    In the same way that there are no users of V, there is no single V. At this point in time, and as far as I can tell, everyone working on The Most Serene Republic's Reference Implementation also has their own implementation of V's base functionality. Stan has his own, Phil idem, Shane one that bears the imprimatur of The Foundation (by virtue of his signing a zillion copies of it), punkman his own variant descended from Stan's, and me my own cobbled-together version.

    I know that this is misformatted. Thank you for your concern.

    NB: this list is actually ascii-betically sorted before it's passed to this function.

    My deep and abiding hatred of OOP stems from having learned it in systems that implement it miserably. Python and Ruby: I'm looking at you.

    And in point of fact when rewriting my own V implementation was bit more times than I care to remember by careless bits of global state lying around.

    The function `get-descendents` takes a node and a list of patches, and examines the patchset to find the node's descendents. This is done by sucking the patch-in-question's hash out of its body (read in during patchset instantiation), and regexing all patches bodies for that hash in the antecedent position (you may remember my referencing '—/+++' in describing vpatches: hashes in the '—' line are antecedent hashes, and hashes in the '+' line are descendent patches. `get-descendents` now takes the patchlist as an argument in a vain attempt to burn the errant global state out of this program.

    Loop macro aside, this is probably the most notable divergence between Stan and my toposort implementations: I have to name the block from which I want to break out, and in Python breaking out of a loop is trivial.

    1. patches unsorted [↩]

     

  5. Mod6's v.pl that I genesised and packaged including a starter pack for total newcomers, Phf's vtools that got meanwhile included in my starter pack, Esthlos' lisp implementation, Asciilifeform's Python implementation. 

  6. E.g. talking about V like you would talk about the Bible, the matter of identity as it finds itself reflected in a single-genesis for each V-tree, the relevance of textuality and intertextuality, implementation specifics, spaces-vs-tabs in vpatches, admissible length of code lines and the inadmissible injection of control characters in text and therefore in code or even the ideal code that should apparently be an ast in sexpr directly 

  7. Note that this is at times still disputed in some discussions as preserved in the logs, sometimes quite explicitly as in asciilifeform: mircea_popescu: keep in mind that it ain't 'text', tho, it's a proggy, the line is a semantic unit

  8. See for instance: the original V release, the textuality and intertextuality, the policing of code as the currently relevant form of speech

  9. Sure, at times the two may coincide but that's irrelevant really. 

  10. Identity itself being predicated on the RSA keypair and presence in the WoT. 

  11. I don't buy this as it is stated. I think it's really the poor code-writing habits that should cause pain, not generically any writing of code. While I get it that one view is to regard all and any code as the evil and therefore all code writers as deserving of as much pain as they can have heaped upon them, I think this is frankly rather idiotic. 

  12. If I missed something important, do shout in the comments below, please! 

  13. I'm sorely tempted to add here, based on some previous experience: bitch about others not signing your own vpatches but do not sign yourself vpatches in trees genesised by others, if you can help it all. Perhaps this is not yet an emerging practice though, I should hope. 

  14. It's ~2.3k words without any footnotes included, ~8k words with all the footnotes included and I don't even want to count all the references. 

10 Responses to “A Walk among the Trees of V”

  1. Pun făianță bat la cuie fut in cur si dau la muie

    1 sounds rather like the need for a mp-wp plugin, which isn't in itself a crime. if (( starts a footnote, let [[ start a code or w/e neh ?

    2 seems entirely spurious, let people have a rss feed on their code shelves and just subscribe via feedbot. it's trivial to add moar rss feeds in mp-wp, you have one for articles and one for comments by default but srlsy nao...

    I think 3 is legitimate but also slowly resolving under significant pressure. We're stuck first throwing off the shitty communicators before useful communication can become a self-supporting activity.

    Re 5, it doesn't need helping. it comes when it comes, like jesus. any kind of jesus that actually needs my halp for anything isn't worth urinating upon.

    Also, this was nothing like a mammooth of an article! It was a cute baby elephant fights dog of an article!

    Re footnote 11 -- god bless you, if we ever manage to find the way to make stupidity more painful than sanity in some sort of manner perceptible by the bovines we've won all the wins possible.

  2. Diana Coman says:

    Lolz.

    1. A plugin it might be, I kind of hesitated on this one and mainly because I'm not particularly fond of eating up mp-wp too after the previous brief dive.

    2. Or that. Currently not even everyone has a shelf to start with and moreover I have no idea if really ~any incarnation of that can be set to trigger the rss feed. If it can, perfect.

    3. Good if it turns out to be exactly the case. I can see how it might, indeed.

    5. Yeah, hence "it can wait".

    And where/what's the dog?!! Lolz.

  3. Nitpicks: first discussion of V per se was here. Shortly after, TRB genesis in the proper form. (Earlier TRB was also in form of patches/sigs, but without the Vtronic antecedent hashes.)

    Re: "make myself a script to format the content of each vpatch properly for article-inclusion, having at the very least: lines numbered and with html hooks for reference, ALL the antecedent code shown..." : IMHO pertinent thread re subj. Are you writing this proggy currently? Presently I've returned from UY, and making a rather spartan one for own use (will genesis.)

    • Diana Coman says:

      Thanks for the additional historical links. I had looked at the starting document rather than practice/previous mentions or discussions, indeed.

      Re viewer, sadly there was no space I could make for it so far so it's waiting in the (long) line. I still see it however as part and parcel of publishing and therefore I don't see much point in having it not play along with mp-wp and/or importing its own additional set of libraries or whatevers.

  4. billymg says:

    This was a very helpful article for me. So much of the information about V is sprawled across logs, blog articles, and comments on those articles. It's nice to have such a comprehensive reference in one place now.

    One small thing I wanted to point out: while I don't have a single code shelf I have been maintaining two separate code shelves for mp-wp and mp-wp-tests.

    Bvt also maintains four code shelves for individual projects, linked to from the sidebar on his site (one of which is already linked in this article).

    I'm personally no longer happy with the format of my code shelves and will probably consolidate them into a single page. I'll also provide links back to patch announcements where relevant, which should help the code shelf serve as a central (at least to my blog) reference point for coding work.

  5. Diana Coman says:

    Glad to hear it's helpful to someone other than me, thank you!

    Re shelves - my original idea was of having one single place to access all signed code from. I guess the practice shows it wasn't all that clearly expressed. I did avoid on purpose already entrenched terms such as "library" or even "bookcase" because they come with additional baggage that I didn't want to import but on the other hand I can see how "shelf" can make one think of multiple shelves. The thing is - if you have one for each V-tree, then it's simply each V-tree on its own, there's no need to use any additional term ("shelf") for it.

  6. [...] a collection of software, the entire system is controlled by V 4 , which has a manual genesis of it's own, to quote : 0x00] Software is the property of people [...]

  7. [...] enumerating goodness 1 and owning any software worth the mention under V, TMSR OS allows the operator to manage his investment of trust and sets the basis of a framework [...]

  8. [...] in fact refer to the same message. These properties are exploited in present manifestations of the V version control [...]

  9. [...] for building up one's own history and reputation, younghands.club as easy access to blogging, V for code versioning and effective collaboration, the various V trees as code library, the full library of related [...]

Leave a Reply