A Basic Requirement for the Literate Introduction of New Tools

This started as a comment1 on bvt's very useful work building up the vtools suite. As I really liked his approach2 and I would really prefer to have just one suite (vtools) with everything V-related instead of various bits and parts, I carved out some time this week to take the reground vpatches, go through the whole code (to see how it fits in my head) and then give it quite a few spins and tests (to see how it fits my hand3, too).

Going through those usual steps of exploring new code and tools meant that my list of notes also went through its usual cycle: first it exploded with all sorts of questions; then it shrunk as most of those questions were answered either at fitting in head time or at fitting in hand time; at the very end, there were only a few questions standing and - more troublesome - a few things that were useful in the previous tool and are missing from the current one without any good reason that I see. And as I was trying to write up those remaining notes into a coherent comment on bvt's site, the text kept pushing away from the concrete bits of code in question and towards the wider issue of what it means and what it takes to switch from a working, tested, often-refined tool (or setup even) to a new (or even only partially new, as is the case with this addition to vtools) tool.

The concrete itch that grew into all this is the fact that bvt's new utilities and v.sh wrapper provide only one way to see the whole tree of a set of patches - visually, using graphviz that produces a pretty .svg illustration (the command4 would be v.sh graph dot-file svg-file). While I agree that the visual illustration is very nice and all that, I have never installed the graphviz & deps on *any* of my machines and I can't say I ever found a reason for which I should - nor do I really see such valid reason now. The old v.pl simply spit out upon request either the list of leaves or the full flow of vpatches in the console and even though it didn't bother to indent it as a tree, it was useful nevertheless. Certainly, it didn't make for much of a *tree* image as such and the lack of indentation also meant that the order was not necessarily the one of a press (since it wouldn't be clear which patches are the leaves - if/when there are several leaves). Nevertheless, it was very useful and so I would logically expect that any new tool would pick it up and either improve on it or keep it the same. Failing either of that, I'd still expect at the very least to read upfront and before any code the *reasons why* that functionality is not present in the newly proposed tool.

For the case at hand, I am aware that the flow command morphed into "flow leaves" but the helpful listing of leaves themselves got silently dropped and without it, the "flow leaves" command is not enough really, since it requires that one identifies/finds/knows what the leaves are to get anything useful. I honestly can't see: why was the listing of leaves just dropped? and why did the basic functionality of printing out a full (or the longest or all possibilities or...) flow morphed instead into a flow-to-known-leaf without the support of a "print leaves" command?

There are indeed otherwise the antecedents/descendents commands (that had equivalents in the old tool) that allow *some* exploration of a given tree and eventual finding of leaves too but it's very tedious -and I can't see any good reason for it, either- to actually rely on those and effectively walk the tree node by node from the genesis down (assuming at least the genesis to be easy to pick out of the pile just by name of the vpatch).

To answer my own question above, I can see the approach of "oh, I never used that and I always/anyway have/like/find-useful graphviz so didn't bother with any text tree or listing of leaves". Which is possibly fine and nice when one either starts a new tool from scratch - ie there is no other already-iterated-upon tool covering that part and therefore nothing to review and no reference point to start from - or otherwise works alone and intends to keep it that way. But when there is already a tool that is the result of a lot of public discussion and quite a few iterations, it strikes me that there should be a minimum requirement to at least *clearly state* from the very start any different decisions made with respect to functionality provided and *why* exactly are such changes *justified*. Because reviewing new code and trying out (not even adopting, just reading and trying stuff out!) new tools is a *cost* that the code author asks any potential readers of code (hence, proper users) to pay and moreover to pay upfront, whether they end up finding the new code useful & fit for purpose or not!

While it might seem at first sight that this is just asking yet-another-thing of the code writer, this is not just for the sake of asking. First of all, the way I see it, this is in the writer's interest very much too - the very exercise of purposefully reviewing and properly considering the why-s of an existing tool *that is not picked at random nor is it the result of a random/unknown process* is likely to *help* with any new design and at the very least spark some further discussion if nothing else. Second, since this is not just any previous tool picked at random, but a tool that is designed, made, vetted and used by L1 (not to mention more generally by people presumably highly ranked in one's own WoT), it seems to me a basic and quite mandatory part of any literate approach to coding to reference the original choices properly from the very beginning - meaning from the very design and conception of any new tool.

Considering the above and given that the scarcity is always a scarcity of *time* and not one of code or of tools, I'd go as far as to set this on my list of prerequisites for even considering any new tool that is touching on the functionality of an existing in-WoT tool: either it's introduced with a proper referencing (that may include more or less discussion, too) of relevant existing history or it's not introduced at all as far as I'm concerned5.

  1. And it ends up as this article PLUS a comment on his site - that's a plus as far as I'm concerned and not a random one either but directly linked to why I like bvt and his work. 

  2. His proposed tools effectively extract out of vpatches the information strictly required for the V-work so that all calculations are not encumbered by whatever amount of code the vpatches may contain. This strikes me as not only very useful when the code base is large (as his own timings clearly show) but also inherently sane and the correct way to go since there's no real reason why one should carry the code where it's not directly needed and/or not use instead the hashes of files and hunks of code exactly for what they are best for, after all. 

  3. And I'd link directly Trinque's coinage of the term except the select still doesn't work on his blog. 

  4. The full current list of command as helpfully provided when running v.sh without any parameters:

    v patches -- list patches
    v wot -- print WoT
    v flow leaves -- print patch flow
    v graph dot-file svg-file -- graphviz plot of dependencies
    v antecedents patch -- list patch antecendents
    v descendants patch -- list patch descendants
    v origin hash -- find patch that introduced the hash
    v press directory leaves -- press all patches up to head


  5. It's quite interesting how people even come to me bent on "making tools" but don't stop one minute to consider that their "making of tools" is more of a drain on other people's time and therefore a request they are trying to impose on others rather than any "help". You'd think they have found otherwise a ton of people clamoring to use their great tools or something. Anyways, I'm too busy to care much about assigning blame and all that but at least for starters, I'll lay this at the already-filthy door of the open sores approach

7 Responses to “A Basic Requirement for the Literate Introduction of New Tools”

  1. bvt says:

    Thanks for the review! Indeed, I started out the work with this deficiency of not providing the review of existing tools and specifications. Basically, dropping all the existing and historical (not in the sense of bad/unused!) material. Another glimpse into this was dropping of init command, which Mircea Popescu has spotted.

    A short explanations of reasons, which don't go to the core argument of the article, more of a curious fact: basically, I have always used Ben Vulpes' Vtronics 101 as the canonical specification of V, and have never seriously used any V-presser implementation of others. (Here, of course, there is a problem that I also dropped a command (roots), without any explanation...)

    I did use v.pl/vk.pl, but not for development and exploration, but for pressing things I already know, on remote machines. When I first learned about V, I very much liked the idea that you could spec the whole tool, so it can be implemented without looking into any other implementation. Which I did without looking (I only grepped out gpg invocations out of v.pl), in Scheme Shell - the vpatches are here 1 and 2, but there was never a writeup, for no good reason really, as the fact that I liked neither the result nor scsh itself speaks more in favor of a writeup, not against. I've been using it whole time for development, but always wanted to replace with something proper, which v.sh is supposed to be.

    For printing full flow, there is a vpatch name of - (v.sh flow -), which yes, may be arguable, but its output is equivalent to vk.pl flow. Press supports the same: v.sh press dirname -. Curiously, I have never personally relied on the neither on graphviz output, nor antecedents/descendants.

    And yes, not knowing about "leaves" command is not an excuse, but only puts one (me) into worse situation - i.e. why did I decide to work on a tool for others with such an approach; and whether someone needs the command or uses it would have been mentioned during the discussion. So I agree with your conclusion as well.

    I guess, what flows out of your article for me is a general improvement in method (not ignoring the existing and the past, to generalize the problem), and admittance that I have one article way overdue. I hope you will continue reading v.sh series, as your comments (or articles) are most welcome.

    (But yes, I can add that command, and try to come up with something for the tree representation)

  2. I don't think in principle keywords are verboten. I do believe adding - to any lexicon is a very bad ideea however, because - is widely relied on by posix command line for a purely semantic (rather than lexical) role, and all sorta well known and historically documented unpleasantness has flown from people mixing its roles. Use something other than -.

  3. bvt says:

    Clear. Is "all" instead of "-" OK?

  4. Yeah, that's probably the correct chocie.

  5. Diana Coman says:

    bvt - why not even the default ie if you don't give it any "leaf"? Anyway, "all" also works, sure.

    No worries otherwise, it is indeed mainly a request to please not ignore the past but quite explicitly link it in because it matters. And I'm looking forward to your writeups and future work for sure!

  6. bvt says:

    I'll give it a try as well.

  7. [...] itself not an insignificant liability. While work had been underway to replace that, the results fell short of completeness, and from the ensuing discussion I decided it would be best to shore up my own grounding in the [...]

Leave a Reply