Software Editions



May 4th, 2023 by Diana Coman

How about approaching the hot problem of 'getting people to care about what software they run'1 from a place of meaning rather than mere ease of use?

To get to such place of meaning, one would need at the very least to be able to tell confidently at any time what a software version even is and what does it mean. Surprisingly or not, the current way in which software versions are 'defined' and reported doesn't really provide much to be confident about and even less to rely on. Luckily though, the needed tools to change this for the better are in fact available, so let's look at the whole trouble and then the working solution to it all, too.

Through historical accident rather than planning of any sort, software ended up with a rather weak approach to versioning that seems good enough only as long as one doesn't try to lean on it too strongly. Generally, versions for software code are given simply as numbers picked more or less transparently2, embedded in the code and then simply read and reported to the user at runtime, on request. So the code says that it's version x.y and the user... takes the code at its word because there isn't much else to do.

Well, there *wasn't* much else to do until now but from now on, there is, courtesy of VaMP: software editions rather than mere versions, offering cryptographic guarantees and embedded historical records that you can check at any time and as often as you like.

Let's start directly with an illustration of use, checking the software version and edition of VaMP itself - and not just any edition but precisely the one that makes available... software editions:

$ vamp version vamp_sw_ed/
************
The code in vamp_sw_ed/ matches a signed software edition and has the following version description from manifest.vamp in vamp_sw_ed/:

787977 vamp_sw_ed Diana Coman Adds software editions, check and reporting via the new 'version' command including authorship (from patch), readership (from sigs) and version info (from manifest file). Further cleanup and improved usability including consistent usage help and order of parameters for all commands (generally the aim is to have command input(s) resource(s) output(s) where resources are supporting the task rather than direct inputs to the task at hand and are usually given as paths, e.g. location of patches, sigs or keys to consider). Adds better support for multiple signatures of the same file through a naming scheme for sig files that includes the base name of the relevant keyfile (which is by default the key's fingerprint).

The software edition producing this exact code version is:
data/patches/vamp_sw_ed.patch
authored by 7ddde85da027e48a9a23aa8573079fff
signed by 7ddde85da027e48a9a23aa8573079fff

Software editions are the code equivalent of literary editions. Just as there are any number of drafts produced before an edition of a book is published, so there are any number of code versions made before a software edition is published. Just as there are others besides the author of a literary work that have a role in and responsibility for a literary edition of a book, so there are other roles and responsibilities for a software edition besides that of writing code. In the most direct terms, a software edition contains and requires more than the code itself - it's a more rounded product that includes as a minimum and in cryptographically verifiable forms, three main parts: the record of the code's history, the published code and the personal responsibility assumed by the publisher, whether that is the same person as the writer or not.

In more concrete terms and as a standard minimum, a software edition will therefore include:

  1. The cryptographically verifiable means to obtain a specific code version, consisting in:
    1. A VaMP patch.
    2. The publisher's3 matching signature of the above patch.
    3. The publisher's public key that can be used to verify the above signature.
  2. An embedded record of the code's versioning history in the form of a plain text file called manifest.vamp that is part of the code version published in this edition. Each code version adds a new line to the manifest.vamp file, including:
    1. The Bitcoin blockheight at the time when the new code version was created, as the only universal timestamp currently available. Wherever you are in the world when you write a new code version, the blockheight is a more accurate description of time than any 'local time zone' and as such, it's entirely immune to any and all 'challenges' of traditional timestamps, of course.
    2. A short name for the new code version. This tends to serve as the filename for the patch that results in the new code version but note that this is just a convention, not mandatory in any way. The patch can really be called anything at all and one can certainly have any number of different patches that still result, each of them, in the same code version4.
    3. The name of the writer of this new code version
    4. A summary of the main changes/additions to the code brought in by the new code version.

In addition to the above, a software edition can include, of course, just about anything else that the publisher wants to include. There is for instance a reasonable case to make perhaps for a record of editions themselves, seeing how they are essentially on a different level than software versions and as such not included in or made visible through the manifest.vamp file. To make the distinction clear, the manifest.vamp file is a historical record of all changes made to the code, hence strictly speaking of all code versions whether published or not. By contrast, editions are signed packagings of specific code versions for which the publisher assumes personal responsibility by signing them with their own key.

Different publishers can maintain their own sets of editions even for the same software and the patches involved may very well be different but the same version of the code has to have the exact same manifest.vamp file in all editions - if that's not the case, then it's simply not at all "the same version of the code" regardless of what one might claim. So publishers may well keep an additional record of their own software editions, whether signed or not, whether in file format or web format and so on. In doing so, they build up their own record, of course and as such can reap the accumulated benefits for their own work over any period of time.

For some additional clarity, let's see also side by side what would the differences be between software changes, versions and editions as supported by VaMP:

Description Format
Change Any addition, deletion or substitution. A change can be as small as one character in one single file and as big as a full replacement of all files on one's computer, there is no limit to its scope. There is also no responsibility assumed explicitly and verifiably by anyone for it. The thing changed and you as a user might know or might not know about it but you'll certainly have to live with the results of it, regardless. Automated updates and other viruses.
Version A code state that is fully described and signed for in a cryptographically verifiable manner and that includes a relevant line in the manifest.vamp file describing the changes made from the previous code state. Whether some code is or not in the described state matching a specific version can be checked reliably at any time and as often as desired without any side effects and detecting any change regardless of its size. A VaMP patch together with its signature by a RSA key in your WoT. Alternatively, an index of the relevant code state together with its signature by a RSA key in your WoT.
Edition The cryptographically verifiable and signed means of obtaining a specific code version including as a minimum its historical record of changes, its known authorship, readership and the identity of the publisher. One or several VaMP patches together with their signatures by the publisher and possibly by others, as well as the publisher's public RSA key.

In a nutshell, by providing software editions, VaMP leverages cryptography to empower personal users, writers, readers and publishers of software or in other words, everyone who uses computers at all. Combined with RSA keys for personal identity and the Web of Trust (WoT) as public record, software editions are a part of the wider structure and approach that can give people as opposed to bots (or the 'AI') the edge even in this digital environment which is otherwise not at all human-friendly.

Whether enough people figure it out and make good enough use of it before the deluge of bot-created 'content' is too great to even find one's way to any meaning anymore is something that depends on each individual person, as simple as that. As my public record shows, I have always been firmly on the human side on this and I don't plan to stop any time soon so I'll continue working to increase and support people's leverage as connected individuals, first and foremost, not the bots' training data sets or the like. What you do though, as always, is entirely your choice and note that you make it with every action you take or refuse to take, whether you realise what you are thus choosing or not.


  1. It is actually a hotter problem than it might seem. It's not *just* that so much runs today in and on software of all sorts but also that the bots (alternatively termed 'AI') have the natural advantage when it comes to the digital environment in general and code or machines in particular. The popular current approaches to 'make coding like plumbing' or to rely on 'AI' to 'write code' can only make it worse, not because they'll take away jobs but because they'll end up inevitably by drowning everyone in such an endless amount of meaninglessness that it would take several lifetimes to even filter through it, let alone get to understand or use or be in control of any part of it. And if you go along with it and get therefore well and truly caught, what you'll end up doing under the guise of 'using it' is much more 'being used by it.' There are better options for sure but they tend to require work

  2. The main approach uses two numbers, one considered 'major' and thus changing only when there are 'significant' differences and the other 'minor' and thus changing in principle for any code change. Note that there isn't any clear definition for what 'major' or 'minor' mean in the first place and I doubt there can even be some reliable definition that holds for everyone involved since quite often the versioning is not even a matter of coding really but one of marketing and/or responding to market pressures of all sorts.

    Even leaving this aside for the moment and to make matters more confusing, it's quite clear that not all changes get to increase even the minor number since there are plenty of code changes that are lumped together at a later stage into what counts then as a new minor version.

    Worse even, there is no reliable way to actually *check* a version, whatever that might be. The traditional way in which versions are 'checked' is simply by interrogating the code itself at runtime and such self-reporting is nothing more than reading and reporting some embedded numbers, without checking anything and without any real link to the whole as such - whether the code changed or not overall or in any of its parts, it can certainly keep reporting the same number for ever and ever.

     

  3. Note that this may or may not be the same as the writer of the code. More importantly here is the *reader* of the code and this means specifically the one who is willing to publicly and personally assume responsibility for a software edition by putting their signature on it. 

  4. A VaMP patch fully describes and guarantees a *resulting* code state starting from an equally well defined *starting* code state. Hence, to obtain the code version you want, you can use one patch or another depending on what code state you are starting from, there's no problem with that and there are clear benefits to it, too. For instance, there's no reason to have to apply 101 patches every time just because the version you want happens to be, indeed, the 101st - you just make a 'shortcut' patch that gets you straight to version 101st and use that without losing for it any of the history or any of the guarantees, it's as simple as that. 

Comments feed: RSS 2.0

10 Responses to “Software Editions”

  1. Diana Coman says:

    For the record, the 1st edition of Eulora2's client is now published for those actively involved.

  2. Jacob Welsh says:

    Between

    Just as there are any number of drafts produced before an edition of a book is published, so there are any number of code versions made before a software edition is published.

    and

    Even leaving this aside for the moment and to make matters more confusing, it's quite clear that not all changes get to increase even the minor number since there are plenty of code changes that are lumped together at a later stage into what counts then as a new minor version.

    and looking at the stated differences in format between a version and an edition, what I'm having trouble with is whether publishing an edition is a more weighty process, akin to the prior "releasing a version" and with similar marketing or communication functions ("this is a stable, believed-good code state that's ready for use and supported"), or whether it could be said that *every* version could be called an edition, when taken in combination with the signatures and keys that produce it. Does the publisher sign every patch in the history that leads to an edition (it seems like they must, unless signing a shortcut instead), and does that mean the version produced by each such patch creates in effect its own separate edition, based on the definitions here?

  3. Diana Coman says:

    Publishing in general is meant to be indeed a more weighty process but what exactly that process is and what it means is quite on purpose up to each publisher to define for themselves and up to each user to discern for themselves. It is in fact precisely an opportunity for publishers to differentiate and for users to support or reject, in a way which is quite bot-resistant, too (meaning: not directly or easily 'ranked' in any meaningful way automatically, not that it can't or won't be tried and thus done of sorts, I'm sure).

    If *some* publishers decide for some reason to make an edition out of each and every version, they can, certainly. It would hardly bring any value though as far as I can currently tell and as such, I think it would reflect rather poorly on those publishers but this is up to the users (the market, if you prefer) to decide, in the end. I don't think that there can or should be a technical solution to this sort of issue or at least not if one wants to empower humans rather than bots.

    Essentially, the main difference between an edition and a version is one of meaning and added value first and foremost, possibly one of access, secondary, but certainly not that much one of pure form. I suppose in time and if it all follows indeed the book publishing model, it might get to the point where there is some sort of mechanically-verifiable way to distinguish as well (eg ISBN-like) but I'm not sure that this is necessarily needed or an improvement even. It's open for now and it will go the way it needs to.

    whether it could be said that *every* version could be called an edition, when taken in combination with the signatures and keys that produce it.

    No 'could' ever made a 'would' or 'should' and this is no exception. There isn't even any requirement for each version to ever be published (or at least to be published to the same public as it were), nor for each writer of code to be a publisher, just like there isn't any requirement for every change to get its own version. Moreover, it's quite likely it makes sense to have a 'publisher key' vs a 'developer key' even as sub-identities of one single person and to make available all versions to collaborators but only editions to the wider public, for instance.

    Does the publisher sign every patch in the history that leads to an edition (it seems like they must, unless signing a shortcut instead), and does that mean the version produced by each such patch creates in effect its own separate edition, based on the definitions here?

    The publisher signs only what they publish and they get to pick that as they consider best. There is no requirement for the publisher to sign or even have/read/ever see every patch mentioned in the manifest file, although it is indeed their choice as to what they require for producing an edition. Whatever they require though, what they are signing on is a final code state (which includes a certain version of its history that the publisher supports thus publicly) and the provided means to obtain that, nothing else.

    Thanks for the question, it's quite a good one really, pointing as it does to the options opened up as to what each may make on top of the fixed, underlying structure. I get it that this same sort of open options space is potentially confusing at first exactly because the exact paths are yet to be marked in it through practice rather than being even traced as such from start, so hopefully the discussion informs and then the practice proceeds, too.

  4. Diana Coman says:

    For clarity, I should add that technically speaking a code version *can* be given separate from a VaMP patch, basically in what VaMP currently calls index form. If the need for such a signed index becomes pressing enough, it's quite straightforward to produce even now since VaMP already produces indices for code states anyway and it certainly offers its users the option to sign any content they wish.

    I updated the table in the article to include this as well.

  5. Note that there isn't any clear definition for what 'major' or 'minor' mean in the first place and I doubt there can even be some reliable definition that holds for everyone involved since quite often the versioning is not even a matter of coding really but one of marketing and/or responding to market pressures of all sorts.

    The most striking example that comes to my mind here is how the fuck faces dicking around with the shithub repo incorrectly called bitcoin managed to dangerously soft fork the damn thing several times by now, but according to them, those weren't major changes and they're still on "major" version 0.x.x.

    For instance, there's no reason to have to apply 101 patches every time just because the version you want happens to be, indeed, the 101st - you just make a 'shortcut' patch that gets you straight to version 101st and use that without losing for it any of the history or any of the guarantees, it's as simple as that.

    Do you see such a shortcut more for streamlining development workflows ? I.e. work with the shortcut to patch 101, but when it comes time to publish, include it on top of the the full history. Seems like from an operator perspective, I'd prefer to have the whole history there, even if I can rely on the input hash on vpatch 101 to find the full history in case there's some problem encountered and I want to track down when and where it was introduced.

    What you do though, as always, is entirely your choice and note that you make it with every action you take or refuse to take, whether you realise what you are thus choosing or not.

    Word.

    @Jacob Welsh, this differentiation between version and edition seems to me to already be your practice. For example, with bitcoin, you're not typically issuing a new release with each new vpatch because most problems you're tackling there ought to be broken up into several vpatches. Same with yrc and your signed tarball releases there.

    As far as multiple subkeys for a single person, you're practice there already paid once.

    P.S. Looks like a space is missing in footnote #1.

    There arebetter

  6. Diana Coman says:

    That bitcoin example of "versioning" is quite something indeed, one of those that I would find hard to believe if I hadn't seen it first hand.

    Do you see such a shortcut more for streamlining development workflows ?

    More for user-workflows even, hence why more naturally in an "edition". From a dev perspective, I can say from experience that I have 0 problems applying 80+ patches, I never felt the need for a shortcut at dev time. And as you notice, at dev time one would rather want to have the history in full, if at all possible, indeed, especially since otherwise it's not really in the way at all, the machine works with it all without complaint. My point with shortcuts is really that I don't think that all users *have to* be directly and at all times developers as well and similarly, a deployment doesn't have to be same as development, hence the *option* (because it's that, an option, not an imposition) of having one single patch where the final state is simply what one wants mainly.

    Fixed the missing space in footnote #1, thanks.

  7. Alright, makes sense about development vs deployment.

  8. [...] publishing the code as a 1st edition of MiniZipLib, in VaMP format, of course, since it can be perhaps useful to others if/when they [...]

  9. [...] one full year later, on the 4th of May 2023, the versioning and software editions capabilities were fully in [...]

  10. [...] the state of the art in software publication advanced, and having got a little practice with the tools already, I took advantage of this as a [...]

Leave a Reply