Introduction

Welcome to the Pijul book, an introduction to Pijul, a distributed version control system that is at the same time theoretically sound, fast and easy to learn and use.

A version control system is a tool that tracks changes in your files, can revert them, and can merge them with your coauthors’ changes.

Why Pijul?

Pijul is the first distributed version control system to be based on a sound mathematical theory of changes. It is inspired by Darcs, but aims at solving the soundness and performance issues of Darcs.

Pijul has a number of features that allow it to scale to very large repositories and fast-paced workflows. In particular, change commutation means that changes written independently can be applied in any order, without changing the result. This property simplifies workflows, allowing Pijul to clone sub-parts of repositories, to solve conflicts reliably, to easily combine different versions.

Change commutation

In Pijul, for any two changes A and B, either A and B commute, (in other words, A and B can be applied in any order), or A depends on B, or B depends on A.

  • [Use case: early stage of a project] Change commutation makes Pijul a highly forgiving system, as you can “unapply” (or “unrecord”) changes made in the past, without having to change the identity of new changes. A reader familiar with Git will understand “rebasing”, in git terms).

    This tends to happen pretty often in the early stages of a project, when most things are still uncertain. With Pijul, exploring new features and new implementations comes at no extra cost in time.

  • [Use case: the project becomes stable] As your project grows, change commutation saves even more time: imagine a project with two main branches, a stable one containing only the original product, and bugfixes, and an unstable one, where new features are constantly added.

    The team working on the unstable branch is likely to discover old bugs, and fix them in the stable branch too.

    In Pijul, maintainers of the stable branch can simply pull only the changes they are interested in. Those changes do not change when imported, which means that pulling new changes in will work just as expected.

Associativity

In Pijul, change application is an associative operation, meaning that applying some change A, and then a set of changes (BC) at once, yields the same result as applying (AB) first, and then C.

With branches, the first scenario looks like this: Bob creates A, while Alice creates B, C, and Bob finally merges both B and C at once.

The second scenario would look like the following, with Bob creating commit A, and then pulling B. At that moment, Bob has both A and B on his branch, and wants to pull C from Alice.

Note that this is different from change reordering: here, we apply A, then B, then C, in the same order in both scenarios.

Using math words such as “associative” for such a simple operation may sound like nitpicking, because intuition suggests that it should always be the case. However, Git doesn’t guarantee that property, even if A, B, and C do not conflict. Concretely, this means that Git (and relatives) can sometimes shuffle lines around, because these systems only track versions, rather than the changes that happen between the versions. And even though one can reconstruct one from the other, the following example (taken from here) shows that tracking versions only does not yield the expected result.

Git merge (which A is which?)
Pijul merge

In this diagram, Alice and Bob start from a common file with the lines A and B. Alice adds G above everything, and then another instance of A and B above that (her new lines are shown in green). Meanwhile, Bob adds a line X between the original A and B.

This example will be merged by Git, SVN, Mercurial… into the file shown on the left, with the relative positions of G and X swapped, where as Pijul (and Darcs) yield the file on the right, preserving the order between the lines. Note that this example has nothing to do with a conflict, since the edits happen in different parts of the file. And in fact neither Git nor Pijul will report a conflict in this case.

The reason for the counter-intuitive behaviour in Git is that Git runs a heuristic algorithm called three-way merge or diff3, which extends diff to two “new” versions instead of one. Note, however, that diff has multiple optimal solutions, and the same change can be described equivalently by different diffs. While this is fine for diff (since the patch resulting from diff has a unique interpretation), it is ambiguous in the case of diff3 and might lead to arbitrary reshuffling of files.

Obviously, this does not mean that the merge will have the intended semantics: code should be still reviewed and tests should still be run. But at least a review of the change will not be made useless by a reshuffling of lines by the version control tool.

Modelling conflicts

Conflicts are a normal thing in the internal representation of a Pijul repository. Actually, after applying new changes, we even have to do extra work to find where the conflicts are.

In particular, changes editing sides of a conflict can be applied without resolving the conflict. This guarantees that no information ever gets lost.

This is different from both Git and Darcs:

  • In Git, conflicts are not really handled after they are output to files. For instance, if one commits just after a conflict occurs, git will commit the entire conflict (including markers).

  • In Darcs, conflicts can lead to the exponential merge problem, which might cause it to take several hours to merge even a two-lines change.

Comparisons with other version control systems

Pijul for Git/Mercurial/SVN/… users

The main difference between Pijul and Git (and related systems) is that Pijul deals with changes (or patches), whereas Git deals only with snapshots (or versions).

There are many advantages to using changes. First, changes are the intuitive atomic unit of work. Moreover, changes can be merged according to formal axioms that guarantee correctness in 100% of cases, whereas commits have to be /stitched together based on their contents, rather than on the edits that took place/. This is why in these systems, conflicts are often painful, as there is no real way to solve a conflict once and for all (for example, Git has the rerere command to try and simulate that in some cases).

Pijul for Darcs users

Pijul is mostly a formally correct version of Darcs’ theory of changes, as well as a new algorithm for merging changes. Its main innovation compared to Darcs is to use a better data structure for its pristine, allowing for:

  • A sane representation of conflicts: Pijul’s pristine is stored in a “conflict-tolerant” data structure. Many changes can be applied to it, and the presence or absence of conflicts are only computed afterwards, by looking at the pristine.

  • Conflicting changes always commute in Pijul, and never commute in Darcs.

  • Fast algorithms: Pijul’s pristine can be seen as a “cache” of applied changes, to which new changes can be applied directly, without having to compute anything on the repository’s history.

However, Pijul’s pristine format was designed to comply with axioms on a specific set of operations only. As a result, some of darcs’ features, such as darcs replace, are not (yet) available.

Installing

Releases

The main channel to compile Pijul from source is the Rust Language package manager: cargo. You’ll need to download rustup from here to install cargo.

On installing cargo, there are also three different channels to choose from, namely: stable, beta and nightly. You can find more information about it here. If you’re unsure of which channel to choose, go with stable as your default rust toolchain:

rustup default stable

If you’ve already installed rustup before, make sure to update it.

Next, you’ll need the necessary libraries and header files installed on your computer and clang compiler:


On Debian:

sudo apt install make libsodium-dev libclang-dev pkg-config libssl-dev libxxhash-dev libzstd-dev clang

On OSX:

brew install llvm libsodium openssl xxhash zstd

On Arch Linux:

sudo pacman -S clang libsodium gcc-libs rustup pkgconf diffutils make xxhash

On openSUSE Tumbleweed:

sudo zypper in clang-devel libopenssl-devel libsodium-devel libzstd-devel pkgconfig xxhash-devel

On Void (Linux):

sudo xbps-install libgcc-devel libressl-devel libsodium-devel libzstd-devel xxHash-devel

After installing the necessary libraries and header files, it’s time to install Pijul!

Run the following command in your terminal:

cargo install pijul --version "~1.0.0-alpha"

You might need to add the install directory to your $PATH environment variable. For example: adding the following to your ~/.config/environment.d/envvars.conf:

export PATH="\$PATH:$HOME/.cargo/bin/"

Distribution packages

Nix and NixOS

Nix is a package manager that can be installed on any linux distribution, and on OSX.

nix-env --upgrade
nix-env -iA pijul

Debian

curl https://nixos.org/nix/install | sh

If it fails with the following error:

nix error: cloning builder process: Operation not permitted

then create a file as root:

sudo su root -c 'echo "kernel.unprivileged_userns_clone=1" >> /etc/sysctl.d/nix.conf'

Arch Linux

Pijul is packaged in AUR. It should be up-to-date or at least close.

openSUSE

For the development version (1.0.0-alpha.X) of Pijul, a package for openSUSE Tumbleweed in the openSUSE Build Service:

sudo zypper addrepo -f -r https://download.opensuse.org/repositories/devel:/tools:/scm/openSUSE_Tumbleweed/devel:tools:scm.repo
sudo zypper install --allow-vendor-change pijul

Use the development version at your own risk. As soon as Pijul 1.0.0 is published, the package will be updated and submitted to openSUSE Tumbleweed.

Void (Linux)

xbps-install pijul

Windows

We do not provide official Windows binaries at the moment, but community-provided binaries are available.

Getting started

Here is how to start a project with Pijul: first run the following command:

pijul init

In the project’s directory. This creates a directory called .pijul, and initialises a few things.

Then, add a few files to track. For a Rust project, this could be for instance:

pijul add Cargo.toml src/lib.rs

Finally, create a patch by recording your changes:

pijul record

Alternatively, one can add and record files in a single record command. For example, importing all the files in a directory named d is done with:

pijul record d

First-Time Setup

Now that pijul is installed and working on your system, you can set your user name, full name, and e-mail address so you don’t have to set it each time you want to record a change.

You can put this information into one of several places; the one you probably want is $HOME/.config/pijul/config.toml (Linux), %AppData%\pijul\config.toml (Windows), or ~/.pijulconfig (MacOS).

Create the file and add the following lines to it:

[author]
name = "<username>"
full_name = "<full name>"
email = "<email address>"

From now on, pijul should pick this up when recording changes and pre-fill the author section for you.

Definitions

Pijul works in four different places of your hard drive: a working copy, where you can edit files, a set of changes, a tree, representing the tree of files currently tracked by Pijul, and a pristine, which is a representation of the current recorded version of the repository.

The differences between the pristine and the working copy are used to produce changes, which can be then applied to the pristine. Changes encode edits made to files of the repository (line insertions and deletions), as well as file additions, deletions and name changes.

At any time, the pristine can be output into the working copy, for instance after receiving new patches, or to cancel changes and reset the working copy to its recorded state.

So, the first command of the above example, pijul init, initialised an empty pristine. pijul add then updated the tree, to tell Pijul to start tracking files “Cargo.toml” and “src/lib.rs”. Finally, pijul record compared the (currently empty) repository with the working copy, producing a patch as a result, and applied this patch to the pristine.

Next steps

In the next chapter, we will see how to exchange patches with others.

Collaboration with Pijul doesn’t have to be centralised, and is not even made easier by centralisation, as patches allow for a completely distributed workflow, which can happen even by email exclusively.

However, the authors of Pijul provide a free web service called the Nest, as one way to share patches with collaborators.

Working with others

Although one can use Pijul to work alone, for instance to store the history of changes of a project, or synchronise work with another machine, it is mainly useful (and more efficient than other solutions) when working with other people.

The only way to collaborate with others in Pijul is to send and receive patches, and apply them to one’s repository. When working on an existing project, it might be necessary to clone it first. For instance, in order to clone Pijul’s main repository, you can run the following command:

pijul clone https://nest.pijul.com/pijul/pijul

This just downloads all the patches currently in the remote repository, applies them all, and outputs the result to a new working copy. Further patch exchange can be done with the two commands pull (receive patches) and push (send patches), as we explain now.

Between local repositories

Patches can also be exchanged between local repositories, in both directions (push and pull). As an example, let’s create two repositories, and exchange two patches between them:

mkdir a
cd a
pijul init
echo "blabla" > file
pijul add file
pijul record
cd ..

pijul clone a b
cd b
cat file # should contain "blabla"
echo "blibli" >> file
pijul record
pijul push ../a
echo "one extra line" >> file
pijul record

cd ../a
pijul pull ../b

SSH

Pijul can work with remote repositories over SSH just like local repositories. A working Pijul needs to be installed on the remote machine, though.

As an example, if you have an account called “me”, and a repository named “repo” under that account on nest.pijul.com, you can run for instance:

pijul clone me@ssh.pijul.com:me/repo

To clone your repository, and:

pijul push me@ssh.pijul.com:me/repo

To send your local patches to your repository there. Just as with a local repository, pulling patches can also be done over SSH:

pijul pull me@ssh.pijul.com:me/repo

HTTP

Pijul is able to download patches from HTTP and HTTPS servers. In the above example, receiving new patches from Pijul’s main repository can be done by running:

pijul pull https://nest.pijul.com/pijul/pijul

However, Pijul is not (yet) able to push patches to an HTTP URL.

Conflicts

An important part of working with others is disagreeing. On text files (such as source code), work can sometimes be split beforehand in such a way that disagreements on a single line in a file never occurs.

However, many teams don’t have everything planned in advance, and their members often edit each other’s code. This might result in conflicts.

In Pijul, there are two kinds of conflicts inside a file:

  • When two different authors add lines at the same position in a file, and it is impossible to tell which comes first in the file.

  • When one author adds a line in a block of text or code, while another author deletes that block.

Conflicts can also occur between files:

  • When authors give the same name to two different files.

  • Or when they rename the same file to two different names.

It is important to note that conflicts in Pijul always happen between changes, for example we might say that “change A conflicts with change B”. A conflict resolution is always a change. One option to fix a conflict between A and B is to run pijul unrecord A (respectively pijul unrecord B), which removes A (respectively B) from history.

One of the main features of Pijul is that its internal representation of repositories fully models conflicts. Patches can even be applied to a conflicting repository, leaving the conflict resolution for later.

Theory

As a first approximation, one can think of a repository as a single file represented by a directed graph $G = (V, E)$ of lines of text, where each vertex $v\in V$ represents a line of text, and an edge from $u \in V$ to $v\in V$, labelled with a change (also called patch) number $c$, could be read as “according to change $c$, line $u$ comes before $v$”.

This means that changes may introduce both vertices and edges, as in the following example, where a line of text $D$ is introduced between $A$ and $B$:

Here, the thick arrow represents a change $c_0$ from a file containing the lines $A$, $B$, $C$, to a file which includes the line $D$. As mentioned above, the edges are labelled with the change that introduced them, in this case $c_0$. An important feature to note is that vertices are uniquely identified, by the hash of the change that introduced them, along with a position in that change. This means that two lines of text with the same content, introduced by different changes, will be different. It also means that a line keeps its identity, even if the change is applied in a totally different context.

Moreover, this system is append-only, in the sense that deletions are handled by a more sophisticated labelling of the edges. In the example above, if we want to delete line $D$, we just need to make a change mapping the edge introduced by $c_0$ to a deleted edge, which is also labelled with the change in which it was introduced, this time $c_1$:

From now on, we call the full edges alive, and the dashed ones dead.

We have just described the two basic kinds of actions in Pijul. There are no other. One kind adds vertices to the graph, along with “alive” edges around them, and the other kind maps an existing edge label onto a different one. The edge labels are fully described by two parameters: (1) their status (alive, deleted, and a few others related to multiple files and technical details explained below), and (2) the change that introduced them.

Dependencies

This scheme allows one to define dependencies between changes:

  • If a change $c$ adds a vertex, we must have its “context”, i.e. the lines before and after it, hence the changes that introduced these lines are in the dependencies of $c$.

  • If a change $c$ deletes a vertex, or in other words maps an existing edge introduced by a change $d$, then $c$ must depend on $d$.

Of course, this is just the minimal set of dependencies needed to make sense of the text edits. Hooks and scripts may add extra language-dependent dependencies based on semantics.

Are edge labels minimal?

Our goal is to find the smallest possible system, both for reasons of mathematical aesthetics (why store useless stuff?) and the other one for performance. Therefore, one immediate question comes to mind: why even keep the change number on the edges?

In order to answer that question, suppose we don’t keep the labels, meaning that the maps happen between statuses only. Then, consider the following two situations:

  • Change inverses

    The first issue happens when two authors delete a line in parallel, and one of the authors reverts their change. Applying these changes yields the following diagram, where the two deletions get merged into one, and the inverse applies to both:

    However, this is not what we expect, since one of the authors explicitly reverted the deletion, while the other performed the same deletion in parallel. By keeping the labels, this is what we get instead:

  • Missing contexts

    For the sake of clarity, in the rest of this post, we name two users Alice (with pronouns “she/her”) and Bob (with pronouns “he/his”).

    In this scenario, Alice writes something in the middle of a paragraph $p$, while Bob deletes $p$ in parallel. One issue here is that the situation is not symmetric: when Bob applies Alice’s change, he can tell immediately that something is wrong, because the context of Alice’s edits is labelled as deleted in his repository.

    However, Alice’s situation is different: indeed, consider the case where instead of deleting $p$ in parallel of her changes, Bob deleted $p$ after applying Alice’s change. The edges deleted are exactly the same, but this is not a conflict, as shown in the following diagram:

    The situation is further complicated by the fact that this system doesn’t behave symmetrically with the contexts above and below the new line. Indeed, if Bob deleted the down context of the line (i.e. if he deleted line $C$) instead of the up context (line $B$), Alice could detect the conflict, since in that case, $C$ would have both an alive and a dead edge pointing to it ($C$ is called a “zombie vertex” internally), as shown in the following diagram:

    Keeping the change identifiers on each edge allows us to solve this. Bob adds the labels of all the edges around the deleted lines to the dependencies of his change. Then, Alice can tell whether Bob knows of her change before applying it. The changes are conflict if and only if Bob doesn’t know of the new lines.

Conflicts and CRDTs

According to what we described so far, there are three types of conflicts in Pijul:

  • Two alive vertices that do not have a (directed) path between them, in either direction.
  • Two alive vertices have paths in opposite directions between them, meaning that the graph has a cycle.
  • Or zombie vertices.

Moreover, it is easy to show that Pijul implements a conflict-free replicated datatype (CRDT): indeed, we’re just adding vertices and edges to a graph, or mapping edge labels which we know exist because of dependencies.

However, Pijul’s datastructure models, in a conflict-free way, the conflicts that can happen over a text file. In fact, Pijul would remain a CRDT with or without the design choices explained above about edge labels: for example, we could decide that the “deleted” status has higher precedence. But as shown above, that wouldn’t model conflicts accurately.

Pseudo-edges

Moving back to the sequential (i.e. single-user) situation, suppose we start a file with many lines. Our user deletes all of them, adds one new line at the very beginning, and one at the very end, as shown in the following diagram:

If we represented this situation naively like in that diagram, the complexity of applying changes and outputting the repository would depend linearly on the size of the graph, as we would need to traverse the entire thing to know about line $C$, and know it comes after $B$.

The trick is to use what we call “pseudo-edges”, which are not part of any change, but are just here to keep the “alive subgraph” (the subgraph of the alive vertices) connected. Every time we delete an edge, we add pseudo-edges between the source of the edge and all the descendants of the target, like the dotted edge in the graph below:

Multiple files

The extension of this scheme to multiple files is done in the following way:

  • We introduce another type of edge label to indicate that edges represent files, and hence are not be transitive. In particular, when we delete a file, all descendant vertices below must be deleted.

  • Each file or directory is represented by two separate vertices: one is its name, the other one is an “inode” vertex representing the file itself. This allows directory renames to commute with file renames, and file renames to commute with edits at the beginning of the file. The naive representation where files are represented as just their name would cause the kind of conflicts described above when the contexts of new vertices are deleted in parallel.

  • The non-conflicting case is where the graph of alive files, reduced by contracting the edges from and to name vertices, is a tree, and each file has exactly one unique name.

    Negating this last sentence yields four different types of conflicts:

    • First, the graph of alive files can be disconnected, when a file is introduced in a directory deleted in parallel. When that happens, we double the deleted graph path with a path of pseudo-edges.
    • Another kind of conflict is when a file $a$ is renamed in parallel, to $b$ by Alice, and to $c$ by Bob. This includes the case where a file gets moved to two different directories, yielding a non-tree DAG.
    • Yet another one is when two different files are given the same name in parallel.
    • Finally, one can create a cyclic graph of directories: starting from two directories $a$ and $b$ at the root of the repository, Alice can move $a$ to a subdirectory $b/a$ of $b$, while Bob moves $b$ to a subdirectory $a/b$ of $a$.

A less naive representation

We now revisit the first two diagrams in this post: adding and deleting lines. Vertices are now referenced by a change number (for example $c_0$) and a byte interval in that change (for example $[0, n[$, which means “bytes from offset $0$ included to offset $n$ excluded”). Note that vertices can now represent an arbitrary number of lines. Moreover, the size they occupy in memory is independent from $n$ (assuming $n < 2^{64}$).

Starting from a single vertex $c_0:[0, n[$ with $n$ bytes (containing an arbitrary number of lines), introduced by change $c_0$, adding a line is done by first splitting $c_0:[0, n[$ at some offset $i < n$, and inserting the new vertex just like before.

This means in particular that the splitting of content into lines is done by the diff algorithm and is encoded in the changes, instead of being set in stone for all repositories. With a different diff algorithm, we could imagine splitting contents according to programming language structure.

Once we know how to split vertices, deletions are handled almost as before: as shown in the following diagram, where we first apply the same change $c_1$ as in the previous example, and go on to applying change $c_2$, which deletes the end of vertex $c_0:[0, i[$ from some $j<i$ to $i$, and the beginning of vertex $c_1:[0, m[$, from $0$ to some $k<m$.

One important difference with before is that our previous edges had two different roles which were not clearly distinguishable from one another until now. One of these meanings was to order the lines, and the other one was the status. However, now that vertices can be split, the “status” role of edges becomes ambiguous: for example, a deleted edge pointing to vertex some vertex $c:[i, j[$ means that bytes $[i, j[$ of change $c$ are deleted, but what if we split that vertex into $c:[i, k[$ and $c: [k, j[$? Should we add an extra deleted edge, and if so, where?

There is a simple solution: by introducing a new kind of edge label (named BLOCK in the source code) we can distinguish between “internal” or “implicit” edges that are only here to order the blocks, and “status” edges informing about the status of their target vertex. I’ll probably explain more about this in a future blog post, or in the manual, or in a paper.

Version identifiers

Version identifiers need more sophistication in Pijul than in other version control systems, because any two patches $c_0$ and $c_1$ that could be produced independently commute, meaning that applying $c_1$ after $c_0$ yields the same repository as applying $c_0$ after $c_1$, and we want version identifiers to reflect that.

This means that meaningful version identifiers must be independent from the order. While this could be achieved easily with any linear function of the hashes, for example taking the bitwise XOR of hashes, some naive schemes have the problem that servers could trick clients into accepting that versions are synchronised, even though they are not. The bitwise XOR described above has this problem, and so do other linear functions: since the hashes are random, there is a high probability that any $n$ hashes form an independent linear basis of the vector space of hashes. The problem of forging a version identity becomes easy, as it is just a matter of solving a linear equation.

Our new scheme for version identifiers comes from the discrete log problem. The version identifier of an empty repository is always the identity element $1$, and applying a change with hash $h$ on top of version $V = e^{v}$ is $V^h = e^{v\cdot h}$.

Now, because getting from $V$ to the exponent $v$ is hard (at least for a classical computer), forging a version identity is hard, since we would need to generate hashes whose multiplication is $v$, without even knowing what $v$ is.

As pointed by Dan Robertson, this scheme is similar to homomorphic hashing.

Diff algorithms

Myers diff

See http://blog.robertelder.org/diff-algorithm/.

Patience diff

See Patience diff on wikipedia.

The Nest

The Nest, hosted at nest.pijul.com, is a platform for hosting and publishing Pijul repositories. This chapter explains how to use it in greater detail.

Creating an account

In order to create an account on the Nest, go to https://nest.pijul.com, and fill in the signup form on the front page.

Alternatively, one can also signin with a GitHub, a Google or a Twitter account, with the same effect.

After confirming your account (if you chose to signup by email), you will be taken to your profile settings page, which allows you to add login details, such as SSH keys, and to create repositories.

At any time, you can go to your profile settings page by clicking on your login in the top right corner of the page.

Uploading public keys

Pijul and the Nest use two kinds of keypairs (public and secret): keys for signing patches, and keys for SSH access.

SSH public keys

The Nest’s profile settings page contains a field to add SSH public keys. At the moment, supported formats are Ed25519 and RSA. Pijul can handle password-protected keys, and can talk to SSH agents.

Creating repositories

In order to create a repository on the Nest, you first have to go to your profile settings page (by clicking on your user name in the top right corner of the page).

Then, just enter the name of your new repository in the field at the bottom of the list of your repositories, and voilà! You are ready to push patches directly to your new repositories (no need to clone it, patches can be pushed directly).

Contributing to a project

Because of the patch-based nature of Pijul, contribution just means sending a change to a repository.

The owner of a repository can give permissions to other users in the “Admin” tab of the repository. If you have permission to push directly to a repository, you can do so with:

pijul push my_login@ssh.pijul.com:owner/repository

If my_login is the same as owner, you can omit owner.

Discussions

Another way to contribute to a repository on the Nest is to open a discussion, and attach changes to it. A change can be attached with the following syntax:

pijul push my_login@ssh.pijul.com:owner/repository --to-channel name:29

Which means “compare the current channel of my repository with channel name on the Nest, and attach the result to discussion number 29”.

In the special case where your current channel and the name of the Nest channel match, you can omit name.

Workflows

To give you a feeling of how pijul can be used in day-to-day situations, this chapter collects a couple of scenarios and how they can be solved with pijul.

These scenarios hopefully also convey a way of thinking about Pijul concepts, such as your repository, remote repositories, channels, changes, merges etc.

Most of these scenarios or ideas for them are taken from the Discourse forum, especially this post was very helpful in creating these pages.

Working with channels

What is a channel

Channels are a way to maintain two related versions of a repository in the same place (a bit like branches in Git).

Formally, a channel is a pointer to a set of changes (the state of a channel is a set of changes).

However, channels are different from Git branches, and do not serve the same purpose. In Pijul, independent changes commute, which means that in many cases where branches are used in Git, there is no need to create a channel in Pijul.

The main differences with Git branches are:

  • The identity of a change doesn’t depend on the branch it is on, or in other words, rebase and merge are the same operation in Pijul.
  • This implies that conflicts do not mysteriously come back after you solve them (which is what git rerere is for).
  • Also, conflicts are between changes, so the resolution of a conflict on one channel solves the same conflict in all other channels.

What you can do with channels

Creating a channel

A channel is created by using pijul fork <new-channel>. A succeeding command does not issue any output.

Listing channels

You can get a list of all channels with pijul channel:

\$ pijul channel

* main
  other

The currently active channel is marked with a * (asterisk).

Switching channels

To work effectively with a channel, you need to be able to switch channels, so that any changes you pijul record goes into the channel by default. This way, there is no chance of forgetting the --channel option, for example. Long story short, to change the channel, use

\$ pijul channel switch other

Do not expect any output here, no news is good news!

Merging channels

There is no simple way to merge all changes from one channel into another. This being said, the usual workflow currently is:

\$ pijul log --channel feature

Change EXKC67TCSWSPRG4JP5AK2MYSRG6RCC3N5W3N62KESUUFN3FN6R4AC
Author: []
Date: 2020-11-14 02:06:21.014281963 UTC

    Some change

...

\$ pijul apply EXKC67TCSWSPRG4JP5AK2MYSRG6RCC3N5W3N62KESUUFN3FN6R4AC

All changes this change depends upon will be applied as well.

Deleting and renaming channels

You can delete a channel by using the command pijul channel delete <channel>.

Rename it with pijul channel rename [<old>] <new>, where <old> is optional and defaults to the current channel.

Recording, splitting and combining changes

When working on code in your repository, you might come to a point where you feel that the pijul log doesn’t look quite as nice as it could: Some changes don’t work on their own, or some change is changing unrelated pieces of code. So you’d like to re-arrange (split and / or combine) your changes so that other, well-cut changes result.

On this page, you can learn how to achieve pijul log nirvana with the tools at hand.

Recording a change

After editing the files in your repository, you want to make pijul aware of the changes. You already came across the command(s) to do that: pijul add any files that aren’t tracked yet (probably using the -r flag for directories), and then pijul record all changes.

pijul record opens an editor for you containing a view of the change as it will be recorded:

message = ''
timestamp = '2020-12-07T06:59:38.257226181Z'

[[authors]]
name = 'me'
full_name = My Name
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3] H6EG7X443HN5LXRZ3TZLRQSIGW46NXGOMFXT2NP6KBGUKVDMF4RAC
[4]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

1. File addition: "file3.txt" in "/" 644
  up 1.0, new 0:11
+ ui
+ ae
+ nr
+ td

2. Edit in file2.txt:2 3.12
B:BD 3.22 -> 3.22:31/3
- uiaeuiae

3. Edit in file2.txt:3 3.12
  up 3.40, new 26:35
+ uiaexvlc

4. Edit in file.txt:4 4.11
  up 2.5, new 36:49
+ xvlcuiaexvlc

You can see the file is divided into three big blocks: the preamble with metadata about the change like message and author; one block describing the dependencies of this change; and one block containing a representation of all changes you made since recording the last change.

To finish recording, this file needs to be edited a bit and saved, and the editor window needs to be closed.

The message field has to be filled in, the fields under the [[author]] heading can be changed. Changes between # Dependenices and # Changes will not be regarded by pijul; the dependencies will be computed after you close the editor window. This part is purely informational.

The diff below # Changes, however, is where you can get creative with your change. The diff is composed of several small sections. Each section is introduced by an enumeration and a very short description of the diff (in the example you can see Edit in file and File addition) and what file the diff refers to.

You can remove any of these enumerated sections. This will leave the diff out of the recorded change, but your working copy will not be touched. This way, you can split up a big chunk of changes into several small ones or - if there were changes you didn’t want to keep in your repo - pijul reset whatever is left.

Amending

Suppose you (or someone else working in the same repository) have already recorded a large change, which you want to split into two separate changes (maybe because you’d like to be able to push just one):

\$ pijul change 5HWYGVSSDKMEQP43H3JDSURPS7FJB4TXMAGMUAKSKWIFH6QRZN3QC

message = 'Big change, should be split up'
timestamp = '2020-12-05T15:34:38.416301967Z'

[[authors]]
name = 'me'
full_name = 'My Name'
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

1. File addition: "file2.txt" in "/" 644
  up 1.0, new 0:11
+ xvlcxvlc
+ uiaeuiae
+ nrtdnrtd

2. Edit in file.txt:4 3.11
  up 2.5, new 41:54
+ xvlcuiaexvlc

To edit the recorded change, you issue the command

\$ pijul record --amend 5HWYGVSSDKMEQP43H3JDSURPS7FJB4TXMAGMUAKSKWIFH6QRZN3QC

An editor will open showing the change (see above, Recording a change). Now, you can remove all sections you don’t want to go into the first change. Whatever you remove, the files themselves will still look the same as before the amend, only the record will have changed.

From here, you can pijul record the next change, again removing any sections you don’t want to have in it. Repeat until done.

Splitting a change

Although amending a change should be enough in most, if not all, scenarios, you might be happy to know there’s an alternative:

By doing pijul unrecord, directly followed by pijul record, and then opening a few instances of your favourite text editor to cut and paste the result into multiple temporary text files, you can pipe them into pijul apply. The whole procedure would look something like this:

Suppose we started with the change from above, and already did pijul unrecord to strip the record of the change, but not the content of the change itself. When you issue pijul record, you will be presented with the following in your editor:

message = ''
timestamp = '2020-12-07T06:59:38.257226181Z'

[[authors]]
name = 'me'
full_name = My Name
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3] H6EG7X443HN5LXRZ3TZLRQSIGW46NXGOMFXT2NP6KBGUKVDMF4RAC
[4]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

1. File addition: "file3.txt" in "/" 644
  up 1.0, new 0:11
+ ui
+ ae
+ nr
+ td

2. Edit in file2.txt:2 3.12
B:BD 3.22 -> 3.22:31/3
- uiaeuiae

3. Edit in file2.txt:3 3.12
  up 3.40, new 26:35
+ uiaexvlc

4. Edit in file.txt:4 4.11
  up 2.5, new 36:49
+ xvlcuiaexvlc

Now, you can split this text up into several files as you like, remembering to copy over the preamble of the original, resulting in something like this:

In file1:

message = 'First change'
timestamp = '2020-12-07T06:59:38.257226181Z'

[[authors]]
name = 'me'
full_name = My Name
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3] H6EG7X443HN5LXRZ3TZLRQSIGW46NXGOMFXT2NP6KBGUKVDMF4RAC
[4]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

1. File addition: "file3.txt" in "/" 644
  up 1.0, new 0:11
+ ui
+ ae
+ nr
+ td

2. Edit in file2.txt:2 3.12
B:BD 3.22 -> 3.22:31/3
- uiaeuiae

In file2:

message = 'Second change'
timestamp = '2020-12-07T06:59:38.257226181Z'

[[authors]]
name = 'me'
full_name = My Name
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3] H6EG7X443HN5LXRZ3TZLRQSIGW46NXGOMFXT2NP6KBGUKVDMF4RAC
[4]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

3. Edit in file2.txt:3 3.12
  up 3.40, new 26:35
+ uiaexvlc

And in file3:

message = 'Third change'
timestamp = '2020-12-07T06:59:38.257226181Z'

[[authors]]
name = 'me'
full_name = My Name
email = 'me@gmail.com'

# Dependencies
[2] 5PI4BUI3SDD23P7V3NNCA3FR6NA7YCMF7PJRIJNMZH5TGDA7AGWQC
[3] H6EG7X443HN5LXRZ3TZLRQSIGW46NXGOMFXT2NP6KBGUKVDMF4RAC
[4]+NM32CQUCZLDJIE3H4SZJU35WJTWES6X2CAVWVZCPDEINQKHYAXTAC

# Changes

4. Edit in file.txt:4 4.11
  up 2.5, new 36:49
+ xvlcuiaexvlc

You don’t need to worry about the # Dependencies block, it just needs to be there. The dependencies themselves will be re-computed by pijul when you record the change.

Now that your 3 files are saved, back out of the original pijul record by closing the editor without a message for your change.

Finish the task by doing

\$ pijul apply <file1
\$ pijul apply <file2
\$ pijul apply <file3

or, if you have a real lot of files, you can do some clever shell script like

\$ for file in file*; do pijul apply <\$file; done

Combining changes

To combine multiple changes into one, there currently is no better way than running pijul unrecord on all changes you want to combine, and then again pijul record them as described above.

Reference

This chapter contains a detailed manual of individual Pijul commands. More succinct help about individual commands can also be obtained directly from a terminal by running pijul --help.

Help for individual commands can also be obtained in the same way. Let’s take record as an example. The command to get help on record is pijul record --help.

pijul add

Adds a path to the tree.

Pijul has an internal tree to represent the files currently tracked. This command adds files and directories to that tree.

Usage

    pijul add [FLAGS] [PATHS]...

Args

<PATHS>...

        Paths to add to the internal tree

Flags

-f, --force

-h, --help

        Print help information

-r, --recursive

-V, --version

        Print version information

pijul apply

Applies changes to a channel

Usage

    pijul apply [FLAGS] [OPTIONS] [CHANGE]...

Args

<CHANGE>...

The change that need to be applied. If this value is missing, read the change in text format on the standard input

Flags

--deps-only

Only apply the dependencies of the change, not the change itself. Only applicable for a single change

-h, --help

Print help information

-V, --version

Print version information

Options

--channel <CHANNEL>

Apply change to this channel

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

pijul archive

Creates an archive of the repository

Usage

    pijul archive [FLAGS] [OPTIONS] -o <NAME>

Flags

-h, --help

Print help information

-k

Do not check certificates (HTTPS remotes only, this option might be dangerous)

-V, --version

Print version information

Options

--change <CHANGE>...

Apply these changes after switching to the channel

--channel <CHANNEL>

Use this channel, instead of the current channel

-o <NAME>

Name of the output file

--prefix <PREFIX>

Append this path in front of each path inside the archive

--remote <REMOTE>

Ask the remote to send an archive

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

--state <STATE>

Archive in this state

--umask <UMASK>

Append this path in front of each path inside the archive

pijul change

Shows information about a particular change

Usage

    pijul change [OPTIONS] [HASH]

Args

<HASH>

The hash of the change to show, or an unambiguous prefix thereof

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--repository <PATH>

Use the repository at PATH instead of the current directory

pijul channel

Manages different channels

Usage

    pijul channel [OPTIONS] [SUBCOMMAND]

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

Subcommands

delete

Delete a channel. The channel must not be the current channel

help

Print this message or the help of the given subcommand(s)

new

Create a new, empty channel

rename

Rename a channel

switch

Switch to a channel. There must not be unrecorded changes in the working copy

pijul channel rename

Rename a channel

Usage

    pijul channel rename <FROM> [TO]

Args

<FROM>

<TO>

Flags

-h, --help

Print help information

-V, --version

Print version information

pijul channel delete

Delete a channel. The channel must not be the current channel

Usage

    pijul channel delete <DELETE>

Args

<DELETE>

Flags

-h, --help

Print help information

-V, --version

Print version information

pijul clone

Clones an existing pijul repository

Usage

    pijul clone [FLAGS] [OPTIONS] <REMOTE> [--] [ARGS]

Args

<REMOTE>

Clone this remote

<PATH>

Path where to clone the repository. If missing, the inferred name of the remote repository is used

<SALT>

Flags

-h, --help

Print help information

-k

Do not check certificates (HTTPS remotes only, this option might be dangerous)

-V, --version

Print version information

Options

--change <CHANGE>

Clone this change and its dependencies

--channel <CHANNEL>

Set the remote channel [default: main]

--path <PARTIAL_PATHS>...

Clone this path only

--state <STATE>

Clone this state

pijul credit

Shows which change last affected each line of the given file(s)

Usage

    pijul credit [OPTIONS] <FILE>

Args

<FILE>

The file to annotate

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--channel <CHANNEL>

Use this channel instead of the current channel

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

pijul diff

Shows difference between two channels/changes

Usage

    pijul diff [FLAGS] [OPTIONS] [PREFIXES]...

Args

<PREFIXES>...

Only diff those paths (files or directories). If missing, diff the entire repository

Flags

-h, --help

Print help information

--json

Output the diff in JSON format instead of the default change text format

--short

Show a short version of the diff

--tag

Add all the changes of this channel as dependencies (except changes implied transitively), instead of the minimal dependencies

--untracked

Include the untracked files

-V, --version

Print version information

Options

--channel <CHANNEL>

Compare with this channel

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

pijul fork

Create a new channel

Usage

    pijul fork [OPTIONS] <TO>

Args

<TO>

The name of the new channel

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--change <CHANGE>

Apply this change after creating the channel

--channel <CHANNEL>

Make the new channel from this channel instead of the current channel

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

pijul init

Initializes an empty pijul repository

Usage

    pijul init [OPTIONS] [PATH]

Args

<PATH>

Path where the repository should be initalized

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--channel <CHANNEL>

Set the name of the current channel (defaults to “main”)

-k, --kind <KIND>

Project kind; if Pijul knows about your project kind, the .ignore file will be populated with a conservative list of commonly ignored entries. Example: pijul init --kind=rust

pijul log

Show the entire log of changes

Usage

    pijul log [FLAGS] [OPTIONS] [-- <FILTERS>...]

Args

<FILTERS>...

Filter log output, showing only log entries that touched the specified files. Accepted as a list of paths relative to your current directory. Currently, filters can only be applied when logging the channel that’s in use

Flags

--description

Include full change description in the output

-h, --help

Print help information

--hash-only

Only show the change hashes

--state

Include state identifiers in the output

-V, --version

Print version information

Options

--channel <CHANNEL>

        Show logs for this channel instead of the current channel

--limit <LIMIT>

        Output at most this many changes

--offset <OFFSET>

        Start after this many changes

--output-format <OUTPUT_FORMAT>

--repository <REPO_PATH>

        Set the repository where this command should run. Defaults to the first ancestor of the
        current directory that contains a `.pijul` directory

pijul ls

pijul mv

pijul pull

Pulls changes from a remote upstream

Usage

    pijul pull [FLAGS] [OPTIONS] [FROM] [-- <CHANGES>...]

Args

<FROM>

Pull from this remote

<CHANGES>...

Pull changes from the local repository, not necessarily from a channel

Flags

-a, --all

Pull all changes

-f, --force-cache

Force an update of the local remote cache. May effect some reporting of unrecords/concurrent changes in the remote

--full

Download full changes, even when not necessary

-h, --help

Print help information

-k

Do not check certificates (HTTPS remotes only, this option might be dangerous)

-V, --version

Print version information

Options

--from-channel <FROM_CHANNEL>

        Pull from this remote channel

--path <PATH>...

        Only pull to these paths

--repository <REPO_PATH>

        Set the repository where this command should run. Defaults to the first ancestor of the
        current directory that contains a `.pijul` directory

--to-channel <TO_CHANNEL>

        Pull into this channel instead of the current channel

pijul push

Pushes changes to a remote upstream

Usage

    pijul push [FLAGS] [OPTIONS] [TO] [-- <CHANGES>...]

Args

<TO>

Push to this remote

<CHANGES>...

Push only these changes

Flags

-a, --all

Push all changes

-f, --force-cache

Force an update of the local remote cache. May effect some reporting of unrecords/concurrent changes in the remote

-h, --help

Print help information

-k

Do not check certificates (HTTPS remotes only, this option might be dangerous)

-V, --version

Print version information

Options

--from-channel <FROM_CHANNEL>

        Push from this channel instead of the default channel

--path <PATH>...

        Push changes only relating to these paths

--repository <REPO_PATH>

        Path to the repository. Uses the current repository if the argument is omitted

--to-channel <TO_CHANNEL>

        Push to this remote channel instead of the remote's default channel

pijul record

Creates a new change

Usage

    pijul record [FLAGS] [OPTIONS] [PREFIXES]...

Args

<PREFIXES>...

Paths in which to record the changes

Flags

-a, --all

Record all paths that have changed

-h, --help

Print help information

--ignore-missing

Ignore missing (deleted) files

-V, --version

Print version information

Options

--amend <AMEND>

        Amend this change instead of creating a new change

--author <AUTHOR>

        Set the author field

--channel <CHANNEL>

        Record the change in this channel instead of the current channel

-m, --message <MESSAGE>

        Set the change message

--repository <REPO_PATH>

        Set the repository where this command should run. Defaults to the first ancestor of the
        current directory that contains a `.pijul` directory

--timestamp <TIMESTAMP>

        Set the timestamp field

--working-copy <WORKING_COPY>

pijul remote

Manages remote repositories

Usage

    pijul remote [OPTIONS] [SUBCOMMAND]

Flags

-h, --help

Print help information

-V, --version

Print version information

Options

--repository <REPO_PATH>

Set the repository where this command should run. Defaults to the first ancestor of the current directory that contains a .pijul directory

Subcommands

delete

Deletes the remote

help

Print this message or the help of the given subcommand(s)

pijul remote delete

Deletes the remote

Usage

    pijul remote delete <REMOTE>

Args

<REMOTE>

Flags

-h, --help

Print help information

-V, --version

Print version information

pijul remove

Removes a file from the tree of tracked files (pijul record will then record this as a deletion)

Usage

    pijul remove [PATHS]...

Args

<PATHS>...

The paths need to be removed

Flags

-h, --help

Print help information

-V, --version

Print version information

pijul reset

Resets the working copy to the last recorded change.

In other words, discards all unrecorded changes.

Usage

    pijul reset [FLAGS] [OPTIONS] [FILES]...

Args

<FILES>...

        Only reset these files

Flags

--dry-run

        Print this file to the standard output, without modifying the repository (works for a
        single file only)

-h, --help

        Print help information

-V, --version

        Print version information

Options

--channel <CHANNEL>

        Reset the working copy to this channel, and change the current channel to this channel

--repository <REPO_PATH>

        Set the repository where this command should run. Defaults to the first ancestor of the
        current directory that contains a `.pijul` directory

pijul unrecord

Unrecords a list of changes.

The changes will be removed from your log, but your working copy will stay exactly the same, unless the --reset flag was passed. A change can only be unrecorded if all changes that depend on it are also unrecorded in the same operation. There are two ways to call pijul-unrecord:

  • With a list of s. The given changes will be unrecorded, if possible.

  • Without listing any s. You will be presented with a list of changes to choose from. The length of the list is determined by the unrecord_changes setting in your global config or the --show-changes option, with the latter taking precedence.

Usage

    pijul unrecord [FLAGS] [OPTIONS] [CHANGE_ID]...

Args

<CHANGE_ID>...

        The hash of a change (unambiguous prefixes are accepted)

Flags

-h, --help

        Print help information

--reset

        Also undo the changes in the working copy (preserving unrecorded changes if there are
        any)

-V, --version

        Print version information

Options

--channel <CHANNEL>

        Unrecord changes from this channel instead of the current channel

--repository <REPO_PATH>

        Set the repository where this command should run. Defaults to the first ancestor of the
        current directory that contains a `.pijul` directory

--show-changes <N>

        Show N changes in a text editor if no <change-id>s were given. Defaults to the value of
        `unrecord_changes` in your global configuration

Licenses

Code exerpts taken from Pijul and reproduced in this manual are licensed under the same license as Pijul, i.e. the GPL-2.0 or any later version at your convenience. See https://www.gnu.org/licenses/old-licenses/gpl-2.0.txt for the full text of the license.

All other parts of this manual is licensed under the Creative Commons Attribution-ShareAlike 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0 or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA. The full text of the license is available at https://creativecommons.org/licenses/by-sa/3.0/legalcode.