How to Write a simple xonsh xontrib


The xonsh docs have a guide on how to write xontribs, but I think this guide is a bit brusk. It says everything it needs to say but leaves out a lot of the detail that someone like me might need to create a xontrib. So, while I write a xontrib for something else, I've decided to write a more detailed xontrib composition guide.

Though this will show you how to create a xontrib, it is presented as a log of me making my way through this process rather than a traditional step-by-step how-to.

When I say "create a xontrib," this phrase deceptively implies there is one step in creating a xontrib everyone can use. If you want to share your xontrib with the world, the involved steps are to:

  1. Literally create xontrib code and load it into your xonsh shell.
  2. Make this xontrib a distribution installable with something like pip.
  3. Host this xontrib distribution on PyPI.
  4. Register the xontrib with the xonsh project.

Once these steps are complete, the xontrib is ready for consumption by the good folks on the internet. If your aim is to write a xontrib solely for your personal use and never share it, you could stop at step 1 (but I'd recommend going to through step 2 as well).

For this guide, I'll focus only on steps 1 and 2, which should be enough to get anyone far down the path of developing and using a xontrib they have developed. Steps 3 is a more general Python distrubtion concern and is outlined in the Python Packaging User Guide section on the topic.

Part 4 is covered in the above linked doc on registering xontribs with the xonsh project.

Software versions

I'm using xonsh 0.9.3 and Python 3.7.3 for this guide.

Step 1: What makes a xontrib?

The xonsh docs have a guide on how to write xontribs!

These docs briefly tell us how to make a xontrib:

Writing a xontrib is as easy as writing a xonsh or Python file and sticking it in a directory named xontrib/.

That seems so simple! But how could that work? The docs are implying that creating any .xsh or .py file in a directory with the special name xontrib/ will allow the xontrib to be imported and loaded by xontrib load.

How could my xontrib/ directory somewhere on my filesystem get picked up and added to a globally importable xontrib package? The answer is implicit namespaces. Implicit namespaces are a convention, introduced in PEP 420, that the import machinery uses to construct a single package from multiple locations (these locations are called "partials").

The specifics of implicit namespaces are a bit more complicated than this, but for xontribs, this means almost exactly what we were told. We need to create a .xsh or .py file inside a directory named xontrib/; however, missing from the docs is the important point that this xontrib/ directory cannot just be anywhere on the file system - it must be available in the Python path. This should all become clear after an example.

So, if we follow the implicit namespace conventions for our xontrib, it will be available on the xontrib package as a module. xontrib load, the line you are probably familiar with if you've used a xontrib, takes the name of a module in the xontrib package, so xontrib load myxontribname will load the xontrib.myxontribname module as a xontrib.

xontrib "Hello World!"

Here's a trivial xontrib "Hello World!" example using these concepts.

My first xontrib, called hello_world has the following directory structure:

``` @eddie-ubuntu ~/source/xontrib-hello-world $ tree . └── xontrib └── hello_world.py

1 directory, 1 file ```

I created this by running the following in xonsh inside the xontrib-hello-world directory:

eddie@eddie-ubuntu ~/source/xontrib-hello-world $ mkdir xontrib eddie@eddie-ubuntu ~/source/xontrib-hello-world $ echo "print('Hello World!')" > xontrib/hello_world.py

Now, I want to load this xontrib, right? Let's see if xonsh has picked it up yet:

eddie@eddie-ubuntu ~/source/xontrib-hello-world $ xontrib load hello_world The following xontribs are enabled but not installed: hello_world To install them run xpip install

Hmm, this is a strange message, but it all makes sense when considering (1) what a xontrib is, (2) how implicit namespaces work with xontribs, and (3) how xontribs are normally installed. To break this message down, let's first tackle the statement that the xontrib is enabled but not installed. Indeed, we just enabled this xontrib with xontrib load hello_world, but by saying the xontrib is not installed, xonsh is effectively saying that the xontrib code cannot be found. The second part of the message, which tells us to install the xontrib via xpip install would be helpful if we were trying to install a xontrib from a distribution (a package with a setup.py that is hosted on PyPI, for instance). In our case, it isn't really helpful since we are trying to load a xontrib locally using the bare minimum components for creating a xontrib, so we can ignore this second message.

So, how do we install our trivial xontrib? Since we aren't installing a distribution package and we're just relying on the import machinery's understanding of implicit namespace packages, we have to make sure all the requirements of that convention are satisfied. So far, we have done everything to create a xontrib except making it discoverable via import machinery under the xontrib package. This next step will do that.

As usual, the way to make a package discoverable by import machinery is to add its containing directory to the sys.path global, so let's do that, picking up immediately where we left off before with the "The following xontribs are enabled but installed..." message:

``` eddie@eddie-ubuntu ~/source/xontrib-hello-world $ xontrib load hello_world The following xontribs are enabled but not installed: hello_world To install them run xpip install eddie@eddie-ubuntu ~/source/xontrib-hello-world $ from xontrib.voxapi import Vox eddie@eddie-ubuntu ~/source/xontrib-hello-world $ from sys import path eddie@eddie-ubuntu ~/source/xontrib-hello-world $ path ['/home/eddie/.virtualenvs/xonsh/bin', '/home/eddie/.pyenv/versions/3.7.3/lib/python37.zip', '/home/eddie/.pyenv/versions/3.7.3/lib/python3.7', '/home/eddie/.pyenv/versions/3.7.3/lib/python3.7/lib-dynload', '/home/eddie/.virtualenvs/xonsh/lib/python3.7/site-packages', '/home/eddie/source/xonsh', '/home/eddie/source/xontrib-z'] eddie@eddie-ubuntu ~/source/xontrib-hello-world $ tree . └── xontrib └── hello_world.py

1 directory, 1 file eddie@eddie-ubuntu ~/source/xontrib-hello-world $ path.append(os.getcwd()) eddie@eddie-ubuntu ~/source/xontrib-hello-world $ path ['/home/eddie/.virtualenvs/xonsh/bin', '/home/eddie/.pyenv/versions/3.7.3/lib/python37.zip', '/home/eddie/.pyenv/versions/3.7.3/lib/python3.7', '/home/eddie/.pyenv/versions/3.7.3/lib/python3.7/lib-dynload', '/home/eddie/.virtualenvs/xonsh/lib/python3.7/site-packages', '/home/eddie/source/xonsh', '/home/eddie/source/xontrib-z', '/home/eddie/source/xontrib-hello-world'] eddie@eddie-ubuntu ~/source/xontrib-hello-world $ xontrib load hello_world Hello World! ```

After adding the current directory, which contains the aforementioned xontrib/ named directory, we are able to enabled the xontrib, and upon loading it, we see that xonsh must think it's installed because it runs the code inside the hello_world.py file and prints "Hello World!" to the screen.

That concludes the creation of a super simple xontrib. The rest of this post will be concerned with building up a distribution out of a simple xontrib. To spoil the fun a bit, packaging up a xontrib is just a matter of getting setuptools to put our xontrib code in a xontrib/ directory that is already in the sys.path so that it can be loaded as a module on the xontrib package.

Step 2: How do we make a xontrib installable with something like pip?

Python packages are described for easy installation through the setup.py file.

I had not created a Python package before this one, though I was acquainted with setup.py files through the many examples I saw in GitHub projects, but I never understood setup.py very well.

In the fictional internet time between the last step and this one, I started working on a real xontrib that I will use for the rest of this guide. This means I'm abandoning the "Hello World!" xontrib and creating one called xontrib-per-directory-history. This will be like per-directory-history for zsh, which I used for a long time in my zsh configuration and now miss dearly. The following steps should be applicable regardless of the goal of the xontrib you are creating.

This is the setup.py configuration I ended up with for this xontrib:

``` xontrib-per-directory-history

Per-directory history for xonsh, like zsh's https://github.com/jimhester/per-directory-history

Restricts history to those that were executed in the current directory, with keybindings to switch between that and global history. """

from setuptools import setup

setup( name='xontrib-per-directory-history', version='0.1', description="Per-directory history for xonsh, like zsh's https://github.com/jimhester/per-directory-history", long_description=doc, license='MIT', url='https://github.com/eppeters/xontrib-per-directory-history', author='Eddie Peters', author_email='edward.paul.peters@gmail.com', packages=['xontrib'], package_dir={'xontrib': 'xontrib'}, package_data={'xontrib': ['*.xsh']}, platforms='any', install_requires=[ 'xonsh>=0.9.3', ], classifiers=[ 'Environment :: Console', 'License :: OSI Approved :: MIT License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Topic :: System :: Shells', 'Topic :: System :: System Shells', ] ) ```

xontrib packages have a particular layout that requires us to write some boilerplate in the setup.py. The xonsh docs link to a cookiecutter template that you can use to generate most of this, as well as the file structure, for you if you want.

Most of the kwargs passed to setup define obvious informational metadata and won't be discussed here. They are useful for attribution, searching on PyPI, etc., but won't affect the installability of your xontrib package. The most interesting (read: least obvious) of these informational pieces is the classifiers list, which you can find out more about on PyPI's classifiers page.

The following triplet effectively tells setuptools to do everything we did manually in Step 1 above:

packages=['xontrib'], package_dir={'xontrib': 'xontrib'}, package_data={'xontrib': ['*.xsh']},

packages says that this package will provide a package named xontrib, package_dir maps this xontrib package to the directory that contains the xontrib package's code, and this package_data setting tells setuptools to not only install .py files into the package's final installation location, but to also include any files matching the glob pattern *.xsh. More details on this setting can be found in the setuptools data files documentation.

Conveniently, setuptools will install multiple distributions that claim to create the same packages via the packages directive without clobbering one another (assuming their contents don't conflict). What this means for a xontrib is that all the xontribs that use packages=['xontrib'] will have their .py and .xsh files installed into the site-packages xontrib directory alongside the other installed xontribs' .py and .xsh files.

Have a look at the xontrib/ directory inside my xonsh virtualenv, for instance, which includes several other xontribs' code:

eddie@eddie-ubuntu ~/.virtualenvs/xonsh $ ls lib/python3.7/site-packages/xontrib fzf-widgets.xsh per-directory-history.xsh thefuck.py

Each of these .xsh and .py files were created by installing different xontrib Python packages using pip, yet their code lives in the same spot, discoverable by import machinery using the implicit namespace packages convention.

platforms allows us to specify that a xontrib should be available on one platform but not another. For mine, I chose 'any' because I didn't expect to introduce functionality that only works on some of xonsh's supported platforms.

So, now my xontrib is installable via pip!

``` eddie@eddie-ubuntu ~/source/xontrib-per-directory-history master $ tree
. ├── setup.py └── xontrib └── per-directory-history.xsh

1 directory, 2 files

eddie@eddie-ubuntu ~/source/xontrib-per-directory-history master $ xpip install .
Processing /home/eddie/source/xontrib-per-directory-history Requirement already satisfied: xonsh>=0.9.3 in /home/eddie/source/xonsh (from xontrib-per-directory-history==0.1) (0.9.3) xontInstalling collected packages: xontrib-per-directory-history rib Running setup.py install for xontrib-per-directory-history ... -ldone Successfully installed xontrib-per-directory-history-0.1 aYou are using pip version 19.0.3, however version 19.1.1 is available. You should consider upgrading via the 'pip install --upgrade pip' command. eddie@eddie-ubuntu ~/source/xontrib-per-directory-history master $ xontrib load per-directory-history
hello world ```

And, yes, the only code in my xontrib's .py file right now is print('hello world'), but that's beside the point.


I hope you've found this guide useful for getting started in developing a xontrib, but obviously I've left out everything regarding what you can do with a xontrib. xontribs allow hooking into shell functionality, such as that provided by prompt toolkit. Using prompt toolkit's keybinding functionality would allow you to install a new keyboard command to run some code, for instance. For more examples and inspiration for what can be done in a xontrib, I suggest checking out the xontribs bundled with xonsh by default. Happy hacking!

For information on sharing your xontrib with the world, check out the links in the introduction.