path: root/blog/threa.md

+++
title = "Writing a Clipboard Manager"
rss = "Raku's concision demonstrated in form of a tutorial"
date = Date(2021, 7, 3)
tags = ["clipboard", "fun", "raku"]
+++

!!! note "A word of protest"

    This was intended to be presented in [The Raku Conference],
    however the organizers insisted on using [Zoom] and [Skype],
    which are privacy invasive platforms running on proprietary software
    and shadily managed.

\toc

## Motivation

Clipboard management is very important to my workflow.  To me,
a clipboard manager is useful in two ways:

1. It extends my (rather poor) temporary mundane memory by caching
   a few dozens of most recent selections.
2. It synchronizes clipboard and primary selections.
   Since some programs only support one kind of selection,
   this is particularly useful.

For the first point, I have to be able to choose from the history
by pressing a few keystrokes.  Having to touch the mouse during
writing sessions is unacceptable.  The menu dropping down from the systray
is also undesirable because I have a multi-monitor setup.  This narrows down
to only one plausible option: [Diodon], which I having been using on Debian
for at least two years.  However, as I was migrating to NixOS earlier
last month, [I was unable to package it for Nix][126190].

Naturally, I went looking for [alternatives], most of which I had
tried before and did not satisfy my requirements.  [clipmenu] got
my attention however: it was made to work with dmenu(-compliant launchers),
which I had a rather nice experience with in [Sxmo] on my [PinePhone].
However, I use [awesome] on my workstation and its widget toolkit covers
my launcher and menu need perfectly.  I don't need [dmenu] and do not
wish to spend time configuring and theming it.  Plus, the architecture
of dmenu scripts and awesome widgets vastly differs: while awesome
executes the input programs, dmenu is called from the scripts.

## Inspirations and Design

As even the most plausible candidate is not a suitable replacement,
I would need to write my own clipboard manager.  clipmenu is not really
a good base though because it's written in shell script, something I ain't
fluent in.[^1]  Its idea is brilliant however:

> 1. `clipmenud` uses `clipnotify` to wait for new clipboard events.
> 2. If `clipmenud` detects changes to the clipboard contents,
>    it writes them out to the cache directory
>    and an index using a hash as the filename.

I later translated [clipnotify] to [Zig][^2] and called it [clipbuzz].[^3]
From clipbuzz's usage,

```sh
while clipbuzz
do # something with xclip or xsel
done
```

and this is exactly how yet another clipboard manager was written,
but before we get there, let's talk about this article's sponsor!

I'm kidding d-; though we cannot jump into the implementation just yet:
we only resolved the first point out of two.  How about the data structure?
Hashing sounds like overengineering in this case: nobody needs more than
a few dozen entries[^4] and hashes are not very memorable.  Printable characters
can serve much better as indices.

What?  What happens when we run out of them?  We reuse/recycle them![^5]
They would also fit within one single line, heck, we just store all of them
in order inside a file and rotate each time there's a new selection.
Picking would just be moving a char to the beginning.  The entire
cache directory can just look something like this:

```console
$ ls $XDG_CACHE_HOME/$project
order
R
A
K
U
```

Wait, is that a sign?  We must use [Raku] to implement `$project` then…
Speaking of `$project`, I planned to use it with awesome and [vicious]
so let's call it something brutal, like a *cutting board*, which is *thớt*
in Vietnamese, an Internet slang for *thread*.  Cool, now we have
the daemon name, and conventionally the client shall be *threac*,
or *threa client*.

## Daemon Implementation

### Reading Inputs

Raku was chosen[^6] for the ease of text manipulation and seamless interfacing
with external programs.  I learned it quite a while ago and has always been
waiting for a chance to do something more practical with it, other than
competitive programming which isn't a good fit due to Rakudo's poor performance.
In Raku, the snippet from clipbuzz's README becomes:

```sh
while run 'clipbuzz' {
    # do something with xclip or xsel
}
```

Out of all languages I know, this is by far the simplest way to [run]
an external program.  Most would require one to import something
or do something with the call's return value, and don't even get me start
on POSIX *fork* and *exec* model.

OK, now what are we gonna do with `xclip`?  One obvious thing would be
to read the current selection.  Raku got you covered, fam:

```sh
my $selection = qx/xclip -out/;
```

Remember when I said Raku can seamlessly interact with external programs?
[qx] is how you capture their standard output, it is really that simple.
But wait, which selection is that?  No worries, `xclip` supports both
primary and clipboard:

```sh
my $primary = qx/xclip -out -selection primary/;
my $clipbroad = qx/xclip -out -selection clipboard/;
```

### Cache Directory Setup

This is when we write those selection down for later use, right?
Well, we need to figure out where to save them first.  According to
[XDG Base Directory Specification][XDG], `$XDG_CACHE_HOME` shall
falls back to `$HOME/.cache`:

```sh
my $XDG_CACHE_HOME = %*ENV<XDG_CACHE_HOME> // path $*HOME / '.cache':;
```

For convenience purposes, I defined the `/` operator as an alias
for path concatination:

```sh
multi sub infix:</>($parent, $child) { add $parent: $child }
```

With `$XDG_CACHE_HOME` defined, we can prepare the base directory as follows:

```sh
my $base = $XDG_CACHE_HOME.IO / 'threa';
mkdir $base: unless $base.e;
die "thread: $base: File exists" when $base.f;
```

As [a wise man once said][roles],

> As it often happens, writing an article ends up with a bug found in Rakudo.

In this case, there's a [bug in mkdir][1507] that makes it happily returns
even if the target path is a file.  I'm trying to fix it at the moment
but [a test][4408] is still failing.  *Update: it passed after a maintainer
bumped the dependencies to the patched version.*

Anyway, back to our clipboard manager.  Here we are using
[flow controllers][control] such as `unless` and `when` in the form
of *statement modifiers*, which can sometimes be easier on eyes
keeping the code flat.  Existence checks like `e` (exists) and `f` (file)
are also really handy.  Next, we check on the `order`:

```sh
constant $ALNUM = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';

sub valid($path) {
    return False unless $path.f;
    so /^\w\w+$/ && .chars == .comb.unique given trim slurp $path
}

my $order = $base / 'order';
spurt $order, $ALNUM unless valid $order;
```

Instead of printable, we only allow alphanumerics and fallback to
the uppercase ones (mainly because my screen can only fit as much vertically),
unless `$XDG_CACHE_HOME/threa/order` is a file, contains at least
two unique alphanumerics (exclusively).  Reading and writing files in Raku
is incredibly trivial, just `slurp` and `spurt` the path.  Since we are not
interested in whitespaces, they are `trim`'ed from `order`.  Notice that
Raku allows subroutines to be called without any parentheses—I love Lisp,
but opening parenthesis *after* the function name always confuses me,
especially when nested.

As you might have guessed, `given` is another statement modifier
setting the [topic variable] that is particularly useful
in [pointfree programming], where regular expressions (e.g. `/^\w\w+$/`)
are matched against directly and methods are called without specifying
the object.  Raku is also a weakly-typed language: `.comb.unique` (a list
of unique characters) is coerced into an integer when compared to one
(number of `.chars`).

### Comparing and Saving Selections

What do we do with the order then?  First we can determine the latest
selection and compare it to the ones we got from `xclip` earlier
to see which one is really new.  We'll also need to rotate the order,
i.e. write the new selection to the `$last` file and move it in front
of the others that we `$keep` as-is:

```sh
my ($first, $keep, $last) = do
    given trim slurp $order { .comb.first, .chop, .substr: *-1 }
my ($other, $content) = do given try slurp $base / $first or '' {
    when * ne $primary { 'clipboard', $primary }
    when * ne $clipboard { 'primary', $clipboard }
}
```

On the first few run, probably the cache files don't exist just yet,
so we fall them back to empty ones using `try ... or ...`.  We need
to know the `$other` selection (outdated one) to later synchronize them both.
In case of reselection, neither is updated and we simply skip this iteration:

```sh
next unless $other;
```

Otherwise, let's go ahead, write down the `$content`, rotate `$order`
and synchronize with the `$other` selection:

```sh
my $path = $base / $last;
spurt $path, $content;
spurt $order, $last ~ $keep;
run <xclip -in -selection>, $other, $path
```

That's it, now put the daemon in `$PATH` and run it in `~/.xinitrc`
or something IDK.  If you're worried that some selection might be
too big to read that you'll the next event, asynchronize the `qx` calls
by prefixing them with `start`, and `await` the results later on.
It is *that* easy.

### Command-Line Interface

Hol up, what if I want to store the cache elsewhere or use another set
of characters?  *"Then you can go right ahead and have an intercourse
with yourself, you ungrateful little piece of [redacted]."*  I would have said
this were I to implement this in other languages, but luckily I got Raku,
and Raku got `sub MAIN`:

```sh
sub MAIN(
  :$children where /^\w\w+$/ = $ALNUM, #= alphanumerics
  :$parent = $XDG_CACHE_HOME           #= cache path
) {
    my $snowflakes = $children.comb.unique.join;
    my $base = $parent.IO / 'threa';
    my $order = $base / 'order';

    while run 'clipbuzz' {
        ...
        spurt $order, $snowflakes unless valid $order;
        ...
    }
}
```

No matter how cool you think this is, it is cooler, I mean, look:

```console
$ thread --help
Usage:
  thread [--children[=Str where { ... }]] [--parent=<Str>]
  
    --children[=Str where { ... }]    alphanumerics
    --parent=<Str>                    cache path
```

## Client Implementation

### Back-End

Following the Unix™ philosophy, `threac` will do only one thing and do it well:
it shall take the chosen selection and *schedule* it to move to the beginning:

```sh
my $XDG_CACHE_HOME = %*ENV<XDG_CACHE_HOME> // path add $*HOME: '.cache':;

sub MAIN(
   $choice where /^\w?$/,    #= alphanumeric
  :$parent = $XDG_CACHE_HOME #= cache path
) {
    my $base = $parent.IO.add: 'threa';
    my $order = add $base: 'order';
    spurt $order, S/$choice(.*)/$0$choice/ with $order.slurp;
    my $path = $base.add: $choice;
    run 'xclip', $path
}
```

The highlight here is the non-destructive substitution `S///`,
which allow regex substitution in a pointfree and pure manner.
Though, instead of moving `$choice` to top of the deque, we place it
at the bottom and use `xclip` to trigger the daemon to do it
and synchronize between selections.

### Front-End

Note that `threac` does not give any output: selection history (by default) 
are stored in a standard and convenient location to be read by any front-end
of choice.  For awesome I made a menu whose each entry is wired to `threac`
and `xdotool` (to simulate primary paste with `S-Insert`) and bind
the whole thing to a keyboard shortcut.

```
local base = os.getenv("HOME") .. "/.cache/threa/"
local command = "threac %s && xdotool key shift+Insert"
local f = io.open(base .. "order")
local order = f:read("*a")
f:close()

local items = {}
for c in order:gmatch(".") do
  local f = io.open(base .. c)
  table.insert(items, {f:read("*a"):gsub("\n", " "), function ()
    awful.spawn.with_shell(command:format(c))
  end})
  f:close()
end
awful.menu{items = items, theme = {width = 911}}:show()
```

## Conclusion

Through writing the clipboard manager [threa], which is released
under [GNU GPLv3+] on [SourceHut], we have discovered a few features
of Raku that make it a great *scripting* language:

* Out-of-box CLI support:
  - Running programs and capturing output
  - Environment variables
  - File system operations
  - Builtin argument parser
* Concision:
  - Statement modifiers
  - Topic variable
  - First-class regex
  - Trivial asynchronization

As a generic programming language, Raku has other classes of characteristics
that makes it useful in other larger projects such as grammars (i.e. regex
on steroids) and OOP for human beings.  It is a truly versatile language
and I really hope my words can convince someone new to try it out!

[^1]: I ain't proud of this, okay?
[^2]: I'm obsessed with exotic languages.
[^3]: The *z*'s are for Zig, how original, I know.
[^4]: [*citation needed*]
[^5]: Wow much environment!
[^6]: By some supernatural being of course!

[The Raku Conference]: https://conf.raku.org
[Zoom]: https://stallman.org/zoom.html
[Skype]: https://stallman.org/skype.html
[Diodon]: https://launchpad.net/diodon
[126190]: https://github.com/NixOS/nixpkgs/pull/126190
[alternatives]: https://search.nixos.org/packages?query=clip
[clipmenu]: https://github.com/cdown/clipmenu
[Sxmo]: https://sxmo.org
[PinePhone]: https://www.pine64.org/pinephone
[awesome]: https://awesomewm.org
[dmenu]: https://tools.suckless.org/dmenu
[clipnotify]: https://github.com/cdown/clipnotify
[Zig]: https://ziglang.org
[clipbuzz]: https://git.sr.ht/~cnx/clipbuzz
[vicious]: https://vicious.rtfd.io
[Raku]: https://raku.org
[run]: https://docs.raku.org/routine/run
[qx]: https://docs.raku.org/syntax/qx
[XDG]: https://specifications.freedesktop.org/basedir-spec/latest/ar01s03.html
[roles]: https://vrurg.github.io/2021/06/16/article-on-roles
[1507]: https://github.com/MoarVM/MoarVM/pull/1507
[4408]: https://github.com/rakudo/rakudo/pull/4408
[control]: https://docs.raku.org/language/control
[topic variable]: https://docs.raku.org/syntax/$_
[pointfree programming]: https://raku-advent.blog/2020/12/22/draft-whats-the-point-of-point-free-programming
[threa]: https://sr.ht/~cnx/threa
[GNU GPLv3+]: https://www.gnu.org/licenses/gpl-3.0
[SourceHut]: https://sourcehut.org