Add new blog post

1 files changed, 278 insertions, 0 deletions

diff --git a/content/posts/2022-01-16-dict-1.md b/content/posts/2022-01-16-dict-1.md
new file mode 100644
index 0000000..37d5660
--- /dev/null
+++ b/content/posts/2022-01-16-dict-1.md
@@ -0,0 +1,278 @@
+---
+title: "Implementing DICT protocol: Part 1"
+date: 2022-01-16
+lang: en
+categories: [ blog ]
+tags: [dict, dictionary, go, golang, rfc2229, tcp ]
+translationKey: "2022-01-16-Dict-1"
+---
+
+## DICT Protocol
+
+What is DICT protocol?
+
+<figure>
+  <blockquote>
+    <p>
+    The Dictionary Server Protocol (DICT) is a TCP transaction based
+    query/response protocol that allows a client to access dictionary
+    definitions from a set of natural language dictionary databases.
+    </p>
+    <figcaption>
+      <cite>
+        <a href="https://datatracker.ietf.org/doc/html/rfc2229">
+          DICT Protocol - RFC 2229
+        </a>
+      </cite>
+    </figcaption>
+  </blockquote>
+</figure>
+
+Notable implementations for this include [dict(d)][dict]
+and [GNU dico(d)][dico]; the former is the reference implementation that
+supports multiple database formats, as listed in [dictfmt (1)][man-dictfmt].
+
+[dict]: https://github.com/cheusov/dictd
+[dico]: https://www.gnu.org.ua/software/dico/
+[man-dictfmt]: https://linux.die.net/man/1/dictfmt
+
+I intend to implement a server and multiple clients (CLI, GUI, ~~web~~) to this
+protocol, as well as some tools to easily create a dictd-readable database.
+
+## Why?
+
+No practical reason, but [dict] is one of the first command line tool
+introduced to me and easily one of my favorite, along with curl and [jq][jq].
+It's basically just a dictionary app, but it's cool:
+
+- works perfectly in terminal
+- easily self-hostable
+- fast
+- has cool dictionaries (though only Debian, Arch and derivatives distribute
+  those)
+
+[jq]: /posts/2021-06-13-jq/
+
+Also, I'm writing dictionaries for my [conlangs][conlang] and I want to
+distribute them via this protocol.  Clearly, implementing a server that is
+already implemented doesn't help, but I tend to go down rabbit holes.
+
+[conlang]: /misc/#conlangs
+
+I also like to explore non-web protocols, and starting with something simple
+like DICT might be a good idea.
+
+## Reading the spec
+
+The spec (linked at the top of this post) is shorter and easier to read than I
+thought.  Ignoring the introduction,  examples and citation, it's les than 20
+pages.  There are five classes of commands:
+
+- Querying the database: `DEFINE`, `MATCH`
+- `SHOW` metadata about the servers and the databases
+- Utilities: informing `CLIENT` name, check `STATUS`, show `HELP`, show
+    `OPTION` and `QUIT`
+- Authentication: `AUTH` and `SASLAUTH`
+
+The authentication ones are optional, and I don't find that useful, so I
+won't implement it anyway, this limits to the first three categories.
+
+## Handling TCP
+
+DICT is based on <abbr title="Transmission Control Protocol">TCP</abbr>,
+and there is a neat interactive <abbr>TCP</abbr> tool called [`telnet`][telnet],
+which I used for testing the commands.
+
+[telnet]: https://en.wikipedia.org/wiki/Telnet
+
+### telnet
+
+DICT runs on port 2628:
+
+```sh
+$ telnet dict.org 2628
+Trying 199.48.130.6...
+Connected to dict.org.
+Escape character is '^]'.
+220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <89168346.27665.1642303045@dict.dict.org>
+```
+
+Let's try out some commands to understand how this work.  Note that I prefix
+the command with `~> ` here so that it stands out of the response, and truncate
+long results with `[...]`.
+
+Let's first show what databases there are
+
+```
+~> SHOW DB
+110 166 databases present
+[...]
+.
+250 ok
+```
+
+There are a lot of dictionaries here, including [GCIDE][gcide], [WordNet][wn],
+[The Jargon File][jargon], [V.E.R.A.][vera], [FOLDOC][foldoc], but most of them
+are [FreeDict][fd] dictionaries.
+
+To  a word, the syntax is
+
+```
+~> MATCH database strategy word
+```
+
+Strategy is how the server will match the word you're looking up.  To list all
+strategies available, send the command:
+
+```
+~> SHOW STRATEGIES
+```
+
+There are various strategies supported by dictd, for example, `substring`,
+which matches if the entry has the queried word as substring:
+
+```
+~> MATCH jargon substring program
+152 13 matches found
+jargon "c programmer's disease"
+jargon "cargo cult programming"
+jargon "mickey mouse program"
+jargon "perfect programmer syndrome"
+jargon "program"
+[...]
+.
+250 ok [d/m/c = 0/13/5775; 0.000r 0.000u 0.000s]
+```
+
+This command only show which words in the database, if any, satisfy the match,
+without showing the definition.  To actually view a definition, one has to
+supply the dictionary name to the `DEFINE` command.  Note that, you can also
+use `*` for both `DEFINE` and `MATCH` command, which will define/match for all
+dictionaries.
+
+```
+~> DEFINE * programming
+150 3 definitions retrieved
+151 "programming" wn "WordNet (r) 3.0 (2006)"
+programming
+    [...]
+.
+151 "programming" jargon "The Jargon File (version 4.4.7, 29 Dec 2003)"
+programming
+ n.
+
+    [...]
+
+.
+151 "programming" foldoc "The Free On-line Dictionary of Computing (30 December 2018)"
+programming
+
+.
+250 ok [d/m/c = 3/0/145; 0.000r 0.000u 0.000s]
+```
+
+That's a gist of how to look up words with DICT protocol.  You can find more
+commands with:
+
+```
+~> HELP
+[...]
+.
+250 ok
+```
+
+Finally, to end the session, the command is:
+
+```
+~> QUIT
+221 bye [d/m/c = 0/0/0; 123.000r 0.000u 0.000s]
+```
+
+Note that, the response always ends with a period and a `250 ok`
+response---this is equivalent to HTTP's 200 OK---except for `QUIT`. These
+response code are defined in [the protocol specification][rfc2229].
+
+Commands other than `HELP` has some additional statistics, though this is
+optional.  I figured out that `d` means definitions, `m` means matches, and `s`
+is probably the time it took to query (why are they always zero, though?), but
+no clues on what `c`, `r`, and `u` mean.  I might check the [source code][dict]
+to figure that out, but let's leave it for another time.
+
+[gcide]: https://gcide.gnu.org.ua/
+[wn]: https://wordnet.princeton.edu/
+[jargon]: http://www.catb.org/~esr/jargon/
+[foldoc]: https://foldoc.org/
+[vera]: https://savannah.gnu.org/projects/vera
+[fd]: https://freedict.org/
+[rfc2229]: https://datatracker.ietf.org/doc/html/rfc2229#page-23
+
+### Go
+
+Of course we are not going to make the users type these commands (though it's
+not too unintuitive and can be easily remembered).  I chose Go to build the CLI
+client, though without any conscious consideration of fitness.  I'm trying out
+new things[^0] after all.
+
+From the [doc][go-net], we can figure out how to make a TCP connection.
+
+```go
+conn, err := net.Dial("tcp", "golang.org:80")
+if err != nil {
+	// handle error
+}
+fmt.Fprintf(conn, "GET / HTTP/1.0\r\n\r\n")
+status, err := bufio.NewReader(conn).ReadString('\n')
+// ...
+```
+
+Let's copy that and replace with DICT command instead of HTTP:
+
+```go
+conn, err := net.Dial("tcp", "dict.org:2628")
+if err != nil {
+	panic(err)
+}
+defer conn.Close()
+buf := bufio.NewReader(conn)
+fmt.Fprintf(conn, "MATCH jargon word programming\n")
+fmt.Fprintf(conn, "QUIT\n")
+
+for {
+	response, err := buf.ReadString('\n')
+	if err != nil {
+		// oftentimes this is EOF error
+		fmt.Println(err)
+		break
+	}
+	fmt.Printf(response)
+}
+```
+
+Running this code, we get response:
+
+```
+220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <89266600.1914.1642341395@dict.dict.org>
+152 4 matches found
+jargon "cargo cult programming"
+jargon "programming"
+jargon "programming fluid"
+jargon "voodoo programming"
+.
+250 ok [d/m/c = 0/4/3814; 0.000r 0.000u 0.000s]
+221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]
+EOF
+```
+
+which is a good start.
+
+There is a problem with this code: currently we are reading line by line,
+rather than reading the whole response for each command.  We can't know if line
+3 is response for the first command or the second this way.  A solution is to
+check if the line is prefixed with a status code, but do we have a better
+solution?
+
+Let's wait till next week!
+
+[go-net]: https://pkg.go.dev/net
+
+[^0]: Not really, I've written a CLI client for Wiktionary API with Go before.