Ever since my daughter learned the word “fast-forward,” meaning to skip ahead in a song or TV show, she took to saying “fast-backwards” to mean “rewind.” Despite the additional syllables, her phrase instantly took hold in our household. So, given the passage of time since my previous post from December 2016, I think some fast-backwarding is in order!

tl;dr XQuery for Humanists is out! And a roundup of my activities and other developments in the world of XQuery, TEI, and digital humanities.

The book!

At the end of my last post, I mentioned that I was “wrapping up” work on a book. In retrospect, that characterization was a tad optimistic. Three more years would pass. Last December, my co-author, Clifford Anderson, and I met at Circle Bistro in Foggy Bottom to review our final set of changes for the manuscript of XQuery for Humanists:

We just sent in the final proofs and index for XQuery for Humanists, due out from @TAMUPress—@andersoncliffb, Dec 10, 2019

Joe and Cliff reviewing the XQuery for Humanists manuscript

These meetings with Cliff—in Nashville or DC or at conferences elsewhere, twice a year or so—were the highlights of working on the book. I couldn’t have hoped for a better co-author.

At long last, at the end of February 2020, a satisfyingly dense package arrived from Texas A&M University Press. My first book, hot off the presses!

A stack of copies of XQuery for Humanists, hot off the press

That box arrived just shy of one year ago—just as COVID-19 began to ravage the world. While the pandemic and other events of 2020 absorbed much of the energy I had planned to put into the book’s launch, I am so pleased with the reception the book has received, and especially that it has reached distant shores and entered classrooms. Here are some selected tweets from readers:

Y’all. This excellent book by @andersoncliffb and @joewiz arrived in mid March. I intended to tweet this photo on the last Friday I was in my office. And then everything happened. So I’m posting it now and saying I’m really glad I brought it home with me—@KathrynTomasek, May 22, 2020

I liked the book very much it’s extremely well written—@ZornGerhard, Sep 19, 2020

Your book was one of five introductions to XQuery that I especially recommended to my class on Wednesday. ;-)—@MonikaBarget, Nov 19, 2020

The moment when #XQuery does not work and you remember the magic of *: from #XqueryForHumanists :). Great book!—@CatoMinor3, Dec 4, 2020

The companion website

Whether you’re a reader of the book, just curious about it, or a seasoned user of XQuery, I hope you have explored the book’s companion website. To make it I used several technologies familiar to readers of the book and this blog. Naturally, I used XQuery to prepare the site’s Code section by transforming the plain text source files into GitHub-flavored Markdown. I also used XQuery to validate that all of the sample code was free of syntax and well-formedness errors and would run in the three XQuery processors we highlighted in the book. We addressed processor-specific issues in our Implementations page. We hope the site is a useful resource for all learners of XQuery.

Under the hood, the book’s companion website is a static site powered by Jekyll and hosted by GitHub. My main criterion was good syntax highlighting for support for XQuery, XML, XSLT, and JSON. Rouge, which Jekyll has used as its default syntax highlighting library since version 3, does a great job on this front; see the sample XQuery page. As a result, both the companion site and this blog now run on Jekyll, version 4. GitHub Pages’s support for HTTPS on custom domains such xquery.forhumanists.org and joewiz.org made it a great platform to host the site’s content.

Other projects and news

Between the book and other developments, there’s now a lot to catch up on in the sphere of XQuery, XML, and digital humanities. So let’s fast backwards, with a sampling of papers, packages, and projects, in roughly reverse chronological order:

Outside of my own projects, numerous tools and projects have entered the scene and become invaluable, among which include:

  • TEI Publisher, created by Wolfgang Meier and now developed under e-editiones, lowers barriers and empowers TEI projects, by ingeniously leveraging the TEI Processing Model and establishing a common framework for digital editions. I discuss TEI Publisher in the Balisage and ICEDD papers linked above. Continuing the work of the late Sebastian Rahtz, TEI Publisher’s community is active and rapidly growing.
  • existdb-vscode, by Wolfgang Meier, brings great XQuery support to Visual Studio Code.
  • generator-exist, by Duncan Paterson, simplifies the task of generating scaffolding for library and application EXPath Packages for eXist. I used this when I prepared airtable.xq and Airlock for publishing.
  • XQuery Fiddle, by Martin Honnen, excels at sharing XQuery snippets and is used widely in Stack Overflow answers.
  • Dash docsets, by Norm Walsh, makes searching the W3C XML and XQuery specifications a snap.
  • exist-stanford-nlp, by Loren Cahlander, brings the Stanford CoreNLP library for named multilingual entity recognition and other natural language processing capabilities like parts of speech tagging and sentence tokenization to XQuery and eXist. Here’s some sample code I wrote.
  • github-xq, by Winona Salesky, lets you interact with the GitHub API from XQuery—making commits, submitting pull requests, creating branches, and responding to webhooks.

XQuery implementations and the community

In addition, the XQuery implementation that I’ve used on a daily basis, eXist, has leaped from version 3 to 4 and now 5, bringing extensive XQuery 3.1 support, improvements in performance and stability, and transformative features like facets and fields. eXist has modernized greatly, jettisoning its legacy build process with a standard Maven build process; you can check eXist’s conformance with the W3C’s XQuery test suite using the exist-xqts-runner; and you can use it with Docker and continuous integration pipelines, including GitHub Actions.

Agreed, the facets & fields facility in eXist 5 has made me rethink search, sort, and other aspects of my XQuery design. The ability to pre-generate facet & field contents means you can skip a lot of XQuery processing at request time. It’s like a customized, persistent cache.—@joewiz, May 1, 2020

While eXist 5.2 has been out for nearly a year, don’t mistake this for a lack of activity; the core developers and community contributors have put considerable energy into new features and bugfixes, and the next version—more likely version 6 than 5.3—is shaping up to be the best yet.

In these past few years, the eXist community has undergone a renaissance, with a very active Slack channel and weekly community calls (see these calendar and invite links). It’s exciting to see talented new users with great DH projects. This community also has a good deal of overlap with the e-editiones and TEI Publisher community, which also has a great Slack channel (see this invite link).

Like eXist, BaseX, Saxon, and oXygen XML Editor have issued a steady stream of stellar releases with great support for XQuery 3.1. These products all are indispensable tools in my toolbox.

Two new XQuery implementations, xqerl, by Zack Dean, and FusionDB, by Adam Retter & co., have entered the scene and look to have a great future. For eXist users, FusionDB is compatible with eXist, and Fusion Studio is a promising IDE and administrative tool that could consolidate many of the tools that eXist users rely on: eXide, Monex, the Java admin client, and possibly more.

Looking ahead, there are some exciting ideas in the community draft proposal of XQuery 4, led by Michael Kay and developed through the XML.com Slack workspace’s #xpath-ng channel and in several repositories on GitHub.) If you haven’t joined the XML.com Slack workspace, please do! If you’ve attended Balisage, you’ll know what I mean if I say, it’s like Balisage, but all year long; if you haven’t, then just know it’s a great, vendor-neutral forum for asking questions and talking about XML and markup technologies.

One final note: Thanks to the Center of Digital Humanities Research at Texas A&M University for supporting the book!

CoDHR is pleased to announce the publication of XQuery for Humanists, the newest release in its Coding for Humanists book series published by the Texas A&M University Press!—@TAMU_CoDHR, May 27, 2020