Writing tools

Lately, I’ve been using a new method for writing up my homework and other memorandum- or paper-like things I need to write. I’m not claiming I’m particularly good at writing, since I’m not, but this particular collection of tools seems to be helping me focus more on actual writing and less on formatting. It’s also helped me be a little more productive, since I can now work on things both with my desktop and my laptop.

I know, you’re about to ask, has it taken him this long to discover Dropbox? Well, yes, but that’s only part of the solution. Instead, it’s a combination of a markup language, a text editor, a text processor, a bibliographic manager, a sharing mechanism, and tools to integrate all of these things together.

First, a brief disclaimer. I use a Windows laptop and a Linux desktop. Since a lot of tools work on MacOS and Windows but not Linux, or MacOS and Linux but not Windows, this puts a constraint on the tools I can use.

Markup Language: Markdown

Markdown is a lightweight markup language that, in some of its dialects, provides the capabilities I need

• Basic formatting, including links and lists,
• Tables,
• Footnotes,
• Mathematical notation, and
• Syntax highlighting.

It’s very simple to write, with very little fluff getting in the way. Even MediaWiki markup often ends up taking up too much space in my typing with formatting code, rather than what I’m actually trying to say. Forget about HTML.

For the past decade, I’ve used LaTeX for pretty much everything I need to write. But that required a lot of looking things up because of a problem with LaTeX: you can do almost anything with it. What that means is I spend a lot of time fiddling with little settings instead of getting down to what I’m supposed to be doing, like my homework. The good news is that I’ve got a template, mostly cribbed from an old CS51 homework template, that I can mostly just fill in to get the formatting roughly okay, but that still needs tweaking. It doesn’t really need tweaking, but, if I see a knob, I fiddle with it. This takes time and makes me unhappy, since it never comes out just right, despite all the twiddling.

The great thing about Markdown is that it’s opinionated. The documents it generates (especially in its Pandoc incarnation), are reasonably well presented, and it doesn’t present as many little temptations for fiddling as any other language or tool I know that has the same breadth of features.

Text Editor: Atom

With what shall we edit our fancy new markup language? I’ve settled on Atom, brought to you by the folks at GitHub. It’s a relatively new editor, and not all the kinks have been worked out yet. I’ve also never been a professional webdev, so I’m not as enamored of JavaScript as some people are.¹

That said, the goal is once again a tool that gets out of the way but reminds me what’s going on if I happen to forget an arcane keyboard command. I used to use Vim for that, but Vim has a single great flaw: It’s difficult to use on Windows. To use Vim well, you need easy access to the escape key. On Linux, which I used to use for all my computing, it’s easy to remap the caps lock key as an escape key. In Windows, this isn’t the easiest thing to do. Even if it were, the caps lock key isn’t where you’d expect it to be on my laptop’s keyboard.

I could have switched to Emacs, but I don’t like having to stretch to type the chords I need to type. Also, it’s got so many knobs and dials and gizmos that it’s almost impossible for me to stop playing with the editor and actually edit.

So, why Atom? It’s free. It does syntax highlighting and obeys the usual keyboard conventions, but it’s mostly through process of elimination. Gedit isn’t featureful enough, Emacs is too featureful, Vim is no good on Windows, and everything else is either less featureful or costs money. That said, Atom is the single part of this process that I’m least attached to, so I’d love to hear suggestions for a better editor that fits my criteria:

1. Syntax highlighting
2. Don’t have to stretch too much (typing escape or C-M-S-v too often)
3. Multiple panes
4. Autocomplete (especially for Markdown)
5. Spell check
6. Works (and same features) on Linux and Windows
7. Has a menu system for when I forget the magic key combinations
8. Free, so the only switching cost is my time while in grad school

In my experience, it’s tough to find something that’s got both autocomplete and a menu system that works.

I like Atom’s package system, which it seems all the editors have these days. It lets other people add features without having to wait for a new version, while simultaneously letting me ignore most maintenance tasks. I use the following for writing Markdown:

1. autocomplete-paths for autocompleting image paths
2. language-pfm for highlighting, autocomplete, and other features
3. markdown-preview-plus for a better preview, though I still can’t get it to work with pandoc-citeproc
4. markdown-writer for, among other things, a decent table template
5. minimap to show you where you are in your file
6. typewriter for smoother writing
7. wordcount so I know how much I’m boring you
8. zotero-citations an absolute necessity for convenient use of Zotero

These packages mostly came from a very useful forum discussion.

Text Processor: Pandoc

Pandoc is a document conversion tool that can translate an extended version of Markdown, along with a host of other formats, into HTML, PDF, DOCX, ODT, Epub, and a slew of other formats too. Need to write a single document that will go in your PDF manual, your project website, and a man page all in one go? Pandoc has you covered. I’m pretty excited about it.

It adds a bunch of extensions to vanilla Markdown, including mathematical notation familiar to LaTeX users,² tables, footnotes, code blocks, footnotes, and many other features. Without them, Markdown would be unusable for most of my writing.

The YAML metadata blocks are useful, too, since they let me control things like font size and margins, which some instructors are sticklers about.

Most of the time I’m working on homework, I’ll use PDF output, but this website, for example, is written in Pandoc Markdown for HTML5 output.

The one thing I would like is the ability to caption blockquotes like you can for figures. I use two layers of blockquote to show the author of a quotation or a citation. That’s only a minor annoyance, though.

I also want to mention the very useful pandoc-citeproc, which takes a bibliographic database (such as a Bibtex file), a document, and a CSL file, and generates a well-formatted bibliography with in-text citations in the style you need. I tend to use Chicago, since I’m not writing for publication, but there are publicly available styles for most any format you need.

Bibliographic Manager: Zotero

Speaking of bibliographies, remember that I said that pandoc-citeproc needs a bibliographic database? The easiest one I’ve found is Zotero. It integrates with your web browser to let you store citations from web pages, and its translators pull out the relevant bibliographic information from pages on sites like the New York Times and Jstor. It will also store PDFs, and just generally make your bibliographic life easier to manage.

It has a built-in syncing mechanism, but I’ve never used it. I imagine it’s pretty useful, since it’s apparently tricky to use Zotero with Dropbox, for example.

It provides great integration with MS Word, so, if you need to write in Word, you can still use your bibliography.

The real trick is Better Bibtex (about which, more, later) and the Zotero-citations package for Atom. This combination will let you add Pandoc Markdown-formatted citations to your writing very easily. You can use the Zotero-citations “pick” command to search through your database by title, author, or most anything else and add a citation to that item. That search feature is hugely important, since it turns something that would be annoying into something incredibly easy.

Sharing mechanisms

If you use multiple devices for your writing, you’ll want to come up with a way to move files back and forth, so you can work on either machine. The simplest thing to do, I imagine, is use a USB stick, but it’ll get lost. So, there’s online mechanisms.

Dropbox

I’ve been using Dropbox. But I hear that there are cases where it’s lost people’s data in the past. Even if it hasn’t, it’s pretty easy for me to accidentally make myself lose data, since a delete will automatically propagate to my other machine.

So I wrote a script to automatically keep backups of my Dropbox files based on this one. That one needed some tweaking, but it should be relatively straightforward to get it working. One change I made was to use zip instead of rsync, since I’m making local copies, and I’d like to save at least some space if I can. I also had to make some changes because I’ve set my desktop to hibernate after inactivity, so I use anacron instead of cron to run my backup jobs. If you’ve got an always-on computer, you don’t need to worry about that.

The security issues associated with Dropbox don’t bother me, since I don’t keep anything on there other than my homework. I’ve heard good things about SpiderOak, though.

Git

For collaborative editing, I think git would be the best thing. Since Markdown files are just plain text, just like source code, source control tools are designed really well to handle revision control and collaboration involving Markdown documents. I haven’t actually done any collaborative authoring this way, but I think it’s the way I’d go. It does require an always-on computer, though, to function as a repository that people’s laptops can speak to.

Integration tools

Better Bibtex

Better Bibtex is a plugin for Zotero to make it better able to handle plaintext writing. It sets static citation keys for documents in your Zotero library.

You can use it to automatically export your library into a format that Bibtex (and also pandoc-citeproc) understand. I just export my whole library into a big CSL JSON file in my home directory, since I think that’s easiest.

If you want to use zotero-citations’s citation picker in Atom, you’ll also need to enable Better Bibtex’s HTTP export feature, which will run a webserver, allowing the Atom package to search your Zotero library. You may want to make sure your firewall is blocking non-local access to port 23119.

Powershell

Believe it or not, the Windows Powershell is actually usable. Which means that you can use the Windows version of Pandoc without too much pain. And that means this whole process actually works on Windows.

Putting it all together

If you’re using Linux, you’ll probably want to use cabal to build the latest pandoc and pandoc-citeproc packages, since the ones in most distros seem to be pretty elderly.

Remember to set up Zotero to automatically export your library.

pandoc -s -S -o output.pdf input.md --bibliography ~/zotero-library.json --filter pandoc-citeproc --csl ~/zotero-styles/chicago-author-date.csl

Other resources

• http://homes.esat.kuleuven.be/~bioiuser/blog/markdown-till-the-end/
• http://blog.cigrainger.com/2014/07/pandoc-markdown.html
• http://pandoc.org/README.html