blog

I use the ~/.Xmodmap file to redefine some keys. I don't know which Linux desktops source this file automatically. Using xmonad, I had to source the file during startup with this command: sleep 3 && xmodmap ~/.Xmodmap &

! This is a comment line
! This mapping swaps Neo's right Mod3 and the Return key on my physical US keyboard.
! I recommend this settings for Neo users who have to work on physical US keyboards.

! :r !xmodmap -pke | grep Level3_Shift
keycode  51 = Return Return Return

! :r !xmodmap -pke | grep Return
keycode  36 = ISO_Level3_Shift ISO_Level3_Shift ISO_Level3_Shift

! You can activate the mapping from VIM using one of these command
! :!xmodmap %
! :w !xmodmap -
! Or from the command line with
! xmodmap THIS_FILENAME

In this configuration file there are comments suggesting Vim commands to generate the configuration because the key codes and identifiers might differ from system to system.

References:

• http://tex.stackexchange.com/questions/1979/good-keyboard-layouts-for-typing-latex
• https://www.math.ucdavis.edu/~greg/pinky-rsi.html

The make program can take options in this format:

make USE_TIKZ=1 BOOK_PRINT=1

Make will automatically define and export these options as environment variables, available for all recepies. For my thesis I use these two options to enable TikZ on demand (because the compilation with TikZ is much slower) and to create a book print with roman and arabic page numbering (as opposed to a PDF with only arabic page numbering which I prefer).

To make this working, I have to manipulate my main thesis.Rtex file. BTW, the file is called .Rtex, not .tex, because the knitr preprocessor first evaluates the embedded R code and generates the .tex file.

At the beginning of my thesis.Rtex I have the following chunks of R code:

... % latex preamble

<<include=FALSE>>=
source('helpers.R')
@

... % latex code

<<echo=FALSE, results='asis'>>=
if (ifdef('BOOK_PRINT')) {
    cat('\\pagenumbering{roman}')
}
@

... % latex code

<<include=FALSE>>=
if (ifdef('USE_TIKZ')) {
    # tikz takes a very long time!
    # Use it for the final version:
    library(tikzDevice)
    opts_chunk$set(dev = 'tikz')
}

... % latex code

The first chunk loads an R file which defines a function ifdef (similar to the ifdef macro in C/C++).

The second chunk changes the page numbering to roman (for table of contents and list of figures, tables, listings, …). Later there is a chunk to reset the page numbering to arabic, starting at chapter 1.

The third chunk enables TikZ as output device for R plots.

This is my helpers.R file:

isSet = function(val) {
    if (is.character(val))
	return(val != '')
    return(as.logical(val))
}
ifdef = function(name) {
    var = Sys.getenv(name)
    isSet(var)
}

So yesterday I finished my thesis! It will be printed in a few hours. A few pages will be printed in color – so I noted their real page numbers (i.e. BOOK_PRINT=0, see my last post). Doing this, I stumbled upon this problem:

Cross references and links are highlighted in a blue color. For black/white print this is not a problem; links will be printed black (or grey). But links on colored pages will be printed in blue! This inconsistency looks horrible for me, so I added another knitr snippet in my thesis.Rtex file:

<<echo=FALSE, results='asis'>>=
if (ifdef('BOOK_PRINT')) {
    cat('\\hypersetup{colorlinks=false}')
} else {
    cat('\\hypersetup{colorlinks=true}')
}
@

It's important to note, that the \hypersetup command can only set this option if it has not been passed to the hyperref package yet.

\usepackage[
    % colorlinks=true, % must not be set here!
    linkcolor=blue,
    % ...
    pdfpagelabels
]{hyperref}

Finally I created the thesis.pdf with the command make BOOK_PRINT=1 and got a PDF file with non-highlighted links, roman page numbering for the first pages and arabic page numbering starting at the first chapter.

In this post, I describe some steps of the fully automated compilation process of my master thesis. I'm referring to the Makefile.

The thesis can be downloaded here (PDF) and this is the GitHub repository: https://github.com/schoettl/master-thesis

• latexmk -bibtex -lualatex thesis.tex
  • latexmk automatically processes the TeX file the correct number of times to enable the cross-references to the bibliography and other links to labels.
  • -bibtex, to use the literature database of my research group and my own one (.bib text files). These text file are not encoded in UTF-8; I think I should have better used bibtex8. With bibtex, I always have to write e.g. German umlauts like this: M\"uller. I wrote a bash script to translate fields like author names, abstract, etc. to the LaTeX ANSI encoding (utf82latex.sh).
  • -luatex, to really enable use of UTF-8 in LaTeX source. For example, in math mode I can write $α \cdot β$ instead of $\alpha \cdot \beta$ which is much nicer to read and write (using the Neo keyboard layout).
• ~R -q -e 'library(knitr); knit("thesis.Rtex", out = "thesis.tex")'~ – I use R in -q=uiet mode and pass an =-e=xpression to convert =thesis.Rtex to thesis.tex using knitr. The .Rtex format allows the use of R code in LaTeX, to generate numbers, tables, plots, and program output.
• I use command line utilities (GNU core utils) like join, awk, cp, grep, sed, etc. to prepare some sources for inclusion in the LaTeX document.
• ctags to generate a tags file to facilitate navigation in LaTeX sources in Vim.
• There are scripts in tools/
  • to extract methods or functions from Java and R files,
  • to remove the indentation from source code,
  • to skip lines of source and replace them with "…",
  • and a tool to convert UMLet XML files to vector graphics (using umlet itself).
• inkscape, to convert SVG files to PDF for inclusion in the document.
• emacs' command line API and Org mode, to convert another .org source to PDF to include it in the document.
• javap, to extract the class definition of a Java class from its binary file.

We use a shared calendar to document work hours in our forest. In this calendar, an event title only consists of the first letters of the first name. For example, the event's title is "js" when Jakob and Simon worked at this time.

$ curl -s "$CALENDAR_URL" | ical2text | work-hours-statistics
Arbeitsstunden insgesamt: 44.5
B: 7.00 h (16 %)
H: 10.00 h (22 %)
J: 8.00 h (18 %)
K: 8.00 h (18 %)
S: 9.50 h (21 %)
Y: 2.00 h (4 %)

We use a shared calendar at work. In this calendar, we add events like "Urlaub Jakob" when I take days off. To get an overview of who took how many vacation days, we also use ical2text. There is a difficulty with multi-day events as well as weekends and holidays but some simple scripts overcome this problems:

ical2text < cal.ics | grep ^2016 \
  | grep Urlaub | grep Jakob \
  | expand-multiday-events \
  | remove-events-on-holidays german-holidays.ics \
  | remove-events-on-weekends \
  | wc -l

• The program expand-multiday-events converts a line with a multi-day event to multiple lines of single full-day events.
• The program remove-events-on-holidays removes lines with events that take place on a holiday.
• The program remove-events-on-weekends does the same for events on weekends.

The final outcome is the number of Jakob's vacation days in 2016.

Note: There are holiday calendars on the web for download, e.g. http://www.schulferien.org/deutschland/ical/

We also use these programs at our start-up to acknowledge extraordinary work hours on holidays or weekends.

We plan to use the program for statistics about voluntary hours for the Bayrisches Rotes Kreuz Wasserwacht in Holzkirchen. This will be a bit more complex because we need to include person names in the event description to get accurate statistics. The hours will also come from a couple sources, not only one calendar. This is because there is a variety of different events, trainings, classes, and rescue missions, and some processes (documentation, protocols) are complex.

For my Yesod web apps, I use yesod devel for running a test server and automatically recompiling the project. And I use ghci to do basically the same thing faster.2 I set up the following Bash aliases:

alias yesod-devel='stack exec -- yesod devel 3>&1 1>&2 2>&3 3>&- | tee >(rewrite-after-timeout 4 .ghc-errors.txt)'
alias ghci-devel='stack ghci                 3>&1 1>&2 2>&3 3>&- | tee >(rewrite-after-timeout 4 .ghc-errors.txt)'

With the weired 3>&1 ... I pipe stderr to tee without loosing stdout.3 In the tee command I use Bash process substitution which allows me to specify a command instead of a file. The command is my program which appends to or rewrites the error file.

In Vim, I :set makeprg=cat\ .ghc-errors.txt to import the error file for quickfix. When I execute :make, the errors are loaded in the quickfix list and I can start working on it.

I'm just getting started with the combination of Haskell, Yesod, GHCi, and Vim. I still have some problems with the Vim quickfix and the error format; e.g. multi-line error messages are not displayed properly in Vim, and GHCi-specific errors confuse Vim. But it's already better than locating the compiler erros by hand in Vim :)

Onboard is not in Raspbian's package index (though it's in Debian's package index*). sudo apt-get install onboard says "package not found" and apt-cache search onboard returns nothing useful.

So this is how to install it manually.

Copy the download link of the archive: https://launchpad.net/onboard/

wget $THE_ARCHIVE_LINK
tar xf onboard...
cd onboard...

Start Onboard's installer:

sudo ./setup.py install

Here comes the first error:

To build Onboard you need https://launchpad.net/python-distutils-extra

Since head setup.py shows that the setup tool uses Python 3, I install the package python3-distutils-extra with sudo apt-get install.

On the next try, the setup tells me:

setup.py: get_pkg_version('dconf'): pkg-config returned exit code 1

So I install the dconf package with sudo apt-get install.

Now the setup still gives the same error. The reason is that the setup uses FreeDesktop's pkg-config to test for the dconf package. However dconf (at least at Raspian) does not support FreeDesktops pkg-config specification1. Therefore pkg-config cannot find the package.

So we have to change setup's source: In setup.py search for get_pkg_version.*dconf, comment this line out and insert major, minor, revision = 1, 0, 0 instead. It's important that the next if condition is not satisfied, otherwise the environment variable DCONF_API_0 is defined which signals the use of an old dconf version which leads to more errors.

Now the setup comes up with a new error:

setup.py: pkg-config returned exit code 1
setup.py: sdist needs libgtk-3-dev, libxtst-dev, libxkbfile-dev, libdconf-dev, libcanberra-dev, libhunspell-dev and libudev-dev

This is easy to fix: Just install the suggested packages using sudo apt-get install.

Now the setup actually compiles and installes successfully!

Now I try to start Onboard with the command onboard. Don't expect it to work in the SSH session; it can only work if you are in a graphical environment (X server is running).

  ...
  File "/usr/local/lib/python3.4/dist-packages/Onboard/utils.py", line 38, in <module>
    from gi.repository import GLib
ImportError: No module named 'gi'

I tried and found out that I have to install two more packages (sudo apt-get install):

python3-gi
python3-gi-cairo

Now when try to start Onboard, the next error is:

ERROR Config: gsettings schema for org.onboard.keyboard is not installed

Apperently, Onboard's setup did not install the dconf config schema. The solution for this is described on reddit:

sudo cp data/org.onboard.gschema.xml /usr/share/glib-2.0/schemas/
sudo glib-compile-schemas /usr/share/glib-2.0/schemas/

Finally it works!

In Onboard's settings dialog, you can customize appearance and many other features.

If you use the keyboard in a kiosk application or something similar, there are some interesting configuration options2:

1. In the general settings, I disabled the tray icon to prevent the enduser from opening settings and help windows.
2. You can "lockdown" some features of Onboard, e.g. the settings dialog. This is done using (graphical) dconf-editor. Open the org.onboard.lockdown directory and disable what you don't want.

As I like reproducible configuration, I use dconf to dump and load Onboard's configuration:

# dump the settings
dconf dump /org/onboard/ > org.onboard.dconf-dump

# reset all settings to their defaults
dconf reset -f /org/onboard/

# load the settings
dconf load /org/onboard/ < org.onboard.dconf-dump

If you want to use the dconf tool from another tty (outside of the X session) or in an SSH session (?), prepend dbus-launch to the commands3.

First, how to start X at all? You can start X with two commands: xinit or startx where startx is like a simplified version of xinit (man startx says it "is a front end to xinit").

There are multiple files related to X start-up:

• ~/.xinitrc – the "run code" for X. It is read when you start X with startx=/=xinit.
• ~/.xprofile – like a ~/.profile or ~/.bash_profile for X. But this file is only read by some display managers (i.e. the visual login screens)! For example, KDE and Gnome might execute the commands in ~/.xprofile but when you start a specific WM directly, the file will not be read at all.
• xmonad's startupHook – this function will be called when xmonad starts. In this monadic function you can =spawn "~/.xmonad/startup.sh"= to execute a script.

exec xmonad should be the last line in ~/.xinitrc to start the WM.

So which file to use for starting the UI software (e.g. the browser, the custom UI, the virtual keyboard, the system tray)?

~/.xprofile does not work as it is not read when we start a WM directly. Unless you have a line like [[ -f ~/.xprofile ]] && source ~/.xprofile in your ~/.xinitrc.

We could start UI software in ~/.xinitrc before we start the WM. Starting graphical programs before the WM is generally not a problem. We could also start UI software in the start-up script called by xmonad's startupHook.

However, ~/.xinitrc seems to be too early for some programs. For example a call to xinput to rotate the touch display seems to have no effect at this point. Therefore, I moved all UI software start-up to my startup.sh script, including xinput, setxkbmap, and xset calls.

The virtual keyboard and the system tray must be ignored by the WM, i.e. the WM should not manage these windows (no tiling, no automatic positioning, etc.).

In xmonad this is done in the manageHook with doIgnore.

But xmonad's behavior seems inconsistent and even non-deterministic at the first glance: Sometimes, ignored windows stay below tiled windows, sometimes they are always on top. Sometimes ignored windows disappear when opening certain dialogs (floating windows), sometimes they show up again when opening or closing other dialogs.

The reason for this seems to be the order in which windows are created. This determines the order in which xmonad's manageHook manages the windows.

Conclusion: Programs with windows that should be ignored must be started before all other programs! If this is guaranteed, my virtual keyboard and system tray work perfectly well and stable.

In practice it can still be a problem. My startup.sh looks something like this:

trayer &
virtual-keyboard &
start-gui &

If, for example, trayer blocks for some reason a few microseconds too long during an I/O operation, the Linux kernel might give CPU time to start-gui which creates the main UI window before other, ignored windows. This would then result in the non-deterministic behavior described above.

Required was a virtual keyboard that can be toggled and that does not overlap with other windows. I already chose Onboard because it excels all other open source virtual keyboards I found in configuration options and design.

I'm not sure if ignoring the keyboard window is the only way to prevent it from getting the focus. Anyway, I have to ignore it and/or make it a dock (xmonad calls it struts). For example the status bar xmobar is a dock window and xmonad leaves this area free. The good thing is that I can toggle struts, i.e. hide or show docks. xmonad can even toggle only the buttom struts which would be the virtual keyboard.

Onboard has a option to make it a dock! It just don't work in version 1.4.1 under Raspbian (although it work on Arch). I already had a lot of trouble installing Onboard on Raspbian, see the earlier blog post.

The bug has already been documented and fixed but the fix seems to be not yet published: https://bugs.launchpad.net/ubuntu/+source/onboard/+bug/1672706

I first tried to set the dock window properties manually which is pretty hacky and did not work properly (see notes below).

Then I found a work-around for using docks. It is a package called XMonad.Layout.Gaps and solved exactly my problem.