blog

1 Blog posts

I'm Jakob and this is my blog. Please drop me an e-mail if you have questions on any of the topics.

This is just a quick HTML export of my blog.org Org mode file which I'm currently managing in Vim.

1.1 Using the Neo keyboard layout on a physical US keyboard [2017-10-17 Tue]

Neo is a keyboard layout which is optimized for German but also great for English, programming, and other languages. As I bought my Laptop in the US, I have a physical US keyboard which is not perfect for using the Neo layout: The enter key on an US keyboard is wide and low. The mod3 key (labeled with |\) is above the enter key and it is hard to reach.

This article shows how the US keyboard layout is actually not good for programming. Using Neo is already a huge optimization, but the mod3 key on a physical US keyboard sucks.

Working on Linux, I decided to swap enter and mod3. The advantages are:

• The two mod3 keys (left and right) are symmetrical now.
• The mod3 key is easier to reach. (And I'm pretty sure I use mod3 more often than enter.)

The rest of this article describes how I managed to swapped these two keys.

1.1.1 How it worked for me

I use the ~/.Xmodmap file to redefine some keys. I don't know which Linux desktops source this file automatically. Using xmonad, I had to source the file during startup with this command: sleep 3 && xmodmap ~/.Xmodmap &

! This is a comment line
! This mapping swaps Neo's right Mod3 and the Return key on my physical US keyboard.
! I recommend this settings for Neo users who have to work on physical US keyboards.

! :r !xmodmap -pke | grep Level3_Shift
keycode  51 = Return Return Return

! :r !xmodmap -pke | grep Return
keycode  36 = ISO_Level3_Shift ISO_Level3_Shift ISO_Level3_Shift

! You can activate the mapping from VIM using one of these command
! :!xmodmap %
! :w !xmodmap -
! Or from the command line with
! xmodmap THIS_FILENAME


In this configuration file there are comments suggesting Vim commands to generate the configuration because the key codes and identifiers might differ from system to system.

References:

1.2 Writing a thesis with LaTeX, knitr, TikZ, R, GNU Make, …

1.2.1 Make-options for the thesis [2017-10-17 Tue]

The make program can take options in this format:

make USE_TIKZ=1 BOOK_PRINT=1


Make will automatically define and export these options as environment variables, available for all recepies. For my thesis I use these two options to enable TikZ on demand (because the compilation with TikZ is much slower) and to create a book print with roman and arabic page numbering (as opposed to a PDF with only arabic page numbering which I prefer).

To make this working, I have to manipulate my main thesis.Rtex file. BTW, the file is called .Rtex, not .tex, because the knitr preprocessor first evaluates the embedded R code and generates the .tex file.

At the beginning of my thesis.Rtex I have the following chunks of R code:

... % latex preamble

<<include=FALSE>>=
source('helpers.R')
@

... % latex code

<<echo=FALSE, results='asis'>>=
if (ifdef('BOOK_PRINT')) {
cat('\\pagenumbering{roman}')
}
@

... % latex code

<<include=FALSE>>=
if (ifdef('USE_TIKZ')) {
# tikz takes a very long time!
# Use it for the final version:
library(tikzDevice)
opts_chunk$set(dev = 'tikz') } ... % latex code  The first chunk loads an R file which defines a function ifdef (similar to the ifdef macro in C/C++). The second chunk changes the page numbering to roman (for table of contents and list of figures, tables, listings, …). Later there is a chunk to reset the page numbering to arabic, starting at chapter 1. The third chunk enables TikZ as output device for R plots. This is my helpers.R file: isSet = function(val) { if (is.character(val)) return(val != '') return(as.logical(val)) } ifdef = function(name) { var = Sys.getenv(name) isSet(var) }  1.2.2 Printing the thesis [2016-10-28 Fri] So yesterday I finished my thesis! It will be printed in a few hours. A few pages will be printed in color – so I noted their real page numbers (i.e. BOOK_PRINT=0, see my last post). Doing this, I stumbled upon this problem: Cross references and links are highlighted in a blue color. For black/white print this is not a problem; links will be printed black (or grey). But links on colored pages will be printed in blue! This inconsistency looks horrible for me, so I added another knitr snippet in my thesis.Rtex file: <<echo=FALSE, results='asis'>>= if (ifdef('BOOK_PRINT')) { cat('\\hypersetup{colorlinks=false}') } else { cat('\\hypersetup{colorlinks=true}') } @  It's important to note, that the \hypersetup command can only set this option if it has not been passed to the hyperref package yet. \usepackage[ % colorlinks=true, % must not be set here! linkcolor=blue, % ... pdfpagelabels ]{hyperref}  Finally I created the thesis.pdf with the command make BOOK_PRINT=1 and got a PDF file with non-highlighted links, roman page numbering for the first pages and arabic page numbering starting at the first chapter. 1.2.3 Finally, the result [2016-11-07 Mon] In this post, I describe some steps of the fully automated compilation process of my master thesis. I'm referring to the Makefile. The thesis can be downloaded here (PDF) and this is the GitHub repository: https://github.com/schoettl/master-thesis • latexmk -bibtex -lualatex thesis.tex • latexmk automatically processes the TeX file the correct number of times to enable the cross-references to the bibliography and other links to labels. • -bibtex, to use the literature database of my research group and my own one (.bib text files). These text file are not encoded in UTF-8; I think I should have better used bibtex8. With bibtex, I always have to write e.g. German umlauts like this: M\"uller. I wrote a bash script to translate fields like author names, abstract, etc. to the LaTeX ANSI encoding (utf82latex.sh). • -luatex, to really enable use of UTF-8 in LaTeX source. For example, in math mode I can write $α \cdot β$ instead of $\alpha \cdot \beta$ which is much nicer to read and write (using the Neo keyboard layout). • ~R -q -e 'library(knitr); knit("thesis.Rtex", out = "thesis.tex")'~ – I use R in -q=uiet mode and pass an =-e=xpression to convert =thesis.Rtex to thesis.tex using knitr. The .Rtex format allows the use of R code in LaTeX, to generate numbers, tables, plots, and program output. • I use command line utilities (GNU core utils) like join, awk, cp, grep, sed, etc. to prepare some sources for inclusion in the LaTeX document. • ctags to generate a tags file to facilitate navigation in LaTeX sources in Vim. • There are scripts in tools/ • to extract methods or functions from Java and R files, • to remove the indentation from source code, • to skip lines of source and replace them with "…", • and a tool to convert UMLet XML files to vector graphics (using umlet itself). • inkscape, to convert SVG files to PDF for inclusion in the document. • emacs' command line API and Org mode, to convert another .org source to PDF to include it in the document. • javap, to extract the class definition of a Java class from its binary file. 1.3 Vim usage of the dot operator [2016-10-28 Fri]vim Recently, I thought, I discovered a great new way to use the dot operator in Vim. Suppose you want to make a number of lines to be LaTeX sections. I would place the cursor at the first line, and then type: • cc, to cut the line and switch to insert mode, and • \section{<C-r>"<BS>}<Esc>, to surround the line with the section command. For other lines, I only have to position the cursor on them and press the dot (.). But it does not work :( :help . does not tell much about the dot operator. Apparently, the dot operator does not actually repeat the last atomic edit but rather reconstruct the effect in a naive way. Consolation: Reading some docs, I just stumbled on the related :help redo-register. This feature of the dot operator enables you to paste words from the "delete history". I guess I will use this feature in future. Here is a use case: Sometimes you delete some words or phrases and then you want to paste one of the phrases in a different place. p pastes the last deleted text – oh, that was the wrong one, undo it with u. ​"2p pastes the text deleted before. Oh, it's still the wrong one! Now you can press u., (repeatedly) and Vim pastes text from the history of deleted text. P.S. Regarding the original problem (converting lines to LaTeX headers) there are other solutions, e.g. :s/$$.*$$/\\section{\1}/ and then & to repeat this command on other lines. 1.4 Using xmonad in a restricted touch user interface [2016-11-25 Fri]xmonad linux At work we develop intensive care units for animal hospitals. Inside, there is an embedded computer which proivides a graphical user interface (amongst other things). The user can interact using the attached touch display. Requirements for the GUI are that it is stable, secure, and hopefully not hackable. It must always display an web app running on localhost. It also must provide additional options such as configuring wifi and activating remote control. Some specs: • OS: Debian GNU/Linux • Display server: X11 • Window manager: xmonad • Desktop environment: xmonad, xmobar, stalonetray, nm-applet, controlpanel, xvkbd • User programs: chromium in app mode with touch mode enabled The operating system and the display server are given. I chose xmonad as the window manager because it is highly configurable and an empty configuration is the perfect starting point for a very restricted and secure desktop environment. These are the steps at system boot: 1. Auto-login the restricted user for the GUI 2. Auto-start X: startx -- -nocursor in ~/.bash_profile 3. Auto-start xmonad: exec xmonad in the ~/.xinitrc 4. Start other programs, including the system tray and chromium in app-mode. This is done in a startup.sh script which is spawn=ed from =~/.xmonad/xmonad.hs in the startupHook. 1.5 iCalendar (*.ics) to text on the Linux command line [2017-01-05 Thu]linux haskell text I googled for a command line tool to convert files from iCalendar format to plain text to analyze a calendar with tools like grep, sed, awk, cut, … I found nothing! At least not on the first result pages. The closest thing was an online tool to convert iCalendar to CSV, but it is not usable for the command line and also you would have to pass your calendar to an unknown webserver. Luckily, Haskell has a great library to parse the iCalendar format. So I developed a tool following the UNIX philosophy: it's a filter program translating ics input to txt output. The output needs to be further processed with command line tools, or for example a statistical programming language like R. Example usage: # Display all meetings in December 2016 ical2text < calendar.ics | grep ^2016-12 | grep -i meeting | sort  More information and code can be found here: https://github.com/schoettl/ical2text For me it is really helpful in many use-cases. 1.5.1 Work hours using a shared calendar We use a shared calendar to document work hours in our forest. In this calendar, an event title only consists of the first letters of the first name. For example, the event's title is "js" when Jakob and Simon worked at this time. $ curl -s "$CALENDAR_URL" | ical2text | work-hours-statistics Arbeitsstunden insgesamt: 44.5 B: 7.00 h (16 %) H: 10.00 h (22 %) J: 8.00 h (18 %) K: 8.00 h (18 %) S: 9.50 h (21 %) Y: 2.00 h (4 %)  1.5.2 Vacation days at work We use a shared calendar at work. In this calendar, we add events like "Urlaub Jakob" when I take days off. To get an overview of who took how many vacation days, we also use ical2text. There is a difficulty with multi-day events as well as weekends and holidays but some simple scripts overcome this problems: ical2text < cal.ics | grep ^2016 \ | grep Urlaub | grep Jakob \ | expand-multiday-events \ | remove-events-on-holidays german-holidays.ics \ | remove-events-on-weekends \ | wc -l  • The program expand-multiday-events converts a line with a multi-day event to multiple lines of single full-day events. • The program remove-events-on-holidays removes lines with events that take place on a holiday. • The program remove-events-on-weekends does the same for events on weekends. The final outcome is the number of Jakob's vacation days in 2016. Note: There are holiday calendars on the web for download, e.g. http://www.schulferien.org/deutschland/ical/ 1.5.3 Acknowledging extraordinary work hours We also use these programs at our start-up to acknowledge extraordinary work hours on holidays or weekends. 1.5.4 Yearly voluntary hours statistics at the Wasserwacht We plan to use the program for statistics about voluntary hours for the Bayrisches Rotes Kreuz Wasserwacht in Holzkirchen. This will be a bit more complex because we need to include person names in the event description to get accurate statistics. The hours will also come from a couple sources, not only one calendar. This is because there is a variety of different events, trainings, classes, and rescue missions, and some processes (documentation, protocols) are complex. 1.6 Managing a file to only contain the latest output of a program [2017-01-23 Mon]vim haskell text It's a bit complicated to describe what this is and why it can be useful. For example, you might start a command so that the GCC compiler automatically recompiles your source whenever you save the source file.1 Suppose you want to view the error output or use it in Vim for quickfix. You would need the output of a single compilation run, not the accumulated error output. This program fixes exactly that issue. Consider this example: # in one terminal: cat | rewrite-after-timeout 4 /tmp/test.txt # in a second terminal: watch -n1 /tmp/test.txt  As long as you type text into cat, your input accumulates in the /tmp/test.txt output file. If you pause for 4 or more seconds and then start typing again, the output file is cleared before your new input is appended. Now you can replace cat with the error output of a compiler to collect error messages from only one compilation run. The code can be found here: https://gist.github.com/schoettl/312467a7fb6ad27c0ab74cbebbc30aac 1.6.1 Use case: save error output of yesod devel or ghci to a file For my Yesod web apps, I use yesod devel for running a test server and automatically recompiling the project. And I use ghci to do basically the same thing faster.2 I set up the following Bash aliases: alias yesod-devel='stack exec -- yesod devel 3>&1 1>&2 2>&3 3>&- | tee >(rewrite-after-timeout 4 .ghc-errors.txt)' alias ghci-devel='stack ghci 3>&1 1>&2 2>&3 3>&- | tee >(rewrite-after-timeout 4 .ghc-errors.txt)'  With the weired 3>&1 ... I pipe stderr to tee without loosing stdout.3 In the tee command I use Bash process substitution which allows me to specify a command instead of a file. The command is my program which appends to or rewrites the error file. In Vim, I :set makeprg=cat\ .ghc-errors.txt to import the error file for quickfix. When I execute :make, the errors are loaded in the quickfix list and I can start working on it. I'm just getting started with the combination of Haskell, Yesod, GHCi, and Vim. I still have some problems with the Vim quickfix and the error format; e.g. multi-line error messages are not displayed properly in Vim, and GHCi-specific errors confuse Vim. But it's already better than locating the compiler erros by hand in Vim :) 1.7 Install onboard virtual keyboard on a Raspberry Pi [2017-04-28 Fri]raspberrypi I want the virtual keyboard Onboard on a Raspberry Pi 3. This is my system: $ uname -a
Linux intensobox4 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux
$lsb_release -a No LSB modules are available. Distributor ID: Raspbian Description: Raspbian GNU/Linux 8.0 (jessie) Release: 8.0 Codename: jessie  1.7.1 Installation Onboard is not in Raspbian's package index (though it's in Debian's package index*). sudo apt-get install onboard says "package not found" and apt-cache search onboard returns nothing useful. So this is how to install it manually. Copy the download link of the archive: https://launchpad.net/onboard/ wget$THE_ARCHIVE_LINK
tar xf onboard...
cd onboard...


Start Onboard's installer:

sudo ./setup.py install


Here comes the first error:

To build Onboard you need https://launchpad.net/python-distutils-extra


Since head setup.py shows that the setup tool uses Python 3, I install the package python3-distutils-extra.

Now the setup tells me:

setup.py: get_pkg_version('dconf'): pkg-config returned exit code 1


So I install the dconf package.

Now the setup gives still the same error. The reason is that the setup uses FreeDesktop's pkg-config to test for the dconf package. However dconf (at least at Raspian) does not support FreeDesktops pkg-config specification1. Therefore pkg-config cannot find the package.

So we have to change setup's source: In setup.py search for get_pkg_version.*dconf, comment this line out and insert major, minor, revision = 1, 0, 0 instead. It's important that the next if condition is not satisfied, otherwise the environment variable DCONF_API_0 is defined which signals the use of an old dconf version which leads to more errors.

Now the setup comes up with a new error:

setup.py: pkg-config returned exit code 1
setup.py: sdist needs libgtk-3-dev, libxtst-dev, libxkbfile-dev, libdconf-dev, libcanberra-dev, libhunspell-dev and libudev-dev


This is easy to fix: Just install the suggested packages.

Now the setup actually compiles and installes successfully!

1.7.2 Start the virtual keyboard

Now I try to start Onboard with the command onboard. Don't expect it to work in the SSH session; it can only work if you are in a graphical environment (X server is running).

  ...
File "/usr/local/lib/python3.4/dist-packages/Onboard/utils.py", line 38, in <module>
from gi.repository import GLib
ImportError: No module named 'gi'


I tried and found out that I have to install two more packages:

python3-gi
python3-gi-cairo


Now when try to start Onboard, the next error is:

ERROR Config: gsettings schema for org.onboard.keyboard is not installed


Apperently, Onboard's setup did not install the dconf config schema. The solution for this is described on reddit:

sudo cp data/org.onboard.gschema.xml /usr/share/glib-2.0/schemas/
sudo glib-compile-schemas /usr/share/glib-2.0/schemas/


Finally it works!

1.7.3 Tipps for configuration

In Onboard's settings dialog, you can customize appearance and many other features.

If you use the keyboard in a kiosk application or something similar, there are some interesting configuration options2:

1. In the general settings, I disabled the tray icon to prevent the enduser from opening settings and help windows.
2. You can "lockdown" some features of Onboard, e.g. the settings dialog. This is done using (graphical) dconf-editor. Open the org.onboard.lockdown directory and disable what you don't want.

As I like reproducible configuration, I use dconf to dump and load Onboard's configuration:

# dump the settings
dconf dump /org/onboard/ > org.onboard.dconf-dump

# reset all settings to their defaults
dconf reset -f /org/onboard/


If you want to use the dconf tool from another tty (outside of the X session) or in an SSH session (?), prepend dbus-launch to the commands3.