Sunday, December 2, 2018

Using the GOG.com installers for Linux, on NetBSD

GOG.com prefers that you use their GOG Galaxy desktop app to download, install and manage all of your GOG games. But customers always have the option to install the game on their own terms, with a platform-specific installer.

GOG offers these installers for Mac, Windows and/or Linux, depending on which platforms the game is available for. The installers truly are platform-specific:

  • macOS games are distributed in a standard .pkg
  • Windows games are distributed in a setup wizard .exe
  • Linux games are distributed in a goofy shell archive

Of course, none of those are NetBSD. So, if I wanted to even attempt to play a game distributed by GOG.com on NetBSD, which one should I pick? The obvious choice is the Linux installer, since Linux is the most similar to NetBSD, right? Au contraire! In practice, I found that it is easier to download the Windows installer.

Here’s what I mean. For example, I ported the open source version of Aquaria to pkgsrc, but that package is only the game’s engine, not the multimedia data. The multimedia data is still copyrighted. Therefore, you need to get it from somewhere else. GOG is usually a good choice, because they distribute their games without DRM. And as mentioned earlier, picking the Linux installer seemed like a natural choice.

But I quickly discovered that the Linux installer, that goofy shell archive, is not portable.

charlotte@sakuracity:~/games$ sh ./pyre_1_50427_11957_23366.sh
Verifying archive integrity... All good.
Uncompressing Pyre (GOG.com)     0% gzip: unknown compression format
Extraction failed.
 100%  Signal caught, cleaning up

Ah, maybe you just need a different shell…

charlotte@sakuracity:~/games$ bash ./pyre_1_50427_11957_23366.sh 
Verifying archive integrity... All good.
Uncompressing Pyre (GOG.com)     0% gzip: unknown compression format
Extraction failed.
 100%  Signal caught, cleaning up

Lol no? OK fine whatever.

That’s when I discovered that GOG’s Windows installers are actually easier to deal with on NetBSD, with the help of the archivers/innoextract package. It really does Just Work. So, I’ve been using innoextract on GOG’s .exe files almost exclusively in order to obtain multimedia data for all sorts of games that have open source reimplementations of their game engines: Jazz Jackrabbit, Jade Empire, Grim Fandango, and more.

That was my only way forward, until I remembered that NetBSD has a Linux compatibility layer!

You just have to install one of the Linux compat packages. I’m on a 64-bit machine and I want the newest stuff, so I installed the emulators/suse_base package from pkgsrc.

When that package is done installing, it writes you a big fat note:

===========================================================================
$NetBSD: MESSAGE.NetBSD,v 1.1 2013/12/05 11:42:14 obache Exp $

Do not forget to include COMPAT_LINUX in your kernel configuration file.
Linux binaries require this option in order to work.

Most Linux binaries also require procfs to be mounted with -o linux.
This can be automated by adding the following to your /etc/fstab:

procfs /emul/linux/proc procfs ro,linux

Some Linux binaries also require tmpfs to be mounted on /dev/shm.
This can be automated by adding the following to your /etc/fstab:

tmpfs /emul/linux/dev/shm tmpfs rw,-m1777

===========================================================================

I’m not positive if those extra entries in fstab(5) are 100% necessary for the task at hand (run GOG.com’s game installers for Linux), but it certainly doesn’t hurt. I have them enabled.

Alright, it’s go time!

charlotte@sakuracity:~/games$ bash pyre_1_50427_11957_23366.sh 
Verifying archive integrity... All good.
Uncompressing Pyre (GOG.com)     0% gzip: unknown compression format
Extraction failed.
 100%  Signal caught, cleaning up

Oh, right, derp, I should put the Linux binaries in the front of my PATH.

charlotte@sakuracity:~/games$ LINUX=/usr/pkg/emul/linux

charlotte@sakuracity:~/games$ PATH=${LINUX}/bin:${LINUX}/sbin:${LINUX}/usr/bin:${LINUX}/usr/sbin:${PATH}
bash ./pyre_1_50427_11957_23366.sh 
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Verifying archive integrity... All good.
Uncompressing Pyre (GOG.com)  100%  
/bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Collecting info for this system...
Operating system: linux
CPU Arch: x86_64
trying mojosetup in bin/linux/x86_64
USING en_US
Error opening terminal: screen.
Error: Couldn't run mojosetup

It can’t run “mojosetup” (whatever that is) because it doesn’t like my $TERM? Also, what’s that deal about the locale, doesn’t know what en_US.UTF-8 is?

At this point I decided to use the simplest setup I could envision, and it worked! That means a locale of “C” and a TERM of “vt100”.

charlotte@sakuracity:~/games$ \
  PATH=${LINUX}/bin:${LINUX}/sbin:${LINUX}/usr/bin:${LINUX}/usr/sbin:${PATH} \
  TERM=vt100 \
  LC_ALL=C \
  bash ./pyre_1_50427_11957_23366.sh 

It’s this insane little setup wizard which has interesting curses support (??!!)

┌───────────────────────────────── Pyre ─────────────────────────────────┐
│                                                                        │
│ Pyre                                                                   │
│ Welcome to GOG.com installer!                                          │
│                                                                        │
│ For technical support for this game, please visit our Support page at: │
│ http://www.gog.com/support/pyre                                        │
│────────────────────────────────────────────────────────────────────────│
│                          < Cancel > < Next >                           │
└────────────────────────────────────────────────────────────────────────┘

You use the arrow keys to highlight the option you want and hit SPACE and/or ENTER to select it.

Pyre                                                                                                                                    
 ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
┌──────────────────────────────────────────────────────────────── EULA ────────────────────────────────────────────────────────────────┐
│                                                                                                                                      │
│ GOG.com End-User License Agreement                                                                                                   │
│ ========================================                                                                                             │
│                                                                                                                                      │
│ Your use of this product/service is subject to the GOG User Agreement. Copy and paste this link to your browser to check it out:     │
│ http://www.gog.com/support/policies/gog_user_agreement                                                                               │
│                                                                                                                                      │
│                                                                                                                                      │
│ MojoSetup License Terms                                                                                                              │
│ ========================================                                                                                             │
│                                                                                                                                      │
│ Copyright (c) 2006-2010 Ryan C. Gordon and others.                                                                                   │
│                                                                                                                                      │
│ This software is provided 'as-is', without any express or implied warranty.                                                          │
│ In no event will the authors be held liable for any damages arising from                                                             │
│ the use of this software.                                                                                                            │
│                                                                                                                                      │
│ Permission is granted to anyone to use this software for any purpose,                                                                │
│ including commercial applications, and to alter it and redistribute it                                                               │
│ freely, subject to the following restrictions:                                                                                       │
│                                                                                                                                      │
│ 1. The origin of this software must not be misrepresented; you must not                                                              │
│ claim that you wrote the original software. If you use this software in a                                                            │
│ product, an acknowledgment in the product documentation would be                                                                     │
│ appreciated but is not required.                                                                                                     │
│                                                                                                                                      │
│ 2. Altered source versions must be plainly marked as such, and must not be                                                           │
│ misrepresented as being the original software.                                                                                       │
│──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────(  3%)──│
│                                                     < Cancel > < Back > < Next >                                                     │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Wow, incredible. I didn’t know you could do this in a shell archive?

The installer asks you if you’d like to install the game data to somewhere other than the default ~/GOG Games. I just left it at the default.

At a certain point, you’re invited to create a desktop shortcut and a “menu item” (most be some Ubuntu thing) and, well, I don’t know what happens with those options enabled on NetBSD, but I turned them off for safety.

┌──────────────── Options ─────────────────┐
│ [ ] Create a desktop shortcut            │
│ [ ] Create a menu item                   │
│──────────────────────────────────────────│
│ < Cancel > < Toggle > < Back > < Next >  │
└──────────────────────────────────────────┘

And then the fun part, when the game is actually installed!

┌ Installing ─┐
│ Pyre        │
│     12%     │
│  Music...   │
│             │
│─────────────│
│ < Cancel >  │
└─────────────┘

And sure enough, the game is installed in the place it said it would!

charlotte@sakuracity:~/GOG Games/Pyre$ ls -F
docs/                   gameinfo                support/
game/                   start.sh*               uninstall-Pyre.sh*

I suspect that the uninstall script there would require Linux compatibility, too.

Now, actually PLAYING the games on NetBSD is a separate matter entirely. The game I’ve got here, though, my current obsession Pyre, is built with MonoGame and therefore could theoretically work on NetBSD, too, with the help of a library called FNA and a script for OpenBSD called fnaify. I do hope to create a pkgsrc package for FNA and port the fnaify script to NetBSD at some point.

UPDATE!

So, lol, haha, it turns out that if you install the meta-pkgs/suse131 package, then we get a hold of the Linux version of glib2, which enables the GOG installer to spit out a GRAPHICAL SETUP WIZARD ON NETBSD

Sunday, November 11, 2018

Custom console fonts with wscons(4)

A very long time ago, I made a VGA or “console” font out of Leah Neukirchen’s font called “sq”. I even wrote a simple script which helped me do the conversion. The whole point of that exercise was to turn the “sq” font into a font for the console on my old NetBSD machine. I was successful all those years ago. But I couldn’t remember how I did it. So let me quickly show you what I had to do to get the font working on my current NetBSD box.

You could run several commands on the command line in order to achieve this effect, but it’s way simpler to simply edit wscons.conf(5) and restart the wscons system service.

First, towards the top of wscons.conf, we declare the existence of “sq” and the boldfaced version, “sqb”. It also ensures that the fonts will be loaded.

font  sq   - - iso /home/charlotte/fonts/sq.816
font  sqb  - - iso /home/charlotte/fonts/sqb.816

Then, towards the bottom of wscons.conf, we set the “font” variable for each tty to be “sq”.

setvar  ttyE0 font    sq
setvar  ttyE1 font    sq
setvar  ttyE2 font    sq
setvar  ttyE3 font    sq  

Then, we simply restart the wscons system service, and we’re all done!

charlotte@sakuracity:~$ sudo service wscons restart
wsconscfg: screen 1 is already configured
wsconscfg: screen 2 is already configured
wsconscfg: screen 3 is already configured
wsconscfg: screen 4 is already configured
Keyboard encoding -> us
ttyE0: font -> "sq"
ttyE1: font -> "sq"
ttyE2: font -> "sq"
ttyE3: font -> "sq"

Saturday, October 27, 2018

Attaching a Bluetooth keyboard to a NetBSD machine

I am FINALLY able to use my Apple Bluetooth keyboard with NetBSD! I am writing this blog post right now using it. The NetBSD Guide helped me a LOT with this, but I believe it requires a little bit of updating… so in the meantime, let me show you how I did it.

Overall strategy

  • Enable the Bluetooth system service.
  • Figure out what your machine’s Bluetooth device node is called.
  • Put your keyboard into discover mode.
  • Figure out what your keyboard’s Bluetooth address is.
  • Register a temporary PIN to be used when pairing the keyboard.
  • Type the PIN in with the keyboard.
  • Attach the device.

In detail:

Enable the Bluetooth system service.

This is simply adding “bluetooth=YES” in your rc.conf and then turning it on with sudo service bluetooth start.

Figure out what your machine’s Bluetooth device node is called.

That’s just a matter of dmesg. We are interested in finding out where a Bluetooth remote device hub (bthub(4)) has set itself up:

$ dmesg | grep bthub
[    15.588860] bthub0 at ubt0 local-bdaddr 48:45:20:52:f6:60

ubt0 is where our Bluetooth remote device hub is located. That’s the part we’re interested in. We’ll make a note of it.

Put your keyboard into discover mode.

In the case of my keyboard from Apple, that means pressing and holding the power button until the status light blinks. When that happens, the keyboard will be discoverable when my laptop’s Bluetooth device tries to discover it (and anything else in the vicinity).

Figure out what your keyboard’s Bluetooth address is.

Apparently, the technical term for “discover all the discoverables” is actually an “inquiry.”

$ btconfig ubt0 inquiry
Device Discovery from device: ubt0 ... 3 responses
  1: bdaddr e4:8b:7f:2e:bb:60
   : name "Apple Keyboard"
   : class [0x002540] Peripheral Keyboard <Limited Discoverable>
   : page scan rep mode 0x01
   : clock offset 31094
   : rssi 0

  2: bdaddr b0:67:2f:00:54:ff
   : name "Audio Device"
   : class [0x080428] HiFi Audio <Capturing>
   : page scan rep mode 0x01
   : clock offset 23232
   : rssi 0

  3: bdaddr 60:6b:bd:c2:89:de
   : name "DTVBluetooth"
   : class [0x08043c] Video Display and Loudspeaker <Capturing>
   : page scan rep mode 0x01
   : clock offset 24943
   : rssi 0

I’m interested in the first device, the keyboard with the BDADDR “e4:8b:7f:2e:bb:60”. At this stage, we can register an alias or “nickname” for this device, so that we don’t have remember the address:

 $ sudo echo e4:8b:7f:2e:bb:60 CharlotteAppleKeyboard >> /etc/bluetooth/hosts

Register a temporary PIN to be used when pairing the keyboard.

The btpin command sets up a number to be typed in. I typed the number on my Bluetooth keyboard and pressed ENTER.

$ btpin -d ubt0 -a CharlotteAppleKeyboard -r -P
PIN: (numbers)
Pairing.. done

Pairing is done. Nice!

Attach the device.

The btdevctl -A command will do that for us.

$ sudo btdevctl -d ubt0 -a CharlotteAppleKeyboard -s HID -A

You can confirm that the device has attached as a keyboard (rather than something else) by inspecting /var/log/messages or dmesg. This is the third keyboard attached to my laptop (my laptop’s own keyboard and a USB keyboard were already attached). So the fact that it’s wskbd2 makes sense:

Oct 27 10:26:53 sakuracity /netbsd: [ 510454.1260370] btkbd0 at bthidev0 reportid 1
Oct 27 10:26:53 sakuracity /netbsd: [ 510454.1260370] wskbd2 at btkbd0 mux 1
Oct 27 10:26:53 sakuracity /netbsd: [ 510454.1260370] wskbd2: connecting to wsdisplay0

At this point you can try typing! That’s what I’m doing right now!

Unfortunately, there are a few shortcomings with this keyboard. For example, The media keys don’t work. Volume up, volume down, you name it. Page up and page down, even. All of these nonworking keys are “function” keys (not F1-F12, but, like, “extended” functionality) and none of them work because every time you press the Fn key, this happens:

[ 512317.5222648] bthidev0: autoconfiguration error: report id 17, len = 1 ignored

Not sure what that’s all about, but I’m willing to dig deeper into it.

Conclusions

So, the instructions here are NOT exactly what is detailed in the NetBSD Guide. I (or someone) should fix that.

Also, I’ve been inspired to write a tool or two which makes this process A LOT simpler. I feel like this should be a task that requires 1 command rather than 3 or 4. I also believe it would be useful to automate maintaining /etc/bluetooth/hosts somehow.

Thursday, October 11, 2018

Porting Keybase to NetBSD

Keybase significantly simplifies the whole keypair/PGP thing and makes what is usually a confusing, difficult experience actually rather pleasant. At its heart is an open-source command line utility that does all of the heavy cryptographic lifting. But it's also hooked up to the network of all other Keybase users, so you don't have to work very hard to maintain big keychains. Pretty cool!

So, this evening, I tried to get it to all work on NetBSD.

The Keybase client code base is, in my opinion, not very well architected... there exist many different Keybase clients (command line apps, desktop apps, mobile apps) and for some reason the code for all of them are seemingly in this single repository, without even using Git submodules. Not sure what that's about.

Anyway, "go build"-ing the command line program (it's written in Go) failed immediately because there's some platform-specific code that just does not seem to recognize that NetBSD exists (but they do for FreeBSD and OpenBSD). Looks like the Keybase developers maintain a Golang wrapper around struct proc, which of course is different from OS to OS. So I literally just copypasted the OpenBSD wrapper, renamed it to "NetBSD", and the build basically succeeded from there! This is of course super janky and untrustworthy, but it seems to Mostly Just Work...

I forked the GitHub repo, you can see the diff on top of keybase 2.7.3 here: bccaaf3096a

Eventually I ended up with a ~/go/bin/keybase which launches just fine. Meaning, I can main() okay. But the moment you try to do anything interesting, it looks super scary:

charlotte@sakuracity:~/go/bin ./keybase login
▶ WARNING Running in devel mode
▶ INFO Forking background server with pid=12932
▶ ERROR unexpected error in Login: API network error: doRetry failed,
attempts: 1, timeout 5s, last err: Get
http://localhost:3000/_/api/1.0/merkle/path.json?last=3784314&load_deleted=1&load_reset_chain=1&poll=10&sig_hints_low=3&uid=38ae1dfa49cd6831ea2fdade5c5d0519:
dial tcp [::1]:3000: connect: connection refused

There's a few things about this error message that stuck out to me:

  • Forking a background server? What?
  • It's trying to connect to localhost? That must be the server that doesn't work ...

Unfortunately, this nonfunctional "background server" sticks around even when a command as simple as 'login' command just failed:

charlotte@sakuracity:~/go/bin ps 12932
  PID TTY STAT    TIME COMMAND
  12932 ?   Ssl  0:00.21 ./keybase --debug --log-file
  /home/charlotte/.cache/keybase.devel/keybase.service.log service --chdir
  /home/charlotte/.config/keybase.devel --auto-forked 

I'm not exactly sure what the intended purpose of the "background server" even is, but fortunately we can kill it and even tell the keybase command to not even spawn one:

charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --standalone
   --standalone                         Use the client without any daemon support.

And then we can fix wanting to connect to localhost by specifying an expected Keybase API server -- how about the one hosted at https://keybase.io?

charlotte@sakuracity:~/go/bin ./keybase help advanced | grep -- --server
   --server, -s                         Specify server API.

Basically, what I'm trying to say is that if you specify both of these options, the keybase command does what I expect on NetBSD:

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io login
▶ WARNING Running in devel mode
Please enter the Keybase passphrase for dressupgeekout (6+ characters): 

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io id dressupgeekout
▶ WARNING Running in devel mode
▶ INFO Identifying dressupgeekout
✔ public key fingerprint: 7873 DA50 A786 9A3F 1662 3A17 20BD 8739 E82C 7F2F
✔ "dressupgeekout" on github:
https://gist.github.com/0471c7918d254425835bf5e1b4bcda00 [cached 2018-10-11
20:55:21 PDT]
✔ "dressupgeekout" on reddit:
https://www.reddit.com/r/KeybaseProofs/comments/9ng5qm/my_keybase_proof_redditdressupgeekout/
[cached 2018-10-11 20:55:21 PDT]

Hey, that's pretty cool!

I verified that signing data works:

charlotte@sakuracity:~/go/bin echo "SOYLENT GREEN IS PEOPLE" > /tmp/secret.txt

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io sign -i /tmp/secret.txt -o /tmp/secret.signed.txt 
▶ WARNING Running in devel mode
charlotte@sakuracity:~/go/bin cat /tmp/secret.signed.txt 
BEGIN KEYBASE SALTPACK SIGNED MESSAGE. kXR7VktZdyH7rvq v5weRa0zkV2Q3rQ yJlaULSlqaeTnR5 enipWZdks4GrGGp MDHBBLInD3IXYih w4EXhfbZNlo99Pk n3P8ntUP9rt4DWl bWnVnS5T52YVtNB QiKoN7LMVa60GG8 ZvOGShKaIKZNG1n kWMcxKWUTPOLK0N JPAPxafF9Y25RBO ZAzpsfkd26FRFUO OW54leXr5YA3MK8 VTMvU4b4. END KEYBASE SALTPACK SIGNED MESSAGE.

And verifying that signature works:

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io verify -i /tmp/secret.signed.txt 
▶ WARNING Running in devel mode
Signed by dressupgeekout (you).
SOYLENT GREEN IS PEOPLE

But unfortunately, encrypting data (and I guess decrypting it, too) is not functional at this time:

charlotte@sakuracity:~/go/bin ./keybase --standalone -s https://keybase.io encrypt -i ~/pictures/random/unicorn.jpg -o /tmp/unicorn.jpg.encrypted dressupgeekout
▶ WARNING Running in devel mode
▶ ERROR Can't run command in standalone mode

But you can't turn off standalone mode because then we're back to where we started:

charlotte@sakuracity:~/go/bin ./keybase -s https://keybase.io encrypt -i ~/pictures/random/unicorn.jpg -o /tmp/unicorn.jpg.encrypted dressupgeekout
▶ WARNING Running in devel mode
▶ INFO Forking background server with pid=4222
▶ ERROR Keybase services aren't running - KBFS client not found.

It is kinda cool that some of the interesting functionality of Keybase does work on NetBSD though!

New blog for technical writings

Hey gang, this is Charlotte. And I decided that Tumblr wasn't exactly the best place to do technical writing after all. Like, talking about open source and BSD and multimedia science and all of that. So, enjoy! And I do promise to update this from time to time!