8 June 2025
June 2025 Benben Update
Ohhhh boy, has there been a lot of activity with Benben in the past six and a half weeks. Like, seriously, there’s been a LOT of work put into Benben and its support libraries since the last update. These (almost) monthly updates also seem to be becoming a tradition, so I may as well continue it with an overview of what all has been happening with Benben’s development. Let’s dive in!
Lisp Port Merged Into Trunk
The biggest news is that the Common Lisp branch has been merged into the trunk. The code is far enough along that users that power users (is that term still used?) who are comfortable compiling it from source can use it as a daily driver. There are still bugs, of course, and still work to complete, but it’s far enough along that I felt it was ready to be merged.
I’ll get back to releasing development AppImages soon, once things quiet down for me at home, and once I have an updated remote-benben program. For now, if you’re feeling brave and adventurous, you can use the Lisp Build Instructions file. This file will eventually become the wiki entry that tells you how to build Benben.
Remaining Formats Implemented
On the format front, the last few missing formats have been ported over and are now working. These are:
- Vorbis (now written entirely in Common Lisp - no C bindings needed!)
- MIDI (also pure Common Lisp)
- C64 SID (via libsidplayfp and a very small C library I wrote)
MIDI support is still a little shaky at the time of writing just because I haven’t tested it too much. But, it should be stable enough for general use.
Improved Relative Volume Adjustment Support (“ReplayGain”)
ReplayGain support is a bit better in Benben v0.7.0, as it’s now supported with XQAF files, and also for WavPack files. Also, Opus files with both ReplayGain tags and the format’s own internal Output Gain adjustment are properly handled.
Rendering Implemented for All Supported Formats
The rendering system has been ported over, and also improved, at least internally. Rendering files still happens in parallel, and all formats are supported.
The internal design has been overhauled so that much more code is shared between the rendering jobs and the normal playback code. In fact, the rendering code has dropped from about 2300 LOC down to 946 at the time of writing. But, also take that with a grain of salt since ~2300 lines of Crystal code isn’t necessarily equivalent to ~2300 lines of Common Lisp code. Still, it’s a dramatic reduction, and makes it a lot easier to add additional formats in the future.
Anyway, just for reference, here are all of the supported input formats in Benben v0.7.0:
- VGM (.vgm, .vgz, .vgb, and .vgzst, all chips that my SatouSynth library supports)
- Module/tracker files (all formats that libxmp supports)
- FLAC
- MPEG-1 (i.e. MP3, MP2, and MP1 files, via libmpg123)
- Ogg Vorbis
- Opus
- General MIDI
- MUS
- QOA
- Extended QOA Format (XQAF)
- WavPack
- RIFF WAVE (.wav files)
- Sun Au
Rendering to Wavpack or QOA Removed
Some of the more recent Crystal versions of Benben had the ability to render to WavPack or QOA as well as the usual WAV and Au. QOA output was originally added simply because I didn’t know of any good, general-purpose QOA encoders out there except for the reference encoder. But now that XQATool is a thing (and it’ll support output to normal QOA very soon), it doesn’t make much sense for Benben to also offer QOA output.
Similarly, WavPack is being removed because it also doesn’t make much sense to keep in Benben. The official encoder has more options, and if you have libwavpack installed, you probably also have the command line WavPack encoder at your disposal as well. You know, Unix philosophy and all that. Truth be told, I was somewhat hesitant to add it to begin with for this very reason, but it seemed like a fun idea at the time, and I did at least learn more about WavPack with it.
So, long story short, Benben v0.7.0 will only offer rendering to WAV and Au. Use XQATool for creating XQAF and QOA files, and the official WavPack encoder for making WavPack files. Playback of these formats is still fully supported.
Wad File Support
Related to formats is a completely new feature for v0.7.0: support for Doom WAD files. Basically you tell Benben to play a WAD file, and it’ll find all supported files (lumps) within it and play them. So you can do things like this:
$ benben doom.wad $ benben scythe.wad $ benben kill.wad
Neat, eh? This should make it a bit easier to listen to the soundtracks to Doom levels that you enjoy.
WAD files do have a few limitations, however:
- Normal PCM (WAV/Au) file in WADs are NOT played since there’s no way to tell what’s a sound effect and what’s an actual music track.
- SID files are also not supported because selecting a subsong from a SID file embedded in a WAD file gets to be over complicated from a UI standpoint in my opinion.
- WavPack files in WAD files are not supported, but since I’ve never actually seen a WavPack file in a WAD, I don’t think this will matter much. Also, I would want to support hybrid WavPack files here, and I’m not sure there’s a community standard for how to store the WVC sidecar file.
- The file data is loaded into memory before playback, so it results in slightly higher memory usage than if played the file directly. But in practice it isn’t much.
So, in the end, these are the file types that are supported in WAD files:
- MIDI (and MUS)
- FLAC
- MPEG-1 (mp3, mp2, mp1…)
- Module/tracker
- Vorbis
- Opus
- QOA
- XQAF
New Output Drivers
There are now three different output “drivers” in the Common Lisp port of Benben:
- libout123, which is also the default. It’s called “out123” by Benben.
- libao. Benben calls this one “ao”.
- ALSA. Benben calls it “alsa”.
The ALSA driver is the most bare bones, but is also the closest “to the hardware”, so to speak. The out123 driver is the default since it’s very fast and very robust. The ao driver is a good alternative.
Also, if you use the special “any” driver, Benben will cycle through the available drivers as-needed. This means it tries out123 first, and if that fails, it tries ao. If that fails, it’ll try alsa. If that fails, well, you’re out of luck and you probably have sound system problems outside of Benben :-P
New Config Options
The config file also got a few new additions since the last update.
Banner animations can now be selected in the config file. This is done in the
ui
section with the banner-anim
key. Just like many of the other UI
options, it can take both a string or an array of strings. When it’s a single
string, then only that one animation is used. If it’s an array of strings, then
a random animation from that array will be chosen each time Benben is started.
Putting an animation more than once in an array increases its chance of being
selected. The possible animations are: slide-in, dissolve,
dissolve-from-nothing, wipe-from-left, and wipe-from-right.
There’s also a new key in the vgm
section called max-file-size
. This key
tells Benben not to load any VGM file that’s larger than max-file-size
bytes
before decompression. This was added because I found myself accidentally
opening compressed files a few times, which were being tested as VGMs. It
should also provide some protection against “zip
bomb” VGMs. The default is 62914560
(60 megabytes), which is far larger than any VGM I have in my collection¹. I
intend to implement the same option for module files before release.
Speaking of modules, the modules
config section now has a post-track-seconds
option. This option only takes effect when the fade-out-songs
option is
false
and a reverb unit is enabled when the song starts. What it does is
append post-track-seconds
seconds of silence to the end of the file, thereby
allowing the reverb tails to finish playing.
Port of Remote-Benben Started
I’ve started porting the remote-benben program (the one that provides remote control of Benben via a Unix Domain Socket) from Crystal to Common Lisp. I had intended to keep it in Crystal, but some annoyances with LLVM and libgc on one of my boxes that impacted all my Crystal programs kinda ticked me off recently.
The port of remote-benben probably won’t be finished for a while since it’s kinda low priority, but it will be done in time for the Benben v0.7.0 release. And while the current code in the trunk builds, but only supports a few commands. But it’s a start! Just use the old AppImage of it for now, it’s fully compatible with the Common Lisp version of Benben.
Code of Conduct Updated
One last thing, the Code of Conduct has been updated. This is important for anyone wishing to contribute to Benben, whether it be through tickets, community interaction, code submissions, or so on. Benben’s Code of Conduct was adapted from by ~keith, and was chosen based on my desire to create a safe and equitable space, and also on my own experience as a neurodivergent individual who is uhh… well, let’s go with “rather eccentric”.
The new addition to the Code of Conduct is point number six, which covers AI-generated code. Basically, it’s not allowed in Benben. No vibe coding, no using an LLM to debug a piece of code, and so on. It’s not allowed within Benben. Now obviously I can’t control code in third-party libraries where I’m not aware that it’s AI-generated. For example, if I write bindings for libfoo to use in Benben, and the libfoo authors decide to add some LLM-generated code in the future, and I’m not aware they did, then I can’t help that. But for the code that goes into Benben itself, it must be written by a human to the best of my knowledge and the submitter’s knowledge.
I fully intend to extend this Code of Conduct to all other projects I maintain in the very near future. This means nearly all of the Lisp libraries Benben uses will also be covered by this policy. They kinda are already, but it’s more of a de facto thing at the moment.
Next Steps
At this point, the porting process is mostly
complete.
There’s just a few VGM chip emulators left to port and it will be feature-parity
with the old Crystal version (plus some new features). This is why the old
lisp-rewrite
branch was merged into trunk
.
Given how far along the port is, I’m now shifting my attention to the milestone goals for v0.7.0. Obviously some of these are already completed (mostly ones that made sense to do during the port, like the Minimal UI), but there are still a number of new features left to add before the new release will be ready. My plan for the coming month is to focus on these features specifically:
- Show total time elapsed on the “Total time” line.
- For looping files, show two time values in the play queue in this format:
(time per loop / total time)
There are also a number of known bugs and issues in Benben’s current trunk code that I want to address:
- CL-RemiAudio’s WAV and Au I/O needs one final rewrite to further improve performance.
- There’s a known
deadlock
when
--vgm-strict-gd3-loading
is used. - MIDI files act strange with
--loop
currently, with them looping one extra time. - The progress bar when rendering isn’t always correct. This is just a visual bug, the results are correct.
- There’s some SIMD code in CL-RemiChips that’s consing way too much. It’s currently commented out, but needs to be addressed so it can be re-enabled.
Lastly, while not strictly related to Benben, there are a few other things that are tangentially related:
- I want to add normal QOA encoding and QOA<->XQAF conversions to XQATool (and do an official release of it).
- CL-Remi-Slang and CL-RemiMarshal are probably ready for their first official releases.
So there you have it. Plans for the coming month, and what has been happening since the end of April.