30 August 2024

Upcoming Core Changes to My Projects

I’ve been doing a lot of thinking lately. Personal mid-life sort of stuff (I’m 40 years old currently), like where I want to go in life, what I want to do with the rest of my time on this planet, and what’s actually important to me and gives me joy. To put it plainly, it’s time for a change in direction.

The personal stuff is covered in a separate Gemlog post on my Gemini site, so I’ll leave most of that there and instead focus on what this change is going to mean for my programming/Doom/Quake projects. I’ll update this post with the Gemini link once I’m finished writing that.

EDIT: personal post is here: gemini://nanako.mooo.com/gemlog/2024-08-30-a.gmi

High-level Overview - Benben is Queen

Programming has been nothing more than a hobby for me since 2013 when I realized that doing it for work was far, far too stressful and unfulfilling. An endeavour I still took seriously to be sure, especially since it’s given others stuff they can use and enjoy, but a hobby nonetheless. Moving it to hobby status really helped my anxiety, depression, and stress, mostly because A) I’m essentially my own boss, B) I work almost entirely alone, and C) I can set my own schedule.

I’m extremely proud of Benben. Seriously. It’s exactly what I want out of an audio player, it’s fun to maintain, and it’s just fun to use. It and its related libraries are essentially my primary project, with my others (Aya, RemiMonitor, a few minor tools, and two Matrix bots) being nothing but secondary stuff I sometimes tinker with when I need a break. Nothing here is going to change much, except that I’m going to be emphasizing that Benben is my Big Project™. It’s where 90% of my focus will be. It’s my baby, and I’m in it for the long haul. The other stuff are all things I’ll continue to maintain as long as I can (and am interested), at least as personal tools that I happen to upload in case someone finds them interesting, but I don’t plan to add anything new. Benben is my child. It’s my focus, at least as far as programming is concerned.

That’s the big birds-eye view: focus on Benben and not get involved in any new projects. Just stick with the thing that gives me joy and polish it.

This leads into some more specific changes…

Library Changes

As I’ve mentioned before, Benben v0.6.0 is now in development. Things are going fine on this front, with a lot of the smaller bullet points in the milestone having already been taken care of. The next items I had planned to do two things this week were to integrate WavPack support, and to add YM2413 support into YunoSynth to solve this ticket.

But, after a lot of thought, I’ve realized I need to actually take a step back and do some deeper core cleanups first.

Benben has a lot of dependencies, and all of them except for the zstd.cr bindings are written/maintained by me. Initially it made sense to keep them separate, but since I’ll be refocusing exactly what my hobby will be about, it really is making less and less sense to keep them all separate. It’s a bunch of maintenance, confuses end users who want to make tickets, and it causes headaches with the Shard files (something I’m not the biggest fan of - give me pkg-config and shared libraries any day).

Here’s a list of the libraries I’ve written and what they do, plus what’s directly inside of Benben:

• libremiliacr: My personal standard library that extends Crystal’s stdlib. This is where things like my BZip2 reader/writer live.
• RemiAudio: My core audio library. It has basic audio processing, all my DSP effects, WAV and Au reading/writing, the FLAC codec, the QOA codec, a G.711 codec (aka µLaw/A-Law; not used by Benben), a YM2610 ADPCM-A codec (not used by Benben), my semi-broken old MPEG-1 codec in pure Crystal (not used by Benben), resampling, Vorbis Tag parsing, CUE file handling. Aside from three tables that are stored as C arrays for compilation time reasons, it’s all pure Crystal.
• RemiConf: My configuration file and XDG handling library.
• RemiXspf: XSPF and JSPF handling.
• RemiSound: An audio backend abstraction library. This contains libao bindings, PulseAudio bindings, and an abstraction layer for them all.
• RemiPortAudio: PortAudio bindings. RemiSound depends on this to provide PortAudio support in a transparent way.
• RemiSlang: My S-Lang bindings (for the TUI stuff).
• RemiXmp: My libxmp bindings, for module file support.
• RemiMpg123: My libmpg123 bindings, which Benben uses instead of my native Crystal codec because libmpg123 is just so, so much more complete.
• RemiWavPack: My WavPack bindings. This is new for Benben v0.6.0.
• YunoSynth: My VGM playback and various emulation cores. Entirely native Crystal. This is why Benben originally existed.
• Haematite: My MIDI playback and SoundFont synthesis. It’s actually a port of MeltySynth from C# to pure native Crystal.
• RemiHjson: My HJSON parsing and serialization library. The serialization part is broken. This technically isn’t used by Benben anymore, but it was in v0.1.0, and is still technically a dependency that gets pulled because of RemiConf. I fully plan to fix the serialization code.
• Benben: OK not actually a library, but the player itself. It has minimal bindings for libopus and libvorbis inside, as well as a custom Ogg demuxer.

Yeah, it’s a lot, and it’s messy. Like why are my codecs spread across different modules? Why are my audio backend bindings split between two libraries? Why is RemiHjson still even included? These are all questions I’ve been asking myself. I’m also looking at the maintenance cost in terms of hours and sanity of having all these different components, and I’m not too happy.

So, to remedy this (and to also put more emphasis on “I write code for Benben’s sake and not much else”), I’m proposing a consolidation of most of these libraries:

• libremiliacr: Still my personal standard library, but this will absorb RemiConf and RemiHjson.
• RemiAudio: This becomes my personal standard library for audio. It will have basic audio processing, DSP effects, WAV and Au read/write, resampling, Vorbis tag parsing, and CUE file handling still, but will now absorb RemiXspf; RemiSound, and RemiPulseAudio; and my Ogg demuxer. It will also no longer contain any codecs.
• RACodecs: A new library that will be closely tied to RemiAudio and will be exclusively for codecs. This will contain my FLAC codec, my QOA codec, my G.711 codec, my NeoGeo ADPCM codec, libmpg123 bindings, libvorbis bindings, libopus bindings, and libwavpack bindings. Essentially all non-sequenced digital audio formats.
• RemiSlang: This one stays the same. I may add a high-level TUI abstraction to it eventually, though.
• YunoSynth: The VGM playback and various emulation cores stay here and remain 100% native Crystal. But this will also absorb Haematite for MIDI playback and SoundFont synthesis, and RemiXmp for libxmp bindings. Essentially YunoSynth becomes a library focused on sequenced formats and “retro” formats. I’ll be placing my libsidplayfp bindings here.

This takes thirteen libraries down to five. I may also make RACodecs part of RemiAudio (haven’t fully decided yet), which would mean thirteen down to four.

The only real downside is that consolidating these will mean implementing new APIs to smooth over and abstract some differences, so it will take a bit of time. It will also probably disrupt other users of these libraries, though with how Crystal does a sort of tree-shaking sort of thing when compiling, it shouldn’t impact them too much. But I feel the benefits far outweigh the negatives.

My Other Projects

As I said, most of my focus going forward will be on Benben. Most of my other projects will remain, but will be deemphasized. Some will also just be abandoned.

Here’s what’s staying:

• RemiMonitor: It’s really just a personal tool at this point, and that’s where I want to keep it. But obviously I’ll keep sharing the source.
• RemiQoa: Did you know I had this? It’s a command line QOA encoder/decoder tool. Sources are here. Feature complete.
• VgmConv: Just a tool to convert VGM files to various compressed formats. I’ll probably move this into the Benben tree. I don’t have sources up for it, I don’t think…? Or they’re part of YunoSynth? I forget.
• Aya: My Gemini server. 90% feature complete, I just want to add some virtual hosting stuff eventually.
• RemiGemini: My Gemini library. Feature complete, just maintenance updates now. Needed for Aya.
• Sakuya: The bot I use in my Matrix space. She’ll get the occasional new feature, but she’s so stable that I almost never have to touch her. OK that sounded bad…
• SlackBot: Another Matrix bot I maintain for a Slackware Matrix room. Also stable as heck, and feature complete, so it’s just maintenance now.
• CL-RemiMatrix: My Matrix library for Common Lisp. SlackBot needs it, so I’ll keep maintaining it.
• RemiMatrix: My Matrix library for Crystal. Sakuya needs it, so I’ll keep maintaining it.
• mdcat: Like cat, but for Markdown. Feature complete, I’ll just do maintenance now.
• CL-SDM: My personal standard library for Common Lisp. It’s used by a LOT of things I’ve written, so I’ll keep maintaining it.
• CL-MMapped-File: mmap() with a streaming interface for Common Lisp. Feature complete, maintenance only.
• RemiCatbox: I never actually posted this anywhere, but this is a command line uploader I wrote a long time ago for catbox.moe. I’ve found it to be quite useful.

And what’s going away:

• CL-MeltySynth: I’ll keep the code up, but I don’t plan to do anything else here. This will be delisted from my website.
• CL-RemiAudio: Same as CL-MeltySynth. Was never listed on my site to begin with.
• CL-Remi-XSPF: Same as CL-MeltySynth.
• CL-HJSON: Same as CL-MeltySynth.
• CL-Remi-RSS: Same as CL-MeltySynth.
• CL-Remidict: Same as CL-MeltySynth.
• SBDumper: Same as CL-MeltySynth. This was just a tool I wrote to extract some music from a game anyway.
• RemiGemFeeds: Same as CL-MeltySynth. Was never listed on my site to begin with.
• RemiKeyBot: Same as CL-MeltySynth. Was never listed on my site to begin with.
• Dwaddle and CL-Dwaddle: Same as CL-MeltySynth.
• CrDoom: Same as CL-MeltySynth.

Speaking of Doom…

Speaking of Doom, I think my mapping days for it (and Quake) are mostly over. The way the community has shifted since Doom 2016 came out, especially over the past 4-5 years, has kinda taken the fun out of it for me. Other still enjoy it, and it still brings newcomers, which is exciting to see and makes me happy. But the community and I are just in two different places. So, I’ll probably not map much any more, or if I do, it will be entirely one-off stuff, and probably only for Doomsday Engine (or maybe for the original ROTT someday). I’m extremely proud with everything I’ve accomplished in Doom), and I’ll still play the game, but it’s time for me to move onto greener pastures ^_^

I do have an unreleased Heretic map I need to still post, though…