SFXR/Sound effect generator format direct support

[Blockhead]

Background
I got started trying to make a game for Minetest in just 4096 (4Ki) bytes yesterday. Things progressed pretty well with a texture atlas/sheet graphic file and a bunch of code golfing, but then I hit a stumbling block: sound.

If I open Audacity and save an empty file as Ogg Vorbis, it comes out at 4119 bytes, which on its own is beyond the 4 Ki limit. The thought occurred to maybe steal some sounds off the engine, but there are none to use that ship with the engine, and stealing sounds from another game would defeat the point. Maybe the game would need to be 16-32 Ki just to have a few sounds, or maybe a single sound with pitch variations or offsets would work.

My thinking then goes a few directions: generating the Ogg files myself in Lua with some kind of highly-compressed encoder, trying to Muntz the Ogg file down, or trying to push for a much smaller format to be supported in the engine. This thread is about the last of the those.

sfxr
Back in December 2007, it was the 10th Ludum Dare competition and one entrant, DrPetter, needed a tool to make sound effects quickly to meet the competition's 48 hour time limit. He named it sfxr (sfx = sound effect, r = -er suffix; it makes sound effects). It worked quite well, a lot of entrants used it, and it has seen wide popularity since.

sfxr isn't even new to Minetest. It was mentioned by Melkor in a forum thread back in 2013, and in the OP of Zoonami.

These days, you can get a lot of programs, even web apps, that can give you the same results as the original SFXR program. Debian has sfxr-qt.

The sound effects produced by SFXR are output in WAV format. SFXR-qt uses SDL to play the samples back in the preview window. You can also save the SFXR info in the original binary SFXR format or the newer JSON-based SFXJ. These files are quite small. In fact, the original binary format is so small that encoding it with a compression codec like gzip or xz will increase its size due to the fixed cost of overheads. Sizes are in bytes:

Code: Select all

1073 a.sfxj
108 a.sfxr
464 a.sfxj.xz
168 a.sfxr.xz
133 a.sfxr.gz

This got me thinking - Minetest 5.9.0-dev already uses SDL, why not take the C++ code of sfxr and put it into the Minetest engine, and let mods use it? The playback code is already there. Maybe it's not quite as simple, since Minetest soundspecs support pitch and gain adjustments, but those could probably be arranged by changing the sfxr parameters.

A rationale
With sfxr direct integration, I could drastically reduce file sizes compared to Ogg. Now, it would probably be a bit selfish to ask for it purely so my project could turn out well. I'd also like to point out a similar long-lived proposal, #3873, to add Tracker Music support to Minetest.

Minetest would not be the first game engine to directly integrate sfxr playback. "Chroma 2D" (MIT-licensed) has an integration for it: Chroma.Audio.Sfxr.

Back in the 8-bit era, computers often had audio chips that could be programmed directly, instead of having to feed samples to the sound card. This enabled games to ship with sound effects and music in an era with tiny storage requirements. We no longer have that problem with pretty much any hardware that can run Minetest, but I still like it in principle. The tracker music proposal would enable a large library of music to be played back directly, and save on server and ContentDB bandwidth. My proposal is more modest, but it definitely could save on transfers.

I can also think about two parallels in Minetest: Schematics and graphics. We have an API for serialising schematics into a binary format that are made out of Lua tables, and deserialising them back into the game world from files. We have a png texture modifier for making a whole image out of. I think it would be good to be able to specify a table of sfxr parameters and play it back. The audio system doesn't get as much love as graphics :(

Discussion questions
Have you used sfxr before? Was exporting it to Ogg a significant barrier for you? Maybe an inconvenience?

Is the large size of sound effects a concern for you? What about music?

Do you think it's possible to approach my file size problem in a different way, such as encoding an Ogg from Lua? Do you have a library in mind?

[TPH]

Not exactly related to what you've typed, but I love the idea of compressing stuff into a still functional but small size, bonus for compatibility with earlier versions.
Fan of ".kkrieger" for that.
Yeah I noticed that Audacity unnecessarily inflates sound files - I wonder if there's a better software that let's one at least compress it down into a good quality but small size?

I don't think it's selfish at all, I'd love to see such an implementation. I agree with how sound files don't get much love - it was a culture shock for me when they weren't as object oriented as Roblox. Wish there was a custom end_time for them to end at so one could make a sheet of audio to play at specified times.

[Blockhead]

I experimented a bit more today. The container overhead of Vorbis is so bad that WAV, and especially zstd-compressed WAV will actually do better for a very short sound.

dig.sfxj:

Code: Select all

{
    "properties": {
        "attackTime": 0,
        "baseFrequency": 0.2551724137931036,
        "changeAmount": 0.027586206896552,
        "changeSpeed": 0,
        "decayTime": 0.18275862068965537,
        "deltaSlide": -0.055172413793103114,
        "dutySweep": 0,
        "hpFilterCutoff": 0,
        "hpFilterCutoffSweep": 0,
        "lpFilterCutoff": 1,
        "lpFilterCutoffSweep": 0,
        "lpFilterResonance": 0,
        "minFrequency": 0,
        "phaserOffset": 0,
        "phaserSweep": 0,
        "repeatSpeed": 0,
        "slide": -0.3034482758620686,
        "squareDuty": 0,
        "sustainPunch": 0,
        "sustainTime": 0.10517241379310364,
        "vibratoDepth": 0,
        "vibratoSpeed": 0,
        "volume": 0.42068965517241336,
        "waveForm": "Noise"
    },
    "version": 1
}

Also, the ogg file has been reduced to 8kHz 16-bit PCM whereas the WAV is still 16-bit 44.1 kHz. So it's not only bigger, it also sounds worse. Edit: Let's also add FLAC and Opus(Ogg-Opus) file sizes:

Code: Select all

 3439 dig.flac
 3001 dig.flac.zst
 4284 dig.ogg
 3589 dig.ogg.zst
 1347 dig.opus
 1318 dig.opus.zst
  817 dig.sfxj
 2268 dig.wav
 1123 dig.wav.zst

I don't have a binary sfxr because it seem to not deserialize properly, but a zstd-compressed sfxj would still be pretty small.

You may well ask: Why am I using compressed formats? Won't a decoder take more space? Well, no, because I'm using Minetest and it comes with minetest.compress, so the code size is incredibly minimal. It only takes a few extra bytes to decompress the file and write it back. Although modifying the mod directory is deprecated, and I'm not sure if it will play out of the world directory.

We may well be able to make the case for WAV support, but the inevitable objection would be that the file size is worse in the usual case than WAV, for anything more than a fraction of a second long and especially music. It's kind of on others if they misuse WAV, but then, I also would hate to break a record on ContentDB for mod size just due to having 1-3 WAV songs in it.