[Solved] I managed to hit the ContentDB size limit :-(

Post Reply
User avatar
Wuzzy
Member
Posts: 4934
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

[Solved] I managed to hit the ContentDB size limit :-(

by Wuzzy » Post

Today I was about to release version 2.0.0.beta.1 for my game Lazarr! on ContentDB but I got rejected. :-(
100 MB maximum, it said. My game was about 116 MiB large. The major size comes from the music tracks.

So for ContentDB, I was forced to cut two music tracks to bring down the size and release version 2.0.0.beta.2 instead, going down to about 80 MiB. So if you wondered why I didn’t release “2.0.0.beta.1”: this is why.

Since this is only a beta version, this is OK but I wonder if I could release the game with the full track list when the full version 2.0.0 drops. I hope the game doesn’t strain the storage limits of ContentDB too much.

wsor4035
Member
Posts: 188
Joined: Sun Aug 11, 2019 21:23
GitHub: wsor4035
IRC: wsor
In-game: wsor

Re: I managed to hit the ContentDB size limit :-(

by wsor4035 » Post

appears as though you haven't utilized the basic size reduction steps, though I didn't look in depth and may have missed that you did.

optipng textures
excluding files from cdb release. https://content.luanti.org/help/package ... ding-files (21 .blends, 2 .bbmodel, 1 .py, 2 .svg)
j5uBLfc6NxgersvVj5D5dIsiKDkoQb0o

User avatar
Blockhead
Moderator
Posts: 2252
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
IRC: Blockhead256
In-game: Blockhead Blockhead256
Location: Land Down Under
Contact:

Re: I managed to hit the ContentDB size limit :-(

by Blockhead » Post

In addition to wsor's comment about using optipng (personally, I use oxipng), I also checked the audio tracks and some of them are quite high bitrate, over 400 kb/s in some cases, which is much higher than it needs to be.

Code: Select all

$ find -exec ffprobe {} \; 2>&1 | rg kb/s #rg is ripgrep
  Duration: 00:01:02.40, start: 0.000000, bitrate: 98 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:03:36.96, start: 0.000000, bitrate: 289 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 270 kb/s
  Duration: 00:05:39.27, start: 0.000000, bitrate: 373 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:03:11.92, start: 0.000000, bitrate: 103 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:02:49.76, start: 0.000000, bitrate: 421 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:01:24.25, start: 0.000000, bitrate: 100 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:06:28.00, start: 0.000000, bitrate: 104 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:01:18.00, start: 0.000000, bitrate: 91 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:01:18.60, start: 0.000000, bitrate: 102 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:01:20.73, start: 0.000000, bitrate: 98 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:02:59.50, start: 0.000000, bitrate: 105 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:06:40.05, start: 0.000000, bitrate: 444 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:03:10.13, start: 0.000000, bitrate: 99 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 112 kb/s
  Duration: 00:04:06.27, start: 0.000000, bitrate: 393 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:05:18.63, start: 0.000000, bitrate: 367 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:03:32.36, start: 0.000000, bitrate: 415 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
  Duration: 00:03:36.96, start: 0.000000, bitrate: 471 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s
A well-established bitrate for "near-lossless" mp3 files is 320 kb/s, and vorbis apparently needs less. I'd recommend re-encoding down to say, 280-300 kb/s with ffmpeg or audacity (according to your level of prior experience and willingness to learn command line tools) - ffmpeg should support straight up bitrate targetting, audacity may only have the standard 0-9 quality options available; something like 7 quality may be in the vicinity.

For instance, I re-encoded one file from 471 kb/s to a target of 270 k/bs, and reduced it to 61% the size, with no discernible loss of quality:

Code: Select all

$ ffmpeg -i lzr_ambience_kml_island_meet_and_greet.ogg -b:a 270k lzr_ambience_kml_island_meet_and_greet_lower.ogg

$ ffprobe lzr_ambience_kml_island_meet_and_greet.ogg
...
Input #0, ogg, from 'lzr_ambience_kml_island_meet_and_greet.ogg':
  Duration: 00:03:36.96, start: 0.000000, bitrate: 471 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 499 kb/s


$ ffprobe lzr_ambience_kml_island_meet_and_greet_lower.ogg
...
Input #0, ogg, from 'lzr_ambience_kml_island_meet_and_greet_lower.ogg':
  Duration: 00:03:36.96, start: 0.000000, bitrate: 289 kb/s
  Stream #0:0: Audio: vorbis, 44100 Hz, stereo, fltp, 270 kb/s
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My Content on ContentDB ✝️♂

User avatar
Wuzzy
Member
Posts: 4934
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: I managed to hit the ContentDB size limit :-(

by Wuzzy » Post

Yeah right, the idea of size optimization occured to me after I posted this, lol.

PNG really isn't the bulk of the size, most of the images are very small already. I already ran the optimizer for the big images, but not the small ones; for the small ones, it barely makes a dent (even in total). Not worth the commit noise.

Music is by far the bulk of the package size. Thanks for the bitrate hint, I completely ignored this. Also, I might have picked a very high quality level when transcoding before …

For some of the music files, I actually have the original FLAC or WAV files. :-)
Unfortunately, only for those that are already small. :-(

The "*_kml_*" music files are the biggest ones and yes, they also have the largest bitrate.

These files are already the result of a transcoding from MP3, because originally I only have them from MP3 format. So I will use the files that are "closest" to the source instead. Unfortunately, I don’t have FLAC or WAV for those, so I need to be careful to not add more unneccessary levels of loss.

Let’s see how far I come with my tools. I’ll experiment with ffmpeg and oggenc.

User avatar
Wuzzy
Member
Posts: 4934
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: I managed to hit the ContentDB size limit :-(

by Wuzzy » Post

Using the advice of reducing the bitrate, I manage to reduce the ZIP file size to about 80 MiB.

I’ve just released a new version of the game with all the music tracks included.

Thank you.

FYI: I’ve actually tried two methods: First, I tried the ffmpeg command suggested before.
Second, I tried to first convert to WAV, then to OGG with oggenc <file> --managed -m270 -M300 *.wav, i.e. forcing the bitrate to stay between 270 and 300.

For 5 of the 6 files I touched, ffmpeg resulted in smaller files. With oggenc, only 1 of the 6 files was slightly smaller.

Astrobe
Member
Posts: 643
Joined: Sun Apr 01, 2018 10:46

Re: [Solved] I managed to hit the ContentDB size limit :-(

by Astrobe » Post

Good thing you were able to solve it this way. Out of curiosity, did you consider putting (some of) the music tracks in a separate (optional?) mod?

Looking into this, I realize that although Luanti shows the list of optional mods, it doesn't allow to download them by just clicking them; only CDB allows to get them in a couple of clicks.
My game? It's Minefall.

User avatar
Wuzzy
Member
Posts: 4934
Joined: Mon Sep 24, 2012 15:01
GitHub: Wuzzy2
IRC: Wuzzy
In-game: Wuzzy
Contact:

Re: [Solved] I managed to hit the ContentDB size limit :-(

by Wuzzy » Post

No, I didn’t want to put music in a separate mod. It belongs to the game.

Post Reply