ContentDB - now with package translation support

[LMD]

Using the current system, this can be modelled by splitting up "foo++" in two mods: One for the compatibility layer named foo, and one for the extensions foo_ext.

Alternatively, this can be solved outside of the system by having a mod depending on the extensions simply check whether the extensions are there and either disable itself or throw an error if they aren't.

[Blockhead]

That doesn't solve the problem when a fork of "foo-mod" provides functionality that the original mod lacks,

It some cases the fork can ship a modpack with the old mod as a core and the new functionality as an extension. Anyway, minutiae aside the main reason this is a problem is because ruben doesn't have the time to write a better ContentDB dependency resolution scheme like author/module@version a la NPM. Outside ContentBD, people downloading mods manually should always take care to use the right versions of mods and modpack authors need to ship either versions of mods or clear instructions about the sources of where to install from.

[rubenwardy]

It some cases the fork can ship a modpack with the old mod as a core and the new functionality as an extension. Anyway, minutiae aside the main reason this is a problem is because ruben doesn't have the time to write a better ContentDB dependency resolution scheme like author/module@version a la NPM. Outside ContentBD, people downloading mods manually should always take care to use the right versions of mods and modpack authors need to ship either versions of mods or clear instructions about the sources of where to install from.

ContentDB mirrors the engine, would need to be changed there first

The problem with pinning authors is allowing forks, and the problem with versions is fulfilling conflicting versions (you can't install+enable a mod multiple times)

[Neuromancer]

It some cases the fork can ship a modpack with the old mod as a core and the new functionality as an extension. Anyway, minutiae aside the main reason this is a problem is because ruben doesn't have the time to write a better ContentDB dependency resolution scheme like author/module@version a la NPM. Outside ContentBD, people downloading mods manually should always take care to use the right versions of mods and modpack authors need to ship either versions of mods or clear instructions about the sources of where to install from.

ContentDB mirrors the engine, would need to be changed there first

The problem with pinning authors is allowing forks, and the problem with versions is fulfilling conflicting versions (you can't install+enable a mod multiple times)

So I think I've found a pretty good solution to the mods with the same name problem that causes duplicate incompatible dependencies. We add a subname that works as follows: In the mod.conf file, you have the following line:
name = caverealms-lite
where the "-" or some other special character allows the mod creator to specify a sub-name. Then if someone creates a mod that depends on that mod, they add a line to their depends.txt for either
caverealms
caverealms-lite
or
caverealms*
The engine and ContentDb would need to change so that a dependency of caverealms gets you the original version, caverealms-lite gets you the forked "lite" version, and caverealms* will have a dependency on whatever the user has already downloaded else it uses the rule rubenwardy codes to select any of these mods or their forks. Currently that rule is to select the highest rated mod. Then rubenwardy or his team would need to change the code for contentdb to not allow any new duplicate names. Also they would have to identify existing mods with dup names and change name in the mod.conf of the mod that makes the most sense to change. Then they would have to put in code that if the mod creator tries to upload a new release of their mod that doesn't use the correct name = name-subname line that the new release is rejected. Note that subnames are not required for the original version of a mod.

An alternative more heavy handed approach would be to disable any existing mods who have their duplicate named mod flagged by the ContentDb team as an offending duplicate name until they get resolved.

[doxygen_spammer]

Supported games for Mods

ContentDB now keeps track of which games are supported by a mod. [...]

Where does the information come from?
There's a recursive algorithm to determine support based on hard dependencies. Modders can use supported_games and unsupported_games in mod.conf to override this. See the docs.

Thank you, I appreciate using mod.conf instead of .cbd.json.

• .cbd.json is a hidden file, which is sensible for managing the repository (like .git/ and .gitignore), but bad for storing content (like the mod and its indispensable metadata).
• It duplicates information from the strictly required modpack.conf, violating the DRY principle. (dont-repeat-yourself)
• It uses JSON, which prevents comments like licensing information, and is inconsistent with most of Minetest.

I noticed a problem:
My mods now specify supported_games = *, as demanded by your documentation for mods not interacting with any game.
But the website still declares every single game as “Tested with:”, which is not true.
I suggest any of these solutions:

• Remove the “Tested with:” section if such information does not exist. “No specific game required” is just precise.
• Rename to “Dependencies satisfied by:”.

A minor thing:
The documentation says supported_games is only supported for mods, but it also works for modpacks.

[doxygen_spammer]

That doesn't solve the problem when a fork of "foo-mod" provides functionality that the original mod lacks, and another mod may depend upon the extended functionality.

My interpretation of the rule about technical names is that such forks do not have the right on the name.
An extended fork is similar to an add-on mod disguised as fork/reimplementation.

So I agree with LMD:

Using the current system, this can be modelled by splitting up "foo++" in two mods: One for the compatibility layer named foo, and one for the extensions foo_ext.

[rubenwardy]

.cbd.json is a hidden file, which is sensible for managing the repository (like .git/ and .gitignore), but bad for storing content (like the mod and its indispensable metadata).

I should allow using cdb.json as well

It duplicates information from the strictly required modpack.conf, violating the DRY principle. (dont-repeat-yourself)

For a lot of information, .conf is read instead - including dependencies.
.cdb.json is mostly for power users tbh, it's not really necessary.

It uses JSON, which prevents comments like licensing information, and is inconsistent with most of Minetest.

Yeah, this and the lack of support for trailing commas are my main complaints about JSON. But at least it's a standard format and doesn't require a turing complete parser - can't say the same thing about .conf or Lua. ---------------------------------

My mods now specify supported_games = *, as demanded by your documentation for mods not interacting with any game.
But the website still declares every single game as “Tested with:”, which is not true.

You made a mistake here - a mod that depends on default cannot support all games, as it has a dependency on Minetest Game and derivatives. The web UI validates this but I was missing a check when reading from .conf, added now.

The documentation says supported_games is only supported for mods, but it also works for modpacks.

This was an oversight because there's no distinction between mods and modpacks on ContentDB - I'll update the docs to make this more clear and mention modpack.conf

---------------------------------

Btw, you can use .gitattributes to exclude the LICENSES folder, which would allow the Git import to work

[doxygen_spammer]

My mods now specify supported_games = *, as demanded by your documentation for mods not interacting with any game.
But the website still declares every single game as “Tested with:”, which is not true.

You made a mistake here - a mod that depends on default cannot support all games, as it has a dependency on Minetest Game and derivatives. The web UI validates this but I was missing a check when reading from .conf, added now.

Makes sense.

The modpack actually does not depend on default.
I need to take some time to migrate the crafting recipes to adaptation_lib or move that mod in a separate package.

Btw, you can use .gitattributes to exclude the LICENSES folder, which would allow the Git import to work

I already do that (and git-archive-all understands it correctly), but it still doesn’t work.
The problem must be in ContentDB.

Btw, how about just using “Releases” (from GitHub or GitLab or other platforms) instead of the source code?
That would allow to have arbitrary differences between “git clone” and “released” versions, with the added guarrantee that releases have passed CI.

[rubenwardy]

I already do that (and git-archive-all understands it correctly), but it still doesn’t work.
The problem must be in ContentDB.

ContentDB uses git-archive-all to create the archives, ContentDB itself never parses .gitattributes. It works with other packages - the only thing different here is **/, so I imagine that is causing the issue

The relevant code is:

Code: Select all

from git_archive_all import GitArchiver
archiver = GitArchiver(prefix=release.package.name, force_sub=True, main_repo_abspath=repo.working_tree_dir)
archiver.create(destPath)

Perhaps one of these arguments explains why it doesn't work on ContentDB but does work with your tests of git-archive-all

Btw, how about just using “Releases” (from GitHub or GitLab or other platforms) instead of the source code?
That would allow to have arbitrary differences between “git clone” and “released” versions, with the added guarrantee that releases have passed CI.

You can build your package using CI and then upload to CDB using the API. ContentDB also supports creating releases from tags, which are created when you make a GitHub/GitLab release. I don't want to add support for specific Git hosts though

[doxygen_spammer]

It works with other packages - the only thing different here is **/, so I imagine that is causing the issue

Can you give an example of such a package?

The relevant code is:

Code: Select all

from git_archive_all import GitArchiver
archiver = GitArchiver(prefix=release.package.name, force_sub=True, main_repo_abspath=repo.working_tree_dir)
archiver.create(destPath)

Perhaps one of these arguments explains why it doesn't work on ContentDB but does work with your tests of git-archive-all

Unfortunately not.
I ran that code in the interactive Python interpreter and as shell command with equivalent options, and both work as I intend.
It still works if I modify these options, or if I run from outside the repository directory.

You can build your package using CI and then upload to CDB using the API.

Thanks, I forgot about the API.

[Linuxdirk]

I just noticed that the user avatar icon in the content database does not sync with the forums anymore and there seems to be no way to update the avatar directly in the CDB.

[runs]

Hi,

I would like to be added (considered at least) a new tag for ´Content Warning´:

´Dating/flirting´

Thanks!

[MCL]

Hi,

I would like to be added (considered at least) a new tag for ´Content Warning´:

´Dating/flirting´

Thanks!

I like your idea!
I think there should also be a tag that would allow my mods to go through :(
since the modeartors have been shadow-banning them from CDB ever since I added them :(

[c56]

Hi,

I would like to be added (considered at least) a new tag for ´Content Warning´:

´Dating/flirting´

Thanks!

I like your idea!
I think there should also be a tag that would allow my mods to go through :(
since the modeartors have been shadow-banning them from CDB ever since I added them :(

multiplayer is a feature not a mod

[MCL]

Hi,

I would like to be added (considered at least) a new tag for ´Content Warning´:

´Dating/flirting´

Thanks!

I like your idea!
I think there should also be a tag that would allow my mods to go through :(
since the modeartors have been shadow-banning them from CDB ever since I added them :(

multiplayer is a feature not a mod

Where in my post did I mention multi-player?

[Nininik]

Yall know how can i put my own game so It can be downloaded inside minetest? Am I thinking contendb? Is that right?

[Blockhead]

Yall know how can i put my own game so It can be downloaded inside minetest? Am I thinking contendb? Is that right?

Yes, you will want to have a git repository with your game, hosted on some website like Codeberg, GitHub etc. and then add it as a package to ContentDB. You have to pay attention to licences for ContentDB, so make sure all your original mods are licensed properly - code and art assets.

[ROllerozxa]

You aren't required to use a Git repository to submit a package to ContentDB, although it's highly recommended in order to allow for easy contribution and submission of bug reports.

[c56]

multiplayer is a feature not a mod

Where in my post did I mention multi-player?

´Dating/flirting´ <-- right here (as that is enabled simply by having multiplayer / chat) , a better content tag would be something like 'Mature Content' or 'Romance / Sexuality'

[runs]

Yes, I hadn't realized, obviously my game is a multiplayer-chat game like any other. So it will keep its Sammy Rating of 3 for children.

[Neuromancer]

Would like a little more info on the purpose of "collections" in ContentDb

[rubenwardy]

Collections allow you to create and share lists of packages. You might use it to:

• Share the packages you have installed with others
• Curation - recommend packages to others, perhaps based on your Top 10 or some category. This is another way to improve package discovery on ContentDB, by allowing users to recommend lists
• Keep track of packages to try later
• Favorites list - there's now a heart button on each package to add to a favourites list, this is like starring on github

One of the benefits of collections for me at least is that ContentDB can use them for the following things, which means I can simplify the code in some areas:

• Mod and game jams
• Spotlight and featured packages

In the future, I plan to give users much more control over collection pages. You'll be able to use them to make guides

[Linuxdirk]

Idea: Installable collections (like modpacks). Maybe even handled as one single package.

[rubenwardy]

Yeah, that's planned. I'll need to implement client support for it.

The reason why collections aren't a type of modpack is (1) modpacks can't contain games or texture packs (2) I want to show them in a slightly different way, to avoid users being overwelmed with loads of collections that could be confused with packages

The reason why collections aren't a type of package is because that comes with a lot of assumptions and bloat that I didn't want - in the ContentDB backend at least. Maybe I'll make them a type of package in the future, after a refactor. In any case, this doesn't hold back implementation of installing them

[rubenwardy]

Another cool thing would be to have a button in the Minetest client that opens a URL and allows you to add the packages you have installed to a collection. This would be quite easy to implement if the list of packages fits within the URL limit (2000 characters)