Infinite world size

[Calinou]

Adding the ability to not save unmodified MapBlocks would help for file size, but increase global CPU usage since parts of the world may be generated several times.

[aldobr]

Just use a cache.

Current model:

1st time a player enters a chunk, it asks the server to generate it.
server sends the chunk to the client.
server stores the chunk into the database.
doesnt matter if the chunk is exactly as it was when generated. Just passing by it makes it be stored into the database.

Delta engine:

client enters a chunk.
chunk is generated client side/server side (maybe using GPU) at the same time (no transfer via net needed).
client asks the server for deltas to apply to that chunk.
chunk is placed into a lru cache (least recently used chunk is discarded).
when player breaks/places a node, deltas are generated by the client and sent to the server.
deltas are stored into the database back end.

This is a case of storage x cpu.

Supposedly storage would be faster, but all data must be stored in a data structure (in that case a database) that has finite ammount of access time.

When you reach 10gb sized databases, no matter how efficient the database is, there WILL be huge overheads, reaching the case where database storage is LESS efficient than on-the-fly generation (at least when compared to C++ mapgen)...

You can use other tricks to improve database access in the current model...

Distributed database :

add mariadb or postgres database backend (it would be slower than sqlite initially).
split the world in superchunks and store each superchunk in a different mariadb server.
ie.: make a distributed database system and share the load accross multiple servers connected via internet.

[BrandonReese]

split the world in superchunks and store each superchunk in a different mariadb server.

Could this be done with sqlite also? I've wondered if something like this would be beneficial.

[aldobr]

It can be done (if someone is willing to work on it)

but sqlite is not a RDBMS

so you would need a server at each machine accepting connections and storing that in sqlite (ie.: you end up remaking a RDBMS).

Its easier to work with a RDBMS to start with...

Using a checkerboard distribution of chunks across multiple RDBMSs.

it would be faster and cheaper (for a small server) to use a RAID setup and distribute the load across multiple disks at the sector level or use a SSD, because most overhead for huge maps is at the disk i/o.

[spillz]

The point of a delta engine is that you offload the terrain generation to the client (i.e. someone would need to add client side support for lua terrain gen).

But I don't think we'd need to wait for client-side Lua to have bigger world sizes. As someone put it before, it's basically bigger coordinates, plus some mechanisms to prevent server worlds to grow up insane, be it [/snip]

hmm... let's see. 10 players in different unexplored areas of the map. (And if you think that's unrealistic, it's not clear to me why you want unlimited worlds anyway.) Each decides to do some exploring. Map gen is now doing 10x as many requests. Lag fest. You can't get around having enough CPU and, more importantly, bandwidth, to do map gen in game types that involve a lot of exploration. IMHO, the client approach is the only way to really get the game to scale given the current state of technology. With reasonably modern client hardware you can pretty much lock 60 fps and a long draw distance with terrain generation going on in the background and never see map pop-in.

[Sokomine]

Step3: As seen from the previous posts, there's a lot of different options here, but here's mine. Let's add (optional) timestamp to blocks when writing them to disk (no matter what database backend is used) with a special value (0? MAX_VALUE?) for blocks which have been modified. Don't touch the block loading code.

YES, please! Do this. I've been hoping to get this feature for some time and even considered implementing it myshelf but didn't get around to it. A timestamp for "generated at" and "last modified at" at database level (add new columns for those) would help to clean up the map from time to time. With a server growing, more and more territory is explored - while the likelyhood of someone ever passing by there again decreases. Backup and distribution of a map also gets more difficult if the size is in the GB range. Just look at my "manual" copy of Redcrabs world: around 100 MB, while the actual world file is several tens of GB.

Whenever an existing chunk is changed, the server could do a backup of the old chunk on database level, thus saving an edit history and allowing to restore individual chunks to previous states.

Backups/downloads of the map could be limited to chunks that where actually changed, resulting in a very small map file, saving bandwidth and storage space.

When a mod is added that changes mapgen significantly, it wil be possible to delete all the old unmodified blocks done by the old mapgen, while keeping all the structures players built. The actual mapgen changes would show up in existing but unmodified areas and not force players to walk further and further away from spawn.

All the above could be done with pretty simple changes. There's something else I'd like to have as well: Client-side mapblock cache. Download of mapblocks is still slow, and it destroys immersion if you notice after half an hour that there's a giant mountain right in front of you, or that the lake you considered to be an ocean is smaller than expected. The copy of the map you explored last time will still be accurate in most areas - and where it's outdated, the server'd send an update anyway once it gets around to. Such a client-side caching is already done with sfan5's special map-saving client. Now let that one store its mapblocks for each server independently and let it load mapblocks from there when connecting to a server - and immersion ought to increase significanlty. There will be a world to look at instead of white fog.

Both of the two wishes above have nothing at all to do with increasing the square area MT may cover. In my opinion, around 30km in each direction are far more than what even huge servers use up. The players are just drawn to the borders of the map because those borders exist. Sure it would be nice to have a larger area, but it's not really that urgent.

[Wuzzy]

I am not taking any “The Minetest world is too small and needs to be larger.” discussion serious until anyone comes with a REAL example which proofs there is actually a real need for larger worlds. The examples which have been shown have all been pretty hypothetical.

For example, a Minetest world where JUST the surface is almost completely claimed/built upon is yet to be seen …
Can anyone show me a Minecraft server where a surface of 66000×66000 is almost completely covered?

[Thermal_Shock]

Can anyone show me a Minecraft server where a surface of 66000×66000 is almost completely covered?

I don't think it exists. Plus most large Minecraft servers limit the size of their maps to less than that anyway in order to keep the maps filesize down.

Even this server with a few thousand active players on a town server barely makes use of their 16000x16000 map.

The potential size of a Minecraft map sounds great as a feature. But Minetests current map limit seems to fall on the more practical side of things.

[Sokomine]

I am not taking any “The Minetest world is too small and needs to be larger.” discussion serious until anyone comes with a REAL example which proofs there is actually a real need for larger worlds. The examples which have been shown have all been pretty hypothetical.

It's a psychological problem, not a physical one. The area around spawn gets crowded, and then suddenly players start out calling for a new map even tough far less than 100x100m around spawn has ever been built on.

[TenPlus1]

I like the idea of vertical realms appearing in the world so players can explore upwards or downwards to find new blocks to play with... As for map functions, I'd rather have time spent on making maps load/save quicker instead of making them larger so that multi-player servers could use a headerless setup and maybe even use the GPU to do all the work...

[Linuxgamer94]

And what is the advantage? Did ANYONE ever experience a too small world? No.

yes. I like big worlds for factions, also why would anyone want an hampered experiance. If mods can do it then why can't the devlopers, and don't get all savy on the bloody end users.

[kaeza]

yes. I like big worlds for factions, also why would anyone want an hampered experiance. If mods can do it then why can't the devlopers, and don't get all savy on the bloody end users.

Did you even try exploring the entirety of the *surface* of the map? I'm not even talking about underground constructions, or floating castles; I'm talking about *just the surface*.

For reference, one of the oldest servers still alive, Redcrab's server (first post is dated April 2012, more than 2 years ago), still hasn't exhausted the available *surface* space as far as I know.

Think about it.

[thetoon]

For reference, one of the oldest servers still alive, Redcrab's server (first post is dated April 2012, more than 2 years ago), still hasn't exhausted the available *surface* space
as far as I know.

Sure thing, but still : not every MT server strives to be some block-based Second-life, with nicely packed lots. Some of us like freedom, especially freedom to explore in any direction without fear to meet world's end.

64km is a really small world, from that perspective. Mind you, it's not even a country (well, you _could_ reproduce the entirety of Vatican in it, granted, but still).

Vertical space is a great thing, one that MC kinda misses, so yeah : MT wins here. But just because we have a looot of vspace don't mean we couldn't use a little more horizon.

On the upper "factions" stance, I do get it as well. It's nice to spawn players to remote locations, with rather low odds of meeting one another (hence putting a lot more "stress" on such meetings when they occur).

No-one "needs" more horizontal space, that's for sure. But for that matter, no-one really needs a Minecraft-inspired block-game either. Nor Minecraft itself, or any other video game.

Think about it.

[Krock]

64km is a really small world, from that perspective.

I don't get what you mean.

Area
- Total 587,041 km2 (47th)
Population
- 2012[4] estimate 22,005,222 (53rd)
- Density 35.2/km2 (174th)

And minetest has 64km * 64km * 35.2 = space for 144'179 real minetest players in with very much nature.

[twoelk]

Think about it.

Well ther you are.
Thats's the problem right there!
So instead of enjoying what is there and exploring what is given, all what a certain kind of people can think is "oh my God, a hardcoded limit. This must be bad."

The world MT offers in theory is, as far as I know, if fully explored, include up and down, larger than any practically accessable hardware can hold at the moment.

I guess this sort of shows, among other things, that the horizontal extends are way more important than vertical limits. MC has horizontal "soft" limits that are farther away and yet people complain there is a limit, are totally fascinated about reaching the farlands and seem pretty content with what vertical limits are offered in contrast to the discussion noise around the horizontal limits.

What can be learned from this constantly coming up though might be that this description of the game may need some rewording:

Minetest is an infinite-world block sandbox game

[Vazon]

The only way I think the world could be enlarged would be leave the map as is but people want some thing that is similar to a "Nether" parallel added to the game. It would not be access about just by walking all the way to it though. Blocks can be added at the edge of the map that are not mineable and does not let you see the "parallel" world. This would give you a full nether world, would be the same size as the regular world, and other things can be added since there are 3 other sides.

[Linuxgamer94]

you devlopers are just giving us BS. In my latest world the world gen was all screwed up. I ended up spawning up near the end of the world. I could littery walk over to it and the eniter world was one stupid mountain island which I could not dig yet alone walk up. Again this is bull CRAP. I walked to the edge in minetest 3.x and it was not that hard, I even fell out of the world witch does not seem to happen any more. At least that time there was land on my spawn. Minetest is a clone of minecraft, GET OVER IT and stupid things like this is why no one wants to do a lets play of it. While it is fun at first you need mods to make the game usefull. Just look at the world gens:

v6 ok
v7 just why
math wtf
indev what is even diffrent
singlenode ahhh I am falling and I can't get up, or move, or fly or place blocks.
mods like fire, hud, wool, and xdoors should not have to exist! What is worse is that you don't give a single reason why, and yes I am complaining to the few devlopers who actualy read the forums! Everybody just calls Minetest a cheap low budget minecraft clone that is a waste of time from youtubers, LAS, and most open source jounalests. As an end user who likes the game it furstates me that the realy small user base has to defend and make mods for minetest. How offten do you here news about minetest making a new update or adding a new item or feature, I have never, the only way I know if there is a new version is when someone ports it over to the bloody repos!

[drkwv]

I don't get what you mean.

In minecraft, when playing on the public servers, I usually travel 20'000 blocks away from spawn to just not get griefed (territory privates are for little girls). Map boundaries is not about square territory. It's about how far you can travel away from spawn. For example, one can simply travel 300'000 blocks in a straight line for a 1-2 weeks of gaming.

All this nagging you can see here is not about available place. It's about feelings of freedom and game immersion.

Personally, I believe infinite world size is not something that is required on current stage of minetest development, but is something that necessarily needs to be implemented in future (and that is why it is better to think about this today).

Also, I do think that solution of this problem is not to add 2^32*2^32 maps, but to implement servers shared maps. Which means that every 64*64km map will be processed by a different server instance to allow scaling map processing through different servers. This would bring not only infinite map, but also an infinite players server limit. Or even ability to merge friendly servers together. Isn't that would be great?

[twoelk]

...
Isn't that would be great?

If Minetest would be designed that way yes ...

but

... the way it is built by design now - that would be pretty difficult to achieve and would probably not work as you might expect or hope. Besides we would need some dedicated volanteers that run a 24/7 grid service, unpaid of course.

On the other hand why play on a server if you want to play alone?

(says a guy who more often than not builds at some distance from the busy spawn area)

for further thoughts and discussions you might want to study these forum topics to name only the more recent threads on subjects related to world size and the concept of the word infinity.

Mineteset as global virtual world
Multi Map Teleport System
[idea] a second world!

oh and not to forget these three pages worth of discussion:
Infinite world size but you might have read that allready.

edit:
Just noticed your join date so you even may know almost all forum discussions on this subject ever made
or at least since this one: When the map will be infinite? from August 23rd 2012.

oh well
/me runs and hides

[Kilarin]

My walkabout from around the middle of my singleplayer map to the edge of the world.
viewtopic.php?f=3&t=9651
It's a LONG way there.

[thetoon]

My walkabout from around the middle of my singleplayer map to the edge of the world.
viewtopic.php?f=3&t=9651
It's a LONG way there.

Not everyone starts from the middle of the map. On multiplayer environments, it makes sense to dispatch players here and there, especially if you don't want them to meet too soon.

64km is a long trip by foot, that for sure. But people want to add cars, choppers and UFOs.

[HeroOfTheWinds]

To me, the logic goes something like this:
I want to build an amazing building that will make people green with envy.
If I want people to see it and go green, I better build it nearby, or they won't want to trek out into the middle of nowhere to see something someone *claims* is awesome.

In other words, I don't think it's a matter of whether the edge is there; people should focus on being a community of friends, not hermits. :/

[Sokomine]

or they won't want to trek out into the middle of nowhere to see something someone *claims* is awesome.

Exactly. I want to be able to find and enjoy good buildings, and I hope that other builders will enjoy mine as well. Singleplayer is diffrent, and I guess other server types (PvP?) have diffrent needs. Such a long walk seems like a nice idea for a singleplayer world.

[Kilarin]

A possible suggestion for those who want more horizontal exploration space. An approach that MIGHT require a smaller engine rewrite.

Increasing the number of bytes in the coordinates for each node would be a major rewrite of the system. But what if we could have a LOT more horizontal exploration space WITHOUT changing the size of the coords at all?

Currently minetest is a 64k cube. And a lot of players only use a thin layer of this. Usually no more than a few thousand nodes below the surface and perhaps a few hundred nodes up. So what if we split the vertical space up into slices and connnected them via wraparound logic?

So, for example, if we divided our 64k cube into 9 vertical slices, they would each be about 7,000 nodes tall. Each of these 9 slices is a 64k x 7k x 64k cube. Put the surface at about 6000k, since users dig down more than they climb up and you would have room for caverns and mines going down 6000 nodes, and enough sky that most players would never get even close to the top. Now we take those 9 slices and arrange them in a horizontal matrix like this:

Code: Select all

01 02 03
04 05 06
07 08 09

And now you would have a map that is 192k x 7k x 192k

So if you were in slice 01 and walked along the X axis, instead of hitting the "end of the world", you would just walk into slice 02. Smoothly, with no visible transition. And you could keep walking for another 64k distance until you hit slice 03. You wouldn't hit an "end of the world" until you finally hit the far end of slice 03.

We don't need to change the size of the coordinates here, we just need to add in some wraparound logic that tells the engine that slice 01 wraps into slice 02 on the X axis and slice 04 on the Z axis. I don't know how complicated coding that would be, but I suspect it might be simpler than changing the size of the coordinates.

We could get bigger than that. Make your vertical slices only 2560 high and you get 25 64k x 2.5k x 64k slices, which you could map like this:

Code: Select all

01 02 03 04 05
06 07 08 09 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25

Into a 320k x 2.5k x 320k map. Put the surface at 2400 and that leaves 160 up for sky, which is still a bit higher, and a LOT deeper than minecraft.

Of course, if you write the wraparound logic right, you could allow virtually any arangement of slices to be determined at map creation time. Maps that want a lot of deep caves and sky islands would stick with the original 64k cube. Games that want to still have quite a bit of mining space but not as much sky could go with the 192k x 7k by 192k cube. You could even go all the way up to 81 regions 790 nodes high and have a 576k x .5k x 576k map that was STILL deeper and higher than minecraft.

And would the map even have to be square? Perhaps some game developer wants to create a riverworld that is 1 slice wide but 9 slices long?

A 15 x 16 slice map would match minecrafts depth, for a total area of 960k x .266k x 1024k.

Of course, this all depends upon the idea that writing wraparound logic would be simpler than changing the size of the coordinates. I haven't messed with the minetest c code, so I may be way off base with that.

Just an idea for consideration that IF it worked, would let people have a much greater flexibility in determining the dimensions of the game world.

[Sokomine]

That slices idea might currently be impossible due to the way everything is drawn. Best talk to the devs directly about it.