Advice needed for large texture maps

[xenoglyph]

I'm porting another game as a learning exercise. I've got 400+ MiB in tiles that I'll eventually be loading. This includes raw bitmap data plus an additional (don't recall exactly) 16 bytes or so of metadata per tile....stuff like flags and indexes to other linked systems (like animations).

All my height info is on a per tile basis, complex/large objects are collections of single tiles.

What's going to be the smartest and most efficient way to get that data into IGE? As I'm going to have to code some special tools to export/convert the data I want to head down the right path the first time.

[xenoglyph]

For now I've converted the data into a proper tileset and I generate a JS layout like map2.js in 13.4-texturemaps-load-data example...which results in a 200MiB javascript file and doesn't really work, lol. Guess JSON is the proper way?

edit...how would I go about loading a texturemap from a JSON? I see there's an easy way to do it with the tiled component....but what about regular IGE format JSON?

[xenoglyph]

Screw it, I'll code a custom binary encoder/decoder for map objects. Curious how far I can push this engine....and how horrible the memory usage will be.

A more elegant solution would be a tile of smaller maps, but it would have to be absolutely seamless to both the end user and the engine itself. Which is likely a lot of work, especially since I don't know my way around this thing very well yet.

For anyone curious, I'm talking map sizes of 20+ million tiles on the base layer.

[coolbloke1324]

Hey ya!

So... this is a big question.

Basically the first answer is don't. It would take AGES just to download the 200mb of data for most players. I am lucky enough to have a 120mbps line (12MB/s) at home and it would still take me around 17 seconds to download that map... most users aren't going to have more than about ~2mpbs if they are on ADSL and maybe ~30mpbs on normal cable.

That aside, from a technical aspect it is an interesting problem. The data should obviously be loaded as needed like Google Maps does. You will need a "buffer zone" around your screen so that map loading is not noticeable. I "started" a system to do just that but it's not even more than a skeleton with almost no code at the moment.

Obviously memory usage is a major issue as well. A management system needs to be in place to load and unload sections of the map as the viewport is panned.

To set you on the right track you can get the current world "visible area" of a viewport via: viewport.viewArea() or if you are in a tick method you can rely on ige._currentViewport.viewArea() being the view area of the viewport currently being rendered to.

[xenoglyph]

It's something I'll deal with later as it's obviously something that's possible, but I have other things to work out first. Binary loader for maps is kicking butt though. Maps are loaded as an ArrayBuffer with a Uint16Array view. File format is as simple as possible: WORD width, WORD height, WORD[] texture.

Since I have to generate my maps from a proprietary format into an IGE supported format already, I may as well use a format that's smaller and faster. Oh, and regarding the large tilesets I mentioned before, luckily the tile data is all raw bitmaps and not optimized much so my final reindexed and compressed tilesets are going to be in the ballpark of 5% of the original's size or less.

[coolbloke1324]

It's something I'll deal with later as it's obviously something that's possible, but I have other things to work out first. Binary loader for maps is kicking butt though. Maps are loaded as an ArrayBuffer with a Uint16Array view. File format is as simple as possible: WORD width, WORD height, WORD[] texture.

Since I have to generate my maps from a proprietary format into an IGE supported format already, I may as well use a format that's smaller and faster. Oh, and regarding the large tilesets I mentioned before, luckily the tile data is all raw bitmaps and not optimized much so my final reindexed and compressed tilesets are going to be in the ballpark of 5% of the original's size or less.

Very cool!

If you care to share, a new component would be a great addition so others can benefit from your awesome work! :)

If you want to share but don't know / want to wrap in a component I would be happy to take your code and do it for you.

[xenoglyph]

I'll put something together for you. It's really just a few lines of code. The important other half of the implementation is the map generator which is currently coded in C#. I might be able to implement it in JS (this is literally the first javascript I've ever done), not sure how well supported (client wise) it will be.

Here's what I have right now, just an extra function tacked onto IgeTextureMap:
could use a bit of error checking prolly, but my whisky and ambien cocktail is hitting me hard and I gotta bail

Code: Select all

loadMapBinary: function (path) {
		var self = this;
		function reqListener () {
			var data = new Uint16Array(this.response);
			var width = data[0];
			var height = data[1];
			var len = width * height;
			var map = new Array();
			var x = 0;
			var y = 0;
			map[0] = new Array();
			map[0][0] = new Array();
			for (var i = 0; i < len; i++) {
				map[y][x] = new Array();
				map[y][x][0] = 0;
				map[y][x][1] = data[2 + i];
				x++;
				if (x == width && i < len - width) {
					x = 0;
					y++;
					map[y] = new Array();
				}
			}
			//alert(JSON.stringify(map));
			self.map.mapData(map);
			return self;
		};

		var oReq = new XMLHttpRequest();
		oReq.responseType = "arraybuffer";
		oReq.onloadend = reqListener;
		oReq.open("get", path, true);
		oReq.overrideMimeType("text/plain; charset=x-user-defined");
		oReq.send();
	},