Code Monkey home page Code Monkey logo

Comments (5)

isiyu avatar isiyu commented on August 21, 2024

@david-rhodes this is "phase 3" outlined here: mapbox/mapbox-sdk-cs#66 (comment)

"file system" and "on disk" are effectively the same. Based on the TOS, we can use a system similar to Mapbox mobile's native SDKs, which is pre-downloaded areas approx. the size of London

(The tile ceiling is set to 6,000 tiles by default but can be raised for paid plans.)

from mapbox-unity-sdk.

david-rhodes avatar david-rhodes commented on August 21, 2024

@isiyu Ticketed here because I suspect it will be a unity layer implementation (accessing files from various OSs).

from mapbox-unity-sdk.

derekdominoes avatar derekdominoes commented on August 21, 2024

Here is my implementation of local file based caching. I wanted to validate and see if it can be efficiently done. It is in no way optimal. It is fully implemented in MapController.cs and so architecturally not likely the correct way in that it might be better to subclass or add an interface to MapController.

I use it with the SlippyVectorTerrain sample.

To get it to work you must first repair a bug in MapImageFactory.cs:83. Assignement of tile.ImageData causes ImageDataChanged event to be fired before ImageData is loaded Fix this in your version of MapImageFactory.cs

VectorData has not been set in the current UnityTile so vectors are not cached.

Try it out and me know what you think.

`namespace Mapbox.Unity.MeshGeneration
{
using UnityEngine;
using System.Collections.Generic;
using Mapbox.Unity.MeshGeneration.Data;
using Mapbox.Unity;
using Mapbox.Platform;
using Mapbox.Unity.Utilities;
using Utils;
using System;
using System.IO;

/// <summary>
/// MapController is just an helper class imitating the game/app logic controlling the map. It creates and passes the tiles requests to MapVisualization.
/// </summary>
public class MapController : MonoBehaviour, IFileSource, IAsyncRequest
{
    public static RectD ReferenceTileRect { get; set; }
    public static float WorldScaleFactor { get; set; }

	public MapVisualization mapVisualization;
	public float TileSize = 100;

    [SerializeField]
    private bool _snapYToZero = true;

    [Geocode]
    public string LatLng;
    public int Zoom;
    public Vector4 Range;

	private MapVisualization localMapVisualization;
	private string cacheRoot;
    private GameObject _root;
    private Dictionary<Vector2, UnityTile> _tiles;

	/// <summary>
	/// Resets the map controller and initializes the map visualization
	/// </summary>
	public void Awake() {
		cacheRoot = (Application.isEditor ? Application.dataPath.Substring( 0, Application.dataPath.Length - 7 ) : Application.persistentDataPath) + @"/cache/v1/";
		// Need unique Factorie instances in our caches MapVisualization so that they can have a IFileSource not shared with the MapboxAccess MapVisualization
		// Clone the same factories that have been specified in the Unity inspector for MapVisualization
		localMapVisualization = ScriptableObject.CreateInstance<MapVisualization>();
		localMapVisualization.Factories = new List<Factories.Factory>();
		foreach (var factory in mapVisualization.Factories) {
			var clone = ScriptableObject.Instantiate( factory );
			localMapVisualization.Factories.Add( clone );
		}
		localMapVisualization.Initialize( this );   // Our caches IFileSoure will be handled by this classes implementation MapControler.IFileSource.Request
		mapVisualization.Initialize(MapboxAccess.Instance);
       _tiles = new Dictionary<Vector2, UnityTile>();
    }

    public void Start()
    {
        Execute();
    }

    /// <summary>
    /// Pulls the root world object to origin for ease of use/view
    /// </summary>
    public void Update()
    {
        if (_snapYToZero)
        {
            var ray = new Ray(new Vector3(0, 1000, 0), Vector3.down);
            RaycastHit rayhit;
            if (Physics.Raycast(ray, out rayhit))
            {
                _root.transform.position = new Vector3(0, -rayhit.point.y, 0);
                _snapYToZero = false;
            }
        }
    }

    public void Execute()
    {
        var parm = LatLng.Split(',');
        Execute(double.Parse(parm[0]), double.Parse(parm[1]), Zoom, Range);
    }

	public void Execute(double lat, double lng, int zoom, Vector2 frame)
    {
        Execute(lat, lng, zoom, new Vector4(frame.x, frame.y, frame.x, frame.y));
    }

    public void Execute(double lat, double lng, int zoom, int range)
    {
        Execute(lat, lng, zoom, new Vector4(range, range, range, range));
    }

    /// <summary>
    /// World creation call used in the demos. Destroys and existing worlds and recreates another one. 
    /// </summary>
    /// <param name="lat">Latitude of the requested point</param>
    /// <param name="lng">Longitude of the requested point</param>
    /// <param name="zoom">Zoom/Detail level of the world</param>
    /// <param name="frame">Tiles to load around central tile in each direction; west-north-east-south</param>
    public void Execute(double lat, double lng, int zoom, Vector4 frame)
    {
        //frame goes left-top-right-bottom here
        if (_root != null)
        {
            foreach (Transform t in _root.transform)
            {
                Destroy(t.gameObject);
            }
        }

        _root = new GameObject("worldRoot");

        var v2 = Conversions.GeoToWorldPosition(lat, lng, new Vector2d(0, 0));
        var tms = Conversions.MetersToTile(v2, zoom);
        ReferenceTileRect = Conversions.TileBounds(tms, zoom);
        WorldScaleFactor = (float)(TileSize / ReferenceTileRect.Size.x);
        _root.transform.localScale = Vector3.one * WorldScaleFactor;

        for (int i = (int)(tms.x - frame.x); i <= (tms.x + frame.z); i++)
        {
			for (int j = (int)(tms.y - frame.y); j <= (tms.y + frame.w); j++) {
				var tile = new GameObject( "Tile - " + i + " | " + j ).AddComponent<UnityTile>();
				_tiles.Add( new Vector2( i, j ), tile );
				tile.Zoom = zoom;
				tile.RelativeScale = Conversions.GetTileScaleInMeters( 0, Zoom ) / Conversions.GetTileScaleInMeters( (float)lat, Zoom );
				tile.TileCoordinate = new Vector2( i, j );
				tile.Rect = Conversions.TileBounds( tile.TileCoordinate, zoom );
				tile.transform.position = new Vector3( (float)(tile.Rect.Center.x - ReferenceTileRect.Center.x), 0, (float)(tile.Rect.Center.y - ReferenceTileRect.Center.y) );
				tile.transform.SetParent( _root.transform, false );
				if (InCache( tile ))
					localMapVisualization.ShowTile( tile );
				else {
					tile.ImageDataChanged += Tile_ImageDataChanged;
					tile.HeightDataChanged += Tile_HeightDataChanged;
					mapVisualization.ShowTile( tile );
				}
			}
		}
    }

    /// <summary>
    /// Used for loading new tiles on the existing world. Unlike Execute function, doesn't destroy the existing ones.
    /// </summary>
    /// <param name="pos">Tile coordinates of the requested tile</param>
    /// <param name="zoom">Zoom/Detail level of the requested tile</param>
    public void Request(Vector2 pos, int zoom)
    {
        if (!_tiles.ContainsKey(pos))
        {
            var tile = new GameObject("Tile - " + pos.x + " | " + pos.y).AddComponent<UnityTile>();
            _tiles.Add(pos, tile);
            tile.transform.SetParent(_root.transform, false);
            tile.Zoom = zoom;
            tile.TileCoordinate = new Vector2(pos.x, pos.y);
            tile.Rect = Conversions.TileBounds(tile.TileCoordinate, zoom);
            tile.RelativeScale = Conversions.GetTileScaleInMeters(0, Zoom) / Conversions.GetTileScaleInMeters((float)Conversions.MetersToLatLon(tile.Rect.Center).x, Zoom);
            tile.transform.localPosition = new Vector3((float)(tile.Rect.Center.x - ReferenceTileRect.Center.x),
                                                       0,
                                                       (float)(tile.Rect.Center.y - ReferenceTileRect.Center.y));
			if (InCache( tile ))
				localMapVisualization.ShowTile( tile );
			else {
				tile.ImageDataChanged += Tile_ImageDataChanged;
				tile.HeightDataChanged += Tile_HeightDataChanged;
				tile.VectorDataChanged += Tile_VectorDataChanged;
				mapVisualization.ShowTile( tile );
			}
        }
		else {
			var tile = _tiles[ pos ];
			if (tile.ImageDataState == Enums.TilePropertyState.Error)
				Debug.Log( "Offline Tile" );	// Tile previously loaded when offline and data for it was not cached and couldn't be retieved from the internet
		}
    }


	// Cache Support ---------------------------------------

	private void Tile_HeightDataChanged( UnityTile sender, object param ) {
		CacheImage( sender, true );
	}

	private void Tile_ImageDataChanged( UnityTile sender, object param ) {
		// Bug: MapImageFactory.cs:83 assignement of tile.ImageData causes ImageDataChanged event to be fired before ImageData is loaded; fix this in your version of MapImageFactory.cs
		CacheImage( sender );
	}

	private void Tile_VectorDataChanged( UnityTile sender, object param ) {
		CacheVectors( sender ); // Never gets called in the current implementation of MeshFactory.CreateMeshes
								// VectorTile.data is uncompressed in VectorTile.ParseTileData. No vector data seems to be stored in the UnityTile.VectorData
								// To make this work, the compressed data needs to be stored in UnityTile.VectorData
	}

	// Implements IFileSource
	IAsyncRequest IFileSource.Request( string uri, Action<Response> callback ) {
		// uir format = https://api.mapbox.com/styles/v1/mapbox/satellite-v9/tiles/14/3402/6200
		// or			https://api.mapbox.com/v4/mapbox.terrain-rgb/14/3402/6200.pngraw
		// or			https://api.mapbox.com/v4/mapbox.mapbox-streets-v7/14/3402/6200.vector.pbf
		int lonStart = uri.LastIndexOf( '/' );
		int latStart = uri.Substring( 0, lonStart - 1 ).LastIndexOf( '/' );
		int scaleStart = uri.Substring( 0, latStart - 1 ).LastIndexOf( '/' );
		string localPath = cacheRoot + uri.Substring( scaleStart );
		if (uri.EndsWith( ".pngraw" ))
			localPath = localPath.Substring( 0, localPath.Length - 7 ) + @"/H.png";
		else if (uri.EndsWith( ".vector.pbf" )) {
			localPath = localPath.Substring( 0, localPath.Length - 11 ) + @"/V.pbf";
			//Debug.Log( "Vector cache unhandled: " + uri );
			return this;
		}
		else
			localPath += @"/I.jpg";

		using (var fs = File.Open( localPath, FileMode.Open )) {
			int fileLen = (int)fs.Length;
			byte[] image = new byte[ fileLen ];
			fs.BeginRead( image, 0, fileLen, ar => {
				fs.EndRead( ar );
				callback.Invoke( new Response() { Data = image } );
			}, callback );
		}
		return this;
	}

	void CacheImage( UnityTile tile, bool height = false ) {
		try {
			var localPath = CachedTilePath( tile );
			CreatePath( localPath );
			localPath += height ? @"/H.png" : @"/I.jpg";

			using (var fs = File.Open( localPath, FileMode.Create )) {
				var image = height ? tile.HeightData.EncodeToPNG() : tile.ImageData.EncodeToJPG();
				fs.BeginWrite( image, 0, image.Length, ar => {
					fs.EndWrite( ar );
				},  null );
			}
		}
		catch (Exception e) {
			Debug.Log( e.Message );
		}
	}

	void CacheVectors( UnityTile tile ) {
		try {
			var localPath = CachedTilePath( tile );
			CreatePath( localPath );
			localPath += @"/V.pbf";

			using (var fs = File.Open( localPath, FileMode.Create )) {
				string vectorsString = tile.VectorData;
				var vectors = System.Text.Encoding.ASCII.GetBytes( vectorsString );
				fs.BeginWrite( vectors, 0, vectors.Length, ar => {
					fs.EndWrite( ar );
				}, null );
			}
		}
		catch (Exception e) {
			Debug.Log( e.Message );
		}
	}

	bool InCache( UnityTile tile ) {
		return Directory.Exists( CachedTilePath( tile ) );
	}

	string CachedTilePath( UnityTile tile ) {
		var coords = tile.TileCoordinate;
		return cacheRoot + tile.Zoom.ToString() + @"/" + coords.x + @"/" + coords.y;
	}

	void CreatePath( string path ) {
		var parts = path.Split( new char[] { '/' } );
		string startPath = "";
		foreach (var part in parts ) {
			startPath += part;
			if (!Directory.Exists( startPath ))
				Directory.CreateDirectory( startPath );
			startPath += "/";
		}
	}

	// Implements IAsyncRequest
	void IAsyncRequest.Cancel() {
	}

}

}`

from mapbox-unity-sdk.

david-rhodes avatar david-rhodes commented on August 21, 2024

@derekdominoes Cool! We've recently added memory caching, and it's in a layer below MapController. This allows us to cache data regardless where the query comes from. Check MapboxAccess.cs in develop to see how it is implemented. We don't yet have disk caching, but the abstractions should easily enable it (and we'll be working on this soon).

One thing to consider with disk caching is how often you access the disk. You don't want to hit too frequently or your frame rate will suffer and you will drain more battery (on mobile).

from mapbox-unity-sdk.

david-rhodes avatar david-rhodes commented on August 21, 2024

Initial implementation of disk caching has been added! #125

from mapbox-unity-sdk.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.