.. index::
   pair: Optimization; Tile indexes

.. _tileindex:

*****************************************************************************
 Tile Indexes
*****************************************************************************

:Author:        HostGIS 
:Last Updated: 2013/07/04

.. contents:: Table of Contents
    :depth: 2
    :backlinks: top

Introduction
------------
An introduction to tileindexes, MapServer's method for doing on-the-fly mosaicing.

What is a tileindex and how do I make one?
------------------------------------------

A tileindex is a shapefile that ties together several datasets into a single
layer. Therefore, you don't need to create separate layers for each piece of
imagery or each county's road data; make a tileindex and let MapServer piece
the mosaic together on the fly.

Making a tileindex is easy using `gdaltindex`_ for GDAL data sources (rasters)
and `ogrtindex`_ for OGR data sources (vectors). You just run them, specifying
the index file to create and the list of data sources to add to the index.

.. index::
   single: gdaltindex

For example, to make a mosaic of several TIFFs:

::

    gdaltindex imagery.shp imagery/*.tif
    
.. index::
   single: ogrtindex

And to make a mosaic of vectors:

::

    ogrtindex strees.shp tiger/CA/*.shp tiger/NV/*.shp
    
.. note::
    ogrtindex and gdaltindex **add** the specified files to the index.
    Sometimes you'll have to delete the index file to avoid creating duplicate
    entries.

.. index::
   pair: Optimization; TILEINDEX

Using the tileindex in your mapfile
-----------------------------------

Using a tileindex as a layer is best explained by an example:

.. code-block:: mapfile

    LAYER
        NAME "Roads"
        STATUS ON
        TYPE LINE
        TILEINDEX "tiger/index.shp"
        TILEITEM "LOCATION"
    END 

There are two items of note here: :ref:`TILEINDEX <tileindex_keyword>` and
:ref:`TILEITEM <tileitem>`. TILEINDEX is simply
the path to the index file, and TILEITEM specifies the field in the shapefile
which contains the filenames referenced by the index. The TILEITEM will
usually be "LOCATION" unless you specified the *-tileindex* option when
running gdaltindex or ogrtindex.

Two important notes about the pathnames:
  
  * The path to TILEINDEX follows the same conventions as for any other data
    source, e.g. using the SHAPEPATH or else being relative to the location of
    the mapfile.
  * The filenames specified on the command line to gdaltindex or ogrtindex will
    be used with the same conventions as well, following the SHAPEPATH or else
    being relative to the mapfile's location. I find it very useful to change
    into the SHAPEPATH directory and then run ogrtindex/gdaltindex from there;
    this ensures that I specify the correct pathnames.
  
Tileindexes may make your map faster
------------------------------------

A tileindex is often a performance boost for two reasons: 
  
  * It's more efficient than having several separate layers. 
  * MapServer will examine the tileindex to determine which datasets fall into
    the map's view, and will open only those datasets. This can result in a
    great savings for a large dataset, especially for use cases where most of
    the time only a very small spatial region of the data is being used. (for
    example, citywide maps with nationwide street imagery)
  
A tileindex will not help in the case where all/most of the data sources will
usually be opened anyway (e.g. street data by county, showing states or larger
regions). In that case, it may even result in a decrease in performance, since
it may be slower to open 100 files than to open one giant file.

The ideal case for a tileindex is when the most typically requested map areas
fall into very few tiles. For example, if you're showing state and larger
regions, try fitting your data into state-sized blocks instead of county-sized
blocks; and if you're showing cities and counties, go for county-sized blocks.

You'll just have to experiment with it and see what works best for your use
cases.

.. _tileindex_mixed_srs:

Tileindexes with tiles in different projections
-----------------------------------------------

Starting with MapServer 6.4 for raster layers, and MapServer 7.2 for vector
layers, a tileindex can contain rasters/layers in
different projections. Such tileindex can be generated with `gdaltindex`_ (GDAL
1.11 or later) and `ogrtindex`_ (GDAL 2.2 or later), with the -t_srs and
-src_srs_name options. The -t_srs instructs
gdaltindex/ogrtindex to write the envelope of each tile into a common target
projection, so that the geometries written in the tile index are consistent.
This common projection must be the projection of the raster/vector layer.

::

    gdaltindex -t_srs EPSG:4326 -src_srs_name src_srs imagery.shp imagery/*.tif

The corresponding LAYER definition will need to specify the
:ref:`TILESRS <tilesrs>` keyword with the value of the -src_srs_name option.

e.g:

.. code-block:: mapfile

    LAYER
        NAME "My Imagery"
        STATUS ON
        TYPE RASTER
        TILEINDEX "imagery.shp"
        TILEITEM "LOCATION"
        TILESRS "src_srs"
        PROJECTION
            AUTO
            # or :
            # "+init=EPSG:4326"
        END
    END 

MapServer will then be able to proceed to on-the-fly mosaicing and reprojection.

For layers that must be exposed as WCS layers, a few metadata fields
("wcs_extent", "wcs_size", "wcs_resolution") must be specified in the LAYER
definition, so as to define a "virtual dataset" coverage (see :ref:`wcs_server`).
The GDAL wcs_virtds_params.py sample script can help generating those metadata
fields.

Note: this support of tileindex with mixed projections is only available for
raster layers for now.

.. _`gdaltindex`: http://www.gdal.org/gdaltindex.html
.. _`ogrtindex`: http://www.gdal.org/ogrtindex.html
