| Home | Products | Issue Tracker | Service Providers | FAQ | Twitter | Download

# Virtual File System

Author

  Jeff McKenna

Contact

  jmckenna at gatewaygeomatics.com

Last Updated

  2016-06-23

Table of Contents

- Virtual File System

  - Virtual File System

    - More Information on Virtual File System

    - Display Compressed File Directly in MapServer

      - Step 1: Verify Format is “Virtual” Enabled

      - Step 2: Test Access through ogrinfo/gdalinfo

      - Step 3: Configure MapServer Layer

      - Step 4: Test your Mapfile

    - Display Remote Compressed File Directly in MapServer

      - Test Access through ogrinfo/gdalinfo

      - Configure MapServer Layer

    - Display Remote (RASTER) File Directly in MapServer

      - Test Access through gdalinfo

      - Configure MapServer Layer

## Virtual File System

Starting with GDAL version 1.8.0, it is possible to read and display “virtual” VECTOR and RASTER files, such as to a remote file by pointing to a URL, or a compressed local .zip file that contains an OGR data source, and display that in MapServer. Useful mapfile parameters include /vsizip/, /vsigzip/, /vsitar/, /vsicurl/

### More Information on Virtual File System

- Blog post by initial developer Even Rouault

- GDAL Trac page

### Display Compressed File Directly in MapServer

The following example uses a compressed shapefile, “test.zip”, that contains 3 files:

    test.shp
    test.dbf
    test.zip

#### Step 1: Verify Format is “Virtual” Enabled

Execute ogrinfo –formats (for vector) or gdalinfo –formats (for raster) to check if the format lists a “v” beside the driver name, such as:

    > ogrinfo --formats

    Supported Formats:
      PCIDSK -raster,vector- (rw+v): PCIDSK Database File
      netCDF -raster,vector- (rw+s): Network Common Data Format
      PDF -raster,vector- (rw+vs): Geospatial PDF
      DB2ODBC -raster,vector- (rw+): IBM DB2 Spatial Database
      ESRI Shapefile -vector- (rw+v): ESRI Shapefile
      MapInfo File -vector- (rw+v): MapInfo File
      ...

The “(rw+v)” beside shapefile indicates that virtual files are supported.

#### Step 2: Test Access through ogrinfo/gdalinfo

Use ogrinfo with /vsizip/ to summarize the features of the zipped shapefile. Follow the OGR document examples for how to use the ogrinfo command with layers, such as:

    > ogrinfo /vsizip/test.zip/test.shp test -summary

Note that “test.zip” exists in the same directory where we are running the ogrinfo command.

    INFO: Open of `/vsizip/test.zip/test.shp'
          using driver `ESRI Shapefile' successful.

    Layer name: test
    Metadata:
      DBF_DATE_LAST_UPDATE=2000-03-17
    Geometry: Polygon
    Feature Count: 665
    Extent: (-2750564.750000, -936638.500000) - (3583872.500000, 4673125.000000)
    Layer SRS WKT:
    (unknown)
    AREA: Real (15.3)
    PERIMETER: Real (15.3)
    ...

#### Step 3: Configure MapServer Layer

Once you have verified that you can read the features with ogrinfo (or gdalinfo for raster), then we can try creating a map in MapServer.

Warning

A full (absolute) path is required for the CONNECTION parameter, for these virtual layers.

    LAYER
      NAME "test"
      TYPE POLYGON
      STATUS ON
      CONNECTIONTYPE OGR
      CONNECTION "/vsizip/C:/ms4w/apps/test/data/test.zip/test.shp"
      DATA "test"
      CLASS
        NAME "test"
        STYLE
          COLOR 240 240 240
          OUTLINECOLOR 199 199 199
        END
      END
    END # layer

Note

You can omit the filename from the connection (test.shp) if there are no other files inside the zip than expected for a shapefile. When you omit it, the zip is recognized as a directory and it is the same logic as executing ogrinfo on a directory.

#### Step 4: Test your Mapfile

Verify that MapServer can generate a map image by using the shp2img utility:

    shp2img -m mymapfile.map -o test.png -all_debug 5

    msDrawMap(): rendering using outputformat named png (AGG/PNG).
    msDrawMap(): WMS/WFS set-up and query, 0.000s
    GDAL: GDALOpen(/vsizip/C:/ms4w/apps/test/data/test.zip/test.shp, this=07D691D8) succeeds as ESRI Shapefile.
    Shape: 492 features read on layer 'test'.
    GDAL: GDALClose(/vsizip/C:/ms4w/apps/test/data/test.zip/test.shp, this=07D691D8)
    msDrawMap(): Layer 0 (test), 0.016s
    msDrawMap(): Drawing Label Cache, 0.000s
    msDrawMap() total time: 0.031s
    msSaveImage(ttt.png) total time: 0.000s
    msFreeMap(): freeing map at 06289688.

### Display Remote Compressed File Directly in MapServer

We can also display an external compressed file, that is accessible through HTTP, using /vsicurl/ with /vsizip/

#### Test Access through ogrinfo/gdalinfo

    > ogrinfo -ro /vsizip/vsicurl/http://download.osgeo.org/mapserver/docs/vsicurl-test.zip/test.shp test -summary

#### Configure MapServer Layer

    LAYER
      NAME "test"
      TYPE POLYGON
      STATUS ON
      CONNECTIONTYPE OGR
      CONNECTION "/vsizip/vsicurl/http://download.osgeo.org/mapserver/docs/vsicurl-test.zip/test.shp"
      DATA "test"
      CLASS
        NAME "test"
        STYLE
          COLOR 240 240 240
          OUTLINECOLOR 199 199 199
        END
      END
    END # layer

Warning

Virtual layers have a “cost” / are not as fast to display in MapServer as regular layers.

### Display Remote (RASTER) File Directly in MapServer

Remote raster files can also be accessed through HTTP, using /vsicurl/

#### Test Access through gdalinfo

    > gdalinfo /vsicurl/http://download.osgeo.org/gdal/data/gtiff/small_world.tif

    Driver: GTiff/GeoTIFF
    Files: /vsicurl/http://download.osgeo.org/gdal/data/gtiff/small_world.tif
    Size is 400, 200
    Coordinate System is:
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4326"]]
    Origin = (-180.000000000000000,90.000000000000000)
    Pixel Size = (0.900000000000000,-0.900000000000000)
    ...

#### Configure MapServer Layer

    LAYER
      NAME "test"
      TYPE RASTER
      STATUS ON
      DATA "/vsicurl/http://download.osgeo.org/gdal/data/gtiff/small_world.tif"
      CLASS
        NAME "test"
      END
    END # layer