.. _rfc44:

=======================================================================
MS RFC 44: Restore URL modification of mapfiles to pre-5.0 levels
=======================================================================

:Date: 2008/08/19
:Author: Steve Lime
:Contact: Steve.Lime at dnr.state.mn.us
:Last Edited: 2008/08/19
:Status: Adopted

:ref:`rfc31` introduced a new syntax for modifying :ref:`mapfiles <mapfile>` 
via URL. Object parameters could specified together in mapfile snippets making
it easier to make changes with far fewer characters. At the same time access 
to a number of parameters, particularly those that mapfile parsing did no 
value checks on (mostly strings) were removed for security purposes. 
In hindsight I underestimated the degree to which that functionality was used 
by developers. This RFC aims to restore that functionality albeit with 
security in mind.

Proposed Changes
----------------

Presently a few widely modified (and risky) parameters (e.g. layer TEMPLATE 
and DATA) can be changed via URL **if** a regular expression 
(e.g. TEMPLATEPATTERN and DATAPATTERN) is set to validate the incoming value. 
I propose using the same approach for all un-checked mapfile input. 
Parameters that represent numbers, colors or have a value domain 
(e.g. ON/OFF/DEFAULT) are subject to the same checks as when a mapfile is 
read from disk and as a result should be ok. Those that don't would require 
specific validation values be set before input would be allowed. For example, 
the LAYER VALIDATION block below defines patterns that would be used to 
validate DATA or FILTER parameter changes. If the appropriate validation key 
doesn't exist the value cannot change. 

Grouping all validation in a new VALIDATION block will ease use by 
simplifying key names to match MapServer token names. The block would be 
valid for :ref:`MAP`, :ref:`WEB`, :ref:`LAYER` and :ref:`CLASS` objects and 
its core type would be a hashTableObj. The MAP level VALIDATION
block would be useful for applying a pattern for all LAYERs or CLASSes (since
there is only 1 WEB object there is no need to rely on the MAP object). This
would save lots of duplication in cases where a mapfile contains similar
layers and the same data validation pattern applies to all. The logic would 
simply be: look for validation pattern in layer, if not found then
look for validation pattern at map level, if not found then no modifications
are allowed.

::

    VALIDATION
    data 'my pattern'
    filter 'another pattern'
    ...
    END
    
The validation would only be invoked if the token source is a URL. 
Mapfile file or string-based processing would be unaffected. An example of 
how this would work can be seen in mapfile.c near line 2683 with the 
DATA/DATAPATTERN parameters.

Files Affected
--------------

* maplexer.l: all parameters (a few will never be modifiable, like 
  VALIDATION) will be changed to be recognized in the URL_VARIABLE lexer 
  state; VALIDATION token needs to be added
* mapfile.h: add VALIDATION token
* mapfile.c: all non-value checked parameters will require regex validation 
  before changes will be allowed via URL; recognize validation token; 
  write validation hash with mapfile
* mapserv.c: update code for runtime substitution and qstring validation to 
  check the validation hash as well

A complete list of parameters affected will be attached to this document in the
post-implementation notes below.

Mapfile Changes 
-----------------

New VALIDATION token will be recognized.

MapScript Changes
-----------------

None. MapScript already has a general class for hashTableObj management.

Backwards Compatibility Issues
-------------------------------

The parameters DATAPATTERN and TEMPLATE pattern will become deprecated 
though. The objects in question (LAYER and CLASS) already contain
validation blocks that can be used for this. 

URL runtime substitution and qstring validation are currently supported 
through metadata, This would become deprecated as well. The runtime variables 
and the word "qstring" can be used as keys in the validation block instead.

Post-Implementation Notes
-------------------------

A HowTo will be developed that covers this topic and run-time substitutions.

Bug ID
------

https://github.com/mapserver/mapserver/issues/2699

Voting History
--------------

+1 Lime, Woodbridge, Morissette, Assefa

