| Home | Products | Issue Tracker | Service Providers | FAQ | Twitter | Download

# MS RFC 95: Support for lists in expression parser

Date

  2013/04/04

Author

  Thomas Bonfort

Contact

  thomas.bonfort@gmail.com

Status

  Adopted

Version

  MapServer 6.4

## 1. Overview

We have a relatively low hanging performance speedup related to expressions when doing class filtering. The context relates to when there’s the need to apply a given CLASS to multiple attribute values.

Currently, to apply a single class to multiple values of an attribute, you can either use regular expressions, or use the `IN` operator, like this:

    LAYER
      ...
      CLASSITEM "type"
      CLASS
        EXPRESSION /primary|secondary|tertiary/ #regular expression on CLASSITEM
        EXPRESSION ("[type]" IN "primary,secondary,tertiary") #"complex" parser expression
        ...
      END
    END

Both methods require quite a bit of overhead, either in the regex system calls, or by using quite a few mallocs when going through the IN parser operation.

## 2. Proposed solution

We can cut down on this overhead by adding a “list” expression type, denoted by the `{ }` delimiters, where the previous layer definition becomes:

    LAYER
      ...
      CLASSITEM "type"
      CLASS
        EXPRESSION {primary,secondary,tertiary}
        ...
      END
    END

In that case, the expression parsing is a simple string lookup inside `msEvalExpression()` instead of having to go down into the system regex or the parser.

## 3. Implementation Details

The modifications to the code are fairly straightforward:

- Support for the {} delimiters in the lexer (maplexer.l)

- New expression type `MS_LIST` in mapserver.h

- support for finding an attribute value in the given list in `msEvalExpression()` in maputil.c

- Loading, saving and copying the expression in mapfile.c and mapcopy.c

## 4. Limitations

- setters and getters will not be initially added for mapscript (use `classObj.updateFromString("EXPRESSION {foo,bar,baz}")`

- Quoting and escaping inside the list is not supported, i.e. attribute values cnanot contain the `'` , `"` or `,` characters.

## 4. Backwards Compatibility Issues

None, new functionality

## 5. Performance implications

The expression parser had already been sped up in 6.0 with the use of precompiled expressions, this goes one step further for the specific case where a given CLASS should match multiple attribute values

## 6. Bug ID

https://github.com/mapserver/mapserver/issues/4648

## 7. Voting history

+1: MikeS, TomK, DanielM, SteveW, ThomasB