Evaluation of coastal area modelling systems at an estuary mouth

Most numerical models are run and compared to data in a subjective manner. This paper demonstrates how model performance statistics can be used to calibrate and/or validate hydrodynamic models in a more objective way. Statistics were also used to compare model runs that used different amounts of field data in order to inform the debate about the optimum mix of modelling and measurement. The hydrodynamics around the mouth of the Teign estuary (UK) have been simulated using two coastal area numerical modelling systems. Model performance statistics were calculated to assess the accuracy of the predictions of the measured currents at a number of locations around the estuary mouth. The relative mean absolute error was used as it is applicable to vectors as well as scalars and measures all types of errors. An adjusted relative mean absolute error was also used to reduce the effect of measurement error. A classification table was adopted that categorises the results according to the size of the error. In addition, time series and scatter plots were used to judge the performance of the modelling systems. Calm conditions during a spring tide were simulated, as was a relatively large storm. The two modelling systems gave more or less equal performances when run in engineering mode (where default values were used for most of the system settings). In each case, one modelling system performed better than the other at some locations and worse than it at other locations. One model was also run using a scientific approach, where different amounts of information were used to alter the model settings and sensitivity tests were performed. The model performance statistics showed that using more data does not necessarily lead to better model predictions. New methods for incorporating data into the operation of a model need to be evaluated thoroughly before they can be used without site-specific calibration.