We develop a Lagrangian model evaluation tool, assessing models by their ability to predict transport properties of the flow. The method is based on the comparison of coherent structure boundaries, given by effective invariant manifolds. The separation area and the average distance between the manifolds are used to quantify the accuracy with which the model manifolds represent the corresponding structures of the true flow. The manifolds accumulate dynamical information and are sensitive to the details of the (spatial and temporal) structure of the flow. We apply the manifold comparison technique to analyze the effects of finite resolution and uncertainty in the wind forcing on the Lagrangian structures and transport properties of a double gyre flow in a rectangular basin. The base flow (truth) is given by a solution of a single layer reduced gravity shallow water equations forced by a steady wind. It was modeled by projections of the velocity field on an incomplete basis of geometrical orthogonal functions (the size of the basis controlled the resolution), and by the solutions to the same set of equations but with the wind forcing amplitude modulated in time. The results of the Lagrangian evaluation showed that, in the resolution experiments, the manifold accuracy correlates with the average Eulerian error (L_2 norm of the velocity error field). However, for the models with modulated wind forcing, the Eulerian error was found to be an inadequate indicator of the quality of transport predictions.