The Center for Operational Oceanographic Products and Services (CO-OPS) of the National Ocean Service (NOS)/NOAA routinely runs an Operational Forecast System (OFS) for the Chesapeake Bay four times daily. This model is based on a three-dimensional baroclinic implementation of Rutgers University's Regional Ocean Modeling System (ROMS). The OFS was developed and evaluated using a non-operational, research counterpart. Since temperature (T) and salinity (S) variables predicted by this model are useful for physically-forced ecological applications in the Chesapeake Bay, an examination of: (i) the sensitivity of the T and S predictions (including stratification) to various model configuration options, (ii) whether these predictions could be further refined and improved relative to observations and, (iii) whether a meaningful set of model-observations performance metrics could be designed and tested, was carried out. The study involved performing a matrix of model runs to vary the river forcing volume discharge and salinity values, the vertical terrain-following sigma grid prescriptions and the vertical advection schemes for tracers in ROMS. All of these test simulations employed the research version of the Chesapeake Bay model and covered the 2004 and 2005 calendar years. Thereafter, the best (relative to observations) temperatures and salinities from this model and several other Chesapeake Bay models (two ROMS-based models, one Environmental Fluid Dynamics Code (EFDC)-based model and one Curvilinear-grid Hydrodynamics model in three dimensions (CH3D)-based model) were compared with each other and with observed data. All of these comparisons were carried out as part of the NOAA/US-Integrated Ocean Observing System (IOOS) modeling testbed Estuarine Hypoxia group agenda. This presentation will provide a summary of the above mentioned numerical experimentation, its findings and the various comparisons and some potential recommendations for physically forced ecological simulations in the Chesapeake Bay.