Methods for Assessing the Quality and Consistency
of Ocean Color Product s

Bryan Franz (SAIC)
Ocean Biology Processing Group
18 January 2005

Introduction

This document provides details on several of the standard methods used by the Ocean Biology Discipline Processing Group (OBPG) at NASA/GSFC to evaluate the oceanic optical properties derived from spaceborne ocean color sensors. Many of these analyses are performed routinely for standard products, but they are also used to evaluate changes in processing algorithms or calibration. The analyses serve to verify the implementation of proposed changes and to provide quantitative feedback as to the impact of those changes on field-data comparisons, sensor-to-sensor agreement, temporal and spatial stability in derived product retrievals, and long-term sensor stability.

Although not discussed here, any evaluation of instrument calibration or processing algorithm changes is normally preceded by a re-evaluation of the vicarious calibration (Eplee, 2003). This effectively removes any bias on the mission-mean normalized water-leaving radiance retrievals at the MOBY vicarious calibration site. When comparing products from different sensors, any algorithm changes that are applicable to both sensors are applied equally, and both sensors are vicariously recalibrated at MOBY.

The plots and images shown in this document come from various processing and testing events. They are provided as examples only, and thus they do not reflect the current state of product quality. This document is intended to describe the analysis methods. The analysis results are posted elsewhere.

I. Comparison with in situ Observations

The primary mechanism for assessing the quality of retrieved ocean color properties is through comparison with ground-truth measurements. A description of the in situ match-up process and current operational results is available here. It should be recognized, however, that the temporal and geographic distribution of the in situ dataset is limited. These match-ups are generally not sufficient for assessing the quality of satellite remote sensed ocean color data over the full range of geometries through which the spaceborne sensor views the earth, or over the full temporal and geographic distribution of the Level-3 products, nor do they account for the effects of temporal and spatial averaging or systematic errors associated with Level-3 masking decisions.

II. Temporal Trending and Annual Repeatability

This analysis looks at long-term trends in Level-3 products and the consistency of those trends from year to year. It provides a standard mechanism for evaluating derived product and sensor stability, and it quantifies the relative impact of calibration and algorithm changes on global spatial scales and life-of-mission time scales. The approach begins with standard Level-3 products. These products are global binned, multi-day averages at 4.6 or 9-km resolution, with bins distributed in an equal-area, integerized sinusoidal projection (Campbell et al., 1995). The typical composite period is 8-days, but for quick turn-around test processing the OBPG uses a temporal subset of the mission lifespan consististing of 4-day composites generated from the start of each consecutive 32-day period (i.e., 12.5% of the mission dataset). The temporal subset is generated at 9-km resolution, and it can be processed within 1-day. The 4-day compositing period generally provides sufficient opportunity to observe most of the daylit side of the earth, including coverage in orbit and glint gaps.

From these global, multi-day composites, a subset of the filled bins is selected and standard ocean products are averaged and trended with time. The analysis is focused on the trends in normalized water-leaving radiances (nLw), but trends in chlorophyll and atmospheric products are also evaluated. For bin selection and averaging, three global subsets are defined, corresponding to clear water, deep water, and coastal water. The deep water subset consists of all bins where water depth is greater than 1000 meters. Clear water is defined as deep water where the retrieved chlorophyll is less than 0.15 mg/m^3. Coastal water is defined as all bins where water depth is between 30 and 1000 meters, as defined by a shallow water mask and the deep water mask.

Figure 2: SeaWiFS 5-Year Annual Cycle, Clear and Deep-Water

A good example of this analysis is the SeaWiFS 5-year annual cycle for nLw shown above. In the absense of any major geophysical events, we expect the trend in global deep-water or global clear-water nLw to repeat from year to year. Low-level differences may be due to geographic sampling biases or real geophysical changes, but on the large-scale these plots tell us that SeaWiFS products are self-consistent over time (i.e., there is no long-term drift).

III. Sensor-to-Sensor Temporal Comparisons

This analysis looks at average values of sensor-to-sensor coincident retrievals on global and regional spatial scales, and presents the results as a comparative time-series over the common mission lifespan. The goal is to produce a statistically rigorous comparison of equivalent ocean color products, to provide a quantitative assessment to the users of these datasets as to their relative agreement, and to evaluate the impact of calibration and algorithm changes on sensor-to-sensor consistency at global spatial scales and life-of-mission time scales. The ocean color products compared are the standard chlorophyll products derived from each mission data set, as well as the normalized water-leaving radiances in the four closest visible wavelengths. The equivalent wavelengths are listed in Table 1. The chlorophyll products compared are the standard "chlor_a" products produced for each sensor, which are all emperical max-band-ratio algorithms developed by O'Reilly (O'Reilly et al., 2000). Additional insight into the atmospheric correction performance is gained by comparing aerosol optical thickness (AOT) retrievals and single-scattering epsilon of the near IR band pair (Gordon & Wang, 1994).

Band	SeaWiFS	MODIS	OCTS
nLw 1	412	412	412
nLw 2	443	443	443
nLw 3	490	488	490
nLw 4	510	531	520
nLw 5	555	551	565
nLw 6	670	667 & 678	670
Chlor_a	OC4v4	OC3M	OC4O
AOT	865	870	865
Epsilon	765/865	750/870	765/865

Table 1: Band Correspondence (nm)

The analysis begins with standard Level-3 products composited over a common time period (usually 4 or 8 days). All OBPG Level-3 ocean color products use the same, equal area binning approach (Cambell, et. al), but standard MODIS products are distributed at 4.6-km resolution while SeaWiFS is distributed at 9-km resolution. To allow for a direct, bin-for-bin comparison, the MODIS products are rebinned to the SeaWiFS 9-km resolution using standard binning algorithms. For quick turn-around test processing, the OBPG processes a temporal subset of the common mission lifespan. Again, the temporal subset consists of 4-day composites generated from the start of each consecutive 32-day period, generated at 9-km resolution.

With Level-3 composited data products in an equivalent form, the datasets are further reduced to a set of common bins. This means that only those bins for which a retrieval exists for both sensors are included in subsequent averaging and trending. This is critical to the statistics, as some sensors show systematic data gaps even after 8-days of compositing, and this can result in geographic sampling bias if both sensors are not equivalently masked.

Finally, with the products in common bin form, the data are divided into geographic subsets for averaging and trending. The subsets include the global deep, clear, and coastal-water subsets described in Section II, as well as a set of standard regions and a set of latitudinally distributed zones. When comparing the clear-water subsetted data, it should be noted that anomalously high chlorophyll retrievals from either sensor can significantly alter the geographic distribution of selected bins. In contrast, the deep-water and coastal subsets are purely geographic in selection criteria. The coastal subset, however, is more likely to contain regions of significant variability in water structure and atmospheric conditions, as well as case 2 water types, all of which can lead to greater retrieval uncertainty and larger differences between the two sensors. The deep-water subset is, therefore, the most stable subset for cross-sensor comparison of retrieved oceanic optical properties. The geographic extent of all three global subsets will vary, however, with the seasonal change in earth illumination and thus sensor imaging duty cycle.

retrievals Note that no effort is made to force the sensor retrievals to a common bandpass. Some level of difference is expected whan comparing the nLw retrievals between two sensors, particularly in the case where the nominal center wavelengths are not identical. A discussion of inherent differences between SeaWiFS and MODIS is available here. And a discussion of differences in the chlorophyll algorithms can be found here.

Figure 3 presents a sample pair of MODIS and SeaWiFS deep water subsetted chlorophyll images for one 8-day period in May of 2003, after mapping to the more familiar platte carre projection. The images show the geographic extent of the common-binned, deep-water subset, and they provide some insight into the qualitative agreement between the two sensors.

SeaWiFS

MODIS

Log10(Chla), 0.01 - 1.0 mg/m^3

Figure 3: Sample Chlorophyll Images, Deep-Water Subset Days 137-144 of 2003

The regional and zonal subsets are generally visible at all times of year. The regions were chosen for relative homogeneity (Fougnie, et al., 2002), and they are all in relatively clear water. A region is also included at Hawaii, to verify performance at the point of vicarious calibration (Eplee et al., 2003), where calibration biases should be minimal. The regions are described in Table 2 and shown graphically in Figure 4. The zonal subsets were added to provide a systematic means for investigating latitudinally-dependent differences between the two sensors. These are shown in Table 3 and Figure 5.

Region ID	Minimum Latitude	Maximum Latitude	Minimum Longitude	Maximum Longitude
Hawaii	18.0	19.9	-158.5	-156.5
PacN	15.0	23.0	-180.0	-159.4
PacNW	10.0	22.7	139.5	165.6
PacSE	-44.9	-20.7	-130.2	-89.0
AtlN	17.0	27.0	-62.5	-44.2
AtlS	-19.9	-9.9	-32.3	-11.0
IndS	-29.9	-21.2	89.5	100.1

Table 2: Regional Subset Definitions

Figure 4: Distribution of Regional Subsets

Region ID	Minimum Latitude	Maximum Latitude	Minimum Longitude	Maximum Longitude
PacN50	40.0	50.0	-170.0	-150.0
PacN40	30.0	40.0	-170.0	-150.0
PacN30	20.0	30.0	-170.0	-150.0
PacN20	10.0	20.0	-170.0	-150.0
PacN10	0.0	10.0	-170.0	-150.0
PacS10	-10.0	0.0	-170.0	-150.0
PacS20	-20.0	-10.0	-170.0	-150.0
PacS30	-30.0	-20.0	-170.0	-150.0
PacS40	-40.0	-30.0	-170.0	-150.0
PacS50	-50.0	-40.0	-170.0	-150.0

Table 3: Zonal Subset Definitions

Figure 5: Distribution of Zonal Subsets

For each sensor Level-3 product, the common, filled bins associated with a particular subset are identified and used to compute the mean, standard deviation, and average observation time. Figure 6 shows an example of a typical trend plot derived from this analysis. For the plot on the left, the common MODIS and SeaWiFS bins for the deep-water subset were spatially averaged for each 8-day-binned water-leaving radiance product, and the resulting means were then plotted as a function of time. MODIS is shown as dashed lines. The colors indicate different bands, as summarized in Table 1. The plot on the right shows the same data as a ratio, with MODIS means normalized by SeaWiFS means.

Figure 6: SeaWiFS and MODIS/Aqua Water-Leaving Radiance Comparison, Deep-Water Subset

Taken alone, the comparitive temporal analysis can not be used to determine absolute error, since relative differences may be due to errors in either dataset or real geophysical effects which are not yet understood. However, when taken in concert with the self consistency analyses described in section II and the in situ comparisons of Section I, the sensor-to-sensor comparisons can serve to identify and isolate the likely cause for differences. An example of this is Figure 7, which shows results for the PacN50 zonal subset for two test cases. The plot on the left is before a correction was made to the MODIS/Aqua polarization sensitivity, while the plot on the right is after the correction.

Figure 7: MODIS/Aqua Water-Leaving Radiance Ratio to SeaWiFS, PacN50 Zonal Subset

IV. Latitudinal Trend Comparisons

In addition to the global and regional temporal trends, it has been found useful to look at the mean latitudinal distribution of derived products between sensor. This analysis begins with the global deep or clear-water common-bin datasets described in Section III. The deep-water or clear-water common bins from a pair of 4-day, 8-day, monthly, or seasonal Level-3 products are divided into 5-deg-latitude by 360-deg-longitude zones, which are then averaged and plotted as a function of zone-center latitude. As an example, Figure 8 shows the results from a pair of consecutive MODIS/Aqua 4-day test runs, which were defined to evaluate the effect of the Morel f/Q correction (Morel et al., 2001). The plot shows the ratio of nLw with and without f/Q applied, for the 4-day period beginning on day 225 of 2002.

Figure 8: MODIS/Aqua Latitude Trends, Deep-Water Subset, Effect of f/Q

Note that in the example shown in Figure 8, the common bins are for the same sensor. The reduction to common bins is still neccessary when evaluating changes within sensor, because any change in calibration or algorithm can reduce or increase retrievals, and we want to minimize extraneous differences when evaluating such changes.

V. Cross-scan and Detector Dependent Residuals

This analysis seeks to quantify and track changes in residual cross-scan trends and, in the case of MODIS and OCTS, detector-to-detector relative differences (i.e., striping). The approach takes advantage of the fact that such Level-2 residual errors will tend to average-out over time and space. Software was developed to generate match-ups between Level-2 observations and Level-3 bins, where the Level-3 product is typically a 7-day mean at 9-km resolution, temporally centered on the date of the Level-2 granule. The software gathers all relevant information relating to the match-up, including scan-pixel, detector number, and mirror side of the Level-2 observation. Match-ups for all granules collected over a complete day are screened, and those cases corresponding to deep, clear water (chlorophyll < 0.15 mg/m^3) with minimal glint contamination are accepted. Standard binner masking is also employed, with the object being to obtain a large number of Level-2 to Level-3 match-ups from homogeneous, temporally stable waters, where the Level-3 retrieval is likely to be a good representation of what the Level-2 retrieval should be. The Level-2 to Level-3 ratios for each derived products can then be averaged within scan-pixel or detector number. Figure 9 shows an example of cross-scan trends derived in this manner. The case shown is nLw at 443 and 667 nm for MODIS/Aqua, from a recent test processing.

Figure 9: Example of Scan-Dependence, MODIS/Aqua Test, 2003 173

The dotted line in Figure 9 is the standard deviation within each scan-pixel mean (the dotted line is off-scale for the 667-nm case). As another example, Figure 10 shows the same data as a function of detector number.

Figure 10: Example of Detector-Dependence, MODIS/Aqua Test, 2003 173

Note that uncertainty in 667 is very high, since we are looking at ratios of small numbers; however, the general patterns seen in these plots have been found to be relatively consistent over time. These analyses are typically done for 6 dates covering start and end of mission and one set of solstice and equinox occurrences.

References

Campbell, J.W., J.M. Blaisdell, and M. Darzi, 1995: Level-3 SeaWiFS Data Products: Spatial and Temporal Binning Algorithms. NASA Tech. Memo. 104566, Vol. 32, S.B. Hooker, E.R. Firestone, and J.G. Acker, Eds., NASA Goddard Space Flight Center, Greenbelt, Maryland

Gordon, H.R. & M. Wang, "Retrieval of water-leaving radiance and aerosol optical thickness over the oceans with SeaWiFS: a preliminary algorithm," Appl. Opt., vol. 33, pp. 443-452, 1994

Eplee, R.E., Jr., R.A. Barnes, S.W. Bailey, and P.J. Werdell, 2003: "Changes to the vicarious calibration of SeaWiFS." In: Patt, F.S., R.A. Barnes, R.E. Eplee, Jr., B.A. Franz, W.D. Robinson, G.C. Feldman, S.W. Bailey, P.J. Werdell, R. Frouin, R.P. Stumpf, R.A. Arnone, R.W. Gould, Jr., P.M. Martinolich, and V. Ransibrahmanakul, Algorithm Updates for the Fourth SeaWiFS Data Reprocessing, NASA Tech. Memo. 2003--206892, Vol. 22, S.B. Hooker and E.R. Firestone, Eds., NASA Goddard Space Flight Center, Greenbelt, Maryland, (in press).

Fougnie, B., P. Henry, A. Morel, D. Antoine, and F. Montagner, 2002: Identification and Characterization of Stable Homogeneous Oceanic Zones: Climatology and Impact on In-Flight Calibration of Space Sensors over Rayleigh Scattering. Ocean Optics XVI, Santa Fe, NM, November 18-22, 2002.

Morel, A., D. Antoine, and B. Gentilli, 2002: Bidirectional reflectance of oceanic waters: accounting for Raman emission and varying particle scattering phase function. Appl. Opt., 41, 6289-6306

O'Reilly, J., S. Maritorena, M. O'Brien, D. Siegel, D. Toole, D. Menzies, R. Smith, J. Mueller, B. Mitchell, M. Kahru, F. CHavez, P. Strutton, G. Cota, S. Hooker, C. McClain, K. Carder, F. Muller-Karger, L. Harding, A. Magnuson, D. Phinney, G. Moore, J. Aiken, K. Arrigo, R. Letelier and M. Culver (2000). SeaWiFS postlaunch technical report series, Volume 11, SeaWiFS postlaunch calibration and validation analyses, Part 3, NASA Technical Memorandum.

Bryan A. Franz

Curator: OceanColor Webmaster

Authorized by: gene carl feldman

Methods for Assessing the Quality and Consistency of Ocean Color Product s