Semi-streaming quantization for remote sensing data

We describe a strategy for reducing the size and complexity of very large, remote sensing datasets acquired from NASA's Earth Observing System. We apply the quantization paradigm from, and algorithms developed in, signal processing to the problem of summarization. Because data arrive in discrete chunks, we formulate a semi-streaming strategy that partially processes chunks as they become available and stores the results. At the end of the summary time period, we re-ingest the partial summaries and summarize them. We show that mean squared errors between the final summaries and the original data can be computed from the mean squared errors incurred at the two stages without directly accessing the original data. The procedure is demonstrated using data from JPL's Atmospheric Infrared Sounder.