Ad hoc testing of algorithms globally

Home > Blog > Ad hoc testing of algorithms globally
Prototyping pixel-based algorithm on EO Browser: a decision tree from A. Hollstein et al. (top left) is coded into a custom script (top right), and immediately visualized in the EO Browser (bottom).

The main goal of the Sentinel-Hub services is to facilitate the use of satellite imagery.

This seemingly straightforward objective in reality encompasses a number of very non-trivial steps: finding the appropriate images depending on the requested time range, cloud coverage etc., extracting the needed pixels from the massive amount of data, combining them to create something sensible like the true colour image, projecting the image to the required coordinate reference system and so on.

When implementing the step in which the user would “create something sensible”, we spent quite some time thinking about various wishes that the users might have and how to organize the parameters and processing steps best. We wanted to give our users as many options as possible to define what “something sensible” is. What we came up with is the so-called “custom scripting”.

By employing the dynamically interpreted JavaScript language, and providing some specialized functions, we have liberated the users from pre-defined EO product formulas and simple 3-band combinations, and gave them an unrestricted playground to combine the bands of multispectral satellite data in unprecedented ways.

Testing your (pixel based for now) algorithm has never been simpler, nor faster. Let's try it on an example.

A paper about cloud detection (A. Hollstein et al.: Ready-to-Use Methods for the Detection of Clouds, Cirrus, Snow, Shadow, Water and Clear Sky Pixels in Sentinel-2 MSI Images), shows a fairly complex classification decision tree (figure 8 in the article), based on derived feature space resulting in 91% of correctly classified spectra.

Figure 1: The decision tree from A. Hollstein et al: Ready-to-Use Methods for the Detection of Clouds, Cirrus, Snow, Shadow, Water and Clear Sky Pixels in Sentinel-2 MSI Images.

The solution includes only band math functions for selecting a single band (B), the difference between two bands (S), and the ratio of two bands (R), and can be written in JavaScript as

function S (a, b) { return a - b };
function R (a, b) { return a / b };
var naturalColour = [3*B04, 3*B03, 3*B02];
var CLEAR  = naturalColour; //[0.1,0.9,0.1];
var SHADOW = naturalColour; //[0.8,0.0,0.7];
var WATER  = [0.1,0.1,0.7];
var CIRRUS = [0.8,0.1,0.1];
var CLOUD  = [0.3,0.3,1.0];
var SNOW   = [1.0,0.8,0.4];

return (B03 < 0.319)
? (B8A < 0.166)
    ? (S(B03,B07) < 0.027)
        ? (S(B09,B11) < -0.097) ? CLEAR : SHADOW
        : (S(B09,B11) < 0.021) ? WATER : SHADOW
    : (R(B02,B10) < 14.689)
        ? (R(B02,B09) < 0.788) ? CLEAR : CIRRUS
        : CLEAR
: (R(B05,B11) < 4.33)
    ? (S(B11, B10) < 0.255)
        ? (S(B06, B07) < -0.016) ? CLOUD : CIRRUS
        : (B01 < 0.3) ? CLEAR : CLOUD
    : (B03 < 0.525)
        ? (R(B01, B05) < 1.184) ? CLEAR : SHADOW
        : SNOW;

Here the colours correspond to the colours from the article, apart from the colours for clear and shadow, which we made into natural (true) colour from red, green and blue bands.

Figure 2: Results of custom script for classification of snow, clouds, cirrus, water and shadows can be now inspected over all Sentinel-2 archive globally. Image shows Alps between Turin and Annecy on 3.1.2017. (view on Sentinel Playground)

Another interesting example deals with volcano eruptions (or fires). Visualization like the one in figure 3.

Figure 3: Vulcano eruption on Mount Etna (Image source on Twitter)

shows eruption very clearly, but the blue slopes of the Mount Etna distract the viewer. Tweaking the same bands into

return [
  B04*2.5 + Math.max(0,B12-0.1),
  B03*2.5 + Math.max(0,B11-0.1),
  B02*2.5];

where red and green bands are overlaid with short-wave infrared bands 11 and 12 make much more pleasing visualization, as shown in figure 4:

Figure 4: Etna volcano eruption, dated 16. 3. 2017. Image combined from true colour image, overlaid with SWIR bands 11 and 12. (view on Sentinel Playground)

Additional tweaking the gamma for snow and more gradients in the lava:

function s(v, min, max) {
  var a = Math.max(0,Math.min((v-min)/(max-min),1));
  return Math.sqrt(a);
}
return [
  0.8*s(B04,0.04,0.8) + s(B12,0.2,2),
  0.8*s(B03,0.04,0.8) + s(B11,0.2,5),
  0.8*s(B02,0.04,0.8)
];

finally results in eye-pleasing visualization:

Figure 5: Visually pleasing interpretation - bright snow is toned down using gamma - of Etna volcano eruption, dated 16.3.2017. Image combined from true colour image, overlaid with SWIR bands 11 and 12. (view on Sentinel Playground)

The third example is a bit more math oriented. We would like to visually interpret how the normalized density vegetation index (NDVI) is affected by the uncertainties in detector reflectances of the L1C products.

Since NDVI is defined as a ratio of difference over sum of bands 8 and 4 (near infrared and red):

the uncertainty propagation gives us the uncertainty of the index itself as

where ΔB4 and ΔB8 are uncertainties of red and near infrared bands respectively (reported by ESA to be 0.02 and 0.03). We left out the mixed part ΔB4B8 as if the two uncertainties were not correlated.

Implementing this in JavaScript function, together with some tweaking for nicer visualization, we get

function clamp (a) {
  return a<0 ? 0 : a > 1 ? 1 : a;
}
function sigNDVI (b4, s4, b8, s8) {
  var sum = b8 + b4;
  var ndvi = (b8 - b4)/sum;
  var s_ndvi = 2 / (sum*sum) *
      Math.sqrt(b8*b8*s4*s4+b4*b4*s8*s8);
  var darkness = clamp(1-2*s_ndvi);
  return [
    0.9*clamp(1-ndvi)*darkness,
    0.8*clamp(ndvi)*darkness,
    0.1*darkness];
}
return sigNDVI(B04, 0.02, B08, 0.03);

The resulting visualization represents NDVI uncertainty stemming from red and near infrared bands. The legend shows how the colour darkness characterizes the uncertainty: the darker the image the larger the error is.

It is clearly seen from the image how terrain affects the NDVI calculations – slopes and shadows carry much larger uncertainty with respect to flat areas. The image also illustrates how NDVI index has no meaning over water areas. The last image demonstrates the effect of cloud shadows on NDVI calculation.

Figure 6: Legend to images 7 and 8. The darker the colour, the higher the uncertainty.

Figure 7: Area near Lago di Garda, Italy. Visualization from custom script showcases how terrain effects NDVI uncertainty. (view on Sentinel Playground)

Figure 8: Area north-west from Cork, Ireland. Visualization from custom script showcases how cloud shadows effect NDVI uncertainty (view on Sentinel Playground)