Downloads the odc-colab Python module and runs it to setup ODC.
!wget -nc https://raw.githubusercontent.com/ceos-seo/odc-colab/master/odc_colab.py
from odc_colab import odc_colab_init
odc_colab_init(install_odc_gee=True)
Downloads an existing index and populates the new ODC environment with it.
from odc_colab import populate_db
populate_db()
This notebook demonstrates the Australian Water Observations from Space (WOFS) algorithm. This water detection algorithm is an improvement over the Landsat QA water flag or the Normalized Difference Water Index (NDWI) for water identification. For more information, visit this website: http://www.ga.gov.au/scientific-topics/hazards/flood/wofs
This notebook uses global Landsat-8 data from Google Earth Engine which has been indexed to work with the Open Data Cube. The selected data is used to create a time series water product that reflects the percent of time each pixel has been water over the entire time window. If a pixel is cloudy, it not considered in the analysis. Users should review the "Cloud Statistics" notebook for more information about the temporal cloud cover for a given region.
This baseline notebook runs in about 4 minutes after the Google authorization step is complete. The default region (0.10 degrees square) and time window (3 years) uses about 15% of the allocated RAM memory. Selecting larger regions and time windows should be done carefully to avoid exceeding the system limits or having execution times.
# Suppress Warning Messages
import warnings
warnings.filterwarnings('ignore')
# Load Data Cube Configuration
from odc_gee import earthengine
dc = earthengine.Datacube(app='Water_Observations_from_Space')
# Import Data Cube API
import utils.data_cube_utilities.data_access_api as dc_api
api = dc_api.DataAccessApi()
# Import Utilities
import datetime
import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
import pandas as pd
# Define the Product and Platform
# This data is indexed from Google Earth Engine data sources
product = "ls8_google"
platform = "LANDSAT_8"
# MODIFY HERE
# This is the primary code block for user modification
# Select a Latitude-Longitude point for the center of the analysis region
# Select the size of the box (in degrees) surrounding the center point
# Mombasa, Kenya
lat_long = (-4.00, 39.65)
box_size_deg = 0.10
# Calculate the latitude and longitude bounds of the analysis box
latitude = (lat_long[0]-box_size_deg/2, lat_long[0]+box_size_deg/2)
longitude = (lat_long[1]-box_size_deg/2, lat_long[1]+box_size_deg/2)
# Select a time range
# The inputs require a format (Min,Max) using this date format (YYYY-MM-DD)
# The Landsat-8 allowable time range is: 2013-04-07 to current
# NOTE: It is common to use a long time window for time series water analyses. Users may also desire to look
# at annual products or seasonal products to see water extent variations for those shorter time windows.
# The baseline analysis uses 3 years to minimize run time
time_extents = ('2018-01-01', '2020-12-31')
# The code below renders a map that can be used to view the region.
# It is possible to find new regions using the map below.
# Use your mouse to zoom in/out to explore new regions
# Click on the map to view Lat-Lon coordinates of any location that could define the region boundary
from utils.data_cube_utilities.dc_display_map import display_map
display_map(latitude,longitude)
landsat_dataset = dc.load(latitude=latitude,longitude=longitude,platform=platform,time=time_extents,
product=product,measurements=['red', 'green', 'blue', 'nir', 'swir1', 'swir2', 'pixel_qa'])
# View a sample of the loaded dataset (this is an XARRAY)
# The dimensions of the loaded dataset (as a sub-cube) will be shown
# The number of pixels in each dimension (latitude and longitude) is shown
# The number of time slices in the analysis is shown (time)
landsat_dataset
<xarray.Dataset>
Dimensions: (time: 55, latitude: 372, longitude: 372)
Coordinates:
* time (time) datetime64[ns] 2018-01-15T07:31:40.430000 ... 2020-12...
* latitude (latitude) float64 -3.95 -3.95 -3.951 ... -4.05 -4.05 -4.05
* longitude (longitude) float64 39.6 39.6 39.6 39.6 ... 39.7 39.7 39.7 39.7
spatial_ref int32 4326
Data variables:
red (time, latitude, longitude) int16 1061 1082 1101 ... 890 1152
green (time, latitude, longitude) int16 1088 1101 1116 ... 811 911
blue (time, latitude, longitude) int16 998 1013 992 ... 543 457 570
nir (time, latitude, longitude) int16 2257 2301 2237 ... 3295 3162
swir1 (time, latitude, longitude) int16 1556 1560 1575 ... 2112 2747
swir2 (time, latitude, longitude) int16 951 965 1008 ... 1418 2098
pixel_qa (time, latitude, longitude) uint16 480 480 480 ... 322 322 322
Attributes:
crs: EPSG:4326
grid_mapping: spatial_refarray(['2018-01-15T07:31:40.430000000', '2018-01-31T07:31:30.720000000',
'2018-02-16T07:31:25.380000000', '2018-03-20T07:31:09.340000000',
'2018-04-05T07:31:02.080000000', '2018-04-21T07:30:52.500000000',
'2018-05-23T07:30:31.940000000', '2018-06-08T07:30:25.770000000',
'2018-06-24T07:30:36.770000000', '2018-08-11T07:31:01.180000000',
'2018-08-27T07:31:08.820000000', '2018-09-12T07:31:14.010000000',
'2018-09-28T07:31:20.150000000', '2018-10-14T07:31:27.390000000',
'2018-10-30T07:31:31.880000000', '2018-11-15T07:31:33.560000000',
'2018-12-01T07:31:32.260000000', '2018-12-17T07:31:29.650000000',
'2019-01-02T07:31:29.620000000', '2019-01-18T07:31:26.790000000',
'2019-02-03T07:31:23.320000000', '2019-02-19T07:31:21.020000000',
'2019-03-07T07:31:16.160000000', '2019-03-23T07:31:11.850000000',
'2019-04-08T07:31:07.730000000', '2019-04-24T07:31:00.940000000',
'2019-05-26T07:31:18.080000000', '2019-06-11T07:31:25.420000000',
'2019-06-27T07:31:30.610000000', '2019-07-13T07:31:33.792000000',
'2019-07-29T07:31:39.330000000', '2019-08-14T07:31:45.150000000',
'2019-09-15T07:31:53.913000000', '2019-10-01T07:31:58.676000000',
'2019-11-02T07:32:00.620000000', '2019-12-04T07:31:56.384000000',
'2020-01-05T07:31:49.649000000', '2020-01-21T07:31:46.277000000',
'2020-02-06T07:31:40.851000000', '2020-02-22T07:31:37.577000000',
'2020-03-09T07:31:31.680000000', '2020-03-25T07:31:23.329000000',
'2020-04-10T07:31:15.317000000', '2020-04-26T07:31:07.984000000',
'2020-05-28T07:31:04.868000000', '2020-06-13T07:31:15.039000000',
'2020-06-29T07:31:23.340000000', '2020-07-15T07:31:29.668000000',
'2020-09-01T07:31:46.777000000', '2020-09-17T07:31:52.772000000',
'2020-10-03T07:31:56.505000000', '2020-10-19T07:31:57.558000000',
'2020-11-20T07:31:56.086000000', '2020-12-06T07:31:58.610000000',
'2020-12-22T07:31:56.951000000'], dtype='datetime64[ns]')array([-3.950099, -3.950368, -3.950638, ..., -4.049542, -4.049811, -4.050081])
array([39.600027, 39.600296, 39.600566, ..., 39.69947 , 39.699739, 39.700009])
array(4326, dtype=int32)
array([[[1061, 1082, 1101, ..., 976, 1876, 2495],
[1293, 1175, 1264, ..., 1666, 2508, 2848],
[1594, 2278, 2751, ..., 1961, 2571, 3019],
...,
[ 987, 962, 967, ..., 896, 1170, 1170],
[1035, 1017, 1040, ..., 930, 1110, 969],
[1266, 1288, 1300, ..., 1208, 1229, 1410]],
[[5948, 5849, 5344, ..., 8143, 8226, 8302],
[6043, 5745, 5250, ..., 8240, 8289, 8329],
[5926, 5737, 5534, ..., 8228, 8313, 8341],
...,
[6018, 5940, 5842, ..., 961, 1266, 1438],
[6053, 5972, 5641, ..., 948, 1082, 1082],
[6039, 5992, 5779, ..., 1219, 1277, 1526]],
[[7651, 7629, 7454, ..., 1860, 1721, 2361],
[7442, 7329, 7253, ..., 1658, 1614, 2551],
[7353, 6943, 6857, ..., 1568, 1525, 1853],
...,
...
...,
[ 718, 738, 685, ..., 646, 1138, 1166],
[ 703, 688, 668, ..., 735, 862, 602],
[ 703, 704, 723, ..., 878, 862, 1310]],
[[ 664, 656, 764, ..., 2240, 1661, 1234],
[ 750, 675, 750, ..., 1712, 1323, 990],
[ 579, 525, 609, ..., 1505, 1046, 866],
...,
[ 436, 466, 516, ..., 2717, 2470, 2118],
[ 435, 462, 520, ..., 1853, 2055, 1746],
[ 467, 518, 590, ..., 1234, 1447, 1709]],
[[ 886, 1317, 1449, ..., 684, 630, 657],
[1345, 1519, 1717, ..., 771, 713, 661],
[1621, 1677, 1763, ..., 986, 852, 782],
...,
[3005, 3355, 3660, ..., 679, 1021, 1378],
[3819, 3626, 3878, ..., 809, 708, 929],
[4199, 3649, 3605, ..., 977, 890, 1152]]], dtype=int16)array([[[1088, 1101, 1116, ..., 931, 1870, 2366],
[1307, 1190, 1241, ..., 1589, 2427, 2736],
[1578, 2174, 2661, ..., 1926, 2529, 2893],
...,
[1144, 1129, 1146, ..., 886, 1007, 994],
[1172, 1161, 1172, ..., 938, 992, 924],
[1281, 1284, 1288, ..., 1181, 1191, 1090]],
[[5762, 5673, 5194, ..., 7916, 7984, 8062],
[5850, 5573, 5086, ..., 7963, 8030, 8069],
[5727, 5527, 5343, ..., 8011, 8084, 8069],
...,
[5877, 5794, 5711, ..., 932, 1063, 1129],
[5917, 5829, 5520, ..., 952, 1001, 977],
[5915, 5851, 5637, ..., 1141, 1172, 1169]],
[[7409, 7388, 7200, ..., 1574, 1437, 2101],
[7224, 7125, 7016, ..., 1255, 1213, 2365],
[7090, 6697, 6598, ..., 1237, 1186, 1515],
...,
...
...,
[ 909, 934, 883, ..., 714, 908, 944],
[ 930, 919, 903, ..., 832, 798, 709],
[ 918, 915, 915, ..., 891, 822, 924]],
[[ 849, 792, 894, ..., 2210, 1730, 1373],
[ 907, 839, 880, ..., 1795, 1416, 1059],
[ 871, 802, 832, ..., 1522, 1099, 953],
...,
[ 662, 694, 758, ..., 2656, 2474, 2073],
[ 664, 693, 764, ..., 1798, 1930, 1681],
[ 699, 753, 843, ..., 1161, 1440, 1653]],
[[1012, 1414, 1524, ..., 778, 811, 825],
[1428, 1610, 1809, ..., 842, 832, 826],
[1717, 1769, 1823, ..., 975, 902, 881],
...,
[3032, 3374, 3666, ..., 700, 849, 1045],
[3770, 3611, 3849, ..., 784, 748, 818],
[4137, 3653, 3634, ..., 944, 811, 911]]], dtype=int16)array([[[ 998, 1013, 992, ..., 667, 1586, 2035],
[1178, 1082, 1246, ..., 1389, 2090, 2482],
[1425, 2177, 2575, ..., 1677, 2288, 2611],
...,
[ 813, 784, 787, ..., 462, 519, 492],
[ 817, 804, 807, ..., 576, 554, 466],
[1002, 1008, 1008, ..., 829, 816, 647]],
[[5607, 5553, 4951, ..., 7643, 7723, 7744],
[5633, 5321, 4942, ..., 7724, 7760, 7788],
[5557, 5405, 5149, ..., 7747, 7802, 7809],
...,
[5762, 5657, 5544, ..., 535, 598, 653],
[5774, 5675, 5355, ..., 571, 549, 536],
[5774, 5727, 5546, ..., 806, 729, 606]],
[[7165, 7092, 6914, ..., 1383, 1348, 1731],
[6896, 6754, 6652, ..., 834, 843, 2126],
[6790, 6295, 6209, ..., 800, 855, 1214],
...,
...
...,
[ 645, 664, 615, ..., 406, 545, 587],
[ 678, 663, 643, ..., 432, 426, 382],
[ 662, 659, 652, ..., 559, 481, 509]],
[[ 400, 371, 437, ..., 2085, 1471, 1324],
[ 491, 384, 420, ..., 1690, 1301, 1013],
[ 409, 334, 363, ..., 1390, 942, 855],
...,
[ 574, 598, 644, ..., 2591, 2281, 1951],
[ 571, 593, 635, ..., 1937, 1966, 1623],
[ 593, 627, 684, ..., 1133, 1427, 1525]],
[[ 879, 1323, 1478, ..., 414, 434, 447],
[1385, 1499, 1654, ..., 474, 429, 448],
[1594, 1638, 1720, ..., 562, 491, 476],
...,
[2993, 3262, 3504, ..., 353, 488, 667],
[3685, 3487, 3703, ..., 400, 400, 527],
[3957, 3451, 3412, ..., 543, 457, 570]]], dtype=int16)array([[[2257, 2301, 2237, ..., 2170, 3185, 3818],
[2546, 2423, 2586, ..., 2906, 3817, 4192],
[2904, 3732, 4180, ..., 3240, 3914, 4359],
...,
[ 717, 665, 662, ..., 2922, 3039, 3128],
[ 717, 677, 681, ..., 3176, 3147, 3223],
[1049, 1206, 1326, ..., 3067, 3288, 3134]],
[[6741, 6740, 6235, ..., 8510, 8605, 8712],
[6861, 6608, 6197, ..., 8617, 8664, 8713],
[6791, 6636, 6431, ..., 8564, 8678, 8705],
...,
[6348, 6288, 6183, ..., 2970, 3459, 3443],
[6361, 6311, 5984, ..., 3169, 3560, 3688],
[6352, 6308, 6121, ..., 3189, 3388, 3367]],
[[8189, 8218, 8065, ..., 3508, 3345, 3715],
[7992, 7916, 7911, ..., 3467, 3337, 3899],
[8029, 7711, 7726, ..., 3436, 3239, 3318],
...,
...
...,
[ 366, 369, 337, ..., 3452, 3350, 3598],
[ 345, 330, 318, ..., 3627, 3417, 3762],
[ 337, 331, 321, ..., 3389, 3531, 3287]],
[[4324, 3811, 3838, ..., 3925, 3374, 2844],
[4144, 4006, 4067, ..., 3477, 2881, 2524],
[4525, 4249, 4084, ..., 3192, 2424, 2234],
...,
[ 220, 232, 261, ..., 3816, 3542, 3252],
[ 222, 235, 259, ..., 3150, 3341, 3013],
[ 227, 248, 271, ..., 2624, 2854, 2874]],
[[2910, 3157, 3253, ..., 3515, 3860, 3748],
[3211, 3364, 3527, ..., 3683, 4005, 3788],
[3456, 3516, 3601, ..., 3624, 3621, 3636],
...,
[3403, 3780, 4103, ..., 2943, 3353, 2963],
[4319, 4083, 4325, ..., 3241, 3398, 3202],
[4708, 4039, 3963, ..., 3111, 3295, 3162]]], dtype=int16)array([[[1556, 1560, 1575, ..., 1877, 2993, 3504],
[1858, 1803, 1757, ..., 2433, 3460, 3854],
[2228, 2695, 3330, ..., 2843, 3513, 4033],
...,
[ 514, 513, 516, ..., 1961, 2451, 2714],
[ 509, 504, 511, ..., 2054, 2273, 2167],
[ 789, 821, 916, ..., 2221, 2385, 2954]],
[[5672, 5773, 5406, ..., 6094, 6119, 6196],
[5891, 5797, 5407, ..., 6137, 6145, 6181],
[5854, 5737, 5613, ..., 6156, 6173, 6140],
...,
[5534, 5496, 5420, ..., 2167, 2691, 2857],
[5579, 5543, 5283, ..., 2057, 2451, 2505],
[5561, 5527, 5275, ..., 2500, 2453, 3095]],
[[6537, 6534, 6340, ..., 3906, 3682, 3795],
[6365, 6334, 6190, ..., 4076, 3794, 3878],
[6337, 6133, 6089, ..., 3977, 3926, 4044],
...,
...
...,
[ 241, 247, 225, ..., 2233, 2784, 3271],
[ 227, 214, 203, ..., 2175, 2596, 2414],
[ 213, 202, 195, ..., 2443, 2327, 3124]],
[[2576, 2575, 2638, ..., 2972, 2522, 1884],
[2604, 2464, 2542, ..., 2626, 2164, 1622],
[2304, 2282, 2408, ..., 2430, 1828, 1512],
...,
[ 126, 138, 158, ..., 3314, 3159, 2845],
[ 127, 136, 158, ..., 2230, 2569, 2492],
[ 131, 151, 174, ..., 2071, 2119, 2390]],
[[1632, 2020, 2149, ..., 2751, 2613, 2542],
[2030, 2218, 2491, ..., 2954, 2858, 2724],
[2392, 2448, 2519, ..., 3418, 3196, 2873],
...,
[3049, 3425, 3746, ..., 2115, 2516, 2728],
[3839, 3781, 3898, ..., 2208, 1975, 2433],
[4482, 3855, 3716, ..., 2199, 2112, 2747]]], dtype=int16)array([[[ 951, 965, 1008, ..., 1148, 2131, 2565],
[1217, 1156, 1122, ..., 1669, 2578, 2934],
[1519, 2048, 2698, ..., 2046, 2720, 3202],
...,
[ 370, 362, 369, ..., 1257, 1730, 1970],
[ 372, 364, 360, ..., 1285, 1578, 1413],
[ 487, 500, 588, ..., 1534, 1613, 2260]],
[[4306, 4459, 4180, ..., 4187, 4214, 4276],
[4519, 4496, 4176, ..., 4211, 4218, 4244],
[4485, 4426, 4402, ..., 4280, 4226, 4189],
...,
[4427, 4325, 4206, ..., 1517, 1828, 1908],
[4483, 4375, 4069, ..., 1343, 1648, 1670],
[4475, 4394, 4122, ..., 1736, 1711, 2464]],
[[4723, 4799, 4614, ..., 2588, 2406, 2704],
[4695, 4707, 4576, ..., 2537, 2465, 2946],
[4781, 4641, 4691, ..., 2518, 2509, 2712],
...,
...
...,
[ 184, 191, 172, ..., 1385, 1865, 2192],
[ 176, 164, 157, ..., 1245, 1736, 1451],
[ 158, 152, 148, ..., 1610, 1514, 2404]],
[[1294, 1318, 1414, ..., 2118, 1691, 1168],
[1317, 1233, 1283, ..., 1761, 1384, 972],
[1094, 1010, 1109, ..., 1586, 1116, 880],
...,
[ 90, 99, 112, ..., 2969, 2800, 2398],
[ 90, 95, 115, ..., 1854, 2156, 2030],
[ 95, 109, 128, ..., 1498, 1622, 1919]],
[[ 981, 1329, 1446, ..., 1490, 1348, 1281],
[1331, 1513, 1776, ..., 1625, 1469, 1368],
[1693, 1740, 1786, ..., 2079, 1870, 1548],
...,
[2607, 2944, 3189, ..., 1439, 1588, 2025],
[3408, 3295, 3383, ..., 1477, 1217, 1622],
[3976, 3365, 3191, ..., 1413, 1418, 2098]]], dtype=int16)array([[[480, 480, 480, ..., 416, 480, 480],
[480, 480, 480, ..., 480, 480, 480],
[480, 480, 480, ..., 480, 480, 480],
...,
[352, 352, 352, ..., 322, 322, 322],
[352, 352, 352, ..., 322, 322, 322],
[480, 480, 480, ..., 322, 322, 322]],
[[480, 480, 480, ..., 480, 480, 480],
[480, 480, 480, ..., 480, 480, 480],
[480, 480, 480, ..., 480, 480, 480],
...,
[480, 480, 480, ..., 322, 322, 322],
[480, 480, 480, ..., 322, 322, 322],
[480, 480, 480, ..., 352, 352, 352]],
[[480, 480, 480, ..., 480, 480, 480],
[480, 480, 480, ..., 352, 352, 480],
[480, 480, 480, ..., 352, 352, 352],
...,
...
...,
[328, 328, 328, ..., 352, 352, 352],
[328, 328, 328, ..., 328, 328, 328],
[328, 328, 328, ..., 328, 328, 352]],
[[352, 322, 322, ..., 480, 480, 480],
[352, 322, 322, ..., 480, 480, 480],
[352, 322, 322, ..., 480, 480, 480],
...,
[328, 328, 352, ..., 480, 480, 480],
[328, 328, 352, ..., 480, 480, 480],
[328, 328, 352, ..., 480, 480, 480]],
[[416, 480, 480, ..., 322, 328, 328],
[480, 480, 480, ..., 322, 322, 328],
[480, 480, 480, ..., 322, 322, 322],
...,
[480, 480, 480, ..., 322, 322, 322],
[480, 480, 480, ..., 322, 322, 322],
[480, 480, 480, ..., 322, 322, 322]]], dtype=uint16)# Find the cloudy pixels and remove them from the time series calculation
from utils.data_cube_utilities.clean_mask import landsat_qa_clean_mask
cloud_mask = landsat_qa_clean_mask(landsat_dataset, platform=platform)
Time series output of the Australian Water Observations from Space (WOFS) results. The results show the percent of time that a pixel is classified as water over the entire time series. Cloudy pixels are ignored in the calculations.
# Apply WOFS water classification. Only consider non-cloudy pixels
from utils.data_cube_utilities.dc_water_classifier import wofs_classify
ts_water_classification = wofs_classify(landsat_dataset, clean_mask=cloud_mask.values)
# Ignore "no data" values
ts_water_classification = ts_water_classification.where(ts_water_classification != -9999).astype(np.float16)
# Calculate the time series water classification product
water_classification_percentages = (ts_water_classification.mean(dim=['time']) * 100).wofs.rename('water_classification_percentages')
# Import color-scheme and set "no data" to black color
# "No data" pixels are only found when clouds exist for the entire time series
from matplotlib.cm import jet_r
jet_r.set_bad('black', 1)
# Show the WOFS time series product
# Areas of RED have experienced little or no water over the time series
# Areas of BLUE have experience significant or constant water over the time series
# The figure size can be adjusted based on the dataset dimensions
water_classification_percentages.plot(cmap=jet_r, figsize=(12,10))
plt.title("Percent of Samples Classified as Water")
plt.axis('off')
plt.show()