{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "02_pktools_colab.ipynb", "provenance": [], "collapsed_sections": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "PBuSNMIUmJ8r" }, "source": [ "# Use PKTOOLS for raster/vector operations - colab" ] }, { "cell_type": "markdown", "metadata": { "id": "kYp80rx3mJ8y" }, "source": [ "## Setting working directory for the Google Collaboratory\n", "\n", "PKTOOLS is a suite of utilities written in C++ for image processing with a focus on remote sensing applications. It relies on the Geospatial Data Abstraction Library (GDAL and OGR, http://www.gdal.org) (source http://pktools.nongnu.org/html/index.html)" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "HNGQAq1GmJ8z", "outputId": "d3a0bfa1-88f1-4e02-b925-729f7af015bd" }, "source": [ "from google.colab import drive\n", "drive.mount('/gdrive')" ], "execution_count": 1, "outputs": [ { "output_type": "stream", "text": [ "Mounted at /gdrive\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "5sWFKdlr0Rvw" }, "source": [ "List the mounted gdrive. If you have folders and files in you gdrive you should be able to see them\n", "\n" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "iwhEWdc8mJ80", "outputId": "3060522c-0a3f-44d5-e274-f0e91735251d" }, "source": [ "! ls /gdrive/MyDrive" ], "execution_count": 2, "outputs": [ { "output_type": "stream", "text": [ " auto\n", " Big_blue\n", "'Colab Notebooks'\n", "'Dataset for global stream network and MOSQLAND.gdoc'\n", "'Discharge Tasks.gdoc'\n", " Elena_intro.gdoc\n", " GeoCompCourse\n", " GeoCompYaleMay2020\n", " grants\n", "'ID PASSPORT'\n", " LandscapeGenetic\n", " Lawsuit\n", " Margosa\n", " panniers-trailers.pdf\n", " RDOC_CMS\n", " review\n", " Sofia_intro.gdoc\n", " Stream_Variables_dataset_selected.gsheet\n", " Taino\n", " yale\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "o7Jo_jFg0bOP" }, "source": [ "Download the SE_data folder yousing git\n", "\n", "First remove the directory if already exist, then enter in the directory." ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "FpDEyHKZiGma", "outputId": "039def93-f4f9-4606-92d6-864be7c4c4ab" }, "source": [ "%rm -rf /gdrive/MyDrive/SE_data\n", "%cd /gdrive/MyDrive\n", "%ls" ], "execution_count": 3, "outputs": [ { "output_type": "stream", "text": [ "/gdrive/MyDrive\n", " \u001b[0m\u001b[01;34mauto\u001b[0m/\n", " \u001b[01;34mBig_blue\u001b[0m/\n", "\u001b[01;34m'Colab Notebooks'\u001b[0m/\n", "'Dataset for global stream network and MOSQLAND.gdoc'\n", "'Discharge Tasks.gdoc'\n", " Elena_intro.gdoc\n", " \u001b[01;34mGeoCompCourse\u001b[0m/\n", " \u001b[01;34mGeoCompYaleMay2020\u001b[0m/\n", " \u001b[01;34mgrants\u001b[0m/\n", "\u001b[01;34m'ID PASSPORT'\u001b[0m/\n", " \u001b[01;34mLandscapeGenetic\u001b[0m/\n", " \u001b[01;34mLawsuit\u001b[0m/\n", " \u001b[01;34mMargosa\u001b[0m/\n", " panniers-trailers.pdf\n", " \u001b[01;34mRDOC_CMS\u001b[0m/\n", " \u001b[01;34mreview\u001b[0m/\n", " Sofia_intro.gdoc\n", " Stream_Variables_dataset_selected.gsheet\n", " \u001b[01;34mTaino\u001b[0m/\n", " \u001b[01;34myale\u001b[0m/\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "IEjDLFRPiE4X" }, "source": [ "Then perform the downloading" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "nFT2E1IZ0tL9", "outputId": "dbd18277-69aa-49d7-a336-734510792e05" }, "source": [ "!git clone https://github.com/selvaje/SE_data.git" ], "execution_count": 4, "outputs": [ { "output_type": "stream", "text": [ "Cloning into 'SE_data'...\n", "remote: Enumerating objects: 195, done.\u001b[K\n", "remote: Counting objects: 100% (195/195), done.\u001b[K\n", "remote: Compressing objects: 100% (155/155), done.\u001b[K\n", "remote: Total 262 (delta 22), reused 190 (delta 20), pack-reused 67\u001b[K\n", "Receiving objects: 100% (262/262), 78.99 MiB | 17.46 MiB/s, done.\n", "Resolving deltas: 100% (25/25), done.\n", "Checking out files: 100% (162/162), done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "wmtvLq9l02Cb" }, "source": [ "Check if the SE_data folder is full" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e6RcVABL07rx", "outputId": "945b5788-74b5-4453-d888-10ea47f9a20b" }, "source": [ "! ls -l SE_data/*" ], "execution_count": 5, "outputs": [ { "output_type": "stream", "text": [ "-rw------- 1 root root 58 Jan 19 22:45 SE_data/README.md\n", "\n", "SE_data/exercise:\n", "total 187\n", "-rw------- 1 root root 8591 Jan 19 22:45 00_Setting_Colab_for_for_Spatial_Ecology_course.ipynb\n", "-rw------- 1 root root 153327 Jan 19 22:45 01_gdal.ipynb\n", "-rw------- 1 root root 24271 Jan 19 22:45 02_pktools.ipynb\n", "drwx------ 7 root root 4096 Jan 19 22:45 geodata\n", "\n", "SE_data/pktools_local:\n", "total 15\n", "-rw------- 1 root root 11141 Jan 19 22:45 00_pktools_gdrive_install.ipynb\n", "drwx------ 5 root root 4096 Jan 19 22:45 usr\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "ySAS8F0x06y5" }, "source": [ "## Install pktools\n", "\n", "pktools is not present in Colab but is possible to install via the \"apt install\" comand. Pay attention that if you re-open another colab you have to re-install pktools" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ddnLcpzM1UiX", "outputId": "7a8a01e7-5777-46f4-ec25-9a2c0aa7772f" }, "source": [ "! apt install pktools" ], "execution_count": 7, "outputs": [ { "output_type": "stream", "text": [ "Reading package lists... Done\n", "Building dependency tree \n", "Reading state information... Done\n", "The following additional packages will be installed:\n", " libalgorithms1 libfann2 libfileclasses1 libgsl23 libgslcblas0\n", " libimageclasses1 liblas3 liblasclasses1\n", "Suggested packages:\n", " libfann-dev libfann-doc gsl-ref-psdoc | gsl-doc-pdf | gsl-doc-info\n", " | gsl-ref-html libgeotiff-epsg\n", "The following NEW packages will be installed:\n", " libalgorithms1 libfann2 libfileclasses1 libgsl23 libgslcblas0\n", " libimageclasses1 liblas3 liblasclasses1 pktools\n", "0 upgraded, 9 newly installed, 0 to remove and 12 not upgraded.\n", "Need to get 2,347 kB of archives.\n", "After this operation, 9,702 kB of additional disk space will be used.\n", "Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libgslcblas0 amd64 2.4+dfsg-6 [79.7 kB]\n", "Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libgsl23 amd64 2.4+dfsg-6 [823 kB]\n", "Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libimageclasses1 amd64 2.6.7.3+ds-1 [57.9 kB]\n", "Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libalgorithms1 amd64 2.6.7.3+ds-1 [130 kB]\n", "Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libfann2 amd64 2.2.0+ds-3 [64.5 kB]\n", "Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libfileclasses1 amd64 2.6.7.3+ds-1 [12.7 kB]\n", "Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblas3 amd64 1.8.1-6build1 [270 kB]\n", "Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblasclasses1 amd64 2.6.7.3+ds-1 [15.2 kB]\n", "Get:9 http://archive.ubuntu.com/ubuntu bionic/universe amd64 pktools amd64 2.6.7.3+ds-1 [894 kB]\n", "Fetched 2,347 kB in 2s (1,435 kB/s)\n", "Selecting previously unselected package libgslcblas0:amd64.\n", "(Reading database ... 146364 files and directories currently installed.)\n", "Preparing to unpack .../0-libgslcblas0_2.4+dfsg-6_amd64.deb ...\n", "Unpacking libgslcblas0:amd64 (2.4+dfsg-6) ...\n", "Selecting previously unselected package libgsl23:amd64.\n", "Preparing to unpack .../1-libgsl23_2.4+dfsg-6_amd64.deb ...\n", "Unpacking libgsl23:amd64 (2.4+dfsg-6) ...\n", "Selecting previously unselected package libimageclasses1.\n", "Preparing to unpack .../2-libimageclasses1_2.6.7.3+ds-1_amd64.deb ...\n", "Unpacking libimageclasses1 (2.6.7.3+ds-1) ...\n", "Selecting previously unselected package libalgorithms1.\n", "Preparing to unpack .../3-libalgorithms1_2.6.7.3+ds-1_amd64.deb ...\n", "Unpacking libalgorithms1 (2.6.7.3+ds-1) ...\n", "Selecting previously unselected package libfann2:amd64.\n", "Preparing to unpack .../4-libfann2_2.2.0+ds-3_amd64.deb ...\n", "Unpacking libfann2:amd64 (2.2.0+ds-3) ...\n", "Selecting previously unselected package libfileclasses1.\n", "Preparing to unpack .../5-libfileclasses1_2.6.7.3+ds-1_amd64.deb ...\n", "Unpacking libfileclasses1 (2.6.7.3+ds-1) ...\n", "Selecting previously unselected package liblas3.\n", "Preparing to unpack .../6-liblas3_1.8.1-6build1_amd64.deb ...\n", "Unpacking liblas3 (1.8.1-6build1) ...\n", "Selecting previously unselected package liblasclasses1.\n", "Preparing to unpack .../7-liblasclasses1_2.6.7.3+ds-1_amd64.deb ...\n", "Unpacking liblasclasses1 (2.6.7.3+ds-1) ...\n", "Selecting previously unselected package pktools.\n", "Preparing to unpack .../8-pktools_2.6.7.3+ds-1_amd64.deb ...\n", "Unpacking pktools (2.6.7.3+ds-1) ...\n", "Setting up liblas3 (1.8.1-6build1) ...\n", "Setting up liblasclasses1 (2.6.7.3+ds-1) ...\n", "Setting up libfann2:amd64 (2.2.0+ds-3) ...\n", "Setting up libfileclasses1 (2.6.7.3+ds-1) ...\n", "Setting up libgslcblas0:amd64 (2.4+dfsg-6) ...\n", "Setting up libgsl23:amd64 (2.4+dfsg-6) ...\n", "Setting up libimageclasses1 (2.6.7.3+ds-1) ...\n", "Setting up libalgorithms1 (2.6.7.3+ds-1) ...\n", "Setting up pktools (2.6.7.3+ds-1) ...\n", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ...\n", "/sbin/ldconfig.real: /usr/local/lib/python3.6/dist-packages/ideep4py/lib/libmkldnn.so.0 is not a symbolic link\n", "\n", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "yF0t0hOF1ecH" }, "source": [ "Test if pktools is working fine by running a pktools command and getting the help" ] }, { "cell_type": "code", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Y-Ni3Hv51ppV", "outputId": "4a5356d1-3044-4cab-fb12-762da5475c43" }, "source": [ "! pkgetmask --help" ], "execution_count": 8, "outputs": [ { "output_type": "stream", "text": [ " -i --input Input image file\n", " -o --output Output mask file\n", " -min --min Values smaller than min threshold(s) are masked as invalid. Use one threshold for each band\n", " -max --max Values greater than max threshold(s) are masked as invalid. Use one threshold for each band\n", " -data --data value(s) for valid pixels: between min and max (default: 1)\n", " -nodata --nodata value(s) for invalid pixels: not between min and max (default: 0)\n", " -b --band band(s) used for mask (default: 0)\n", " -p --operator Operator: [AND,OR]. (default: OR)\n", " -ot --otype Data type for output image ({Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/CInt16/CInt32/CFloat32/CFloat64}). Empty string: inherit type from input image (default: Byte)\n", " -of --oformat Output image format (see also gdal_translate). (default: GTiff)\n", " -co --co Creation option for output file. Multiple options can be specified.\n", " -ct --ct color table (file with 5 columns: id R G B ALFA (0: transparent, 255: solid)\n", "\n", "Usage: pkgetmask -i input -o output\n", "\n", "short option -h shows basic options only, use long option --help to show all options\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "Al7egxCLmJ81" }, "source": [ "## Start to use PKTOOLS\n", "\n", "**Create a mask**\n", "\n", "Create a mask with by manipulate a text file." ] }, { "cell_type": "code", "metadata": { "id": "wb2l2rt5mJ81", "outputId": "4262b7a0-0fd9-4a42-ec86-84457c885e03" }, "source": [ "%%bash\n", "cd /gdrive/MyDrive/SE_data\n", "gdal_translate -of XYZ geodata/vegetation/ETmean08-11_crop.tif geodata/vegetation/ETmean08-11_crop.txt\n", "awk '{if ($3>2 && $3<4) {print $1,$2,50 } else {print}}' geodata/vegetation/ETmean08-11_crop.txt > geodata/vegetation/ETmean08-11_crop_trh.txt\n", "gdal_translate -ot Byte geodata/vegetation/ETmean08-11_crop_trh.txt geodata/vegetation/ETmean08-11_crop_trh.tif " ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "Input file size is 240, 240\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "Input file size is 240, 240\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "SzC0xzJ0mJ82" }, "source": [ "The same operation can be done more efficient using pkgetmask. We can create 3 masks using different thresholds values." ] }, { "cell_type": "code", "metadata": { "id": "8oLciLK-mJ82", "outputId": "5f4e5c9b-f2d4-46f2-8f71-4490d5fb6bc9" }, "source": [ "%%bash\n", "pkgetmask -co COMPRESS=DEFLATE -co ZLEVEL=9 -min 1 -max 2 -data 1 -nodata 0 -ot Byte -i geodata/vegetation/ETmean08-11.tif -o geodata/vegetation/ETmean08-11_01_trhA.tif\n", "pkgetmask -co COMPRESS=DEFLATE -co ZLEVEL=9 -min 5 -max 8 -data 1 -nodata 0 -ot Byte -i geodata/vegetation/ETmean08-11.tif -o geodata/vegetation/ETmean08-11_01_trhB.tif\n", "pkgetmask -co COMPRESS=DEFLATE -co ZLEVEL=9 -min 0 -max 10 -data 0 -nodata 1 -ot Byte -i geodata/vegetation/ETmean08-11.tif -o geodata/vegetation/ETmean08-11_01_trhC.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "UC8VPm5CmJ83" }, "source": [ "**Set a mask to other image**\\\n", "Use the prepared mask and apply to the image." ] }, { "cell_type": "code", "metadata": { "id": "rOCybsZamJ83", "scrolled": true, "outputId": "970b90d4-5153-4b54-8940-dea47cfc8774" }, "source": [ "%%bash\n", "pksetmask -co COMPRESS=DEFLATE -co ZLEVEL=9 \\\n", "-m geodata/vegetation/ETmean08-11_01_trhA.tif -msknodata 1 -nodata -9 \\\n", "-m geodata/vegetation/ETmean08-11_01_trhB.tif -msknodata 1 -nodata -5 \\\n", "-m geodata/vegetation/ETmean08-11_01_trhC.tif -msknodata 1 -nodata -10 \\\n", "-i geodata/vegetation/ETmean08-11.tif -o geodata/vegetation/ETmean08-11_01_msk.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "xWjYbcGimJ84" }, "source": [ "**Composite images**\\\n", "create a mask to apply during the composite" ] }, { "cell_type": "code", "metadata": { "id": "qMEVO1RjmJ84", "outputId": "99d0d72f-4de1-43a8-d9f6-58fc221a5d30" }, "source": [ "%%bash\n", "pkgetmask -co COMPRESS=DEFLATE -co ZLEVEL=9 -min 0 -max 25 -data 0 -nodata 1 -ot Byte -i geodata/LST/LST_MOYDmax_month1.tif -o geodata/LST/LST_MOYDmax_month1-msk.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "c82Wn5bqmJ84" }, "source": [ "Calculate mean and standard deviation with several images" ] }, { "cell_type": "code", "metadata": { "id": "th7g2X8omJ85", "outputId": "58bd6a7d-b1dd-4456-ec73-846df8af7c76" }, "source": [ "%%bash\n", "pkcomposite $(for file in geodata/LST/LST_MOYDmax_month??.tif geodata/LST/LST_MOYDmax_month?.tif ; do echo -i $file ; done ) -m geodata/LST/LST_MOYDmax_month1-msk.tif -msknodata 0 -cr mean -dstnodata 0 -co COMPRESS=LZW -co ZLEVEL=9 -o geodata/LST/LST_MOYDmax_monthMean.tif\n", "pkcomposite $(for file in geodata/LST/LST_MOYDmax_month??.tif geodata/LST/LST_MOYDmax_month?.tif ; do echo -i $file ; done ) -m geodata/LST/LST_MOYDmax_month1-msk.tif -msknodata 0 -cr stdev -dstnodata -1 -co COMPRESS=LZW -co ZLEVEL=9 -o geodata/LST/LST_MOYDmax_monthStdev.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "8Jlxls9pmJ85" }, "source": [ "An alternative way is to use pkstatprofile" ] }, { "cell_type": "code", "metadata": { "id": "foJoTUH0mJ85", "outputId": "40b097b4-c4a2-4ead-9a7d-c8726f825f76" }, "source": [ "%%bash\n", "# Create a multiband vrt\n", "gdalbuildvrt -overwrite -separate geodata/LST/LST_MOYDmax_month.vrt geodata/LST/LST_MOYDmax_month?.tif geodata/LST/LST_MOYDmax_month??.tif\n", "# Calculate mean and standard deviation\n", "pkstatprofile -co COMPRESS=LZW -nodata 0 -f mean -f stdev -i geodata/LST/LST_MOYDmax_month.vrt -o geodata/LST/LST_MOYDmax_month_mean_stdev.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "Do0OMoGLmJ86" }, "source": [ "**Filter/Aggregate images**" ] }, { "cell_type": "code", "metadata": { "id": "RH6nU0xTmJ86", "outputId": "656143a2-ce5f-42d8-9cd9-0d5a41efe287" }, "source": [ "%%bash\n", "# mean aggregation \n", "pkfilter -co COMPRESS=DEFLATE -co ZLEVEL=9 -nodata 0 -ot Float32 -dx 10 -dy 10 -f mean -d 10 -i geodata/LST/LST_MOYDmax_monthMean.tif -o geodata/LST/LST_MOYDmax_monthMean_aggreate10mean.tif\n", "# mean circular moving window\n", "pkfilter -co COMPRESS=DEFLATE -co ZLEVEL=9 -nodata 0 -ot Float32 -dx 11 -dy 11 -f mean -c -i geodata/LST/LST_MOYDmax_monthMean.tif -o geodata/LST/LST_MOYDmax_monthMean_circular11mean.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "opening output image geodata/LST/LST_MOYDmax_monthMean_aggreate10mean.tif with 1 bands\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "opening output image geodata/LST/LST_MOYDmax_monthMean_circular11mean.tif with 1 bands\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "vvikECXdmJ87" }, "source": [ "**Images statistic and histogram**" ] }, { "cell_type": "code", "metadata": { "id": "PuVVTgQpmJ87", "scrolled": true, "outputId": "9997f3ee-750e-4191-9456-c58d8107b24c" }, "source": [ "%%bash\n", "pkstat -hist -src_min 0 -i geodata/temperature/ug_bio_3.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0 0\n", "1 0\n", "2 0\n", "3 0\n", "4 0\n", "5 0\n", "6 0\n", "7 0\n", "8 0\n", "9 0\n", "10 0\n", "11 0\n", "12 0\n", "13 0\n", "14 0\n", "15 0\n", "16 0\n", "17 0\n", "18 0\n", "19 0\n", "20 0\n", "21 0\n", "22 0\n", "23 0\n", "24 0\n", "25 0\n", "26 0\n", "27 0\n", "28 0\n", "29 0\n", "30 0\n", "31 0\n", "32 0\n", "33 0\n", "34 0\n", "35 0\n", "36 0\n", "37 0\n", "38 0\n", "39 0\n", "40 0\n", "41 0\n", "42 0\n", "43 0\n", "44 0\n", "45 0\n", "46 0\n", "47 0\n", "48 0\n", "49 0\n", "50 0\n", "51 0\n", "52 0\n", "53 0\n", "54 0\n", "55 0\n", "56 0\n", "57 0\n", "58 0\n", "59 0\n", "60 0\n", "61 2\n", "62 9\n", "63 31\n", "64 138\n", "65 235\n", "66 288\n", "67 481\n", "68 589\n", "69 803\n", "70 1989\n", "71 3471\n", "72 4490\n", "73 5513\n", "74 6938\n", "75 12332\n", "76 21295\n", "77 23643\n", "78 20937\n", "79 24687\n", "80 36864\n", "81 32858\n", "82 35070\n", "83 33486\n", "84 30120\n", "85 25312\n", "86 19138\n", "87 11203\n", "88 5892\n", "89 5799\n", "90 4372\n", "91 5354\n", "92 2456\n", "93 168\n" ], "name": "stdout" } ] }, { "cell_type": "code", "metadata": { "id": "6UzuZRORmJ87", "outputId": "f1bda511-4de5-4ade-ffc9-ffcdaa20c0aa" }, "source": [ "%%bash\n", "pkstat -hist -nbin 100 -src_min 0 -i geodata/vegetation/GPPstdev08-11.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0.02110012531 616\n", "0.06330037594 0\n", "0.1055006266 30\n", "0.1477008772 624\n", "0.1899011278 650\n", "0.2321013784 555\n", "0.2743016291 589\n", "0.3165018797 602\n", "0.3587021303 634\n", "0.4009023809 695\n", "0.4431026316 761\n", "0.4853028822 1072\n", "0.5275031328 2287\n", "0.5697033834 5055\n", "0.6119036341 9187\n", "0.6541038847 12469\n", "0.6963041353 12988\n", "0.7385043859 11639\n", "0.7807046366 10078\n", "0.8229048872 8994\n", "0.8651051378 7597\n", "0.9073053885 6994\n", "0.9495056391 6978\n", "0.9917058897 8092\n", "1.03390614 9392\n", "1.076106391 10288\n", "1.118306642 11877\n", "1.160506892 13784\n", "1.202707143 15860\n", "1.244907393 17719\n", "1.287107644 18117\n", "1.329307895 16990\n", "1.371508145 14917\n", "1.413708396 11927\n", "1.455908647 8212\n", "1.498108897 4854\n", "1.540309148 2430\n", "1.582509398 1295\n", "1.624709649 1298\n", "1.6669099 1862\n", "1.70911015 2812\n", "1.751310401 3990\n", "1.793510652 5286\n", "1.835710902 6235\n", "1.877911153 6864\n", "1.920111403 7441\n", "1.962311654 7356\n", "2.004511905 7345\n", "2.046712155 8170\n", "2.088912406 9272\n", "2.131112657 9107\n", "2.173312907 7296\n", "2.215513158 4944\n", "2.257713408 3490\n", "2.299913659 2446\n", "2.34211391 1688\n", "2.38431416 1415\n", "2.426514411 1180\n", "2.468714662 995\n", "2.510914912 860\n", "2.553115163 706\n", "2.595315413 564\n", "2.637515664 441\n", "2.679715915 347\n", "2.721916165 318\n", "2.764116416 269\n", "2.806316667 256\n", "2.848516917 217\n", "2.890717168 170\n", "2.932917418 115\n", "2.975117669 112\n", "3.01731792 107\n", "3.05951817 86\n", "3.101718421 63\n", "3.143918672 44\n", "3.186118922 61\n", "3.228319173 53\n", "3.270519423 44\n", "3.312719674 45\n", "3.354919925 41\n", "3.397120175 31\n", "3.439320426 37\n", "3.481520677 42\n", "3.523720927 38\n", "3.565921178 44\n", "3.608121428 55\n", "3.650321679 54\n", "3.69252193 51\n", "3.73472218 46\n", "3.776922431 62\n", "3.819122682 45\n", "3.861322932 49\n", "3.903523183 34\n", "3.945723433 17\n", "3.987923684 15\n", "4.030123935 6\n", "4.072324185 9\n", "4.114524436 1\n", "4.156724687 0\n", "4.198924937 1\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "OYjlbLPnmJ87" }, "source": [ "**Images reclassification**" ] }, { "cell_type": "code", "metadata": { "id": "2oFcBEgHmJ88", "outputId": "25dad608-fc04-4585-a3a0-2b56f9e13f59" }, "source": [ "%%bash\n", "pkstat -hist -i geodata/temperature/ug_bio_3.tif | grep -v \" 0\" | awk '{ if ($1<75) { print $1 , 0 } else { print $1 , 1 } }' > geodata/temperature/reclass_ug_bio_3.txt\n", "pkreclass -co COMPRESS=DEFLATE -co ZLEVEL=9 -code geodata/temperature/reclass_ug_bio_3.txt -i geodata/temperature/ug_bio_3.tif -o geodata/temperature/reclass_ug_bio_3.tif" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "UelAeY4JmJ88" }, "source": [ "**Zonal statistic (polygon extraction)**" ] }, { "cell_type": "code", "metadata": { "id": "zGYUSaTnmJ88", "outputId": "f651593c-e991-4d5f-d2aa-4304dee10f0c" }, "source": [ "%%bash\n", "rm -f geodata/shp/polygons_stat.*\n", "pkextractogr -srcnodata -339999995214436424907732413799364296704 -r mean -r stdev -r min -i geodata/vegetation/GPPmean08-11.tif -s geodata/shp/polygons.sqlite -o geodata/shp/polygons_stat.sqlite\n", " \n", "pkextractogr -f \"ESRI Shapefile\" -srcnodata -339999995214436424907732413799364296704 -r mean -r stdev -r min -i geodata/vegetation/GPPmean08-11.tif -s geodata/shp/polygons.sqlite -o geodata/shp/polygons_stat.shp\n", " \n", "# we can also create a csv that can be manipulate later on with awk\n", "rm -f geodata/shp/polygons_stat.csv\n", "pkextractogr -f CSV -srcnodata -339999995214436424907732413799364296704 -r mean -r stdev -r min -i geodata/vegetation/GPPmean08-11.tif -s geodata/shp/polygons.sqlite -o geodata/shp/polygons_stat.csv" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "processing layer polygons\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "processing layer polygons\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "processing layer polygons\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "xo_AY1bamJ89" }, "source": [ "**Zonal statistic (point extraction)**" ] }, { "cell_type": "code", "metadata": { "id": "dQQp0FR9mJ89", "outputId": "72cb35dd-25f6-473e-c446-fd5bf6d34bb5" }, "source": [ "%%bash \n", "# at point location\n", "rm geodata/shp/point_stat.csv\n", "pkextractogr -f CSV -srcnodata -339999995214436424907732413799364296704 -r mean -r stdev -r min -i geodata/vegetation/GPPmean08-11.tif -s geodata/shp/presence.shp -o geodata/shp/point_stat.csv\n", "# at point location + 1 pixel around \n", "rm geodata/shp/point-buf_stat.csv\n", "pkextractogr -f CSV -buf 2 -srcnodata -339999995214436424907732413799364296704 -r mean -r stdev -r min -i geodata/vegetation/GPPmean08-11.tif -s geodata/shp/presence.shp -o geodata/shp/point-buf_stat.csv" ], "execution_count": null, "outputs": [ { "output_type": "stream", "text": [ "processing layer presence\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n", "processing layer presence\n", "0...10...20...30...40...50...60...70...80...90...100 - done.\n" ], "name": "stdout" } ] }, { "cell_type": "markdown", "metadata": { "id": "gYgDnke_mJ89" }, "source": [ "**Remove all the output**" ] }, { "cell_type": "code", "metadata": { "id": "mWC8G93emJ8-" }, "source": [ "%%bash\n", "rm -f geodata/vegetation/GPPcv08-11.tif geodata/LST/*_crop.tif geodata/vegetation/ETmean08-11_crop_trh.tif geodata/vegetation/ETmean08-11_crop_trh.txt vegetation/ETmean08-11_crop.txt geodata/vegetation/ETmosaic.vrt geodata/vegetation/ETmosaic.tif geodata/vegetation/stack_??.tif geodata/vegetation/stack.vrt geodata/vegetation/tiles.* geodata/vegetation/ETmean08-11_crop_proximity.tif geodata/vegetation/ETmean08-11_crop_buffer.tif geodata/dem/slope.tif geodata/dem/aspect.tif geodata/dem/tri.tif geodata/dem/tpi.tif geodata/dem/roughness.tif geodata/vegetation/ETmean08-11_01_trh?.tif geodata/LST/LST_MOYDmax_month1-msk.tif geodata/LST/LST_MOYDmax_monthStdev.tif geodata/LST/LST_MOYDmax_monthMean.tif geodata/LST/LST_MOYDmax_month_mean_stdev.tif geodata/LST/LST_MOYDmax_month.vrt geodata/LST/LST_MOYDmax_monthMean_aggreate10mean.tif geodata/LST/LST_MOYDmax_monthMean_circular11mean.tif geodata/temperature/reclass_ug_bio_3.tif geodata/temperature/reclass_ug_bio_3.txt geodata/shp/polygons_stat.csv geodata/shp/polygons.sqlite geodata/shp/point-buf_stat.csv geodata/shp/point_stat.csv geodata/shp/polygons_stat.* geodata/shp/TM_LARGE_BORDERS.shp.* geodata/shp/TM_UGANDA_BORDERS-0.3.* geodata/vegetation/ETmean08-11_crop.txt" ], "execution_count": null, "outputs": [] } ] }