Introduction
Welcome to Cloud-Based Remote Sensing with Google Earth Engine: Fundamentals and Applications! This book is the product of more than a year of effort from more than a hundred individuals, working in concert to provide this free resource for learning how to use this exciting technology for the public good. The book includes work from professors, undergraduates, master’s students, PhD students, postdocs, and independent consultants. The book is broadly organized into two halves.
The first half, Fundamentals, is a set of 31 labs designed to take you from being a complete Earth Engine novice to being a quite advanced user. The second half, Applications, presents you with a tour of the world of Earth Engine across 24 chapters, showing how it is used in a very wide variety of settings that rely on remote sensing data.
Using several strategies, we have worked hard to ensure a high-quality body of work across 55 chapters, more than 10,000 lines of code, and 250,000 words:
Fundamentals
The goal of the Fundamentals half of the book is to introduce users to Earth Engine with a set of sequenced labs that are intended to be done in order. We first illustrate the operation of Earth Engine on individual multi-band images, with techniques that form the core vocabulary for Earth Engine image processing. We then introduce more complex options for data representation, data enhancements, and data interpretation. The tools introduced in those sections build on the core analysis ix x Introduction strategies, with important techniques that can be highly valuable for addressing specialized analysis problems. The Fundamentals half of the book is intended to have two main entry points: at Chaps. 1 and 12. It is comprised of six thematic parts, described below:
Part I: Programming and Remote Sensing Basics
Those who are almost entirely unfamiliar with remote sensing data, or are almost entirely unfamiliar with programming, or both, are encouraged to start at the beginning: Part I, Chap. 1 That chapter assumes that the reader has no programming experience, and begins with the most fundamental first steps: defining variables, printing values, etc. The rest of Part I teaches the basics of displaying remote sensing data (Chap. 2), gives a brief survey of Earth Engine’s very large data catalog (Chap. 3), and presents the vocabulary of remote sensing (Chap. 4), intended for both novices and those familiar with remote sensing but just starting with Earth Engine.
Part II: Interpreting Images
Part II assumes that you are aware of the topics, terms, and methods encountered in Part I. This part illustrates the basic operations that can be done on individual satellite images. Like in Part I, the chapters are arranged in sequence, here with a goal of leading you through how bands of remote sensing images can be manipulated to form indices (Chap. 5), which can be classified using a variety of supervised and unsupervised techniques (Chap. 6), giving results that can be assessed and further adjusted (Chap. 7).
Part III: Advanced Image Processing
Part III presents more advanced image processing techniques that can be accessed in Earth Engine using operations on a single image. These include finding a relationship between two or more bands using regression (Chap. 8), linear combinations of bands to produce the tasseled cap and principal components transformations (Chap. 9), morphological operations on classified images to highlight or de-emphasize spatial characteristics (Chap. 10), and object-based image analysis that groups pixels into spectrally similar spatially contiguous clusters (Chap. 11).
Part IV: Interpreting Image Series
Part IV introduces the analysis of time series in Earth Engine. Using long-term series of imagery can inform change and stability in land-use and land-cover patterns around the world. This series of chapters begins by presenting the filter, map, reduce paradigm (Chap. 12), which is used throughout Earth Engine for manipulating collections of both vector and raster data. The next chapter shows strategies for learning about and visualizing image collections (Chap. 13). Subsequent chapters show pixel-by-pixel calculations, including the aggregation of image characteristics in time (Chap. 14); detection and removal of cloud artifacts (Chap. 15), and the detection of change by comparing images at two dates (Chap. 16). Later chapters involve assessments that summarize a pixel’s history, at annual scales (Chap. 17), in harmonically repeating patterns (Chap. 18), and at sub-annual scales (Chap. 19). Long-time series of image classifications can be interpreted for stability and change via Bayesian methods (Chap. 20), and lag effects in time series can be detected (Chap. 21).
Part V: Vectors and Tables
Part V shows how vector data can be used in Earth Engine. This includes uploading vector data and summarizing images within irregular polygons (Chap. 22), converting between raster and vector formats (Chap. 23), and computing zonal statistics efficiently with sophisticated functions (Chap. 24). The chapters conclude with a useful set of advanced vector operations (Chap. 25) and a digitizing tool for drawing features that change across the time span of an image collection (Chap. 26).
Part VI: Advanced Topics
With the foundation provided by the first five parts of the Fundamentals, you will be ready to do more complex work with Earth Engine. Part VI reveals some of these advanced topics, including advanced techniques for better raster visualization (Chap. 27), collaborating with others by sharing scripts and assets (Chap. 28), scaling up in time and space in efficient and effective ways (Chap. 29), creating apps to share to the wider public (Chap. 30), and linking to outside packages (Chap. 31).
The first half, Fundamentals, is a set of 31 labs designed to take you from being a complete Earth Engine novice to being a quite advanced user. The second half, Applications, presents you with a tour of the world of Earth Engine across 24 chapters, showing how it is used in a very wide variety of settings that rely on remote sensing data.
Using several strategies, we have worked hard to ensure a high-quality body of work across 55 chapters, more than 10,000 lines of code, and 250,000 words:
- The text of each chapter has been reviewed for the clarity of the scientific content and instructions, on a minimum of three occasions by people working independently of each other, for more than 350 detailed chapter reviews.
- The code in each chapter has been reviewed at Google for adherence to best practices and subsequently reviewed for consistency with the instructions in the book’s text by two Earth Engine experts.
- A professional copy-editing team has worked through the entire book text, ensuring that all chapters have a consistent sound and approach, while preserving the voice of the authors.
Fundamentals
The goal of the Fundamentals half of the book is to introduce users to Earth Engine with a set of sequenced labs that are intended to be done in order. We first illustrate the operation of Earth Engine on individual multi-band images, with techniques that form the core vocabulary for Earth Engine image processing. We then introduce more complex options for data representation, data enhancements, and data interpretation. The tools introduced in those sections build on the core analysis ix x Introduction strategies, with important techniques that can be highly valuable for addressing specialized analysis problems. The Fundamentals half of the book is intended to have two main entry points: at Chaps. 1 and 12. It is comprised of six thematic parts, described below:
Part I: Programming and Remote Sensing Basics
Those who are almost entirely unfamiliar with remote sensing data, or are almost entirely unfamiliar with programming, or both, are encouraged to start at the beginning: Part I, Chap. 1 That chapter assumes that the reader has no programming experience, and begins with the most fundamental first steps: defining variables, printing values, etc. The rest of Part I teaches the basics of displaying remote sensing data (Chap. 2), gives a brief survey of Earth Engine’s very large data catalog (Chap. 3), and presents the vocabulary of remote sensing (Chap. 4), intended for both novices and those familiar with remote sensing but just starting with Earth Engine.
Part II: Interpreting Images
Part II assumes that you are aware of the topics, terms, and methods encountered in Part I. This part illustrates the basic operations that can be done on individual satellite images. Like in Part I, the chapters are arranged in sequence, here with a goal of leading you through how bands of remote sensing images can be manipulated to form indices (Chap. 5), which can be classified using a variety of supervised and unsupervised techniques (Chap. 6), giving results that can be assessed and further adjusted (Chap. 7).
Part III: Advanced Image Processing
Part III presents more advanced image processing techniques that can be accessed in Earth Engine using operations on a single image. These include finding a relationship between two or more bands using regression (Chap. 8), linear combinations of bands to produce the tasseled cap and principal components transformations (Chap. 9), morphological operations on classified images to highlight or de-emphasize spatial characteristics (Chap. 10), and object-based image analysis that groups pixels into spectrally similar spatially contiguous clusters (Chap. 11).
Part IV: Interpreting Image Series
Part IV introduces the analysis of time series in Earth Engine. Using long-term series of imagery can inform change and stability in land-use and land-cover patterns around the world. This series of chapters begins by presenting the filter, map, reduce paradigm (Chap. 12), which is used throughout Earth Engine for manipulating collections of both vector and raster data. The next chapter shows strategies for learning about and visualizing image collections (Chap. 13). Subsequent chapters show pixel-by-pixel calculations, including the aggregation of image characteristics in time (Chap. 14); detection and removal of cloud artifacts (Chap. 15), and the detection of change by comparing images at two dates (Chap. 16). Later chapters involve assessments that summarize a pixel’s history, at annual scales (Chap. 17), in harmonically repeating patterns (Chap. 18), and at sub-annual scales (Chap. 19). Long-time series of image classifications can be interpreted for stability and change via Bayesian methods (Chap. 20), and lag effects in time series can be detected (Chap. 21).
Part V: Vectors and Tables
Part V shows how vector data can be used in Earth Engine. This includes uploading vector data and summarizing images within irregular polygons (Chap. 22), converting between raster and vector formats (Chap. 23), and computing zonal statistics efficiently with sophisticated functions (Chap. 24). The chapters conclude with a useful set of advanced vector operations (Chap. 25) and a digitizing tool for drawing features that change across the time span of an image collection (Chap. 26).
Part VI: Advanced Topics
With the foundation provided by the first five parts of the Fundamentals, you will be ready to do more complex work with Earth Engine. Part VI reveals some of these advanced topics, including advanced techniques for better raster visualization (Chap. 27), collaborating with others by sharing scripts and assets (Chap. 28), scaling up in time and space in efficient and effective ways (Chap. 29), creating apps to share to the wider public (Chap. 30), and linking to outside packages (Chap. 31).
Applications
Part of the appeal of such an extensive platform in widespread adoption, and one of the most satisfying aspects of editing this book project, has been exposure to the wide range of applications built on this platform. With thousands of papers now xii Introduction published using Earth Engine, the 24 chapters presented here are not intended to be a complete survey of who uses Earth Engine nor how it is best used. We have worked to find applications and authors who fit the following characteristics: (1) they created vibrant work that can be appreciated by both novices and experts; (2) the work has one or more unique characteristics that can have appeal beyond the scope of the book chapter’s application; and (3) the presentation does not seek to be the final word on a subject, but rather as an opening of the subject to others.
Part VII: Human Applications
Earth Engine is used for both large-scale and relatively small-scale interpretations of images and time series of human activity and influence. These include agricultural environments (Chap. 32), the urban built environment (Chaps. 33 and 34), effects on air quality and surface temperature in cities (Chaps. 35 and 36), health and humanitarian activities (Chaps. 37 and 38), and the detection of illegal human activity under cloud cover (Chap. 39).
Part VIII: Aquatic and Hydrological Applications
Earth Engine is also used in the aquatic realm to understand hydrologic patterns across large areas. These include detection of subsurface features like groundwater (Chap. 40) and cover of the sea floor (Chap. 41). Surface water can be detected in satellite series, opening the opportunity to detect rapid changes like floods (Chap. 42) and to map rivers across the globe (Chap. 43). Human influence on hydrological systems can be detected and quantified in patterns of water balance and drought (Chap. 44) and changing patterns of the timing of snow (Chap. 45).
Part IX: Terrestrial Applications
Some of the many terrestrial applications of Earth Engine close out the book. These include the monitoring of active fires using multiple sensors and presented with user-interface apps (Chap. 46). Mangroves are complex systems that have elements of both the terrestrial and aquatic realms; they need to be both mapped (Chap. 47) and monitored for change (Chap. 48). Changes in forests can include both human degradation and deforestation (Chap. 49); the detection of these changes can be aided with the help of multiple sensors (Chap. 50). Analysts often need to use location-specific weather data (Chap. 51) and to create randomly located points for proper statistical analyses (Chap. 52). Rangelands, a major land use worldwide, present subtle changes over long-time periods that require distinctive techniques (Chap. 53). Finally, conservation of natural resources requires Introduction xiii understanding both the effect of precipitation changes on area affected by disturbance (Chap. 54) and the effectiveness of protected areas in conserving places as intended (Chap. 55)
Part of the appeal of such an extensive platform in widespread adoption, and one of the most satisfying aspects of editing this book project, has been exposure to the wide range of applications built on this platform. With thousands of papers now xii Introduction published using Earth Engine, the 24 chapters presented here are not intended to be a complete survey of who uses Earth Engine nor how it is best used. We have worked to find applications and authors who fit the following characteristics: (1) they created vibrant work that can be appreciated by both novices and experts; (2) the work has one or more unique characteristics that can have appeal beyond the scope of the book chapter’s application; and (3) the presentation does not seek to be the final word on a subject, but rather as an opening of the subject to others.
Part VII: Human Applications
Earth Engine is used for both large-scale and relatively small-scale interpretations of images and time series of human activity and influence. These include agricultural environments (Chap. 32), the urban built environment (Chaps. 33 and 34), effects on air quality and surface temperature in cities (Chaps. 35 and 36), health and humanitarian activities (Chaps. 37 and 38), and the detection of illegal human activity under cloud cover (Chap. 39).
Part VIII: Aquatic and Hydrological Applications
Earth Engine is also used in the aquatic realm to understand hydrologic patterns across large areas. These include detection of subsurface features like groundwater (Chap. 40) and cover of the sea floor (Chap. 41). Surface water can be detected in satellite series, opening the opportunity to detect rapid changes like floods (Chap. 42) and to map rivers across the globe (Chap. 43). Human influence on hydrological systems can be detected and quantified in patterns of water balance and drought (Chap. 44) and changing patterns of the timing of snow (Chap. 45).
Part IX: Terrestrial Applications
Some of the many terrestrial applications of Earth Engine close out the book. These include the monitoring of active fires using multiple sensors and presented with user-interface apps (Chap. 46). Mangroves are complex systems that have elements of both the terrestrial and aquatic realms; they need to be both mapped (Chap. 47) and monitored for change (Chap. 48). Changes in forests can include both human degradation and deforestation (Chap. 49); the detection of these changes can be aided with the help of multiple sensors (Chap. 50). Analysts often need to use location-specific weather data (Chap. 51) and to create randomly located points for proper statistical analyses (Chap. 52). Rangelands, a major land use worldwide, present subtle changes over long-time periods that require distinctive techniques (Chap. 53). Finally, conservation of natural resources requires Introduction xiii understanding both the effect of precipitation changes on area affected by disturbance (Chap. 54) and the effectiveness of protected areas in conserving places as intended (Chap. 55)
Uses of This Book
We have strived to produce a book that can be used by people working in a very wide range of settings: from pre-college students in a classroom, to university courses, to professional development workshops, to individuals working alone. The first seven Fundamentals chapters are tightly sequenced and assume no programming or remote sensing experience; their intention is to build a base of knowledge for novices to succeed. The remaining Fundamentals chapters are sequenced within their larger sections. In their introductions, each Fundamentals chapter lists earlier topics and chapter numbers that form the foundation for understanding that chapter’s concepts, under the heading “Assumes you know how to…” However, although the concepts in the chapters do build on each other, each chapter’s steps and data stand alone and do not require that you follow every preceding chapter in order to succeed. This should permit you to use individual sections of the book to solve specific problems you encounter. The Learning Outcomes index contains listings such as “Attaching user-defined metadata to an image when exporting,” “Computing zonal summaries of calculated variables for elements in a Feature Collection,” and “Using reducers to implement regression between image bands,” among more than 150 other examples. The Applications chapters each stand alone and provide a guide both to the use of Earth Engine for that issue and to the technological details of how that application has been addressed by those authors.
We have strived to produce a book that can be used by people working in a very wide range of settings: from pre-college students in a classroom, to university courses, to professional development workshops, to individuals working alone. The first seven Fundamentals chapters are tightly sequenced and assume no programming or remote sensing experience; their intention is to build a base of knowledge for novices to succeed. The remaining Fundamentals chapters are sequenced within their larger sections. In their introductions, each Fundamentals chapter lists earlier topics and chapter numbers that form the foundation for understanding that chapter’s concepts, under the heading “Assumes you know how to…” However, although the concepts in the chapters do build on each other, each chapter’s steps and data stand alone and do not require that you follow every preceding chapter in order to succeed. This should permit you to use individual sections of the book to solve specific problems you encounter. The Learning Outcomes index contains listings such as “Attaching user-defined metadata to an image when exporting,” “Computing zonal summaries of calculated variables for elements in a Feature Collection,” and “Using reducers to implement regression between image bands,” among more than 150 other examples. The Applications chapters each stand alone and provide a guide both to the use of Earth Engine for that issue and to the technological details of how that application has been addressed by those authors.
We Want Your Feedback
We would like to get your feedback on chapters of the book. This feedback is useful in two ways: to make your experience better and to help others who will come after.
We would like to get your feedback on chapters of the book. This feedback is useful in two ways: to make your experience better and to help others who will come after.
- Helping others: How hard is a chapter? How long did it take? Would you recommend it to someone else? What did you learn? Is something confusing? Has code stopped working? To review a lab, visit https://bit.ly/EEFA-review when you’ve finished. By doing this, you will keep the book current and provide future book users with your assessment of the strengths and weaknesses of each chapter. Please give feedback!
- Helping yourself: What have others already said about a chapter you are planning to do? How long will it take? How interesting is it? You can see what others have said at https://bit.ly/EEFA-reviews-stats.