We get asked daily which remote sensing product is the best – the short answer is ‘it depends’.
CANBERRA, AU, August 9, 2022 — FLINTpro relies on multiple different inputs from remote sensing providers, including land cover, disturbances (fire, harvest etc), soils, plant biomass, and evapotranspiration to name a few.
So, how do you decide what remote sensing product and provider to use for your carbon accounting needs?
The FLINTpro team has worked for over 20 years with different remote sensing products, starting with supporting development of the world’s first continental scale forest cover change products for use in carbon accounting in 1999/2000.
The key underpinning data remains; the mapping of land cover and use change through time (See HERE for a short article on the differences between land cover and land use).
The past 12 months has seen an explosion of new companies offering time-series land cover products. This has been enabled by the ready access to Landsat, Sentinel and other data through groups like Google, Amazon and Microsoft and the operationalisation of artificial intelligence and machine learning. But not all products are created equal.
So, what is our secret to assessing these different products? We assess products according to ‘usefulness’.
Over the past 8 years we have spoken to over 50 remote sensing groups that have products that could be used in FLINTpro. All claim, and can show, great accuracy (typically >80%), but few, (really only 5 to 7), have passed our ‘usefulness’ metric.
Those that do pass have a few things in common: deep experience and expertise in the field, backgrounds in land sector measurement and modelling, and an understanding of reporting policy and guidelines.
How does the FLINTpro team assess usefulness?
Below are the top 6 factors we consider when looking at a new remote sensing product and provider.
Our assessment is that there has been too much focus on resolution, much to the detriment of users/clients and investors. There is no doubt that resolution is important, but it is only part of the decision when choosing a product.
We hope this analysis proves ‘useful’ not only to those looking at using remote sensing, but also to those that are developing new products.
1/ Historic time series length (i.e., how far back in time does the product go?)
Longer time-series mean more accurate carbon estimates, better baselines and more flexibility. This is particularly the case in landscapes with multiple changes in land use or management over time, because there are often long lag times in carbon stocks due to past actions.
It is not enough to know what the carbon stock of an area is, but why it is that stock. If a product does not go back at least 10 years, it may still be useful in carbon assessments, but not for determining the past and potential future impacts of change.
2/ Spatial and temporal resolution (i.e., how small and how often?)
Spatial resolution is often the focus, mainly because higher-resolution data ‘looks’ more real when viewed on a computer. However, for frequent, long time-series products, 10-30 meter resolution data is often best. This does not discount the need for much higher resolution data for calibration or for specific tasks (such as land use planning etc) and there are great groups working in this area, but this is a different use case.
For carbon accounting, temporal resolution is often overlooked. The key is to ensure that the time between maps does not lead to missed events or prevent the reliable calculation of baselines. Large gaps between maps (say >2-3 years) make baselines harder to calculate and require interpolation. They may also lead to some events being missed, such as where forest cover is lost but then subsequently regrows. This last one is particularly important in tropical countries where recovery of vegetation cover can occur very rapidly.
3/ Ability to detect changes, both gain and loss, through time consistently (time-series consistency)
The key to carbon accounting is change. There are several methods for estimating change, but the need to estimate it remains. To be useful, this change data needs to be consistent, reliable and logical. It is surprising how often this is not the case – for example, the same area being ‘deforested’ twice in two years.
Such errors generally occur through two processes:
1) product developers are only comparing two maps and not looking at the time series; and
2) developers simply ‘adding’ a new year of data to their time series without reprocessing the past data to ensure consistency.
To get an accurate picture of carbon stocks you need to see both the losses (such as deforestation), and also the gains (reforestation, regrowth following harvesting or deforestation, even if it is rubber or oil palm).
For example, a map that is 80% accurate in detecting loss but ignores regrowth can lead to gross overestimation of GHG emissions where the same area regrows (and hence sequesters from the carbon) and where this area is subsequently recleared. We have found over 3 fold differences in emissions between maps with similar ‘accuracies’ based on if they include only loss or both loss and gain.
4/ Improvability (and can you drive improvements)
Data and systems are improving daily, and nowhere is this more evident than remote sensing. The ability to process huge volumes of data and improvements in models and statistical techniques is revolutionising the sector, making processes that once took weeks at research labs on supercomputers to days by startups with a cloud compute account and some AI/ML skills.
However, all of these processes won’t necessarily help if:
1) there is insufficient data for calibration and validation; and
2) the developers do not have a detailed understanding of remote sensing, the limitations of the sensors being used and land being monitored.
More data will increase accuracy, but more importantly, so will a continuous feedback loop between the provider and the client of the products to determine where there are issues and to jointly assess how to address these.
The main question here is, if you find errors in the products, which you invariably will, can you work with the provider to improve the product next year? An initially less accurate but improvable product is often more useful than a product that is initially more accurate, but can’t be easily improved. Which one is best will depend on your needs, but you need to know who you are working with.
5/ Cost and timing of updates (cheap or free does not equal good)
In the remote sensing world, you typically get what you pay for. There are, of course, ranges of cost, but thinking that free products are useful for anything more than initial guides is bound to end in disappointment. Prepare yourselves: remote sensing is much cheaper than going to the field, but that does not make it cheap.
Like many products, long-term total costs are more important than initial cost (the “cheap person pays twice” rule applies here). The initial cost will likely be higher as:
1) the provider firstly needs to get the product calibrated and up to specification for the area of interest; and
2) they need to run the data over the time series and check it. Once the system is up and running, annual updates will be far less.
So, when seeking quotes, don’t ask for one run, ask for how much for 5 years of updates at the same time, what is the cost of improvability, and note the timing of when you will get those updates. A long term partnership with your provider will lead to better outcomes.
Finally, ensure that the provider has a clear QA/QC process in place, including QA/QC reports, so that you have certainty in what you are getting. Sending a product back to a provider 3 times for correction will very quickly suck up any perceived cost savings and may jeopardise an entire project in time overruns.
6/ Accuracy (but divided into precision and bias)
There is no point in having an accurate product if it can’t do any of the above. But there is also no point in having a product that can do all the above but is not sufficiently accurate.
How accurate a product needs to be is hard to define and depends on how it is used and the impact on the accuracy of the final outputs (e.g., emissions and removals).
There are two main factors we consider in this assessment.
1) That the accuracy quoted reflects the actual use of the product. Knowing the accuracy of a forest cover map does not mean you know the accuracy of this map for land use and, in particular, land use change; and
2) What is the relative precision and bias of the product? If the product is biased (and many will be), can this be assessed and corrected? In such cases a precise but biased product may be more useful – you can correct for bias but it is often harder to increase precision of the maps themselves without sampling. However, if looking for a general understanding of changes, then less bias may be desirable.
This means that different remote sensing data is useful for different purposes, and by default, it’s unlikely that one remote sensing data product will be useful for all purposes. This is why we didn’t lock FLINTpro to ‘one’ remote sensing product, allowing our clients to use the most useful one!
If you found this article ‘useful’, we encourage you to share it with your team, and save it for future reference when researching remote sensing product options for your environmental projects and business needs.