To Blog

SpaceNet 6: Winning Model Release

July 29, 2020

Preface: SpaceNet LLC is a nonprofit organization dedicated to accelerating open source, artificial intelligence applied research for geospatial applications, specifically foundational mapping (i.e., building footprint & road network detection). SpaceNet is run in collaboration by co-founder and managing partner, CosmiQ Works, co-founder and co-chair, Maxar Technologies, and our partners including Amazon Web Services (AWS)Capella SpaceTopcoderIEEE GRSS, the National Geospatial-Intelligence Agency and Planet.

The SpaceNet 6 challenge officially closes out with the open-source release of our prize winning algorithms for extracting building footprints using a combination of half-meter Capella Space Synthetic Aperture Radar (SAR) and Maxar Worldview 2 Optical Imagery. We believe that the SpaceNet 6 dataset and challenge has helped to further research and expand remote sensing analytics beyond the optical spectrum and into new modalities. With the open sourcing of these algorithms further research can continue and expand.

Models and Weights

All of the top 5 prize winning codebases are now available for download via the SpaceNet github. Each algorithm includes a dockerfile to easily deploy these algorithms to your environments, along with a permissive Apache 2.0 license to encourage their use. Furthermore, the codebase contains detailed readmes written by the prize winning competitors, detailing their algorithms, what worked, what didn’t, and what they would try next.

Additionally, model weights for each of the winning submissions along with the baseline can be found in the SpaceNet s3 bucket:

Building footprint predictions from Zbigniewwojna’s winning SpaceNet 6 algorithm trained on Capella SAR imagery

Model Summary

A full summary of algorithm performance is available here. Each of the top 5 all used ensembles of neural networks, with four of the five relying on slight variants of the newly introduced EfficientNet. EfficientNet has achieved state-of-the-art performance on ImageNet while being markedly smaller and faster than other state-of-the-art convolutional neural networks.

Each competitors overall score (F1 * 100), model architectures, ensemble and pre-training schemes, as well as training and inference time for each of the winning solutions. Note that the total contribution to the total NN’s ensembled is listed in parentheses in the Architectures column. Total training and inference times are calculated based upon an AWS p3.8xlarge instance with 4 NVIDIA Tesla V100 GPUs.

Overall, both the training and inference times were greatly reduced vs. past challenges. Some of that is due to the efficiencies of networks improving, another aspect of this is simply that this is a smaller dataset than some of the previous SpaceNet challenges. Zbigniewwojna’s model was both the best performing and the fastest at inference (~5.4 s/km² on an AWS p3.8xlarge).

For a deeper dive on the winning algorithms and some additional analysis of performance we recommend reading some of our post-challenge SpaceNet 6 blog series:

What’s Next?

The final piece of the SpaceNet 6 puzzle will be put in place as we release an expanded SpaceNet 6 dataset which will introduce both phase information and complex SAR data. Continue to watch the DownlinQ for additional information on the SpaceNet 7 dataset and challenge!

IQT Blog

Insights & Thought Leadership from IQT

Read More