Introduction

Satellites are being launched into space at an exponential rate and are able to produce high quality images in relatively short intervals of time on any part of Earth. The amount of data and types of it are also increasing significantly and in this paper we specifically use Synthetic Aperture Radar (SAR) satellite imagery in order to detect ships traveling through bodies of water. We created a ship counting tool that intakes a start date, end date, and an area of interest and returns the number of ships for each day between the two dates. We propose a new method where the images are first classified into offshore or inshore and a separate object detection algorithm counts the number of ships per image. The classifier and object detection networks are trained using the Large-Scale SAR Ship Detection Dataset-v1.0 (LS-SSDD-v1.0) and deployed on Google Earth Engine. We will be using ships and vessels interchangeably throughout the report.

Applications

Methods

Full Pipeline

To obtain the count of ships within an image, we have broken our pipeline down into 5 major steps. Our final workflow is as follows:

1. The first step of the process is to acquire images from Google Earth Engine. The user can input the coordinates of their area of interest as well as the desired start and end dates into the ship counting script and will start downloading the images.
• (For a more detailed tutorial on how to get the desired coordinates and how to run the ship counting script, please refer to the GitHub Repo)
2. Once the images are downloaded, each image will be preprocessed and split up into sub images one at a time.
3. Then, each sub image is classified as either inshore or offshore.
4. Once classified, the sub image will be fed into its respective object detection model and will be used to detect and count the number of ships present.
5. Finally, the counts for all the sub images will be aggregated together to give the total counts within the full sized image.

The following subsections will explain each step in more detail.

Acquiring Images & Preprocessing

The data we intend to deploy our model on is the Sentinel-1 SAR image collection on Google Earth Engine. To access this data, the user inputs the coordinates of the desired area as well as the start and end dates that the user wants to pull from. The coordinates can be found on the Google Earth Engine Code editor (Please refer to the GitHub Repo for more details.) The set of images will then be downloaded locally as .tif files.

Once downloaded, we have to do some additional cleaning of the image values because as you can see below, the image appears to have a gray tint to it. Fixing this requires clipping images values from -20 to 0 where values less than -20 are set to -20 and values greater than 0 are set to 0. As you can see below, it creates a much more contrasted and clearer image. From there, we pad black bars around the image so that when we split the image in the next step, we will get equally sized sub images.

Inshore-Offshore Classifier

To build our inshore-offshore classifier, we trained on the Large-Scale SAR Ship Detection Dataset-v1.0(LS-SSDD-v1.0) dataset. This dataset contains 15 large SAR images that are broken down into 9000 sub-images with varying presence of land, sea, and ships. Then to find the best model for classification we gathered a list of different machine learning models and assessed the average accuracy of each model on the dataset. From this we found that the K-Nearest Neighbors model performed the best in terms of accuracy out of all the models.

Once we found the best model, we needed to find the best hyperparameters for our model to give us the best performance. To do this we used a technique called Grid Search Cross Validation which essentially tries out different combinations of parameters in the model and outputs the best performing set of parameters. In our case, we found that our K-Nearest Neighbors model worked best with 5 neighbors, kd_tree for the algorithm to compute the neighbors, and with the weights of each datapoint weighted uniformly.

Object Detection Models

For detecting ships within inshore images we trained deep learning object detection model called Faster R-CNN. As we can see below, inshore images are often more complicated with the presence of different land features. Due to this fact, detecting ships can be a little harder of a problem because of that we wanted to use a more robust model that can handle such cases and will allow for more accurate detection of ships when faced with more complex images.

In the opposite case of offshore images, these images are often much less information dense where images feature mostly open ocean and in some cases ships. Since these images are not as complicated, we decided to use the faster and more lightweight model RetinaNet.

Both of these models were built using the deep learning package Pytorch within Python. However, we ran into some problems where we could only use images with ships present in them to train the models. This cut down our total dataset size from 9,000 images to 1,900 images. We tried to fix this problem by introducing image augmentations such as rotations and adding noise to try and increase the size of our dataset, but due to time constraints we were unable to implement these. However, to solve this issue we instead trained our models for more iterations.

Results

Inshore-Offshore Classifier

As we can see from the bar graph and table above, the Decision Tree, Random Forest, and K-Nearest Neighbors models all performed quite similarly in terms of average accuracy and runtime. However, in the end we decided to go with the K-Nearest Neighbors model for its slightly better accuracy despite having a slightly slower runtime. This is because within our application, we want to ensure more accurate inshore/offshore predictions so that the images are fed into their correct object detection models.

Object Detection Models

From the above metrics, we can see that the Faster R-CNN had the highest accuracy scores, but was slowest on inference. Because of this, we decided to go with Faster R-CNN for the more complicated inshore images. For the offshore images, we went with RetinaNet due to the fact that it performs almost as good as well as a little bit faster than Faster R-CNN. However, these speed and accuracy differences are quite small.

Full Pipeline

We can see from above that our tool performs slightly faster than just using Faster R-CNN for both inshore and offshore, and does about the same performance wise. This tells us that our tool predicts just as well as Faster R-CNN, and we hope that the small speed increase will translate to scale.

Note: All results from the inshore-offshore classifier and object detection models are derived from the test set of the Large-Scale SAR Ship Detection Dataset-v1.0(LS-SSDD-v1.0) dataset.

Conclusion

Our goal is for the ship counting tool we have created to be used for other projects such as political policy monitoring, or economic modeling. The modularity of it allows for different parts of the tool to be easily replaced or reconfigured. For example, if a new object detection model is created it can be easily trained and swapped with the current one in the tool. Another example is if a new source of satellite imagery data becomes open source or purchased it can be inputted into the tool instead of Google Earth Engine. Overall, we hope that others continue to build upon this tool or apply it for a beneficial purpose.