Reduce "docker pull" requests to dockerhub

We attempt to run all our build using current versions of docker images.
Unfortunately, now that dockerhub limits pull requests, we need to be
more considerate to when we pull the image or attempt to use the local
copy.

This is the first stab at the problem.  We use stamp files under
/home/shared/docker/ to track times of last image pull and last image
use.  We then run "docker pull" on images that are older than a day
to avoid using terribly-outdated local copies.

We also switch to deleting images that haven't been used for 3+ days in
tcwg-cleanup-stale-containers.sh -- instead of deleting all not-used-at-
the-moment images.

Change-Id: I3839fe030b30adaee318f7453c6d2e47185f4ffc

1 files changed, 16 insertions, 1 deletions

diff --git a/tcwg-cleanup-stale-containers.sh b/tcwg-cleanup-stale-containers.sh
index 2fa246cf..fe626621 100755
--- a/tcwg-cleanup-stale-containers.sh
+++ b/tcwg-cleanup-stale-containers.sh
@@ -176,7 +176,22 @@ else
 fi
 
 if $cleanup_images; then
-    docker image prune -a -f
+    # See start-container-docker.sh for background on image stamp files.
+    stamp_dir=/home/shared/docker
+
+    # Untag and prune images that haven't been used for 3 days or more.
+    for image in $(docker images --format "{{.Repository}}:{{.Tag}}"); do
+	image_stamp="$stamp_dir/$(echo "$image" | tr "/:" "_")"
+	remove_if_not_used_since=$(($(date +%s) - 3*24*60*60))
+	# Use negative comparison to handle non-existent stamp files.
+	if ! [ "$(stat -c %Z "$image_stamp.use" 2>/dev/null)" \
+		   -gt $remove_if_not_used_since ]; then
+	    # Untag the image.
+	    docker rmi "$image"
+	fi
+    done
+    # Prune untagged images.
+    docker image prune -f
 else
     echo "DRY_RUN: NOT REMOVING UNUSED IMAGES"
 fi