summaryrefslogtreecommitdiff
path: root/tcwg-cleanup-stale-containers.sh
AgeCommit message (Collapse)Author
2021-03-22start-container-docker.sh: Move workaround for dockerhub's limitsMaxim Kuvyrkov
... to dockerfiles.git/tcwg-base/tcwg-host/docker-wrapper . Our workaround to dockerhub's pull limits in start-container-docker.sh works well, but with one exception -- it doesn't properly work when container is started on a remote host, which we do a lot in tcwg-benchmark-* jobs. To solve this we migrate workaround logic to docker wrapper script, and add a new docker command for it -- "maybepull". Change-Id: I131f1adc56052164a76c5dfaf9d07a4d2f24ec43
2021-01-04Docker image pull: And one more fixMaxim Kuvyrkov
... to ignore "docker rmi" failures on used images. Change-Id: I69fd40c45fe5f0b36e4b833097c2ac4e5a57b984
2021-01-04Docker image pull: Fix removing previously untagged imagesMaxim Kuvyrkov
... and update "pull" timestamps in tcwg-start-container.sh Change-Id: Ib5638a9fb22c341527443fd3ff2f8431f7aa1fe0
2021-01-03Docker image pull/use: Fix and improveMaxim Kuvyrkov
Change-Id: Ieb39c841151533d7553786f78f80fbb7a62857e2
2021-01-03Reduce "docker pull" requests to dockerhubMaxim Kuvyrkov
We attempt to run all our build using current versions of docker images. Unfortunately, now that dockerhub limits pull requests, we need to be more considerate to when we pull the image or attempt to use the local copy. This is the first stab at the problem. We use stamp files under /home/shared/docker/ to track times of last image pull and last image use. We then run "docker pull" on images that are older than a day to avoid using terribly-outdated local copies. We also switch to deleting images that haven't been used for 3+ days in tcwg-cleanup-stale-containers.sh -- instead of deleting all not-used-at- the-moment images. Change-Id: I3839fe030b30adaee318f7453c6d2e47185f4ffc
2020-04-30Shellcheck fixes round 4David Spickett
Fixes for: https://github.com/koalaman/shellcheck/wiki/SC2207 Are using the bash 4.x 'mapfile' syntax where the input to the array is seperated by newlines. If it's separated by spaces we use 'read' instead. Support for selecting libc in MakeRelease.job was removed in 3fa23da28ad73d0fe4b512c4aab269b5280a6650. So remove references to "libc" which were expanding to nothing anyway. In tcwg-llvm-common.sh PROJECTS->BRANCHED_PROJECTS is not strictly needed. However shellcheck can't tell that the local PROJECTS in this file is different from the PROJECTS in tcwg-llvm-build.sh. Change-Id: Ie55968e3bf185d0bd5795fe48014636cda1009bc
2020-04-29tcwg-cleanup-stale-containers.shDavid Spickett
Don't add empty container names to the list to remove. This was causing us to run docker commands without container names. Change-Id: Iddf2c7aa268aec4a6254db1bf6e4c66850bb7dfa
2020-04-28Shellcheck fixes round 3David Spickett
Now using the version that's present in the bionic images. Meaning we have more warnings but can also set the minimum level to ignore a lot of them. Fix the check for whether shellcheck supports --severity. (we have pipefail on, but we expect shellcheck to fail in this case) Change-Id: I296b8554591b7d327c75393e3924184abc6512b7
2020-04-22Various shellcheck warning fixesDavid Spickett
First round of low hanging fruit from the files with the fewest warnings. Change-Id: I60ad57b5bfb2f0be9bb6506252c820f44d055200
2020-04-20Shellcheck warning fixesDavid Spickett
(all error/style level warnings) SC1035: missing whitespace SC2068: double quote array expansions SC2145: argument mixes string and array SC1066: Don't use $ on the left side of assignments SC2126: Use grep -c instead of grep|wc Change-Id: Ia53070d3e2c9aea9f626fdb7b2b5bfc9e1f8e682
2020-04-10tcwg-cleanup-stale-containers.sh: Improve docker image cleanupMaxim Kuvyrkov
Use Docker's internal command to cleanup images. Change-Id: Ibb4605761d3c638fcf18d061763c1e2594c7b890
2019-03-29tcwg-cleanup-stale-containers.sh: Don't fail on dev machinesMaxim Kuvyrkov
... due to present dangling volumes, which are expected on dev machines. Change-Id: Iec4aefbc025ee0546458299db442597d1fc22397
2019-03-21tcwg-cleanup-stale-containers.sh: Simplify image removalMaxim Kuvyrkov
The current code seems to miss some of the dependencies of dangling images, so we attempt to delete a used image. Rather than being more careful in not attempting to delete used images, we non-forcefully attempt to delete all images, and ignore failures. Change-Id: I240ab7faf6f21d7f60451375992805fcc9124f60
2019-03-07tcwg-cleanup-stale-containers.sh: Make output easier to understand.Christophe Lyon
Add a few more messages, to make the Jenkins console easier to understand. Change-Id: I2bab8aaf5333c4e7f8d8d8681986c961336f1fd6
2019-03-06tcwg-cleanup-stale-containers.sh: Do not fail if there is no ssh-agent ↵Christophe Lyon
process to kill. Handle two cases where killall can fail: - command not found, in which case we want an error so that we know we have to install killall - no ssh-agent process running, which is OK Change-Id: I0d6d7e40eed6db7c6250f2231f05b8cca73c8355
2019-03-04tcwg-cleanup-stale-containers.sh: Report failure to kill stale ssh-agent ↵Christophe Lyon
processes. Without this patch, failure to kill such processes is not reported as part of the script exit status. This happens for instance when the killall command is not installed on the machine. Change-Id: Ic0fa10ce6b69f12fb2382cfa3680fcc1c7d1b4bd
2019-02-26tcwg-cleanup-stale-containers.sh: Fix volume cleanupMaxim Kuvyrkov
"Echo" concatenates all elements into a single string, so we need to break them up for "grep" to match its pattern. Change-Id: I954ca80fc6f0e289185780f76988688ce0f1ed78
2018-06-26Remove last occurences of docker using TCP socket instead of file socket.Maxim Kuvyrkov
Change-Id: I19ed211d69326ff34a239cb4cc80a6f2b72ccd63
2018-06-05tcwg-cleanup-stale-containers.sh: Handle non-zero return codesMaxim Kuvyrkov
... when no matches for "grep" or "killall" are found. Change-Id: I800f06607e01c3a1d167ebdf9a2c3eaeb75559a1
2018-06-05tcwg-cleanup-stale-containers.sh: Add cleanup of ssh-agent processes.Maxim Kuvyrkov
Change-Id: Ia50672fbab9ade7922feb12a6199f4ae581c18cd
2018-06-05tcwg-cleanup-stale-containers.sh: Avoid cleaning up named volumes.Maxim Kuvyrkov
We now start to keep useful data in docker volumes named host-home, home-$USER, etc. Change-Id: I4df6355de4fca9bcdc3073edefb8a6be4d97a61c
2018-05-07tcwg-cleanup-stale-containers.sh: Check if we have more containers than a ↵Christophe Lyon
threshold. Implements TCWG-1404. We want to make sure we do not have more containers left than a given threshold, to make sure there is no problem with automatically created containers becoming too numerous and not properly cleaned up. Change-Id: Ia740b58d1b75cdcd1fa134b551613e9b043ada34
2018-05-07tcwg-cleanup-stale-containers.sh: Fix another typo...Christophe Lyon
Change-Id: I6626b870109f56619930017e6b4e8f40d65e52f2
2018-05-07tcwg-cleanup-stale-containers.sh: Fix typo.Christophe Lyon
Change-Id: I0acd334a54ebf9160ec706bcc0024acebe76e627
2018-05-07tcwg-cleanup-stale-containers.sh: Remove only containers started by Jenkins.Christophe Lyon
Implements TCWG-1404. Containers started by Jenkins jobs have names starting with a number, we want to cleanup only those. This behaviour is hardcoded by this patch, but controlled by a flag we can control later if needed. Change-Id: Idbf8eb6cfc44ccd7cdaad08019522f670046573e
2017-07-24tcwg-cleanup-stale-containers.sh: Use docker via TCP socketMaxim Kuvyrkov
... instead of via local file socket. This works around problem of tcwg-buildslave not being in "docker" group. Change-Id: I4764b0cbc1ba8d6474874d4d1f4550e4cd5de651
2017-05-02tcwg-cleanup-stale-containers: Rework container time calculation.Maxim Kuvyrkov
Thanks, Charles! Change-Id: Iba7f6ba6ad5f51eaf244c38ad68f37150d080a4b
2017-05-02tcwg-cleanup-stale-containers: Improve handling of exit codeMaxim Kuvyrkov
Exit code is now set to non-zero only when unexpected events happen. Expected cleanups do not increase exit code anymore. Change-Id: I009825e7b8d7ba16303ec3dd2aeb7ae69110168b
2017-04-28tcwg-cleanup-stale-containers: Add usageMaxim Kuvyrkov
Change-Id: Ie12e9a64a4ed1d21ab9c155c3f263467c950176d
2017-04-28tcwg-cleanup-stale-containers: Restrict debugging feature.Maxim Kuvyrkov
Change-Id: Icbdf341f268ba3ff18552428e9466cc371450cad
2017-04-26tcwg-cleanup-stale-containers: Remove unused options.Maxim Kuvyrkov
Change-Id: I9169344fbf1657b22de80cd99f209618f2ce6bfd
2017-04-26tcwg-cleanup-stale-containers: Indent.Maxim Kuvyrkov
Change-Id: I18aecdf987ab3a479e26ab7f7c812a17751bdfd7
2017-04-26tcwg-cleanup-stale-containers: Allow override of docker command for testingMaxim Kuvyrkov
E.g., DOCKER="docker -H build-02.tcwglab:2375" ./tcwg-cleanup-stale-containers.sh Change-Id: I7424ab999d3305db9059dbb773a3f4371b4873a7
2017-04-26tcwg-cleanup-stale-containers: Rework to stop running containersMaxim Kuvyrkov
... instead of removing them. This improvement makes cleanup job to stop running containers after [default] 10 hours, and then remove stopped containers after 10 days. Change-Id: I1411a67788890b56cf6d08ee265beccdcdb7bd74
2017-04-19tcwg-cleanup-stale-containers.sh: Fix typos.Maxim Kuvyrkov
Change-Id: I7519d6573aae3b95c304e821fcc5dcd40bfa6f58
2017-04-12tcwg-cleanup-stale-containers.sh: Fix typos.Maxim Kuvyrkov
Change-Id: I49fd34eb85acabd916e61f5eb7f95a3e4398f4fd
2017-04-12tcwg-cleanup-stale-containers: Allow partial cleanupsMaxim Kuvyrkov
In our current workflow we should not generate stale volumes or containers, but we can generate stale images. Add options to allow cleaning up what we need, instead of all or nothing. Change-Id: Id270a7e51f4c931c32a08d2f244e727a7d783750
2016-11-17tcwg-cleanup-stale-containers: Reword removal of unused imagesMaxim Kuvyrkov
... to avoid spurious warning from removing untagged images that are used by non-running containers. Change-Id: I6d8f665c2c1559cd7826236cbfac16cbc48e1f31
2016-11-17tcwg-cleanup-stale-containers: Improve detection of dangling imagesMaxim Kuvyrkov
Change-Id: I68a2df464a9fe1a9563ad2e5b0df068890146082
2016-11-14tcwg-cleanup-stale-containers: And another typoMaxim Kuvyrkov
... what can I say ... Monday morning. Change-Id: Ic6b44cd5c0f9ccec69665fa997b96fdecfd89735
2016-11-14tcwg-cleanup-stale-containers: Fix more typosMaxim Kuvyrkov
Change-Id: I9ed79a09f2a91d7ca66ae8055aab29b6a4bc702d
2016-11-14tcwg-cleanup-stale-containers: Fix typoMaxim Kuvyrkov
Change-Id: I553c12efa9d4933286612b91e3a0faf158e78911
2016-11-14tcwg-cleanup-stale-containers: Cleanup untagged imagesMaxim Kuvyrkov
... which we generate when commiting a container for debugging. If the debug container is still present, then image removal will [expectedly] fail, so only unused images will be removed. Change-Id: If2790d19ff4d944077ebb6f47d36559df20060da
2016-11-11tcwg-cleanup-stale-containers: Remove unnecessary restrictionMaxim Kuvyrkov
Change-Id: I9a763abb102ac33eae9a55fa36bd6754ca7eb61e
2016-11-10tcwg-cleanup-stale-containers: Handle "days" output from "docker ps"Maxim Kuvyrkov
Change-Id: Ia093265c5220910f54d792c8b0fc27f3086bbbbc
2016-11-08tcwg-cleanup-stale-containers: Handle dangling volumes and "docker rm" exit codeMaxim Kuvyrkov
Handling dangling volumes is straight-forward. For unknown to me reasons "docker rm" return non-zero exit code when successfully removing a container. Maybe it is proxying exit code from the container's main process (sshd in our case). In any case, we now switch to the meaning of the exit code to zero -- nothing to do; non-zero -- something was cleaned up. Change-Id: I16eb5946055a828511d320dcfc28933c7af5a2d8
2016-11-08tcwg-cleanup-stale-containers: Fix bugMaxim Kuvyrkov
Change-Id: I086487f4e33e5ba0e9b02e973827a8363bb0fa1e
2016-11-08tcwg-cleanup-stale-containers: Handle verboseMaxim Kuvyrkov
Change-Id: If3898928a9231b90e115e60fd400cbb29ff4b553
2016-11-08tcwg-cleanup-stale-containers: Make executableMaxim Kuvyrkov
Change-Id: I7ab120d24e659bb238dfce6e6c044db69160e270
2016-11-08tcwg-cleanup-stale-containers: Set status for dry-runsMaxim Kuvyrkov
Change-Id: I2d92f4eb5b76a1a62cf09f1ed021756a2f9fcceb
generated by cgit v1.2.3 (git 2.25.1) at 2024-04-27 22:50:50 +0000