summaryrefslogtreecommitdiff
path: root/start-container-docker.sh
AgeCommit message (Collapse)Author
2018-06-01*.sh: Switch to *-tcwg-tested docker images.Maxim Kuvyrkov
Change-Id: I0abd9f0c7ec0da98990459cf175eb4e076bdab8d
2018-05-28Add -H :2375 to remote docker commands.Christophe Lyon
A previous patch replaced docker $host:2375 cmd with ssh $host cmd but this needs to be replaced with ssh $host -H :2375 cmd Change-Id: I40c58c466b1d3d20a9979205f63f11acb8f864ed
2018-05-24Connect to docker via ssh rather than directly to port 2375.Christophe Lyon
This allows to manage containers using ssh only, the docker port does not need to be open to external connections. This allows to start containers on hosts outside of the lan. Basically it is a matter of changing: -DOCKER="docker -H $docker_host:2375" +DOCKER="ssh $docker_host docker" in jenkins-helpers.sh, start-container-docker.sh and start-container-qemu.sh. Change-Id: I1051896521ee47640cdeefc9eed3324661ee0a9f
2018-05-07start-container-docker.sh: Add "--newuser username[:uid]" optionMaxim Kuvyrkov
To simplify developer use of docker / qemu containers. Change-Id: Ib0c4152de5c691cf40e4cb6ecf45dc9126ac9fa6
2018-05-04start-container-docker.sh: Make friendlier to running outside of JenkinsMaxim Kuvyrkov
Don't expect $BUILD_NUMBER and $JOB_NAME to be set, and require $WORKSPACE only for "--task build" containers. Change-Id: I6f56a7550da82693a3223b88869137895eb61f37
2018-02-01start-container-docker.sh: Fix starting containers on localhost.Maxim Kuvyrkov
DNS doesn't resolve localhost.tcwglab, it is parsed only by ssh. Change-Id: I54a455d14aeea39d303678a6d9999d5254b8a3c3
2018-01-31start-container-docker.sh: Improve handling of defaultsMaxim Kuvyrkov
... for --arch, --session-host, --node -- to make it easier to run scripts in local containers (via upcoming docker-run.sh helper script). Change-Id: I0303c40a8e4fe173d64745e8e750f5ef43ddc817
2017-09-20start-container-docker.sh: Add --user option.Christophe Lyon
This is useful for developers using containers with only tcwg-buildslave account, in conjuction with --cp_file to copy the user's authorized_keys into the tcwg-buildslave account in the container. Change-Id: Ie85204660d4352d27672115f131f6f85a640fe64
2017-09-08start-container-docker.sh: Make usable with calling via "bash -x"Maxim Kuvyrkov
"Set +x" disables effect of "bash -x", so remove set -x/+x and use "echo" to print out "docker run" command. Change-Id: I931533d3c0af7616eb041e36e76d8e3c87c671d4
2017-09-07start-container-docker.sh: Fix typo.Maxim Kuvyrkov
Change-Id: If59eba28dd8f929d63481e039649d219d9430285
2017-09-06start-container-docker.sh: Add --cp_file optionMaxim Kuvyrkov
This option is targeted to be used by developers to replace /home/tcwg-buildslave/.ssh/authorized_keys inside container with developer's keys. E.g., this is how I start a QEMU container for ILP32 testing: WORKSPACE=`pwd` bash -x ./start-container-qemu.sh --cp_file ~/.ssh/authorized_keys:/home/tcwg-buildslave/.ssh/authorized_keys:tcwg-buildslave:0600 --task test --session-name qemu-2 --label tcwg-apm_64-build --node tcwg-apm-04 --distro xenial --prefix qemu_ --kernel_url https://cloud-images.ubuntu.com/releases/16.04/release/unpacked/ubuntu-16.04-server-cloudimg-arm64-vmlinuz-generic --initrd_url https://cloud-images.ubuntu.com/releases/16.04/release/unpacked/ubuntu-16.04-server-cloudimg-arm64-initrd-generic > qemu-container.sh Change-Id: I8e264616a55fbb1175d12eb0ebd82a03fca5e619
2017-09-05start-container-docker.sh: Fix parsing of --docker_opts option.Maxim Kuvyrkov
Handle whitespaces. Change-Id: Icfdd17c873ab2b5cc2d8be9c3f9164c4b6c42953
2017-08-09start-container-docker.sh: Add customization necessary for QEMU VMs.Maxim Kuvyrkov
Running QEMU VMs requires container to run in privileged mode (KVM needs this; non-KVM VMs can do without it). For this we pass --docker_opts "--privileged". When running QEMU VM we forward port 2222 of container to port of the VM. Consequently, we need to publish port 2222 to outside of container (on a random port). For this we pass --docker_opts "-p 2222". Finally, we add ${prefix}container_id to the exported interface. We require this to resolve external port mapped to VM's port 22. Change-Id: I3d5d58ec4de02b82acd4ed097892651a656edaf5
2017-08-09jenkins-helpers.sh: Generalize code into helpers.Maxim Kuvyrkov
Add print_cpu_shares, print_memory_limit, print_pids_limit, and print_bind_mounts. These helpers will also be used in upcoming start-container-qemu.sh. Change-Id: I7a442eb1a959fde70003dc2f693df0e3848509a0
2017-07-21jenkins-helpers.sh: Add wait_for_ssh_server.Christophe Lyon
This new helper is used by start-container-docker.sh, and will be used by other scripts later (for instance by benchmarking scripts, to wait until a board has rebooted). Change-Id: I9853a2d1c0ae67df9bb3e90b65651d050c3e35f7
2017-05-18start-container-docker.sh: Source jenkins-helper.shMaxim Kuvyrkov
... to provide remote_exec. Change-Id: I3ed874ac3368665275eabb7bf012b41f6be0e5b5
2017-05-17jenkins-helper.sh, start-container-*.sh: Remove unneeded optionMaxim Kuvyrkov
We forward ssh-agent by default in .ssh/config. Change-Id: I30d2b71ff07b2a2fbee7c130de91059de5d3ade4
2017-05-17start-container-docker.sh:Use remote_exec in ${prefix}container_execMaxim Kuvyrkov
Change-Id: I321bcd7bd9748d0aa4b3328513859540979c95f1
2017-05-04start-container-docker: Don't bind-mount unnecessary directoriesMaxim Kuvyrkov
Cross-testing doesn't need any of the bind-mounts. Change-Id: I1927b1e594704c263c7e1d3a42c87f03a7c3de93
2017-05-04start-container-docker: Create directory on the right host.Maxim Kuvyrkov
When creating test containers (which, arguably, should not have bind-mounts altogether) we run the script on the build host, while creating container on the remote host. Change-Id: If251dff04f076572c9b6786809c3d44f32b6b65e
2017-05-02start-container-docker.sh: Add {stop,cleanup}_all_containers.Charles Baylis
Add support for stopping and cleaning up all containers to the v2 interface. Change-Id: I1deba5c04d4960090b0ff5c7762bf4907cdd1450
2017-05-02start-container-docker: Fix problem with cleaning up stale workspacesMaxim Kuvyrkov
... which are created with root:root owner when bind-mount dir doesn't exist on the host. Change-Id: I63a1a1839c83ca98a997ae4c4302a882b82c315e
2017-04-27start-container-docker: Add container_exec command.Maxim Kuvyrkov
This command allows transparently execute commands inside container via SSH. Arguments are properly quoted to allow drop-in replacement for local/schroot shell. Parameters are carefully quoted so that both will print the same result: $ echo 'A B' A B $ container_exec echo 'A B' A B A naive remote shell would print: $ ssh dev-01.tcwglab echo 'A B' A B Change-Id: Ie367a4566bcaf6af06c4d9c7bfc9e91065e196ff
2017-04-26start-container-docker.sh: Allow seccomp:unconfined on all targetsRenato Golin
Unrestricting seccomp:unconfined to all targets, so we can run sanitizer tests. I'm not sure this is the best thing to do, so please, review with a pinch of salt. https://cdn-images-1.medium.com/max/600/1*snTXFElFuQLSFDnvZKJ6IA.png Change-Id: I63dcc0cfce260a0da859e70c232e31643ea6db39
2017-04-26start-container-docker: Add container_stop functionMaxim Kuvyrkov
Change-Id: Ic000d2a82919743d3818317695c4672e7612dd68
2017-04-25start-container-docker.sh: Fix typos.Maxim Kuvyrkov
Change-Id: I867de465a0646d5627fc51106d04c833b87032ed
2017-04-25start-container-docker.sh: Add --label and --node options.Maxim Kuvyrkov
jenkins-helper.sh: Add more functions and generalize existing. New options to start-container-docker to reduce copy-paste of logic to decided which host/arch to start container on. In jenkins-helpers.sh unify naming of "print_A_for_B" functions. Add print_host_for_node to simplify calling of nodename2hostname. Various other improvements to accomodate changes to tcwg-buildfarm and tcwg-abe-extended. Change-Id: I9e6bac241f70f9b9a992073b6c649c4eefead129
2017-04-20start-container-docker.sh: bind mount LLVM git referenceRenato Golin
Change-Id: Ic5915d62ec2ecc9c48f69ba2e1849d420e3d9fcd
2017-04-20start-container-docker.sh: Add --prefix option and v2 interfaceMaxim Kuvyrkov
Change-Id: I4d16c5e09d5cf155a45e066c918d5238beec72e3
2017-04-20start-container-docker.sh: Remove ssh-agent workaround.Maxim Kuvyrkov
Change-Id: Ia09d11756fdef4aaf1182784215f5dcba53999c7
2017-04-17start-container-docker: Correct memory limit for "test" containersMaxim Kuvyrkov
Change-Id: I8ad00aec56be126e8393e3f8b90c69f24b65d448
2017-04-11start-container-docker.sh: Add --no-ssh-agent option.Christophe Lyon
This script start ssh-agent if it detects that it is not already running, to workaround problems with Jenkins' ssh-agent plugin. Then, it adds tcwg-buildslave's key to ssh-agent. The Jenkins ssh-agent plugin defines SSH_AUTH_SOCK, but not SSH_AGENT_PID. However, some jobs ask Jenkins to forward other credentials to the ssh-agent plugin (eg tcwg-benchmark). Starting ssh-agent here effectively discards this credential (since we can only add tcwg-buildslave's key). We want to experiment with using Jenkins' ssh-agent when we need several credentials, but there is no way of knowing in which case we are: we do not want to call ssh-agent -l here, because it may hang when the Jenkins implementation is broken. Adding a new flag seems to be the simplest way, until we can rely on Jenkins and remove all this. A subsequent patch will be needed in the related Jenkins jobs, to make use of the new flag (tcwg-buildapp). Change-Id: Iae64effb826d902c63c111945ec04b40f4414689
2017-04-11start-container-docker.sh: Execute 'docker run' under 'set -x'Christophe Lyon
For a more verbose output. Change-Id: If628eff398e5a8e72f7c66c4c7c93003714e560f
2017-03-30start-container-docker.sh: Add --weight option.Christophe Lyon
To control the resources limits of the docker container (memory, pids, cpu shares). start-container-none.sh, start-container-schroot.sh: Support, but ignore --weight option. (To keep the same interface as start-container-docker.sh). Change-Id: I3a495d9f9027a2a55449c7e03ad387c4681bfbdf
2017-03-30Revert "start-container-docker.sh: Add --memory option."Christophe Lyon
This reverts commit 8dd72d26d0a1865bdeacb179fedf96324f22eba3. Change-Id: Ic0fce2125b74b98789e9048d74135f3215b87a26
2017-03-29start-container-docker.sh: Add --memory option.Christophe Lyon
To control the memory limit of the docker container. start-container-none.sh, start-container-schroot.sh: Support, but ignore --memory option. (To keep the same interface as start-container-docker.sh). Change-Id: Ie44071eae95103a3ba53b0406e106aa310184116
2017-03-21start-container-docker.sh: Cope with new image name.Christophe Lyon
Change-Id: Idc6d3498d41d19c68ccc6ca089622d66f1333978
2017-03-08start-container-docker.sh: Use --cap-add=SYS_PTRACE when starting docker.Christophe Lyon
This is needed for gdb and gcc's guality tests (which use gdb). Change-Id: I83467b84f4ce4cc3c400eaeb17d96f8288946891
2017-02-20start-container-docker.sh: Fix ssh-agent cleanup.Christophe Lyon
Make sure to kill the just-started ssh-agent in case something goes wrong. Change-Id: Ibb6d659298da97754926d1fca5ff8255f00d5955
2017-02-17start-container-docker.sh: Restore CONTAINER_CLEANUP.Christophe Lyon
Change-Id: I82afb35b783449146a31251d907846e15ee123eb
2017-02-17start-container-docker.sh: Define SSH_AGENT_CLEANUP if needed.Christophe Lyon
Including ssh-agent cleanup code in the same variable as the container cleanup code is tricky because it involves subshell evaluation: putting this in a shell variable with backquotes makes further assignations impossible as they would trigger unwanted calls to the ssh-agent cleanup up because of shell expansion. The convention is that the contents of SSH_AGENT_CLEANUP should be used as: eval `${SSH_AGENT_CLEANUP}` Change-Id: I4ce7afdde37e07621bd3e9b8a2aa42037ec941bb
2017-02-17start-container-docker.sh: Fix ssh-agent invocation.Christophe Lyon
To avoid leaking file descriptors 3 and 4, which would otherwise make an external pipe hang (eg tee). Change-Id: I62921634bdcb8b98b88198c007ba5e628a1475db
2017-02-17start-container-docker.sh: Fix ssh-agent cleanup.Christophe Lyon
Change-Id: Ibf5b270c99417fdceadf49e5a1eab60ede37571a
2017-02-17start-container-docker.sh: Add ssh key to root user for test containers.Christophe Lyon
ABE's test-schroot.sh needs to connect to the 'test' container as 'root', so add the new temporary key to it too. Change-Id: Id018d26beeb93a7cd04b82a21fc31be6adbbd723
2017-02-16start-container-docker.sh: Fix docker cp invocation.Christophe Lyon
Use $DOCKER instead of docker to include the session host name. Change-Id: Ic0628ddbc151a3c1c87aa48604b6508484c7f96a
2017-02-16start-container-docker.sh: Propagate tcwg-buildslave ssh key.Christophe Lyon
This is a workaround to Jenkins/ssh-agent problems. The keys are present on the host where the script is run, we make sure the docker container will accept them. Start ssh-agent if necessary, provide it with the key, and export related environment variables. Change-Id: Ic787a34993700c5878fc450e6023ed6e2c858fd4
2017-02-13start-container-*.sh: Append .tcwglab only if not already present.Christophe Lyon
Change-Id: Id16401100dea2b74ff439aea1ab022b1f4807899
2017-02-13start-container-docker.sh: Use --security-opt seccomp:unconfinedChristophe Lyon
This docker flag reduces security, but is currently needed because GCC's libgo configure has a test that aborts otherwise. See test 'whether setcontext clobbers TLS variables'. Do this on i386 only, since the problem appeared only on this arch. Change-Id: I9b01df3cc6e407923d2adc509f6f15055f12d920
2017-02-08start-container*.sh: Add --task parameter.Christophe Lyon
Change-Id: I869c0693531b05958e4e482d15d901da9b57ca71
2017-02-08start-container-*.sh: Define CONTAINER_CLEANUP instead of installing a trap.Christophe Lyon
It was not possible to create 2 containers with these scripts and benefit from automatic cleanup via a trap handler: the 2nd one would override the 1st one. With this patch, we define CONTAINER_CLEANUP which contains the cleanup command, which can be saved by the caller script and used as needed. Change-Id: I832395baeb6dda2c89a4fa7d29e68017df6c628f
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-11 16:17:07 +0000