ec2_post_init

Introduction

This shell library is useful if you are not a systems administrator but want to spin up an EC2 image to do data analysis or research. ec2_post_init provides a simple easy to use API that can:

• Install system software (see: system.inc.sh :: sys_pkg_install())
• Install Docker (see: docker.inc.sh :: docker_setup())
• Install Miniconda3 (see: miniconda.inc.sh :: mc_install())
• Install STScI pipelines (see: astroconda.inc.sh)
  • ac_releases_install_hst()
  • ac_releases_install_jwst()
  • ac_releases_install_data_analysis()

Supported Operating Systems

• Red Hat
  • CentOS 7+
  • Fedora 19+
• Debian
  • Stretch+
• Ubuntu
  • Bionic+

Installing

System installation

git clone https://github.com/spacetelescope/ec2_post_init
cd ec2_post_init
sudo make install PREFIX=/usr/local

Portable installation

If you don't want to install ec2_post_init permanently, you don't have to. This is especially useful for systems that provide curl and tar by default but lack git and make. Here is how to use ec2_post_init from its source directory:

curl https://github.com/spacetelescope/ec2_post_init/archive/refs/heads/main.tar.gz | tar -x
cd ec2_post_init
export PATH=$(pwd)/bin:$PATH

Using ec2_post_init

Now you can include the library in your own script by sourcing ec2pinit.inc.sh...

#!/usr/bin/env bash
 
# Load ec2_post_init
source ec2pinit.inc.sh
 
# ...

To see how one can use ec2_post_init to populate a system with Miniconda3 and the three major STScI pipeline releases, please refer to the Full example page. The API reference for each library module can be found here.

Developing

To write code for ec2_post_init you should have access to an EC2 instance, or a host with docker or vagrant installed.

git clone https://github.com/spacetelescope/ec2_post_init
cd ec2_post_init
export PATH=$(pwd)/bin:$PATH

To test ec2_post_init using docker:

docker run --rm -it -v $(pwd):/data -w /data centos:7 /bin/bash
[root@abc123 data]# export PATH=$PATH:/data/bin
[root@abc123 data]# cd tests
[root@abc123 tests]# ./run_tests.sh

To test ec2_post_init using vagrant (VirtualBox):

mkdir -p ~/vagrant/centos/7
cd ~/vagrant/centos/7

Create a new Vagrantfile. Be sure to change any paths to match your local system

Vagrant.configure("2") do |config|
  config.vm.box = "generic/centos7"
 
  # Mount the ec2_post_init source directory at /data inside of the VM
  config.vm.synced_folder "/home/example/my_code/ec2_post_init", "/data"
 
  # Change VM resources
  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
    v.cpus = 2
  end
end

Provision the VM, log in, and execute the test suite:

vagrant up
vagrant ssh sudo -i
[root@vagrant123 ~]# export PATH=$PATH:/data/bin
[root@vagrant123 data]# cd /data/tests
[root@vagrant123 tests]# ./run_tests.sh