Just in case anybody else encounters a similar issue in the future, another grad student in the lab asked the HCP-Users mailing list and got advice from Dr. Tim Brown that worked well for us. When it comes to choosing the NITRC AMI from the Community AMIs menu for Spot Requests, there are four options and the one that works for us is: NITRC-CE HCP v0.44-0f23881d-3d66-4624-9b14-6e7532f5e671-ami-0966c41f.4 (ami-935470f6).
HCP-Users Digest, Vol 61, Issue 8, Message 3:
Hi Michelle,
I have not yet had the opportunity to make use of spot instances to run
data through HCP processing pipelines. So please understand that I may
not be able to adequately answer your questions. However, I’ll try to
make some helpful suggestions.
With regard to there being no NITRC option for the AMI selection menu,
I’m assuming that you are starting from your AWS EC2 console
(https://console.aws.amazon.com/ec2). From that page, you should see the
“Spot Requests” link under the “INSTANCES” heading along the left hand
side. You then select the “Spot Requests” link to get taken to the
“Request Spot Instances” page. Again, I’m assuming that you then select
the “Request Spot Instances” button at the bottom of the table of your
current Spot requests.
Near the top of the “Requirements” section on the “Request Spot
Instances” page, there is an AMI specification row with a pull down
selector that gives you choices of AMI’s to use for your spot instance
request. I’m guessing that when you say that “There’s no NITRC option
for the AMI selection menu, and when I try to search for it, there are
no results” you mean that the pull down menu doesn’t include a NITRC
option, and you selected the “Search for AMI” button to the right of the
AMI pull down menu and fill in the search field with “NITRC” or
something similar. This indeed seems to give no results.
The default group of AMIs in which to search (see the pull down just to
the right of the search text field) is “My AMIs”. If you
select/pull-down that menu and choose the AMI group “Community AMIs”,
then I think that your search for a NITRC AMI will return some AMIs from
which to select. (At least it does when I try it.)
However…I don’t think it would be appropriate for you to request a
Spot Instance using one of the NITRC HCP AMIs. (Again, recall that I
haven’t had the opportunity to use spot instances, so I’m hypothesizing
here.) You’ll recall that the NITRC HCP AMI is not fully “ready to go”
for logging in and running pipelines. There is some additional
configuration that you need to do (install the FreeSurfer license, mount
the S3 data, create a user account, etc.) before any instance created
from a NITRC HCP AMI is really ready to use. My guess is that you would
need to have an AMI that is fully “ready to use” prepared to use for a
spot instance.? My thought is that the way to do this would be to start
with the NITRC AMI, get an instance configured so that it is all ready
to use, then use that instance as the starting point for creating your
own AMI. Then the AMI that you created would be the one you would choose
for your spot instances. (The AMI that you created would show up in the
“My AMIs” group, which seems to be why that group is the default for
searching for AMIs when creating a spot request.)
As for your connection issue, I suspect you are correct that there is
something wrong with your security settings. Which type of instance are
you trying to access via DNS and/or SSH? A non-NITRC spot instance or a
NITRC-AMI-based on-demand instance that you’ve created?