Queue Profiles
Last updated
Last updated
IDEA let you create queues and queue profiles. To access this section, click "Scale-Out Compute" > "Queue Profiles" on the left sidebar
You must have admin permission to the Scale-Out Compute module
A queue is is a resource that can handle and execute user jobs. A queue profile extends queue functionalities by specifying default compute/storage values. You can have multiple queues registered to one queue profile.
Each profile have a set of default values (instance_type, instance_ami ...) that are automatically applied to the simulation jobs if not specified by the users during the job submission.
Example1:
In this example, no job resources are specified during the qsub command. IDEA will determine instance type, AMI and all other required parameters based on the default values specified on Instance Info for the queue called "normal".
Example 2:
In this example, IDEA honors the job resources specified by the user (instance_type, instance_ami and scratch_size) Other required parameters will be based on the default values specified on Instance Info for the queue called "normal".
As administrator, you can prevent users to specify specific job resources via Restricted Parameters
To create a queue profile, click "Create Queue Profile" button.
To edit a queue profile, select the profile and click "Action" > "Edit Queue Profile"
Choose a name for the queue profile. Name must be all lowercase and should not contain spaces or special characters except (-)
Choose a user friendly title for the queue profile
Select the queue(s) to add to this queue profile.
A queue cannot belong to more than one queue profile
IDEA will automatically create the queue if needed
Select the queue(s) mode.
First-In First-Out (FIFO): Default, jobs are processed in the order they have been send to the queue
License Optimized: IDEA will try to maximize licenses consumption and run as many job as possible based on license availabilities. Job order may not be honored.
Fair Share: IDEA starts jobs based on your own fair share formula
Choose whether or not you want the compute nodes to be automatically deleted once the simulation(s) are complete
Select the scaling more for the queue(s).
Single Job: Run one job per compute node (EC2 machine)
Batch: Run multiple jobs per compute node (EC2 machine)
Select the number of maximum concurrent running jobs (0 implies no limits)
Select the number of maximum concurrent provisioned instances (0 implies no limits)
Select whether or not you want to allow your users to submit their simulation jobs without specify a project (-P <project_name>)
List of instance type or instance families allowed to be provisioned by your users for their jobs.
Example:
c5.large,m5: End users can provision a c5.large or any m5 instance types (e.g: m5.large, m5.xlarge ..) for their jobs.
List of instance type or instance families your users are not authorized to provision for their jobs.
Example:
c5.large,m5: End users are not authorized to provision c5.large or any m5 instance types (e.g: m5.large, m5.xlarge ..) for their jobs.
List of jobs parameters your users are not able to customize.
Example:
scratch_size, instance_type: End users are not authorized to override scratch_size and instance_type parameters for their jobs and have to use the defaults value specified at the queue profile
You can assign up to 4 additional security groups per job
You must ensure your security groups have been created with the correct TCP inbound/outbound rules. Failure to do so may prevent your jobs to start
You must ensure your IAM role(s) have been created with the correct policies. Failure to do so may prevent your jobs to start
Choose the default AMI to use for the compute nodes.
Choose the default instance type to provision for the compute nodes. Multiple instance types if provided will be used as weighted capacities. Order is important and must be provided in increasing CPU capacity (e.g: c5.large, c5.xlarge)
Select the size of the root partition for the compute nodes.
Choose whether or not you want to retain ephemeral EBS disks associated to the compute nodes once the simulation has completed.
Choose whether or not you want to enable EFA support. You must be using a supported instance type (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/efa.html#efa-instance-types)
Choose whether or not you want to enable hyper threading
Choose whether or not you want to restrict your job to Reserved Instance
Choose whether or not you want to enable support for Spot Fleet.
Select the maximum bid for your spot instances.
Select the spot allocation logic for your spot instances.
Select the spot allocation strategy for your spot instances.
If needed, restrict the default subnet ids that can be used to provision the compute resources. If not set, IDEA will randomly choose on of the privat subnets:
Specify a custom security group if you do not want to use the default one created by IDEA
Specify a custom IAM instance profile if you do not want to use the default one created by IDEA
Choose whether or not you want to enable placement group (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html)
Choose whether or not you want to automatically allocate a local scratch partition to the compute nodes.
Choose what type of storage provider you want to use for your scratch partition (EBS, FSx ...)
Choose the size (in GB) of the /scratch partition you are about to provision.
Select the provisioned IOPs you want to allocate. We recommend to set this value as 3X the scratch size
Select whether or not you want to enable system metrics collection
Select whether or not you want to anonymous metrics collection https://docs.aws.amazon.com/solutions/latest/scale-out-computing-on-aws/collection-of-operational-metrics.html
List of additional security group(s) users can override via parameter.
List of additional IAM instance profile(s) users can override via parameter.
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource:
Associated Job Resource: None if EBS, / for FSx
Associated Job Resource: /
Associated Job Resource: for EBS / for FSx
Asociated Job Resource:
Associated Job Resource: