βš™οΈ
Integrated Digital Engineering on AWS
  • πŸš€First Time Users
    • What is IDEA?
    • Install IDEA
      • Pre-Requisites
      • Standard Installation
    • Access your IDEA cluster
      • APIs
      • Virtual Desktops (Linux/Windows)
      • Web Interface
      • SSH
    • File Browser
    • Let's get started
    • Cluster operations
      • Uninstall IDEA
      • Update IDEA cluster
        • Patch IDEA module (idea-admin.sh patch)
        • Update IDEA configuration (idea-admin.sh config)
        • Update IDEA backend resource (idea-admin.sh deploy)
  • πŸ€–Modules
    • What is a module?
    • Virtual Desktop Interfaces
    • HPC Workloads
    • Cluster Manager
    • Shared Storage
  • πŸ‘‘Best Practices
    • Analytics
      • OpenSearch
        • Configure your OpenSearch
        • Create your own analytics visualizations
    • Budget
      • Review your AWS spend
      • Control your AWS spend
      • Set up budget per project
    • Security
      • Backup IDEA environment
  • 🧱Architecture
    • High Level Overview
  • πŸ› οΈDeveloper Portal
    • IDEA CLI utility
    • Developer Onboarding
  • 🚨Help and Support
    • FAQ
    • Raise a bug
Powered by GitBook
On this page
  • Access your OpenSearch endpoint
  • Initialize OpenSearch indexes (first time accessing OpenSearch)
  • Validate the index
  • Troubleshooting access permissionΒΆ
  1. Best Practices
  2. Analytics
  3. OpenSearch

Configure your OpenSearch

Access your OpenSearch endpoint

Run idea-admin.sh show-connection-info --aws-region <REGION> --cluster-name <NAME> command to retrieve the "Analytic Dashboard" URL.

$ ./idea-admin.sh show-connection-info --aws-region us-east-2 --cluster-name idea-mcdev
Web Portal: xx
Bastion Host (SSH Access): xx
Bastion Host (Session Manager URL): xx
Analytics Dashboard: https://idea-mcdev-external-alb-391482262.us-east-2.elb.amazonaws.com/_dashboards

Alternatively, you can retrieve the OpenSearch dashboard endpoint via your IDEA Web Interface > "Cluster Management" > "Settings" then "Analytics".

Initialize OpenSearch indexes (first time accessing OpenSearch)

Click the hamburger menu on the left to open the sidebar, then click "Stack Management" under "Management" section.

Click "Index Patterns" on the left sidebar then "Create Index Pattern". Refer to the tabs below to create indexes based on your own interest.

User Sessions

As Index Pattern Name, specify your cluster name followed by "_vdc_user_sessions*". For example, if your cluster name is "idea-test" then search for "idea-test_vdc_user_sessions*". Once done, click "Next Step".

Use "created_on" as Time Field then click "Create Index Pattern"

Compute Nodes

As Index Pattern Name, specify your cluster name followed by "_scheduler_nodes*". For example, if your cluster name is "idea-test" then search for "idea-test_scheduler_nodes*". Once done, click "Next Step".

Use "provisioning_time" as Time Field then click "Create Index Pattern"

Job Information

As Index Pattern Name, specify your cluster name followed by "_scheduler_jobs*". For example, if your cluster name is "idea-test" then search for "idea-test_scheduler_jobs*". Once done, click "Next Step".

Use "queue_time" as Time Field then click "Create Index Pattern"

Validate the index

Once your index is created, click the hamburger menu to open the sidebar and click "Discover" within the "OpenSearch Dashboards" section.

Select the Index pattern you have created previously using the dropdown section on the right

All your IDEA data should now be visible.

IMPORTANT

OpenSearch display the events created within the last 15 minutes by default, make sure to update the timeframe accordingly.

OpenSearch will display the various entries ingested by IDEA as well as all the relevant metadata. You can use these metadata as filters to narrow your research to specific key elements.

It's now time to Create your own analytics visualizations.

Access to OpenSearch is restricted to the IP you have specified during the installation. If your IP change for any reason, you won't be able to access the analytics dashboard and will get the following error message:

{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet"}

To solve this issue, log in to AWS Console and go to OpenSearch Service dashboard. Select your OpenSearch cluster and click "Modify Access Policy"

Finally, simply add your new IP under the "Condition" block, then click Submit

Please note it may take up to 5 minutes for your IP to be validated

PreviousOpenSearchNextCreate your own analytics visualizations

Last updated 2 years ago

Troubleshooting access permission

πŸ‘‘
ΒΆ
Scheduler Index will store Scale-Out Workload jobs. VDC pattern is for Virtual Desktops
Make sure to change the Time selection as needed.
Example of IDEA data being ingested on OpenSearch