Setup a Production-Grade Instance

This is just a single architectural example of how to configure QU4RTET in a production environment and by no means represents an absolute stack of technologies or cloud providers. Again, this is only a representation of a Vantage-curated approach to doing this task. QU4RTET can run on any platform, OS, web-server and database back-end.

There are many approaches to setting up production-grade architectures on AWS once you have an idea of how to configure your application layer to communicate with your UI and database layers. The architecture we will outline here is one of many possibilities and we will be doing tutorials like this on other architectural variations that are possible.

Advanced Setup: Using a Replicated, Clustered Database Backend

Adding a high availability, clustered back-end to QU4RTET is pretty easy. In this section we will guide you along. In no time you'll have a production grade QU4RTET instance ready to go.

Add PARAMETERGROUP to _User Data

The Vantage QU4RTET AMI is pre-configured to look for certain variables in your EC2 instance User Data. Here will will set the PARAMETER_GROUP user data value which will allow your instance to pick up secure environment variables from the Amazon Parameter Store during run time. The EC2 instance will use these parameters to connect to the database we will be creating in forthcoming steps.

Step 1: Stop Your Instance

The first step is to stop the EC2 QU4RTET instance. Right click and select stop as in the diagram below.

Stop

Step 2. Add User Data

Next, right click on the stopped instance, and click Instance Settings|View/Change User Data

Add the following data:

PARAMETER_GROUP='GROUP-1'

Add user data

Change add user data

Save and then re-start your server by right clicking and choosing Instance State/Start.

Configure Aurora RDS or PostgreSQL

*QU4RTET supports many databases; however, the Vantage bundle is only configured for PostgreSQL. If you require another database, contact Vantage- we can help set one up for you.

Stand up a PostgreSQL (aurora) instance following the first part of the instructions below:

Creating a PostgreSQL DB Instance and Connecting to a Database on a PostgreSQL DB Instance and set up a new database instance sized to your liking.

To follow along with the guide use the following settings when configuring your database. Getting the security groups configured correctly can be tricky- make sure to follow the instructions on this closely.

  • DB cluster identifier: qu4rtet-cluster
  • database name: qu4rtet
  • port: 5432

Other options are at your discretion.

OPTIONAL: Migrating From the Local DB to the New

This step is not required but is useful if you've configured a system and would like to migrate it from current state into a more production ready state.

If you'd like to migrate the local database and all of its data to the new RDS instance (which is optional), execute the following steps.

Note: This will require you to ssh into your instance using the key you downloaded when you first launched or execute the script steps below using the AWS Systems Manager.

Upgrade Your Client (Optional)

If your postgres client applications are less than version 10, execute the following:

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ $(lsb_release -sc)-pgdg main" > /etc/apt/sources.list.d/PostgreSQL.list'
sudo apt-get update
sudo apt-get install postgresql-client --upgrade
Switch over to the postgres user account:
sudo -i -u postgres
Optional: Test your connection
psql -h [host name] -U qu4rtet
Back Up the Local Database
pg_dump -h localhost -p 5432 -U postgres -F c -b -v -f /tmp/qu4rtet.backup qu4rtet
Create the qu4rtet Database on RDS

Since your new RDS instance does not have a qu4rtet database created, we will create on here.

createdb -e -E UTF8 -O qu4rtet -U qu4rtet --host=[your host] --port=5432  qu4rtet 'The QU4RTET database backend.
Restore to the RDS Instance
pg_restore -h [put the host name of your RDS instance here] -p 5432 -U qu4rtet -d qu4rtet -v /tmp/qu4rtet.backup

Configure an Inbound Rule

In your RDS instance details, under the Connect heading you will see Security Group Rules. Click on the security group link under the Security Group list- don't worry if there are a few. Just click on the first one.

On the subsequent page, click the Inbound tab at the bottom and click Edit. On the next screen select Add Rule and add the following rule:

Name Value
Type PostgreSQL
Protocol TCP
Port Range 5432
Source Enter in the security group of your qu4rtet app server. type sg an you should see a group or list of groups to choose from.
Description QU4RTET App Server as a CIDR (with a /32 at the end)

Note: To allow more than one app server or resource to connect from your VPC , one may add a /16 CIDR value to the security rule.

Get the Cluster URL

On the RDS management page, click the Clusters link to the left and then click on the quartet-cluster-n link. This will take you to a details page. Copy the cluster endpoint value.

AWS Systems Manager Configuration

Your QU4RTET image is already pre-configured to work with the Amazon S3 storage API. It can access this API via values provided in the Amazon EC2 Parameter Store. The steps below will guide you through getting your EC2 instance connected to the Parameter Store.

Create an instance profile for Systems Manager managed instances

A more expansive tutorial on this can be found here... Configuring Access to Systems Manager

  1. Open the IAM console at https://console.aws.amazon.com/iam/.

  2. In the navigation pane, choose Roles, and then choose Create role.

  3. On the Select type of trusted entity page, under AWS Service, choose EC2.

Note: If the Select your use case section appears, choose EC2 Role for Simple Systems Manager, and then choose Next: Permissions.

  1. On the Attached permissions policy page, search and verify that AmazonEC2RoleforSSM is listed, select that and then search and verify that ReadSSMParameters is listed, Select that and then choose Next: Review.

  2. On the Review page, type a name in the Role name box, and then type a description. We recommend QU4RTETInstanceManagement. But you can name it anything.

Note Make a note of the role name. You will choose this role when you create new instances that you want to manage by using Systems Manager.

  1. Choose Create role. The system returns you to the Roles page. When you are done, your role should look like this: Role View

  2. Now go to your EC2 QUARTET instance (if you have one running already) and right click and select Attach/Replace IAM Role.

Attach The Role Attach The Role

  1. Now that your system is able to be managed by SSM and access parameter store values, we are able to configure it to use your RDS instance.

Generate a Secret Key

Your QU4RTET instance uses a random secret key to encrypt sensitive data in the database. It is essential that every QU4RTET instance have it's own unique secret key for this purpose. Use the site below to generate a 50 character randomized string. Copy this value somewhere and save it for the steps below.

Generate a 50 character string here.

Set Up Your System's Parameters

There are a number of parameters that can be set via the Systems Management Parameter Store, the following will get you connected to your RDS instance. There are additional parameters available for the S3 configuration as well which are included in the Configure S3 For Storage

More on the EC2 Parameter Store

The following assumes the User Data in your QU4RTET instance has been set to PARAMETER_GROUP='GROUP-1' and that you've been following the tutorial using the example values.

Parameter Name Value Type
/GROUP-1/DATABASE_HOST Put the URL to your PostgreSQL cluster server here from the cluster step above. String
/GROUP-1/POSTGRES_DB qu4rtet String
/GROUP-1/POSTGRES_USER The name of the user you created above. String
/GROUP-1/POSTGRES_PORT 5432 String
/GROUP-1/POSTGRES_PASSWORD The password you created for the database user. SecureString
/GROUP-1/DJANGO_SECRET_KEY Paste in the secret key you generated above. SecureString

Configure S3 For Storage

Your QU4RTET instance comes pre-configured to use Amazon S3 storage to queue inbound and outbound EPCIS messages and other such task data. This allows you to automatically expire data and gives you a flexible and (almost infinite) storage back-end for your system.

To set up S3, follow the instructions here.

Create The QU4RTET Database Schema

Now that your system is configured to communicate with RDS, you can populate it with the QU4RTET database schema. To do this, we will use the AWS Systems Manager to issue a remote command.

Enter the Command Parameters

Navigate to the Systems Manager Run Command Page,

Select Run a Command

run command

Next, locate the AWS-RunShellScript command document from the list and select it.

command list

After you've selected this, copy this script and paste it into the command parameters box.

workon qu4rtet
sudo ./utility/ec2_create_database.sh

Command Params

Set the Working Directory

In the Working Directory text box, enter the following:

/srv/qu4rtet

Select The QU4RTET Instance to Run Your Command On

You should see a list of available Targets in the Targets section. If you've just added your EC2 QU4RTET instance to the Systems Manager then it may be a few minutes before it shows up in the lis. Select your QU4RTET instance in the Targets section.

target

Set the S3 Bucket for Storage (Optional)

If you've set up an S3 bucket you can select that in the Output options to log any errors or output if you want.

logging

Run The Command

Click the Run button at the bottom of the page and wait for the command to complete. Once it is done, you should see a screen like the one below:

success

IMPORTANT: Inspect The Output

Once your command is done, you can then view the output to ensure that it has executed correctly. Just because a command has run successfully does not mean there were no errors!

Dealing With Issues

If you are at an impasse and need some help getting over the finish line, contact Vantage Solutions or send an email to Vantage Support.

502 Bad Gateway Issue

The 502 Bad Gateway issue (if you use a browser directly to test the install) is typically the result of a missing requirement or a postgresql password that has an illegal character in it such as a # sign.

The fastest way to figure out what is going on here is to log onto the server via SSH, navigate to /srv/qu4rtet and run

python manage.py runserver

This should generate an error of some sort that can be looked into.