Overview
In addition to our unified API, Terminal supports automatically replicating normalized data to an Amazon S3 bucket in your AWS account. This enables:- Ongoing data replication to power downstream data pipelines
- Complete data archival to maintain ownership of your data
- Simplified implementation without the need to implement APIs or orchestrate data ingestion
Configuration Steps
Step 1: Create an Amazon S3 Bucket
- Log in to the AWS Management Console
- Navigate to S3 under Storage services
-
Create a new bucket:
- Click “Create bucket”
- Enter a unique bucket name (e.g.,
your-company-terminal-data
) - Choose your region
- Keep default security settings to block public access
- Enable default encryption
- Click “Create bucket”
-
(Optional) Enable S3 versioning to protect against accidental deletions or overwrites:
- Go to bucket Properties
- Find and enable Versioning to protect against accidental changes
Step 2: Configure the Bucket Policy
- Go to your bucket’s Permissions tab
-
Add the bucket policy:
- Click “Edit” under Bucket Policy
- Paste the relevant policy provided below depending on your environment
- Save changes
Production Bucket Policy
Production Bucket Policy
Sandbox Bucket Policy
Sandbox Bucket Policy
Step 3: Configure your Destination
After configuring your bucket and bucket policy, you’ll need to set up your S3 destination with Terminal. Contact our team with the following information:- Bucket name
- Environment (production or sandbox)
- Configuration preferences (see options below)
Self-service destination management through the Terminal dashboard is coming
soon.
Option | Description |
---|---|
Update Mode | Controls how partition updates are handled. Can choose to either overwrite or append . Both modes include all partition records. Defaults to overwrite .Overwrite: Replaces existing files Append: Adds timestamped versions of files |
Compression | Enable GZIP compression for objects. Defaults to true . |
Include Raw | Include raw data in output, similar to the API raw parameter. It’s important to note raw data can significantly increase file sizes. Defaults to false . |
Key Prefix | Optional prefix for object keys (e.g., /terminal-data/ ). Defaults to no prefix. |