How ActionIQ constructed a really composable buyer information platform utilizing Amazon Redshift

[ad_1]

This put up is written in collaboration with Mackenzie Johnson and Phil Catterall from ActionIQ.

ActionIQ is a number one composable buyer information (CDP) platform designed for enterprise manufacturers to develop sooner and ship significant experiences for his or her prospects. ActionIQ faucets instantly right into a model’s information warehouse to construct good audiences, resolve buyer identities, and design customized interactions to unlock income throughout the client lifecycle. Enterprise manufacturers together with Albertsons, Atlassian, Bloomberg, e.l.f. Magnificence, DoorDash, HP, and extra use ActionIQ to drive progress by means of higher buyer experiences.

Excessive prices related to launching campaigns, the safety danger of duplicating information, and the time spent on SQL requests have created a requirement for a greater resolution for managing and activating buyer information. Organizations are demanding safe, value environment friendly, and time environment friendly options to energy their advertising outcomes.

This put up will reveal how ActionIQ constructed a connector for Amazon Redshift to faucet instantly into your information warehouse and ship a safe, zero-copy CDP. It is going to cowl how one can get began with constructing a really composable CDP with Amazon Redshift—from the answer structure to establishing and testing the connector.

The problem

Copying or transferring information means heavy and sophisticated logistics, together with added incurred value and safety dangers related to replicating information. On the logistics facet, information engineering groups need to arrange further extract, rework, and cargo (ETL) pipelines out of their Amazon Redshift warehouse into ActionIQ, then configure ActionIQ to ingest the information on a recurring foundation. Extra ETL jobs means extra transferring elements, which introduces extra potential factors of failure, corresponding to breaking schema adjustments, partial information transfers, delays, and extra. All of this requires further observability overhead to assist your staff alert on and handle points as they arrive up.

These further ETL jobs add latency to the end-to-end course of from information assortment to activation, which makes it extra doubtless that your campaigns are activating on stale information and lacking key viewers members. That may have implications for the client expertise, thereby instantly affecting their capability to drive income.

The answer

Our resolution goals to cut back the logistics already mentioned and permits up-to-the-minute information by establishing a safe connection and pushing queries instantly right down to your information warehouse. As an alternative of loading full datasets into ActionIQ, the question is pushed to the information warehouse, making it do the laborious querying and aggregation work, and watch for the end result set.

With Amazon Redshift as your information warehouse, you possibly can run complicated workloads with constantly excessive efficiency whereas minimizing the effort and time spent in copying information over to the information warehouse by means of using options like zero-ETL integration with transactional information shops, streaming ingestion, and information sharing. You may as well prepare machine studying fashions and make predictions instantly out of your Amazon Redshift information warehouse utilizing acquainted SQL instructions.

Resolution structure

Inside AWS, ActionIQ has a digital personal cloud (VPC) and you’ve got your personal VPC. We work inside our personal personal space in AWS, with our personal locks and entry restrictions. As a result of ActionIQ goes to have entry to your Amazon Redshift information warehouse, this suggests that an outdoor group (ActionIQ) will be capable to make direct database queries to the manufacturing database atmosphere.

In your data safety (infosec) groups to approve this design, we’d like very clear and tight guardrails to make sure that:

ActionIQ solely has entry to what’s completely vital
No unintended third social gathering can entry these belongings

ActionIQ wants to speak securely to fulfill each data safety requirement. To try this, inside these AWS environments, you will need to arrange AWS PrivateLink with ActionIQ to create a safe connection between the 2 VPCs. PrivateLink units up a safe tunnel between the 2 VPCs, thus avoiding any opening of both VPC to the general public web. After PrivateLink is about up, ActionIQ must be granted privileges to the related database objects in your Amazon Redshift information warehouse.

In Amazon Redshift, you will need to create a definite database, a service account particularly for ActionIQ, and views to populate the information to be shared with ActionIQ. The views want to stick to ActionIQ’s information mannequin pointers, which aren’t inflexible, however nonetheless require some construction, corresponding to a transparent profile_id that’s utilized in all of the views for simple joins between the varied information units.

Getting began with ActionIQ

When beginning a hybrid compute integration with Amazon Redshift, it’s key to align your information to ActionIQ’s desk sorts within the following method:

Buyer base desk: A single dimension desk with one document per buyer that accommodates all prospects.
Person information tables: Dimension tables that describe prospects and be part of to the client base desk. They typically comprise slow-moving or static demographic data and are sometimes matched one to 1 with buyer information.
Occasion tables: Reality or log-like tables comprise occasions or actions your prospects take. The first secret is sometimes a user_id and timestamp.
Entity tables: Dimension tables that describe non-customer objects. They typically present further data to reinforce the information in occasion tables. For instance, an entity desk may very well be a product desk that accommodates product metadata and joins to a transaction occasion desk on a product_id.

Be aware: Person information and occasion tables can be part of on any out there identifier to the client base desk, not simply the bottom consumer ID.

Now you possibly can arrange the connection and declare the views within the ActionIQ UI. After ActionIQ establishes a desk with grasp profiles, customers can start to interpret these tables and work with them to construct out campaigns.

Set up a safe connection

After establishing PrivateLink, the remaining steps to arrange for hybrid compute are the next:

Create a separate database in Amazon Redshift to outline the shared dataset with ActionIQ.
Create a service account for ActionIQ in Amazon Redshift.
Grant READ entry to the service account for the devoted database.
Outline the views that will likely be shared with ActionIQ.

Enable itemizing

In case your information warehouse is on a non-public community, you will need to add ActionIQ’s IP addresses to your community’s enable checklist to permit ActionIQ to entry your cloud warehouse. For extra data on learn how to set this up, see the Configure inbound guidelines for SQL purchasers.

Database arrange

Create an Amazon Redshift consumer for ActionIQ

Check in to the Amazon Redshift console
From the navigation menu, select the Question Editor and connect with your database.

Create a consumer for ActionIQ:

CREATE USER actioniq PASSWORD 'password';

Grant permissions to the tables inside a given schema you need to give ActionIQ entry to:

GRANT USAGE ON SCHEMA 'yourschema' TO actioniq;
GRANT SELECT ON ALL TABLES IN SCHEMA 'yourschema' TO actioniq;

You may then run instructions within the question editor to create a brand new consumer and to grant permission to the information units you need to entry by means of ActionIQ.

The result’s that ActionIQ now has a programmatic question entry to a devoted database in your Amazon Redshift information warehouse, and that entry is proscribed to that database.

To be able to make this straightforward to manipulate, we advocate the next pointers on the shared views:

As a lot as potential, the shared objects needs to be views and never tables.
The views ought to by no means use choose *, however ought to explicitly specify every area desired within the view. This has a number of advantages:
- The schema is strong; even when the underlying desk adjustments, it gained’t provoke a change within the shared view
- It makes it very clear which fields are accessible by ActionIQ and which aren’t, thereby enabling a correct governance approval course of.
Limiting privileges to READ entry means the information warehouse directors could be structurally sure that the information views gained’t change except they need them to.

The significance of offering views as an alternative of precise tables is two-fold:

A view doesn’t replicate information. The entire level is to keep away from information replication, and we don’t need to replicate information inside Amazon Redshift both. With a view, which is actually a question definition on prime of the particular information tables, we keep away from the necessity to replicate information in any respect. There’s a reliable query of “why not give entry to tables instantly?” which brings us to the second level.
Tables and information schema change on their very own schedule, and ActionIQ wants a secure information schema to work with. By defining a view, we’re additionally defining a contract for sharing information between you and ActionIQ. The underlying information desk can change, and the view definition can soak up this variation, with out modifying the construction of what the view delivers. This stability is essential for any enterprise software program as a service to work successfully with a big group.

On the ActionIQ facet, there’s no caching or persisting of information of any form. Which means that ActionIQ launches a question at any time when a scheduled marketing campaign is launched and requires information, and at any time when a consumer of the platform asks for an viewers rely. In different phrases, queries will typically occur throughout enterprise hours, however technically can occur at any time.

Testing the connector

ActionIQ deploys the Amazon Redshift connector and examined queries to validate the success of the connector. After the viewers is outlined and validated, ActionIQ sends the SQL question to Amazon Redshift and it returns data. We additionally validate the outcomes with Amazon Redshift to make sure that the logic is appropriate as meant.

With this, you expertise a lean and extra clear deployment course of. You may see the queries ActionIQ sends to Amazon Redshift, as a result of the queries are logged. You may see what’s happening, what’s attributable to ActionIQ, and may see the expansion of adoption and utilization.

A Connector defines the credentials and different parameters wanted to hook up with a cloud database or warehouse. The Connector display screen is used to create, view and handle your connectors.

Key concerns

Organizations want robust information governance. ActionIQ requires a contract of what the information goes to appear like inside the outlined view. With the dynamic nature of information, robust governance workflows with outlined fields are required to run the connector easily to attain the final word consequence—driving income by means of advertising campaigns.

As a result of ActionIQ is used because the central hub of advertising orchestration and activation, it must course of a whole lot of queries. As a result of advertising exercise can have important spikes in exercise, it’s prudent to plan for the utmost load on the underlying database.

In a single situation, you may need spikey workloads. With Amazon Redshift Serverless, your information warehouse will be capable to scale robotically to handle these spikes. Which means Amazon Redshift can soak up giant and sudden spikes in queries from ActionIQ with out a lot technical planning.

If workload isolation is a precedence and also you need to run the ActionIQ workloads utilizing devoted compute assets, you need to use the information sharing characteristic to create a knowledge share that may be accessed by a devoted Redshift serverless endpoint. This could enable ActionIQ to question up-to-date information from a separate Redshift serverless occasion with out the necessity to copy any information whereas sustaining full workload isolation.

The info staff wants information to run enterprise intelligence. ActionIQ is driving advertising activation and creating a brand new information set for the common contact historical past—primarily the log of all advertising contacts from the exercise. ActionIQ gives this dataset again to Amazon Redshift, which may then be included within the BI stories for ROI measurements.

Conclusion

In your data safety groups, the ActionIQ’s Amazon Redshift connector presents a viable resolution as a result of ActionIQ doesn’t replicate the information, and the controls outlined set up how ActionIQ accesses the information. Key advantages embody:

Management: Select the place information is saved and queried to enhance safety and match current expertise investments.
Efficiency: Scale back operational effort, improve productiveness and minimize down on pointless expertise prices.
Energy: Use the auto-scaling capabilities of Amazon Redshift for working your workload.

For enterprise groups, the ActionIQ Amazon Redshift connector is querying the freshest information potential. With the connector, there’s zero information latency—an necessary consideration with key viewers members which are primed to transform.

ActionIQ is worked up to launch the Amazon Redshift connector to activate your information the place it lives—inside your Amazon Redshift information warehouse—for a zero-copy, real-time expertise that drives outcomes together with your prospects. To study extra about how organizations are modernizing their information platforms utilizing Amazon Redshift, go to the Amazon Redshift web page.

Improve your Amazon Redshift funding with ActionIQ.

Concerning the authors

Mackenzie Johnson is a Senior Supervisor at ActionIQ. She is an modern advertising strategist who’s passionate in regards to the convergence of complementary applied sciences and amplifying joint worth. With intensive expertise throughout digital transformation storytelling, she thrives on educating enterprise companies in regards to the influence of CX primarily based on a data-driven method.

Phil Catterall is a Senior Product Supervisor at ActionIQ and leads product growth on ActionIQ’s foundational information administration, processing, and question federation capabilities. He’s obsessed with designing and constructing scalable information merchandise to empower enterprise customers in new methods.

Sain Das is a Senior Product Supervisor on the Amazon Redshift staff and leads Amazon Redshift GTM for associate applications together with the Powered by Amazon Redshift and Redshift Prepared applications.

[ad_2]