PREP 300: Adobe Experience Platform & Data Distiller Primers
Last updated
Last updated
In this section, we'll delve into the fundamental concepts of Adobe Experience Platform. Data enters the Adobe Experience Platform through edge, streaming, and batch methods. Regardless of the ingestion mode, all data must find its place in a dataset within the Platform Data Lake. The ingestible data falls into three categories: attributes, events, and lookups. The Real-Time Customer Profile operates with two concurrent stores – a Profile Store and an Identity Store. The Profile Store takes in and partitions data based on the storage key, which is the primary identity. Meanwhile, the Identity Store continuously seeks associations among identities, including the primary one, within the ingested record, utilizing this information to construct the identity graph. These two stores, accessing historical data in the Platform Data Lake, open avenues for advanced modeling, such as RFM, among other techniques.
Adobe Experience Platform excels in ingesting data from diverse sources. However, marketers face a significant challenge in extracting actionable insights to enhance their understanding of customers. Data Distiller addresses this challenge by providing the flexibility to query data using standard SQL in the Query Editor.
A valuable addition to this capability is the Data Distiller package, which encompasses a subset of functionalities available in Adobe Experience Platform. Specifically designed to facilitate post-ingestion data preparation, Data Distiller tackles key tasks such as cleaning, shaping, and manipulation. It executes batch query in the Query Service, preparing data for use in Real-Time Customer Profile and other applications.
Utilizing Data Distiller, you gain the capability to join any dataset within the data lake and capture query results as a new dataset. This newly created dataset proves versatile, serving various purposes such as reporting, machine learning, or ingestion into Adobe Experience Platform-based applications like Real-Time Customer Profile Data, Adobe Journey Optimizer, and Customer Journey Analytics.
There are three primary use cases for Data Distiller, and this continues to expand every few releases:
Next, let us get familiar with a few key terms, which will be used throughout this book
Adobe Experience Platform: This is the shorthand for Adobe Experience Platform.
Adobe Experience Platform Data Lake: This denotes the data lake store housed within the Adobe Experience Platform governance boundary. Irrespective of the ingestion mode, all data is directed to the Adobe Experience Platform Data Lake. Currently, Data Distiller interacts with this lake, both reading and writing datasets. Additionally, Data Distiller possesses its own accelerated store designed for business intelligence reporting, allowing seamless reading, and writing of datasets. The Adobe Experience Platform Data Lake contains datasets which can be either attributes, events, or lookups. Each of these datasets must have an associated schema with them.
Query Service: This is a broad set of SQL capabilities in the Adobe Experience Platform. Some of these capabilities may be included in the packaging of various Apps such as Adobe Journey Optimizer but most of it is packaged in Data Distiller. It is referred to as a service as the entire foundation is built on service-oriented architecture.
Derived Attributes: In Data Distiller, derived attributes are calculated or derived from other attributes within a dataset or table, and they are stored in a customized dataset called as a Derived Dataset. These attributes are computed using expressions or mathematical functions applied to existing attributes or events within the same table or through joins with other tables. For example, calculating the Customer Lifetime Value (CLTV) based on the last 5 years of transactions for each customer.
Audiences: Audiences are constructed on top of attributes, events and derived attributes which include logic for metrics such as Customer Lifetime Value (CLTV) or the count of transactions. Audiences can encompass 1st, 2nd, or 3rd party data and may combine data from multiple sources associated with the same person.
Ad hoc queries: Ad hoc queries refer to SQL queries utilized for exploring ingested datasets, primarily for verification, validation, experimentation, etc. These queries, crucially, DO NOT write data back into the Adobe Experience Platform Data Lake.
Batch queries: Batch queries are SQL queries employed for post-ingestion processing of ingested datasets. These queries undertake tasks like cleaning, shaping, manipulating, and enriching data, with the results written back to the Platform data lake. Batch queries can be scheduled, managed, and monitored as batch jobs.
Accelerated Store: SQL queries executed against this reporting layer support interactive dashboards and BI workflows. The results are cached for faster response time. Within the Data Distiller offering, customers can utilize an accelerated store to create insights data models efficiently, including the one employed for RFM analysis in this lab. Directly within our user interface, users can employ a lightweight BI-type dashboard to visualize key performance indicators (KPIs). Additionally, there is the option to seamlessly connect external BI tools, such as Power BI, enhancing flexibility in data visualization and analysis.
Derived Datasets: The Derived Datasets feature can be leveraged for cleaning, shaping, and manipulating specific data from the Adobe Experience Platform Data Lake to generate custom datasets. These datasets can be regularly refreshed at cadence to enrich the Real-Time Customer Profile. By leveraging derived datasets, you can create complex calculations with distributions such as deciles, percentiles, or quartiles or simpler ones such as maximum value, counts, and mean value. These datasets can be tailored to individual users or business entities, associating directly with identities such as email addresses, device IDs, and phone numbers, or indirectly with user or business profiles.
Derived Datasets play a crucial role in various data analysis and enrichment scenarios, especially when analyzing data on the Adobe Experience Platform Data Lake. Furthermore, they can be marked for use in the Real-Time Customer Profile and applied in downstream use cases such as audience targeting. Potential use cases include:
Identifying the bottom 10% of subscribers based on channel viewership to target specific audience segments for new subscription packages.
Identifying top 10% flyers based on total miles traveled and "Flyer" status to target them for new credit card offers.
Analyzing subscription churn rates.
Identifying the top 1% of household income in a region and tracking the number of individuals moving out of that income bracket over a specified period.
Dashboards provide a dynamic and interactive way to review RFM (Recency, Frequency, Monetization) marketing analysis, offering insights and trends at a glance. This approach enables businesses to quickly identify valuable customer audiences and adjust their marketing strategies, accordingly, maximizing both engagement and ROI.
