Customer Data Platform Development

Event streaming architecture is built on Segment, RudderStack, or Snowplow as the event collection layer -- all three produce a consistent event schema across web, mobile, and server-side sources, and all three support identity stitching across anonymous and known users. Snowplow is preferred when you need full data ownership and a self-hosted pipeline without per-event pricing. RudderStack is preferred when you want warehouse-native activation and your data team works primarily in SQL. Segment is preferred for teams that want a large destination catalogue and a fast integration path.

Web events are collected via a lightweight JavaScript SDK that tracks page views, clicks, form submissions, and custom events. Mobile events are collected via iOS and Android SDKs. Server-side events from your backend (order completions, subscription changes, support ticket creation) are sent via the HTTP tracking API or server-side SDK.

Identity resolution uses deterministic matching on known identifiers: the same email address hash (SHA-256), phone number hash, or customer ID found on two records is enough to merge them. Probabilistic matching on device fingerprints and behavioural patterns links anonymous sessions that are likely the same person but have not yet shared a known identifier. Cross-device stitching merges the anonymous pre-login event history with the known customer record at the point of login -- using the login event's user ID as the bridge identifier. The identity graph is maintained as a real-time Kafka topic so profile merges propagate to the unified profile within seconds of the bridging event arriving, not at the next nightly batch run.

Single profile record for each resolved customer aggregates behavioural events from your web and mobile tracking, transactional history from your ecommerce platform or order management system, CRM attributes from Salesforce or HubSpot (synced bidirectionally), and computed properties calculated from the event and transaction history. The profile schema is designed around your data model rather than a fixed schema that requires your data to conform to the CDP's structure -- a custom schema means your product-specific attributes (subscription plan, cohort, referral source, service tier) are first-class profile attributes, not shoehorned into generic fields.

Computed attributes are updated in real time via a stream processing layer (Kafka Streams or Apache Flink) as new events arrive: total lifetime value (LTV) recalculated on each order event, days since last purchase updated on each order, product category affinity scores updated as browse and purchase events accumulate, and engagement score updated as email open and click events arrive. These computed attributes are available in the profile immediately after each event, so an audience segment that requires "customers with LTV over $500 who haven't purchased in 90 days" reflects the current state, not yesterday's batch.

Customer lifetime value prediction uses an XGBoost model trained on RFM (Recency, Frequency, Monetary) features plus additional behavioural signals. The model generates a predicted 12-month LTV score per profile, updated on a scheduled basis, which feeds into high-value audience segments for acquisition lookalike modelling and retention campaigns. The full event history is retained and queryable for analytical use cases, with a partition strategy that keeps queries against large event tables performant. Profile API supports real-time lookup by any downstream system -- personalisation engine, customer support tool, or recommendation service -- that needs to read current customer attributes without querying the data warehouse directly.

Drag-and-drop segment builder lets marketing teams define audiences from any profile attribute, behavioural event, or computed property without writing SQL or filing a data team request. Conditions cover event frequency (performed a purchase event at least 3 times), event recency (last session within 14 days), attribute ranges (LTV between $100 and $500), computed property thresholds (engagement score above 0.7), and Boolean combinations of all of the above.

First-party data collection for segmentation is structured to support cookieless targeting: identity is tied to authenticated events (email, phone, customer ID) and first-party device identifiers rather than third-party cookies, making the audience infrastructure GDPR and CCPA compliant by design. When building lookalike audiences for paid media, the segment is activated to Meta or Google via hashed email and phone -- no cookies or device IDs cross the platform boundary.

Real-time segment membership updates for event-triggered conditions as the matching events arrive: a customer who completes a purchase exits the "no purchase in 30 days" win-back segment and enters the "recent purchaser -- drive second purchase" segment within seconds. Scheduled refresh (hourly or nightly cron) is available for computationally expensive segments that don't need real-time updates. Segment size estimates are calculated and displayed before the segment is saved so the marketing team knows whether they have built an audience of 50,000 or 500 before designing a campaign around it. Segment overlap analysis shows how much two segments share, preventing the same contacts from receiving conflicting messages from two different campaigns.

Audience segment activation to paid media platforms uses server-side conversion APIs rather than pixel-based audience uploads, which addresses both data accuracy (server-side events aren't blocked by ad blockers or browser privacy restrictions) and compliance (no third-party cookies, hashed identifiers only). Meta segments are synced via the Meta Conversions API and the Custom Audiences API. Google segments use the Google Ads Customer Match API. LinkedIn Matched Audiences and TikTok Custom Audiences are also supported. Match rates on first-party email and phone hashes typically run 50 to 80%, significantly higher than retargeting pixels that depend on third-party cookie availability.

CRM bidirectional sync with Salesforce and HubSpot keeps both systems current: CDP segment membership flows into CRM as contact properties for sales outreach lists, and CRM deal stage and lifecycle changes flow back into the CDP as profile attributes for marketing suppression (suppress paying customers from acquisition campaigns) and re-engagement (trigger a win-back campaign when a deal goes closed-lost). Email platform sync to Klaviyo, Mailchimp, or Braze creates and updates audience lists in real time as segment membership changes, so campaign lists are always current without a scheduled export.

Data residency controls for EU/US data split ensure that profile data for EU residents is stored in EU-region infrastructure (AWS eu-west-1 or equivalent) and is not transferred to US-region systems in a way that violates GDPR Chapter V transfer requirements. US resident data can be processed in US-region infrastructure without those constraints. The residency configuration is enforced at the profile storage and event pipeline level, not just at the UI access layer.

The event pipeline uses Kafka as the backbone for event streaming between the ingestion layer and the profile processing layer. Each event passes through a schema validation step using Apache Avro or JSON Schema registry before it reaches the profile update consumers. Malformed events -- missing required fields, incorrect data types, out-of-range values -- are routed to a dead-letter queue with the validation failure reason attached, so the engineering team can review and replay them after fixing the source. Events that fail schema validation never reach the unified profile store.

Deduplication uses event IDs with an idempotency window to catch duplicate events from retry logic, client-side double-fires on slow connections, and SDK retransmissions after a network interruption. An event with a previously seen event ID within the deduplication window is discarded without being processed again.

PII detection and masking runs as a stream processing step on the Kafka pipeline. Fields configured as PII (email, phone, name, IP address) are either hashed before storage using SHA-256 (for use as identity resolution keys) or masked in the event log while the raw value is stored only in the identity graph under access control. Data lineage tracking records the source event and timestamp for each profile attribute update so the provenance of any computed value is auditable -- useful for debugging incorrect segment membership and for demonstrating GDPR data minimisation compliance.

Schema evolution is managed through the Avro or JSON Schema registry with forward and backward compatibility enforcement. Adding a new event type or a new optional property does not require a code change or downtime in the consumers. Breaking schema changes are flagged by the registry before they can be pushed, preventing silent data corruption in downstream systems.

Segment size trends track audience growth and decay over time, making it visible when a win-back segment is growing (more customers lapsing) or when a post-purchase segment is shrinking (fewer new buyers entering). Profile coverage metrics show what percentage of resolved profiles have data from each source -- email platform, ecommerce, CRM, web analytics -- so gaps in the identity graph are visible. A profile with no web behaviour data suggests the SDK is missing from specific pages. A profile with no CRM data suggests the Salesforce sync has a mapping gap. These metrics surface data quality problems before they cause incorrect segment membership.

Activation performance by destination reports match rates for ad platform uploads (the percentage of email/phone hashes that matched a known user in Meta or Google), email deliverability by segment (open rate, click rate, bounce rate), and CRM sync success rates. Low match rates on a Meta Custom Audience upload signal that the email addresses in that segment are old or inactive. This data informs how segments are constructed for activation rather than letting poor match rates silently reduce paid media effectiveness.

LTV distribution by segment identifies which audience definitions correlate with high-value customers -- comparing the median and 90th-percentile LTV of a "high-engagement, no purchase in 60 days" segment against a "low-engagement, no purchase in 60 days" segment tells the marketing team which win-back audience is worth more aggressive spend. CLTV prediction scores (generated by the XGBoost RFM model) are available as a dimension in all segment-level reports. Reporting is designed for the marketing team to understand their data and make decisions from it, not for data analysts to diagnose infrastructure health.