We know rewards visitors retain at 7.4x and transact at 3.2x. But correlation is not causation. This document defines a rigorous framework to determine whether rewards missions cause incremental activity beyond what the missions themselves require.
The 7.4x retention and 3.2x transaction multipliers suffer from two fatal confounders that must be addressed before any conclusions about rewards effectiveness can be drawn.
The question we must answer: After stripping out the mission-completing transactions themselves, do rewards users show more subsequent activity than comparable users who never engaged with rewards? If someone completes SWAP1, do they go on to do a second, third, fourth swap organically? Or do they stop?
We need to demonstrate that the rewards feature generates incremental activity. Specifically:
The analysis requires carefully defined cohorts. The feature flag [Experiment] rewards-feature and Ramp Network app platform filters are the primary segmentation tools.
Users who actively engaged with the rewards feature, subdivided by depth of engagement.
| Cohort | Definition | Identification Method | Current Size (est.) |
|---|---|---|---|
| T1: Mission Completers | Users who clicked Claim rewards mission reward for any mission_code |
click event where label = "Claim rewards mission reward" |
~250 users |
| T2: Mission Actors | Users who clicked Rewards mission action (started a mission CTA) but may not have completed |
click event where label = "Rewards mission action" |
~500 users |
| T3: Rewards Visitors | Users who viewed the Rewards screen but did not interact with any mission | page_view where screen_name = "Rewards", excluding T1 and T2 |
~530 users |
| Cohort | Definition | Purpose |
|---|---|---|
| C1: Flag-On, No Visit | Users with [Experiment] rewards-feature = "on" who never viewed the Rewards screen |
Best available control. Same flag exposure, same platform (mobile app, non-EU, non-partner), but no rewards engagement. Controls for platform but not for motivation. |
| C2: Propensity-Matched | From C1, select users matched on pre-rewards activity (transaction count, volume, tenure, transaction flow mix in the 30 days before March 16) | Gold standard control. Same observable characteristics, different treatment status. Controls for both platform and baseline activity level. |
| C3: Pre-Period Self | Each treatment user compared against their own behavior in the 30 days before their first rewards screen visit | Difference-in-differences baseline. Each user is their own control. Eliminates between-user confounders but vulnerable to time trends. |
The flag is our best natural experiment. The rewards-feature flag (ID 10188193) is a release flag that rolls out to integration_name = "Ramp Landing - iOS" or "Ramp Landing - Android" users outside the EU. Within that eligible population, ~567 daily users have the flag on, while ~12,000+ do not. Flag-on users who never visit rewards (C1) are the closest available control.
The rewards-feature flag targets users matching ALL of these conditions:
residence_country is NOT any EU/EEA country (31 countries excluded including UK, Norway, Switzerland)integration_name IS "Ramp Landing - iOS" or "Ramp Landing - Android"The default (everyone else) gets no variant. There is also a separate ramp.network employee override segment.
Coverage limitation. This means the analysis is restricted to non-EU mobile app users. Web users (35K) and EU users are excluded from the rewards feature entirely. Any findings apply only to this specific population segment.
For each mission type, we must precisely define: what counts as the mission requirement (to subtract), and what counts as incremental organic activity (to measure).
| Mission | Requirement | What to Subtract | Incremental Metric | Claims to Date |
|---|---|---|---|---|
| TOPUP1 | Top up $25+ | First on-ramp transaction >= $25 after mission action click | Count and volume of on-ramp transactions after claim, excluding the first one | ~67 |
| TX1 | Complete 1 transaction | First transaction_completed after mission action click | Count and volume of all transactions after claim, excluding the first one | ~31 |
| SWAP1 | First swap of $50+ | First swap (transaction_flow = SWAP) >= $50 after mission action click | Count of additional swaps after claim. Does completing SWAP1 lead to organic SWAP3? | ~19 |
| SWAP3 | Complete 3 swaps | First 3 swaps after mission action click | Count of swaps beyond the 3rd. Does the habit persist? | ~3 |
| EARN10 | Earn $25 in stablecoins | Earn deposits up to $25 threshold after mission action click | Additional earn deposits beyond mission threshold. Does user continue earning? | ~3 |
| WALLET | Connect wallet | The wallet connection event itself | Any transactions within 30 days of wallet connection. Wallet is a gateway action. | ~131 |
| INV3 / INV7 | Invite 3/7 friends | The invite actions themselves | Do referred users become active? Do inviters increase their own activity? | ~1 / ~0 |
| SEND5 | Send to 5 friends | First 5 send transactions after mission action click | Send transactions beyond the 5th. Does send become a habit? | ~1 |
Critical sample size constraint. Most missions have fewer than 30 completions. Only WALLET (~131 claims), TOPUP1 (~67), and TX1 (~31) have potentially viable sample sizes for statistical analysis. SWAP1 (~19) is borderline. All others are too small for reliable inference. This analysis will need to wait for more data accumulation, or focus on the top 2-3 missions.
For each treatment user, compute these metrics in the 7, 14, and 30-day windows after their mission claim timestamp.
transaction_completed events after claim, minus the mission-completing transaction(s)fiat_amount_eur on post-claim transaction_completed events, excluding mission-completing transaction(s)transaction_completed event that is NOT the mission-completing transactionpage_view) at D7 / D14 / D30 after claim?For each control user (C2, propensity-matched), compute the same metrics in the identical calendar window, anchored to the matched treatment user's claim date.
No single method is definitive given the observational nature of this data. We apply three complementary approaches; if all three agree, we have strong evidence.
For each user in the flag-on population, compute these features for the 30-day window before the rewards feature launch (Feb 14 - Mar 15, 2026):
transaction_completed eventsfiat_amount_eurpage_view eventspage_viewcountryuser_type = "new" vs "returning"P(visited_rewards = 1) ~ pre_tx_count + pre_tx_volume + pre_session_count + tenure + platform + country_groupPSM limitation. Propensity score matching can only control for observable confounders. If unobserved factors (like crypto enthusiasm, portfolio size, or external market conditions) drive both rewards engagement and activity, PSM will not fully eliminate bias. This is why we run multiple methods.
The DiD estimator is:
This controls for: (a) time-invariant differences between groups (treatment users are "more active" people), and (b) common time trends (crypto market conditions affecting everyone). It requires the parallel trends assumption: absent treatment, both groups would have followed similar activity trajectories.
Plot weekly transaction counts for both groups in the 4-6 weeks before rewards launch. If the trends are approximately parallel, DiD is valid. If they diverge before the launch date, the assumption fails.
For each treatment user individually, compare their activity rate in the 30 days before their first rewards screen visit vs. 30 days after their mission claim. This eliminates all between-user confounders but is vulnerable to regression to the mean and concurrent time effects.
Key variant: split users into activity quartiles based on pre-period behavior:
The Q1 test is the most powerful. If we find that previously-inactive users (0 pre-period transactions) go on to make 2+ post-mission transactions (beyond the mission requirement itself), that is strong evidence that rewards caused new behavior. These users had no baseline activity, so there is nothing to select on.
These are the concrete queries needed to execute the analysis. Each query references actual event names, property names, and filter values verified against the Amplitude schema (project 100017324, "New Widget - Production") and Omni model (Ramp Data Platform, 8d49baba-ff1d-4ab8-a234-0b1797646c0e).
Implementation note. Amplitude's UI does not allow direct export of per-event timestamps with user IDs in a single view. You will need either: (a) the Amplitude Export API (Behavioral Cohorts + Profile API), (b) an Amplitude SQL query via the Data Tables feature, or (c) the Amplitude CDP/warehouse sync if available. The chart-based segmentation confirms the volumes but does not give per-user claim timestamps.
| Data Point | Status | Source | Notes |
|---|---|---|---|
| Rewards screen visits | Available | Amplitude page_view where screen_name = "Rewards" |
Working since early March. Good coverage. |
| Mission claim events | Available | Amplitude click where label = "Claim rewards mission reward" + mission_code |
Has mission_code for per-mission breakdown. This is the treatment timestamp anchor. |
| Mission action clicks | Available | Amplitude click where label = "Rewards mission action" + mission_code |
Indicates user started pursuing a mission (clicked the CTA). |
| Feature flag assignment | Available | Amplitude user property [Experiment] rewards-feature |
Values: "on" or (none). Used for treatment/control segmentation. |
| Transaction data | Available | Amplitude transaction_completed + Omni transactions_simplified |
Both sources have transaction_flow, fiat_amount_eur, transaction_id. Omni has richer historical data. |
| Session activity | Available | Omni fct_widget_events |
Has Merged Amplitude ID, session timestamps, screen names, user type. |
| Mission completion timestamp (backend) | Partial | Amplitude click event timestamp is a proxy | The click on "Claim rewards mission reward" is the closest available proxy for mission completion. Backend Prometheus counters track completions but are not user-attributed. |
| Per-user claim timestamps with user IDs | Partial | Requires Amplitude Export API or Data Tables | The Amplitude chart UI shows aggregate counts. Per-user timestamps need an API export, User Lookup, or warehouse sync. |
| Mission completion event (dedicated) | Not Tracked | Would need new Amplitude event | A dedicated mission_completed event with mission_code, reward_amount, and completion_timestamp would be cleaner than inferring from click labels. |
| Mission progress tracking | Not Tracked | Would need new Amplitude event | A mission_progress event (e.g., "SWAP3: 2 of 3 completed") would let us track partial completions and abandonment. |
| Reward amount / type per claim | Not Tracked | Not in Amplitude events | We do not know the reward value (XP? Tokens? Discount?) associated with each claim. This is needed to compute reward ROI. |
| Transaction-to-mission attribution | Not Tracked | Would need new property on transaction events | A property like triggered_by_mission on transaction_completed events would let us directly identify which transactions were mission-driven vs organic. |
| User ID mapping (Amplitude to Omni) | Partial | Both systems have user IDs but in different formats | Amplitude uses amplitude_user_id and user_id. Omni uses User ID (integer) and Merged Amplitude ID. Need to verify the join key. |
mission_completed event to Amplitudemission_code, reward_type, reward_value, completion_timestamp, time_to_complete_seconds. This replaces the current reliance on click label inference.triggered_by_mission property to transaction_completedtriggered_by_mission = "SWAP1" (or whichever mission). This eliminates the need for heuristic transaction-to-mission matching.mission_progress event for multi-step missionsmission_code, current_step, total_steps. This lets us measure abandonment at each step and understand friction points.user_id maps to Omni's User ID or Merged Amplitude ID. Without a reliable join key, cross-system analysis (Amplitude behavioral data + Omni transaction data) is impossible. Test on a sample of 10 known users.rewards-feature flag infrastructure supports this, though it is currently set to 100% rollout, not a 50/50 split.integration_name = "Ramp Landing - iOS" or "Ramp Landing - Android" users outside 31 EU/EEA countries. This limits external validity. Findings cannot be generalized to web users (35K population) or EU users. If rewards is later expanded to web/EU, a separate analysis would be needed.
rewards-feature flag is a release flag at 100% rollout for the eligible segment, not a randomized experiment. All eligible users get the feature; those who engage are self-selected. This makes true causal inference difficult with observational methods alone. The strongest recommendation is to run a proper 50/50 holdout for 4-6 weeks before concluding anything about causality.
triggered_by_mission property on transactions, we must infer which transaction completed a mission. For SWAP1, the heuristic is: "the first swap of $50+ after the user clicked 'Rewards mission action' for SWAP1." This works for simple missions but breaks for multi-step missions (SWAP3) where we cannot be certain which 3 swaps were mission-driven and which were organic. The P0 instrumentation recommendation (adding triggered_by_mission) would eliminate this ambiguity.
amplitude_user_id (integer, e.g., 89562739732) and optionally user_id (email). Omni's transactions_simplified uses User ID (integer, e.g., 10534249) and fct_widget_events uses Merged Amplitude ID (integer matching Amplitude's ID). The join path is likely Merged Amplitude ID in Omni = amplitude_id in Amplitude, but this must be validated before any cross-system queries.
Answerable today with existing data. No new instrumentation required.
Requires new instrumentation (P0 items) and data engineering support.
mission_completed event and triggered_by_mission property. Validate data flow with 48 hours of production data. Coordinate with the rewards backend team (flag owners: jakub.jastrzebski2@ and marek.rycerski@).rewards-feature flag (ID 10188193) from 100% rollout to 50/50 split within the eligible segment. Run for 4-6 weeks. This is the only way to definitively prove causality. Requires product/eng buy-in since it means withholding rewards from half the eligible users.When the analysis is complete, map the findings to one of these four outcomes. Each outcome has a different strategic implication.
| Outcome | Evidence Pattern | Strategic Implication |
|---|---|---|
| Strong Causal Effect | Post-mission incremental activity is significantly higher than control (p < 0.05). Q1 (inactive) users activate. Effect persists at D30. | Double down. Expand to web, EU, partner channels. Invest in mission design and reward economics. Rewards is a growth lever. |
| Moderate Causal Effect | Some incremental activity observed but small effect size. Mainly driven by already-active users. Q1 users do not activate meaningfully. | Proceed with caution. Rewards reinforces existing behavior but does not create new behavior. Focus on retention value rather than activation. Optimize mission design toward lower-activity users. |
| Selection Effect Only | No incremental post-mission activity beyond mission requirements. Treatment and matched control show same post-period behavior. Rewards visitors were already heavy users. | Rewards is a loyalty feature, not a growth feature. It rewards existing behavior but does not change it. Re-frame the ROI case: is the retention uplift among already-engaged users worth the reward cost? |
| Insufficient Data | Sample sizes too small for statistical significance. Confidence intervals span zero. Cannot reject the null hypothesis. | Wait. Accumulate 60+ days of data. Implement the A/B holdout. Do not make resource allocation decisions on inconclusive evidence. |
The honest expectation. Given that the feature is 2 weeks old with ~250 total mission completions, the most likely Phase 1 outcome is "Insufficient Data" for rigorous statistical inference, combined with "directional signals" from individual user audits. Phase 2 and Phase 3 are where the real answers come from. Do not over-interpret early results.