Adobe Data Scientist — Interview Questions

Adobe Data Scientist Interview Questions

Data science at Adobe sits on top of a subscription business, not an advertising one, and that changes almost everything about the questions you get asked. The metrics that matter are trial conversion, seat expansion, retention across annual contracts, and increasingly the unit economics of generative AI features where every Firefly generation has a real GPU cost attached. Adobe also owns Experience Platform, which means some data science roles are analysing Adobe's own customers while others are building the measurement infrastructure Adobe's enterprise customers use on their data. This bank covers the SQL, experimentation, modelling and business-judgment questions that appear across those loops.

Adobe's Interview Process for Data Scientists

Candidates typically report four to five rounds compressed into a fairly short window — several describe the whole loop closing in ten to fifteen days once it starts, which is faster than most large tech companies.

The usual shape is an online assessment covering statistics and Python, then a technical screen of thirty to sixty minutes with a data scientist that mixes SQL and machine learning, then a dedicated round on SQL, statistics and how you reason through a data problem, then a product or case round where you are handed a business scenario and asked how you would measure success. Many loops include a take-home project presented to a panel, and candidates specifically mention being pushed on sensitivity analysis — how your conclusion moves when your assumptions move. A behavioural round with the hiring manager closes it, and some loops add a director round.

The pattern worth internalising: Adobe cares less about whether you know an exotic model and more about whether you can define a metric that survives scrutiny and state how confident you actually are.

Question 1: The trial funnel that does not add up

You have two tables. trial_starts has one row per trial with user_id, product, started_at, trial_length_days, and acquisition_channel. subscription_events has one row per event with user_id, product, event_type — which can be subscribe, cancel, pause, resume, plan_change — and occurred_at. Marketing reports trial-to-paid conversion at 41 percent. Finance reports 28 percent for the same quarter. Both are pulling from these tables. Write the query you would trust, and tell me where the twelve-point gap most likely comes from.

Why interviewers ask this

This is the SQL round and the metrics round folded into one, and the gap is the point. Weak candidates write a join and produce a third number. Strong candidates recognise that conversion rate is underdetermined until you fix the denominator, the attribution window, and the treatment of users with multiple trials — and they interrogate those before writing anything.

Example strong answer

Before writing SQL I would name the four decisions that a trial-to-paid number depends on, because the gap almost certainly lives in one of them rather than in a bug.

First, the denominator. Do we count trials or users? A user who starts a Photoshop trial and an Illustrator trial is two trials but one person. If marketing counts distinct users and finance counts trials, the numbers diverge immediately, and they diverge more in quarters with heavy cross-product promotion.

Second, cohort assignment. A trial started on the last day of the quarter with a thirty-day length cannot possibly have converted within the quarter. If marketing cohorts by trial start date and reports conversions that happened at any time, while finance cohorts by conversion date within the quarter, you get systematically different numbers, and the direction of the gap depends on whether trial volume was growing.

Third, the attribution window. Does a subscribe event three months after the trial expired count as a trial conversion? Marketing usually says yes, finance usually says no. This is often the single largest contributor.

Fourth, what counts as converted. Does a subscribe followed by a cancel inside the refund window count? Does a paid conversion into a discounted educational plan count the same as a full-price one?

The query I would trust fixes all four explicitly:

WITH first_trial AS (SELECT user_id, product, MIN(started_at) AS started_at, MIN(trial_length_days) AS trial_length_days FROM trial_starts WHERE started_at >= '2026-04-01' AND started_at < '2026-07-01' GROUP BY user_id, product), converted AS (SELECT f.user_id, f.product, MIN(s.occurred_at) AS converted_at FROM first_trial f JOIN subscription_events s ON s.user_id = f.user_id AND s.product = f.product AND s.event_type = 'subscribe' AND s.occurred_at >= f.started_at AND s.occurred_at < f.started_at + (f.trial_length_days + 7) * INTERVAL '1 day' GROUP BY f.user_id, f.product) SELECT COUNT(c.user_id)::float / COUNT(f.user_id) AS trial_to_paid FROM first_trial f LEFT JOIN converted c ON c.user_id = f.user_id AND c.product = f.product;

The deliberate choices: one row per user per product using the first trial, a cohort defined by trial start, and a conversion window of the trial length plus a seven-day grace period. I would report the number alongside those three parameters rather than as a bare percentage.

On where the gap comes from, my ranked guess is: attribution window first, denominator second, cancel-within-refund-window third. I would test that by producing the same query with each parameter flipped and seeing which flip reproduces marketing's 41 and which reproduces finance's 28. That reconciliation table is the actual deliverable — not a fourth number, but an explanation of why two reasonable people got two answers.

Follow-up questions

A user pauses their subscription for two months and resumes. Is that churn? Show me where that decision enters your query.
Trial volume grew forty percent quarter over quarter. Which of your four decisions becomes most sensitive to that?

Question 2: An experiment that looks like a win

Firefly gives free users twenty-five generative credits a month. We tested a change: when a user hits the limit, instead of a hard stop, they see a modal offering five bonus credits for connecting a Behance account. The test ran for two weeks, evenly split, with about 900,000 users per arm. Paid conversion in the treatment arm is up 6 percent relative, p equals 0.03. The PM wants to ship it on Monday. What do you say?

Why interviewers ask this

This is the experimentation round, and the setup contains several deliberate traps. Weak candidates check the p-value and approve. Strong candidates ask about the metric hierarchy, novelty effects, the multiple-comparison surface, and whether a two-week test can measure a subscription outcome at all.

Example strong answer

I would not block it, but I would not sign off on Monday either. Here is what I would want to check, roughly in order of how likely each is to change the decision.

First, is paid conversion the pre-registered primary metric, and was 6 percent within the range the test was powered for? With 900,000 per arm and a p of 0.03, the effect is detectable but the p-value is close enough to the threshold that I want to know how many metrics were examined. If the team looked at fifteen metrics and this is the one that crossed, the effective false-positive rate is much higher than 3 percent. I would ask for the pre-registration document. If there is not one, that is the finding.

Second, was the test peeked at? Sequential monitoring without a corrected boundary inflates false positives substantially, and a p of 0.03 after daily checking is not the same evidence as a p of 0.03 at a fixed horizon. I would ask when the team first looked and whether the stopping decision was influenced by what they saw.

Third, the duration problem, which I think is the strongest objection. Two weeks is a short window to measure conversion to a subscription that most users consider over a longer deliberation period. What we have likely measured is a shift in the timing of conversions among users who were already close, not an increase in the number of users who ever convert. Those look identical at two weeks and diverge at eight. I would want to see the treatment effect by day — if it spikes in week one and decays in week two, that is a novelty and pull-forward pattern, and shipping it will produce a bump followed by a flat line.

Fourth, sample ratio mismatch. I would check that the actual split is close to 50-50. A meaningful imbalance points at a bug in assignment or logging, and if assignment is broken the effect estimate is not trustworthy regardless of the p-value.

Fifth, the guardrails. Bonus credits have a direct GPU cost, and connecting Behance is a real friction point. I would want cost per incremental conversion, the Behance connect rate, and whether free-user generation volume rose enough to matter. A 6 percent conversion lift that costs more in inference than it earns in subscriptions is not a win.

What I would propose: leave it running four more weeks to separate pull-forward from a genuine lift, and in the meantime give the PM the point estimate with its confidence interval and an explicit statement of what we do and do not yet know. If there is commercial pressure to ship now, I would suggest shipping to a holdback — roll out to 90 percent, keep 10 percent as a long-run control — which gets the PM the launch and gets me the long-horizon read.

Follow-up questions

The day-by-day effect is flat, not decaying. Does that fully rule out a novelty effect?
Free users share Firefly outputs with each other. Does that threaten your independence assumption, and how would you detect it?

Question 3: Define activation for Adobe Express

Adobe Express has a lot of signups and not enough repeat usage. Leadership wants an activation metric — a single number the team can move that predicts long-term retention. You have full event data: signups, template opens, edits, exports, shares, and returns. Define the metric. Then tell me how you would prove it is the right one rather than just plausible.

Why interviewers ask this

Metric definition is where Adobe separates analysts from data scientists. Weak candidates propose something reasonable like "exported a design in the first week" and stop. Strong candidates treat metric selection as an empirical question with a validation procedure, and they think about how the metric will be gamed once a team is measured on it.

Example strong answer

I would treat this as two problems: finding a candidate activation event, and validating that it is predictive rather than merely correlated.

For the candidate, the general shape of a good activation metric is an action, a count, and a time window — the classic form being something like "performed action X, N times, within T days." I would not guess at X, N and T. I would derive them.

The derivation I would run: take a cohort of users who signed up at least six months ago so I have a real retention outcome, define the outcome as still active at month three, then for every candidate event and every plausible threshold and window, compute how well that condition separates retained from churned users. I would look at the lift in retention above and below the threshold and pick the point where the curve bends most sharply — the threshold beyond which additional actions stop adding much retention. In my experience with tools like this, the winning event is rarely signup-adjacent and rarely the most common action; it is usually the one that represents a completed unit of value. For Express, my prior is exporting or sharing a finished design, not opening a template, because template opens include a large population of people who looked and left.

But I want to be careful about the difference between predictive and causal, because this is where activation metrics go wrong. Users who export three designs in week one retain better than those who export zero, but that is largely because they were more motivated to begin with. If the team optimises the metric by nagging people into exporting, they will move the metric without moving retention. So validation has to go further than a correlation.

I would validate three ways. First, holdout prediction: fit the activation rule on one cohort and check that it predicts retention out of sample on a later cohort, which guards against overfitting the threshold. Second, look for a natural experiment — cases where something exogenous changed how easy it was to export, such as a platform rollout or a performance fix, and check whether the induced change in activation moved downstream retention proportionally. Third, and most convincingly, run an actual experiment: ship an onboarding change designed specifically to raise activation, and check whether month-three retention moves by roughly the amount the correlation predicts. If it moves far less, the metric is a symptom, not a lever, and I would say so plainly.

I would also stress-test the metric for gaming before it ships. If the metric is exports in the first seven days, the team can raise it by auto-exporting, by prompting aggressively, or by defining export loosely. I would define the event narrowly — a user-initiated export of a design with at least one user edit — and I would pair the activation metric with a guardrail on week-four return rate so that a team cannot win on activation while losing on the thing activation is a proxy for.

Finally, I would put a confidence statement on it. Something like: users who cross this threshold retain at roughly twice the base rate, the rule holds out of sample across three cohorts, and we have experimental evidence for a partial causal effect but not the full magnitude. That is more useful to leadership than a clean number with hidden caveats.

Follow-up questions

Your threshold analysis shows the retention curve rising smoothly with no bend. What do you do?
Express has two very different user groups — small business owners and students. Does one metric serve both?

Question 4: A churn model that performs suspiciously well

You build a model to predict which annual Creative Cloud subscribers will not renew, so customer success can intervene. The model gets an AUC of 0.94 on a held-out test set, which is far better than the 0.78 the previous model achieved. Your manager is delighted. You are uneasy. Walk me through what you check, and what you do if the number holds.

Why interviewers ask this

Adobe's data science round consistently probes whether candidates are appropriately suspicious of their own results. A jump from 0.78 to 0.94 on a churn problem is almost always leakage. Weak candidates explain AUC. Strong candidates go straight to leakage, splitting strategy, and the gap between model quality and business value.

Example strong answer

My first instinct is that 0.94 on annual subscription churn is too good, and the most likely explanation is target leakage rather than a genuinely better model.

The leakage I would look for first is temporal. Churn prediction requires that every feature be computed from data available strictly before the prediction point. It is easy to build a feature set from a snapshot table where some columns were updated after the renewal decision — a support ticket about cancellation, a downgrade to a cheaper plan, a payment method removed, a sharp drop in logins in the final week. Any of those is effectively the outcome wearing a disguise. I would audit every feature by asking, for each one, at what moment in real time its value becomes known, and drop anything that could be populated after the prediction timestamp.

The second thing I would check is the split. If the test set was formed by random row sampling rather than by time, and there are multiple rows per account, then the same account appears in both train and test and the model can memorise accounts rather than learn behaviour. For churn I would always split by time — train on accounts whose renewal date falls before a cutoff, test on accounts after it — and additionally group by account so no account spans the split. Re-running with a proper temporal split is usually enough to make an implausible AUC collapse to something believable.

Third, I would check the base rate and whether AUC is even the right metric. Annual churn might be, say, eight percent. AUC can look strong while the model is useless at the operating point that matters, because customer success can only call a limited number of accounts. What they actually need is precision within the top N accounts they have capacity to contact. I would report precision at k and expected retained revenue at k rather than AUC, because that is the number that determines whether this model is worth deploying.

If after all of that the performance genuinely holds, I would still not hand it over as a scoring service. I would ask what the intervention is, because a model that identifies churners perfectly is worth nothing if the outreach does not change their behaviour. Some accounts churn for reasons no phone call fixes — the company went under, the team switched tools by executive decision. What customer success needs is not the highest-risk accounts but the accounts where the intervention has the largest effect, which is an uplift problem, not a classification problem.

So my recommendation would be to run the model in a randomised pilot: score everyone, then among high-risk accounts randomly assign half to outreach and half to no outreach, and measure the difference in renewal. That gives an honest estimate of incremental retained revenue, tells us whether the model plus intervention beats doing nothing, and gives us the data to move toward an uplift model in the next iteration.

Follow-up questions

The temporal split drops AUC to 0.80, barely above the old model. How do you present that to your manager?
Customer success says they can only call 500 accounts a quarter. How does that change what you optimise?

Question 5: Did the price increase cause the churn?

In February we raised the price of the Creative Cloud Photography plan in North America by about fifteen percent. Existing subscribers were moved to the new price at their next renewal. Six months on, churn among affected subscribers is up 2.3 percentage points versus the same cohort last year. Finance wants to know how much of that is the price increase. There was no control group — everyone in the region got it. What can you actually tell them?

Why interviewers ask this

This is causal inference without the luxury of an experiment, which is the normal condition in a subscription business. Weak candidates either compare to last year and call it the effect, or say it is unknowable. Strong candidates identify a usable comparison group and are honest about what the design can and cannot rule out.

Example strong answer

The year-over-year comparison is not the answer, because it attributes every difference between this year and last year to the price change. Six months of macro conditions, competitor moves, product changes and seasonality all sit inside that 2.3 points. So the first thing I would tell finance is that the raw number is an upper bound on plausible attribution, not an estimate.

To do better I need a comparison group that experienced everything except the price change. There are a few candidates and I would try more than one, because agreement across designs is the real evidence here.

The most useful one is geography. If the increase applied only to North America, then subscribers in a region that did not get it — say Western Europe on the same plan — form a comparison group. I would run a difference-in-differences: change in churn for North America before and after, minus the same change for the comparison region. The identifying assumption is parallel trends, so before running anything I would plot both regions' churn for the previous eight quarters and check that they moved together. If they did not, difference-in-differences is not licensed and I should say so rather than run it anyway.

The second design uses the renewal timing. Because subscribers moved to the new price at their next renewal, people with renewal dates in March were exposed months before people with renewal dates in August. Within the same region, in the same calendar month, I can compare already-repriced subscribers against not-yet-repriced ones. That controls for essentially all macro and product confounders because both groups live in the same environment; the only systematic difference is exposure. The weakness is that renewal month is correlated with signup month, which correlates with acquisition channel and tenure, so I would adjust for those or restrict to comparable tenure bands.

The third check is a placebo. I would run the same difference-in-differences on a plan that did not get a price change in the same region. If it shows an effect, my design is picking up something other than price and I should not trust the main estimate.

I would also decompose the outcome, because "churn" here hides a real distinction. Some subscribers cancelled outright, others downgraded to a cheaper plan, others switched to annual prepaid. Only the first is lost revenue at full value. Finance's real question is net revenue impact, and a 2.3 point churn rise alongside a fifteen percent price rise can still be revenue-positive. I would compute revenue per starting subscriber, not just churn, because that is the number that determines whether the decision was right.

What I would deliver: a central estimate with a confidence interval from the renewal-timing design, corroborated or contradicted by the geographic design, a placebo result, and an explicit list of what could still bias it. If the two designs disagree materially, I would report the range and say we cannot narrow it further with observational data — and I would propose that the next price change be rolled out with a randomised holdout so we never have to have this conversation again.

Follow-up questions

Parallel trends visibly fails between North America and Europe. What is your fallback?
Renewal-timing comparison shows a much smaller effect than the geographic one. Which do you believe, and why?

Question 6: Forecasting Firefly's inference cost

Finance needs a twelve-month forecast of Firefly GPU spend to set next year's budget. Generative usage has grown every month since launch but the growth rate is not stable, a new model version halves per-image cost, and product is planning to bundle more credits into paid tiers in Q2. Build the forecast. Then tell me how you would present the uncertainty to a CFO who wants one number.

Why interviewers ask this

Candidates report being pushed on sensitivity analysis in Adobe's final round, and this is that question. Weak candidates fit a time series to historical spend. Strong candidates decompose the forecast into drivers, recognise that the interesting uncertainty is in assumptions rather than in statistical noise, and know how to give a CFO a number without pretending to false precision.

Example strong answer

I would not forecast spend directly, because spend is a product of several things that are each changing for different reasons, and a time series fitted to the total cannot represent a planned model swap or a bundling change. I would decompose it.

Spend equals active generating users, times generations per user per month, times cost per generation. Each of those gets its own forecast and its own uncertainty.

Active generating users I would model from the subscription funnel rather than extrapolating the Firefly curve, because Firefly usage is downstream of Creative Cloud subscriber counts and of what fraction of subscribers have adopted generative features. Adoption curves in a subscriber base typically saturate, so I would fit a saturating curve rather than an exponential — extrapolating early exponential growth is the single most common way these forecasts go badly wrong, and the error is always in the same direction.

Generations per user I would forecast from cohort behaviour, looking at how usage per user evolves with tenure rather than at the aggregate average, because the aggregate is confounded by the mix of new and mature users. This is also where the Q2 bundling change enters: bundling more credits raises the ceiling for users who are currently constrained by it. I would size that by measuring what share of users currently hit their credit limit and how much headroom the change gives them, rather than assuming a uniform uplift.

Cost per generation is the most tractable piece: it is a known step change when the new model version ships, applied from the ship date. The uncertainty is in the date, not the magnitude.

For uncertainty, I would run a Monte Carlo over the driver assumptions rather than over residuals, because the residual variance of a fitted curve badly understates the real risk here. I would put a distribution on the saturation ceiling, on the bundling uplift, and on the model ship date, then simulate. The output is a distribution of annual spend.

Presenting it to a CFO who wants one number: I would give the number, and I would give it as the median with a stated band, framed for the decision rather than for statistical completeness. Something like: the central case is X, there is roughly a one-in-five chance we exceed Y, and the single assumption that drives most of that spread is adoption saturation. Then I would show a tornado chart ranking the assumptions by how much each moves the total. That is the sensitivity analysis, and it is the most useful artifact in the whole exercise, because it tells finance which assumption is worth spending effort to pin down and which ones do not matter.

I would also recommend the forecast be re-run monthly against actuals with a tracked forecast error, so the budget is a living estimate rather than a number that was wrong from March onward and nobody noticed.

Follow-up questions

Adoption comes in twenty percent above your saturation ceiling in month four. What do you change, and how quickly do you tell finance?
The CFO asks for a worst-case number to size a contingency. What do you give them, and what do you refuse to give them?

Question 7: The split that is not fifty-fifty

An experiment on the Acrobat web onboarding flow was configured as a 50-50 split. After a week, control has 1,204,880 users and treatment has 1,186,340. That is about 0.8 percent off. The PM says it is close enough. Is it? Show me how you would decide.

Why interviewers ask this

Sample ratio mismatch is a favourite of experimentation-heavy teams because it looks trivial and is not. Weak candidates eyeball the percentage and agree it is close. Strong candidates test it, and understand that a small imbalance at large N is strong evidence of a bug that invalidates the whole experiment.

Example strong answer

Close enough in relative terms is the wrong lens, because the question is not how large the imbalance is but how likely it is under correct randomisation, and that depends on N.

I would run a chi-squared goodness-of-fit test against the expected 50-50 split. With about 2.39 million total assignments, the standard deviation of the count in one arm under a fair coin is roughly the square root of N times p times one minus p, which is around 773. The observed difference from the expected split is about 9,270 in each direction — more than eleven standard deviations. The p-value is effectively zero. That is not a close call; it is one of the strongest signals you will ever see in an experiment.

So my answer is no, it is not close enough, and the important consequence is that I would not report the treatment effect at all until it is explained. Sample ratio mismatch matters not because unequal arms hurt statistical power — at these numbers the power cost is negligible — but because it means assignment or logging is not doing what we think, and whatever mechanism dropped or misassigned those 9,000 users is very unlikely to have done so at random. If, say, users on slow connections disproportionately failed to log a treatment exposure, then the treatment arm is missing exactly the users most likely to have a bad experience, and the measured effect is biased in a direction that flatters the treatment.

For diagnosis I would segment the imbalance to find where it concentrates. I would compute the ratio by day, by browser and OS, by country, by device type, by entry point, and by whether the user is new or returning. Sample ratio mismatch almost always localises. Common causes I would look for specifically: exposure logged after a redirect that the treatment adds, so users who bounce during the extra hop never get counted; a bot filter applied post-assignment that catches the arms unevenly; caching that serves the control variant from a CDN so those users are logged differently; and assignment on a hashed identifier that is not uniformly distributed, which sometimes shows up as an imbalance that is stable across days rather than random.

The day-by-day pattern is diagnostic. A ratio that is fine on day one and drifts points at something cumulative like caching or a gradual rollout. A ratio that is off by a constant amount from the start points at the assignment mechanism itself.

Once I find the cause, the remedy depends on it. If the loss is a logging artifact that is provably unrelated to the treatment — for example, the same imbalance appears in an A/A test — I might be able to proceed with a caveat. Usually it is not, and the honest outcome is to fix the instrumentation and re-run. I would rather tell a PM we lost a week than let them ship on an effect estimate I cannot defend.

I would also add an automated sample ratio mismatch check that runs daily on every live experiment and alerts, rather than relying on someone noticing. This class of bug is common enough that catching it manually is not a strategy.

Follow-up questions

An A/A test shows the same 0.8 percent imbalance. Does that clear the experiment to be read?
The imbalance is concentrated entirely in one browser making up two percent of traffic. Can you salvage the result?

Question 8: Telling a director the answer is no

You spent six weeks on an analysis that a director sponsored. The hypothesis was that users who engage with tutorial content in their first month have materially higher retention, and the plan was to fund a large content investment on the back of it. Your analysis says the effect is small and mostly explained by selection — motivated users seek out tutorials. The director has already referenced the hypothesis in a planning document. How do you handle it?

Why interviewers ask this

Adobe loops include a director round and a behavioural round, and this question tests whether a candidate can be right without being politically naive. Weak candidates either soften the finding into meaninglessness or describe delivering it bluntly as a virtue. Strong candidates preserve the finding, give the sponsor room, and convert a null into a decision.

Example strong answer

The finding does not change, and I would not soften it. What I would think carefully about is the sequence and the framing, because a director who is surprised in a room full of peers will spend their energy defending the hypothesis rather than absorbing the result.

So first, I would tell them one-on-one and before the meeting. Not to negotiate the conclusion, but so they have time to process it and to adjust the planning document on their own terms. Being blindsided is what turns a disagreement about data into a conflict about people.

Second, I would lead with the decision implication rather than the statistics. Not "the coefficient shrinks when we control for prior engagement," but "the tutorial investment as scoped is unlikely to move retention by the amount the plan assumes, and here is what I think would." Directors are making resource decisions; the analysis is an input, and framing it as an input rather than as a verdict keeps the conversation productive.

Third, I would be precise about what I found and what I did not. I found that the raw retention gap is largely selection — motivated users watch tutorials and motivated users retain. I did not find that tutorials are worthless. Those are very different claims and conflating them would be as bad an error in the other direction. If my analysis leaves a plausible residual causal effect, I would give the range rather than rounding it to zero.

Fourth, I would bring the next step rather than only the bad news. The honest way to settle this is an experiment: promote tutorial content to a randomly selected slice of new users and measure retention against a holdout. That is a small, cheap, three-to-four week test compared to a large content investment, and it converts an unresolvable observational argument into an answer. Proposing that gives the director a path that is neither abandoning their instinct nor spending the full budget on it.

Fifth, I would be genuinely open to being wrong about my own method. I would walk them through the specification, the controls, and what would change my conclusion. If they push back with domain knowledge I do not have — for example that the tutorial population is dominated by one segment I treated as homogeneous — that is useful and I should test it rather than defend.

The outcome I would be aiming for is that the director says the test was a good idea, not that they concede. And if the decision goes ahead anyway despite the evidence, I would document the analysis clearly, make sure the assumption is written down as an assumption, and set up the measurement so that in six months we learn something either way. Being overruled is normal; being overruled without instrumenting the outcome is the part I would push back on.

Follow-up questions

The director asks you to re-run the analysis excluding a segment that happens to strengthen the result. How do you respond?
The experiment you proposed comes back showing a real effect, larger than your observational estimate. What went wrong in your analysis?

Preparation tip

Adobe's data science loop rewards candidates who state their uncertainty out loud. Candidates consistently report being pushed on sensitivity analysis — not "what is your answer" but "how much would your answer move if this assumption were wrong." Build the habit of finishing every response with the assumption your conclusion is most sensitive to and what you would do to check it. The second habit worth building: for any metric question, name the denominator and the time window before you name the number. Most of Adobe's business questions are subscription questions, and subscription metrics are ambiguous until those two things are fixed.

Back