Demetri Pananos Ph.D
About
Consulting
Numbers, Letters, Sometimes Both
Categories
All
(34)
AB Testing
(5)
AB Tests
(3)
Bayes
(2)
Causal Inference
(2)
Machine Learning
(3)
News
(1)
Probability
(2)
Python
(5)
R
(1)
Riddler
(1)
Scikit-Learn
(1)
Stan
(1)
Statistics
(13)
More on Bayesian Statistics
I recently wrote a blog post for my employer in which I made some claims regarding Bayesian statistics.
Mar 18, 2025
Demetri Pananos
A Brief Tour of CUPED and Related Methods (Pt. 1)
Randomized experiments have a rich history, almost none of which is the concern of this post. Rather, I want to focus on a tiny slice of randomized experiment literature…
Mar 3, 2025
Demetri Pananos
Calibration of Mind Changing for Bayesian AB Tests
Bayesian decision making with loss functions was a very popular with stakeholders and data scientists when I was working in AB testing. In short, eschew p values instead ask…
Sep 1, 2024
Demetri Pananos
What Is Confounding?
Thinking back to my
Introduction to Epidemiology
class, confounding was one of the topics of central interest. However, it took the remainder of my PhD (and then some) to…
Jun 26, 2024
Demetri Pananos
Interesting Interview Questions
Probability
I’m about to do some interviews this week which got me reflecting on some of my favorite questions I’ve been asked. Usually, these are little toy problems you’d find in an…
May 21, 2024
Demetri Pananos
Peeking Sets You Up For Dissapointment
Peeking (looking for significance in an AB test before the experiment has enough samples to reach desired power) is a “no no”. Rationales for not peeking typically mention…
Feb 8, 2024
Demetri Pananos
You Just Said Something Wrong About Logistic Regression
Congratulations, you just said something wrong about logistic regression. That’s OK, logistic regression is hard and we all have to learn/re-learn some things from time to…
Feb 6, 2024
Demetri Pananos
The Winner’s Curse Is Easy To Understand From This Picture
Take a look at the photo below, and it i should be easy to understand why The Winner’s Curse (the general tendency for detected effects to be an over estimate of the truth)…
Jan 20, 2024
Demetri Pananos
Survival Analysis With Logistic Regression
This question came up on cross validated. In the course of responding to my answer, OP asks “why can’t I adjust for exposure length in a logistic regression” which kind of…
Jan 20, 2024
Demetri Pananos
The Generalized Gamma Distribution For Parametric Survival
A couple weeks back, I posted a little something something on the Shifted Beta Geometric distribution. That distribution is used in estimating churn in contractual settings…
Dec 2, 2023
Demetri Pananos
Brain Teaser About Frequentist Statistics
Ron Kohavi is somewhat of a prominent figure in the A/B testing community. Armed with having coauthored “Trustworthy Online Controlled Experiments”, experience at places…
Dec 1, 2023
Demetri Pananos
Estimating the Shifted Beta Geometric Model in Stan
The shifted beta geometric model (sBG) is a model that is used to forecast retention/survival of users in contractual settings (think netflix, disney plus, tinder gold…
Nov 20, 2023
Demetri Pananos
Regularization of Noisy Multinomial Counts
I’m reading
Statistical Learning with Sparsity
. Its like Elements of Statistical Learning, but just for the LASSO. In chapter 3, there is this interesting example of using…
Sep 24, 2023
Demetri Pananos
Right In Ways You Don’t Care About
I’ve typically been long on the value of data scientists but I think I’m starting to lose my optimism.
Sep 11, 2023
Demetri Pananos
Difference in Difference Estimates Can Be Inaccurate When Adjusting For Detected Pre-Treatment Differences in Randomized Experiments
AB Testing
Statistics
Causal Inference
Edits:
Sep 3, 2023
Demetri Pananos
Did You Do Your Homework?
This tweet made its way to my corner of the Bird Site. I’m a glutton for math homework punishment so I attempted it (and failed) but do not confuse my interest as…
Jun 26, 2023
Demetri Pananos
A Practical A/B Testing Brain Dump
AB Tests
Statistics
Over the past year or so, I’ve helped move Zapier towards doing better experiments and more of them. I wanted to document some of this for posterity; what I did, what…
May 14, 2023
Demetri Pananos
Choosing the Optimal MDE for Experimentation
AB Tests
Statistics
Planning experiments means deciding how long to run the experiment (by doing a sample size calculation). A big factor in this decision is agreeing on a minimal detectable…
Mar 31, 2023
Demetri Pananos
Causal Risk Ratios in AB Tests with One Sided Non-Compliance
AB Tests
Statistics
Causal Inference
Sometimes we run AB tests where engaging with the treatment (and hence being treated) is optional. Since assignment to treatment is random, we can use the randomization as…
Feb 11, 2023
Demetri Pananos
Journeyman Statistics
A number of people have asked me how to learn statistics. I don’t have a good answer for them. I find all books are deficient in some way: some too theoretical, others not…
Dec 31, 2022
Demetri Pananos
Way Too Many Taylor Series
Statistics
AB Testing
Let
Y
(
a
)
be a potential outcome for a continuous measure under treatment status
A
=
a
, which for the purpose of this blog post can be considered a binary treatment.…
Nov 25, 2022
Demetri Pananos
Bootstrapping in SQL
Statistics
Remember the “Double Down” from KFC? It was bacon and cheese sandwiched between two deep fried pieces of chicken. I’m willing to bet we all conceived of it independently (as…
Nov 16, 2022
Demetri Pananos
PCA on The Tags for Cross Validated
Machine Learning
Julia Silge gave a really good talk in 2018 about PCA ond tags on stack overflow. She was able to interpret some of the components to infer some subgroups of users of stack…
Aug 16, 2022
Demetri Pananos
Interim Analysis & Group Sequential Designs Pt 2
Statistics
AB Testing
This is part 2 of an ongoing series on group sequential designs for AB testing. Previous parts are shown below
Jul 30, 2022
Demetri Pananos
Forecasting Experimental Lift Using Hierarchical Bayesian Modelling
Statistics
AB Testing
Bayes
You’re part of a team at a company who is tasked with improving conversion on some web page. You’ve run a few experiments already with mixed results and now it is time to…
Jul 20, 2022
Demetri Pananos
No matching items
1
2