Demetri Pananos Ph.D - Numbers, Letters, Sometimes Both

More on Bayesian Statistics

I recently wrote a blog post for my employer in which I made some claims regarding Bayesian statistics.

Demetri Pananos

A Brief Tour of CUPED and Related Methods (Pt. 2)

In a previous post, I demonstrated that Deng et. al’s CUPED is formally equivalent to regression adjustment as justified through the Frisch Waugh Lovell theorem. As a…

Demetri Pananos

A Brief Tour of CUPED and Related Methods (Pt. 1)

Randomized experiments have a rich history, almost none of which is the concern of this post. Rather, I want to focus on a tiny slice of randomized experiment literature…

Demetri Pananos

Calibration of Mind Changing for Bayesian AB Tests

Bayesian decision making with loss functions was a very popular with stakeholders and data scientists when I was working in AB testing. In short, eschew p values instead ask…

Demetri Pananos

What Is Confounding?

Thinking back to my Introduction to Epidemiology class, confounding was one of the topics of central interest. However, it took the remainder of my PhD (and then some) to…

Demetri Pananos

Interesting Interview Questions

Probability

I’m about to do some interviews this week which got me reflecting on some of my favorite questions I’ve been asked. Usually, these are little toy problems you’d find in an…

Demetri Pananos

Peeking Sets You Up For Dissapointment

Peeking (looking for significance in an AB test before the experiment has enough samples to reach desired power) is a “no no”. Rationales for not peeking typically mention…

Demetri Pananos

You Just Said Something Wrong About Logistic Regression

Congratulations, you just said something wrong about logistic regression. That’s OK, logistic regression is hard and we all have to learn/re-learn some things from time to…

Demetri Pananos

The Winner’s Curse Is Easy To Understand From This Picture

Take a look at the photo below, and it i should be easy to understand why The Winner’s Curse (the general tendency for detected effects to be an over estimate of the truth)…

Demetri Pananos

Survival Analysis With Logistic Regression

This question came up on cross validated. In the course of responding to my answer, OP asks “why can’t I adjust for exposure length in a logistic regression” which kind of…

Demetri Pananos

The Generalized Gamma Distribution For Parametric Survival

A couple weeks back, I posted a little something something on the Shifted Beta Geometric distribution. That distribution is used in estimating churn in contractual settings…

Demetri Pananos

Brain Teaser About Frequentist Statistics

Ron Kohavi is somewhat of a prominent figure in the A/B testing community. Armed with having coauthored “Trustworthy Online Controlled Experiments”, experience at places…

Demetri Pananos

Estimating the Shifted Beta Geometric Model in Stan

The shifted beta geometric model (sBG) is a model that is used to forecast retention/survival of users in contractual settings (think netflix, disney plus, tinder gold…

Demetri Pananos

Regularization of Noisy Multinomial Counts

I’m reading Statistical Learning with Sparsity. Its like Elements of Statistical Learning, but just for the LASSO. In chapter 3, there is this interesting example of using…

Demetri Pananos

Right In Ways You Don’t Care About

I’ve typically been long on the value of data scientists but I think I’m starting to lose my optimism.

Demetri Pananos

Difference in Difference Estimates Can Be Inaccurate When Adjusting For Detected Pre-Treatment Differences in Randomized Experiments

AB Testing

Statistics

Causal Inference

Demetri Pananos

Did You Do Your Homework?

This tweet made its way to my corner of the Bird Site. I’m a glutton for math homework punishment so I attempted it (and failed) but do not confuse my interest as…

Demetri Pananos

A Practical A/B Testing Brain Dump

AB Tests

Statistics

Over the past year or so, I’ve helped move Zapier towards doing better experiments and more of them. I wanted to document some of this for posterity; what I did, what…

Demetri Pananos

Choosing the Optimal MDE for Experimentation

AB Tests

Statistics

Planning experiments means deciding how long to run the experiment (by doing a sample size calculation). A big factor in this decision is agreeing on a minimal detectable…

Demetri Pananos

Causal Risk Ratios in AB Tests with One Sided Non-Compliance

AB Tests

Statistics

Causal Inference

Sometimes we run AB tests where engaging with the treatment (and hence being treated) is optional. Since assignment to treatment is random, we can use the randomization as…

Demetri Pananos

Journeyman Statistics

A number of people have asked me how to learn statistics. I don’t have a good answer for them. I find all books are deficient in some way: some too theoretical, others not…

Demetri Pananos

Way Too Many Taylor Series

Statistics

AB Testing

Let \(Y(a)\) be a potential outcome for a continuous measure under treatment status \(A=a\), which for the purpose of this blog post can be considered a binary treatment.…

Demetri Pananos

Bootstrapping in SQL

Statistics

Remember the “Double Down” from KFC? It was bacon and cheese sandwiched between two deep fried pieces of chicken. I’m willing to bet we all conceived of it independently (as…

Demetri Pananos

PCA on The Tags for Cross Validated

Machine Learning

Julia Silge gave a really good talk in 2018 about PCA ond tags on stack overflow. She was able to interpret some of the components to infer some subgroups of users of stack…

Demetri Pananos

Interim Analysis & Group Sequential Designs Pt 2

Statistics

AB Testing

This is part 2 of an ongoing series on group sequential designs for AB testing. Previous parts are shown below

Demetri Pananos

Forecasting Experimental Lift Using Hierarchical Bayesian Modelling

Statistics

AB Testing

Bayes

You’re part of a team at a company who is tasked with improving conversion on some web page. You’ve run a few experiments already with mixed results and now it is time to…

Demetri Pananos

Interim Analysis & Group Sequential Designs Pt 1

Statistics

AB Testing

Special thanks to Jacob Fiksel for writing a great blog post which inspired me to write my own.

Demetri Pananos

This Is A Quarto Blog

News

This is a quarto blog.

Demetri Pananos

Flippin’ Fun!

Bayes

Stan

Python

I wrote an answer to a question about sequences of coin flips a couple days back that I was quite chuffed with. In short, the question asked for statistical ways to…

Demetri Pananos

Hacking Sklearn To Do The Optimism Corrected Bootstrap

Statistics

Machine Learning

Python

Scikit-Learn

Its late, I can’t sleep, so I’m writing a blog post about the optimism corrected bootstrap.

Demetri Pananos

On Interpretations of Confidence Intervals

Statistics

The 95% in 95% confidence interval refers not to the probability that any one interval contains the estimand, but rather to the long term relative frequency of the estimator…

Demetri Pananos

Log Link vs. Log(y)

R

Statistics

You wanna see a little gotcha in statistics? Take the following data

Demetri Pananos

Gradient Descent with ODEs

Python

Machine Learning

Statistics

Gradient descent usually isn’t used to fit Ordinary Differential Equations (ODEs) to data (at least, that isn’t how the Applied Mathematics departments to which I have been…

Demetri Pananos

Neat Litle Combinatorics Problem

Python

Probability

I’ll cut right to it. Consider the set \(S = (49, 8, 48, 15, 47, 4, 16, 23, 43, 44, 42, 45, 46 )\). What is the expected value for the minimum of 6 samples from this set?

Demetri Pananos

Coins and Factors

Python

Riddler

I love Fivethirtyeight’s Riddler column. Usually, I can solve the problem with computation, but on some rare occasions I can do some interesting math to get the solution…

Demetri Pananos