topic Would a FAANG company just use a t-test for A/B Testing? in New SAS User

Would a FAANG company just use a t-test for A/B Testing?

sipos — Tue, 05 Mar 2024 10:36:05 GMT

Currently doing a job application test for a FAANG company that asks me to conduct an A/B tests and cruelly says:

There are several equally acceptable stats approaches, so we’re interested to see your approach.

Which is, like -- well, I was going to just do a T-Test, but now I'm terrified to try something that simple.

Any thoughts on what methods a FAANG company would expect on an A/B test like this?

Re: Would a FAANG company just use a t-test for A/B Testing?

ballardw — Tue, 05 Mar 2024 15:22:57 GMT

Sounds like an employment question. Does "FAANG", what ever that may be, actually have any impact on this question?

I had a job interview where they gave me a data set and instructions "do the most complicated test/analysis you can", though of course the software they wanted was something I had never used at all.

I basically answered " an analysis or test without a question is a pretty poor use of time". I did get the job.

Re: Would a FAANG company just use a t-test for A/B Testing?

sbxkoenk — Tue, 05 Mar 2024 17:07:18 GMT

@ballardw wrote:

Does "FAANG", what ever that may be, ...

In finance, “FAANG” is an acronym that refers to the stocks of five prominent American technology companies: Meta (META) (formerly known as Facebook), Amazon (AMZN), Apple (AAPL), Netflix (NFLX); and Alphabet (GOOG) (formerly known as Google).

Koen

Re: Would a FAANG company just use a t-test for A/B Testing?

Reeza — Tue, 05 Mar 2024 17:26:33 GMT

I would probably list a few of the tests that are options and then explain why I choose the one I did over the others.

Also, keep in mind the type of test, ie is it a count/chi square more appropriate or continuous/rate data?

Re: Would a FAANG company just use a t-test for A/B Testing?

sbxkoenk — Tue, 05 Mar 2024 17:27:11 GMT

Paper 1070-2017
Cold-Start Solution to A/B Testing Using Adaptive Sample Size Modification
Bo Zhang, IBM; Liwei Wang, Pharmaceutical Product Development Inc.

https://support.sas.com/resources/papers/proceedings17/1070-2017.pdf

Google : "A/B testing" site:lexjansen.com

A/B Testing is very much present in SAS® Customer Intelligence 360. But you probably don't have access to that product.

Using the A/B Test Node for Control Groups
https://video.sas.com/detail/video/4844229730001/using-the-a-b-test-node-for-control-groups

Koen

Re: Would a FAANG company just use a t-test for A/B Testing?

Ksharp — Wed, 06 Mar 2024 01:55:59 GMT

I think it is like a Crossover Experiment Design, like the following graph:

And Check example of PROC TTEST for it.

SAS Help Center: Example 122.4 AB/BA Crossover Design