I'm a data scientist working on a project where we need to update the probability of a rare event as new, streaming data comes in from user interactions. I believe a Bayesian inference approach would be perfect for this, as it naturally incorporates prior knowledge and updates beliefs, but my team is more familiar with frequentist methods. I'm trying to build a simple, convincing prototype to demonstrate the value, specifically for A/B testing where we have strong priors from historical campaigns. For practitioners who have implemented Bayesian methods in production, what were the biggest practical hurdles in terms of computational complexity or explaining the results to stakeholders? Are there specific libraries or frameworks you'd recommend for implementing this in a Python environment that can handle real-time data?
Big picture: build a lightweight online updating probabilistic model that can ingest streaming events and revise beliefs on the fly. Start with a conjugate‑like setup so a batch of new interactions updates a posterior without re‑fitting. Practical hurdles include explaining uncertainty to stakeholders, detecting concept drift, and keeping latency acceptable in production. For Python, consider PyMC3/4, CmdStanPy, or NumPyro, and run a small inference service that batches updates and exposes posterior intervals to dashboards.
Be mindful that stakeholders may push for p-values; present credible intervals and posterior changes, not single point estimates. Emphasize how new data shifts the belief and how robust the conclusions are to priors.
Approach ideas: let priors from historical campaigns be strong but adjustable; use batch updates to smooth noise, and implement drift checks to trigger alerts or model resets. Keep the model simple enough to explain and maintain.
Possible toolset: PyMC3/4, Stan via CmdStanPy, Pyro, and NumPyro are solid options; for real-time operation you’ll likely pair them with a lightweight microservice and a streaming data queue (Kafka/Kafka-like) to feed dashboards and experiments.