When is an A/B test significance enough to call a winner?

[David_B]

I'm analyzing A/B test results for a website feature change, and while the variant shows a higher conversion rate, I'm not confident in declaring it a winner because the sample size still feels relatively small. I understand the basic concept of p-values and confidence intervals, but I'm struggling to determine what level of statistical significance is truly meaningful for a business decision in this context. For other data analysts, how do you balance statistical rigor with practical decision-making when results are promising but not overwhelmingly definitive? What are your go-to methods for calculating required sample size beforehand and communicating the uncertainty of results to stakeholders who want a clear yes or no answer?