Frequently Asked Questions
Table of contents
Where can I run experiments with ABsmartly?
ABsmartly supports a wide range of environments for experimentation. You can deploy experiments (a.k.a. A/B tests) on web, email, app environments, and more, both server-side and client-side. The platform caters to various testing scenarios, including feature flags, split testing, multi-variant, multi-page, multi-platform, SEO tests, and cross-device testing, among others.
What makes ABsmartly unique in the experimentation tool market?
ABsmartly is the only experimentation platform featuring a Group Sequential Testing engine, which allows tests to run up to 80% faster. Plus, our cloud deployment means your A/B test data is stored securely in your organization's cloud account, ensuring robust information security. We also prioritize features like data privacy, maintenance-free solutions, and expert support and training.
What types of statistical methods does ABsmartly offer?
ABsmartly offers a range of advanced statistical methods tailored for experimentation, online A/B testing in particular. Our platform includes traditional Fixed Horizon Testing as well as innovative Group Sequential Testing. This Group Sequential Testing Engine allows teams to run experiments 20% to 80% faster compared with a fixed horizon approach, and twice as fast when compared with normal sequential testing. Furthermore, when conducting multiple comparison corrections, ABsmartly uses the Dunnett correction over the commonly used Bonferroni, offering greater statistical power. We continuously refine our statistical methodologies to provide accurate and timely insights, ensuring the most robust and efficient experimentation results for your business.
What data analysis capabilities does ABsmartly offer?
ABsmartly is designed for powerful and in-depth data analysis. Users can delve deep into their experimentation data, benefiting from flexible segmentation, precise filtering, and customizable reporting. These features ensure you have all the tools necessary to make well-informed decisions and effectively debug and interpret your results.
How quickly can I see reporting results from experiments on ABsmartly?
ABsmartly ensures a minimal 15 to 30-second lag from tracking to reporting.
Does ABsmartly offer a visual editor?
No, ABsmartly doesn’t offer a visual editor. We believe in robust and performant experimentation, and as such, all our experiments are coded directly into the codebase. This approach ensures optimal performance and allows developers to control every detail of the experiment, ultimately enabling an unbiased test. However, it’s possible to create integrations with your CMS or copywriting tool, simplifying the process of running A/B tests without developer intervention.
How does ABsmartly support integrations with other platforms?
ABsmartly is designed with an API-first approach, enabling seamless integration with various tools. Our SDKs further simplify the integration process.
What kind of support and training does ABsmartly offer?
We offer comprehensive support and training to our clients. Our team has extensive experience in managing large-scale experimentation. Clients have access to real-time support via Slack or video calls, and to a dedicated Technical Account Manager. Additionally, we provide regular online training sessions on topics like experimentation culture, best practices, decision-making, and strategies for effectively scaling experimentation.
What is ABsmartly's subscription model?
ABsmartly operates on a yearly subscription basis. This approach ensures that our users gain consistent access to our comprehensive suite of features, promoting sustained and effective experimentation throughout the year. For more details on our subscription model or pricing, please don't hesitate to get in touch!
Can I try the platform before committing to a yearly subscription?
Yes. You can acquire a 60-day PoV (Proof of Value) to explore the features and experiment with the platform before committing to the annual subscription.
Can you tell me more about ABsmartly's pricing plans?
With our flexible, events-based pricing model, clients have unlimited experiments, metrics and platform users, and they pay per event. We believe this is the best way to scale experimentation and provide a tool that can grow with each unique organisation. We have no pricing tiers or packages, meaning all our customers get all our features, including our advanced Group Sequential Testing engine, real time data processing, collaborative tools and more.
We handle the infrastructure costs and we provide the platform, however if clients wish, they can handle the infrastructure on their side. Our event-based plans start at €60,000/year for 50 Million events per month. We include onboarding, standard training and support in our pricing at no extra cost.
We have 2 categories of billable events:
Exposure events
An exposure event happens every time a call is made to check which variant (of an experiment or a feature flag) to show a visitor. The number of exposure is dependent on the number of unique visitors, the number of experiments and the number of exposure per experiment.
Exposure Events = {# unique visitors} x {# experiments} x {# experiment exposures}
Goal events
A goal event happens every time a goal is triggered by a visitor of your application. A goal is what is used to create your experimentation metrics and they usually represent an action taken by your visitors, a click on a button, a conversion, a cancellation, etc. The number of goals events is dependent on the number of unique visitors, the number of goals and the number of goals exposures.
Goal Events: = {# unique visitors} x {# goals} x {# goal triggers}
How does ABsmartly prioritize data privacy and compliance?
We prioritize data privacy. With ABsmartly, your A/B test data never leaves your data center. Moreover, we're fully GDPR, HIPAA, and SOC2 compliant, with ISO 27001 certification. For more information on security, see https://absmartly.com/security.
Is there a demo I can see to understand ABsmartly's features better?
Absolutely! The best way to get acquainted with our platform is through a live demo. Book a demo with us, here.
Are clients able to provide input into platform development based on their needs?
Absolutely! We greatly value feedback from our customers and actively encourage them to share their requirements and ideas. Many of the features we've introduced in 2023 were directly influenced by client suggestions and needs. We always aim to integrate these insights into our development roadmap, ensuring that our platform evolves in alignment with our customers' aspirations.
