MindsDB: An enterprise AI tool
A tool to interact with multiple, peta-byte scale enterprise data sources using RAG, text-2-SQL, knowledge bases and MCP.
Role
Principal UX Designer / PM
Industry
Augmented analytics / chatbot
Duration
2 months
Testing
We iterated on various approaches to gathering user data. We tried analytics solutions including Hotjar, Hubspot and Amplitude, eventually settling on PostHog. One challenge we faced was testing prototypes of enterprise software like this with actual user data. We used text-2-SQL benchmarks such as BIRD and industry standard datasets like
Challenges
As with many modern AI apps, the UX is only as good as the AI implementation. A product can be beautiful and well-designed, but if the performance, accuracy, latency, etc is poor it will fail. Achieving the necessary performance at scale was extremely challenging for the engineering and ML teams. Similarly, it's difficult to truly test these types of applications because users are hesitant to provide actual data. Testing on Huggingface and Kaggle datasets is insufficient.
Other projects
H2O.ai Data science sensitivity analysis
Sensitivity analysis (aka “what if” analysis) lets users interactively explore their data and see how different hypothetical scenarios might impact an outcome or variable. This is especially important for ensuring the fairness of a dataset.
Robustness Gym: real world NLP benchmarking
A tool for Natural Language Inference researchers
AutoCAD Skill tree
A tool for CAD/BIM Managers to visualize their teams specializations for project staffing