I joined the Remote.com AI and Knowledge Management team in 2023. On this team I was technical lead for a number of projects:
- Knowledge Management & AI Governance Data Pipeline
- AI Support Chatbot
- AI Support Email Auto-Responder
- AI Evaluation Framework
- Ask An Expert (backend)
Remote.com is a global human resources SAAS, and is (at the time of writing) the largest Elixir codebase known to exist.
I [...] witnessed firsthand his exceptional engineering talent and leadership abilities. Brett is a master of his craft - he demonstrates deep technical expertise and a methodical approach to complex, ambiguous challenges. He is the person that others turn to for advice on tough technical issues and consistently sets high standards for engineering excellence. What sets him apart, though, is his natural ability to lead through influence and his strategic thinking. Brett consistently leads by example and elevates the performance of everyone around him.
Knowledge Management & AI Governance Data Pipeline
At Remote.com we had a massive internal knowledge database that covered every country we operated in. This was routinely updated by humans and had a mixture of private internal information and information we wanted to make available to our customers. This internal database spanned tens of thousands of documents across various internal and external platforms; including Zendesk, Notion, Slack and other internal sources…
I was tasked with architecting and building a data pipeline that regularly ingested multiple knowledge sources, scanned it for sensitive information, mistakes and other issues, then published and ingested that data into a vector database for later retrieval by our runtime AI systems.
This was achieved by using:
- ArgoCD to orchestrate the ETL pipelines
- Python and Typer to provide the ETL scripting
- Kubernetes to provide the infrastructure orchestration
- Terraform to orchestrate the provisioning of cloud resources
- Gitlab providing the CI/CD platform
AI Support Chatbot
Remote.com saw massive and sudden growth in 2023. One part of the organization that was struggling to cope with the sudden influx of customers was their Customer Success Department and Support staff.
We built a new chatbot that maintained feature-parity with their old Zendesk support bot, allowing us to run both in parallel and analyse performance before switching our customers fully to the new system.
We integrated the AI Knowledge Database described above as a layer between our customers and our first-line support staff.
What was critical, was the ability to monitor support team schedules, and to provide application-level filtering on which queries and which user roles could be escalated to a human agent. This meant building a sophisticated “switchboard system” to manage the flows of conversations between the AI, human agents, and back again.
In addition, we needed to carefully build and monitor our “user context” only allowing certain information about the user to be exposed to the AI systems.
In doing so we were able to deflect ~55% of all support escalations, and were able to answer ~60% of all knowledge-based queries with AI, confidently, and with sources.
This had a massive impact on the support team and enabled Remote to scale their operation into countries far quicker.
Our chatbot was achieved by using:
- Tight integration with the Remote.com internal APIs using Elixir and Phoenix
- Integration with Zendesk APIs
- Live messaging and UI integration with Smooch Sunshine Conversations
- Deployed with Helm and Kubernetes into Google Cloud.
I implemented a number of governance and monitoring tools to collect business metrics and analyse the performance of our bot in production.
Brett is, without a doubt, one of the best engineers — and overall humans — I’ve had the pleasure of working with in my entire career.
His vast and comprehensive knowledge — spanning the full stack, from front end to infrastructure — is matched only by his ability to adapt and learn quickly. I’ve seen him successfully steer a project in the right direction almost entirely on his own, even though it was written in a language he wasn’t initially proficient in.
He’s detail-oriented in execution, yet always aligned with the bigger goals. With a strong product mindset, he’s unafraid to challenge assumptions and focuses his time on solving real, impactful problems. Thanks to his excellent communication skills, he is highly self-sufficient and capable of navigating even the most complex challenges. He consistently involves stakeholders strategically to accelerate problem-solving and ensure alignment. [...]
AI Email Auto-Responder
On the back of the success of our chatbot system, the support team suggested we implement a similar system around our support emails.
help@remote.com
I worked with Zendesk to integrate our AI Knowledge API to selectively auto-respond to support queries that could be easily answered from our AI Knowledge Database.
The support auto-responder attempted to answer ~40% of customer queries where it felt highly confident in its ability to answer, and the majority of those queries were answered well enough so that no additional support was required.
AI Evaluation Framework
Once our chatbot and email auto-responder had run in production, we realised we needed more in-depth insights into how customers were querying the data, and how the chatbot’s responses were performing in real-world scenarios.
In order for our team, and Remote management, to locate areas of weakness in our AI Knowledge Database, I built Remote’s AI Evaluation Framework.
This would system routinely pull all conversations our users had with our AI systems, perform strict anonymization to remove any sensitive personal data, and then perform a series of analyses on the conversation data.
I was able to use AI vector embeddings to categorize our users’ queries in multi-dimensional space, in temporary vector databases and then perform cluster analysis at various levels (conversation, message, etc).
Once we could categorize conversations we could then evaluate the emotional tone, and success rates in vector space.
This allowed us to pinpoint key areas of our system where knowledge was lacking, and immediately identify painpoints for our customers.
In addition, we didn’t just want the ability to analyse performance retroactively, we wanted the Remote AI Evaluation Suite to be able to project common queries into the system, and evalute responses before changes were deployed.
When working with complex AI LangChain systems that chain multiple prompts together, it can be very difficult to trust the behaviour of the system-as-a-whole.
The evaluation suite was developed to project common edge-cases and senstive topics into our AI bots and ensure they answered with the correct content, the correct emotional tone, and were responding within an appropriate timeframe.
I've had the privilege of working closely with Brett on some of the most foundational, high-impact initiatives in our AI platform, and he stands out as a truly exceptional engineer.
Brett architected and implemented our core data pipelines. More than just basic ingestion, he built a robust, extensible framework that empowers teams across the company to seamlessly contribute knowledge to our AI systems. His work included automated quality checks and clear reporting mechanisms that safeguard the integrity of our data, ensuring only high-quality content is ingested. [...]
Ask An Expert
I was backend lead on the Ask An Expert feature. Ask An Expert allows Remote customers to book in calls with various teams of advisors on-demand using real-time scheduling and dynamic routing across teams based on user queries.
Cleanest code I've ever had the pleasure to work with. Extremely capable at mentoring - the time I passed working alongside him made me a better programmer.
A lot of people who are exceptionally gifted on the technical side might be lacking in vision or end up confusing the technical implementation with the goal and not just the means to achieve that goal: that is definitely not the case with him. Brett doesn't just deliver the optimal solution, but he also had the experience and intuition necessary to foresee future issues way before the rest of the team, management included.
10/10, would love to work with him again, an absolute beast and powerhouse. You learn to trust his judgement pretty quickly, and trust is a powerful thing to have in a team.
