Adopting GitHub Copilot: WellSky's Journey Toward AI-assisted Software Development

In the ever-evolving landscape of technology, Generative AI stands as a beacon of innovation, holding the promise of revolutionizing various sectors. At WellSky, we have embraced this potential by deploying GitHub Copilot for our engineering teams. The results are already visible, and we are excited to share the journey we undertook to integrate this transformative tool into our workflows.

Recognizing the Need and Securing Leadership Investment

The first step in our journey was securing recognition and investment from our leadership. The endorsement for this initiative came from the highest echelons of our organization, including our CTO and the engineering executive team. Their investment and support were instrumental in driving the project forward and ensuring that we had the necessary resources to implement GitHub Copilot effectively.

Establishing a Governance Structure

With the technology selected, we moved on to establishing a governance structure. This involved close collaboration with our legal teams to ensure that all regulatory and compliance aspects were addressed. We developed a specific policy on the use of Generative AI in software development and established the Responsible AI Committee.  The governance framework serves as a guide, ensuring that the deployment of GitHub Copilot aligns with our organizational policies and industry standards.

Evaluating Generative AI Technologies

The next phase involved a thorough evaluation and comparison of various generative AI technologies for code generation. We conducted pilot projects to assess the capabilities and suitability of these technologies for our needs. This rigorous evaluation process enabled us to make an informed decision and select the most appropriate tool for our engineering teams.  After the evaluation, we developed multi-phased roll-out plan.

Training and Onboarding

Training our teams was a crucial step in the successful deployment of GitHub Copilot. We engaged an external expert and our GitHub account management team to conduct training sessions and scheduled office hours for ongoing support. Additionally, we facilitated knowledge sharing among teammates through SkyStudio sessions and during our recently concluded DevDays at WellSky. These efforts ensured that our teams were well-equipped to leverage the full potential of GitHub Copilot.

Measuring Usage and Impact

To gauge the effectiveness of GitHub Copilot, we established a comprehensive set of usage metrics. These included direct metrics such as the number of Copilot users, Copilot usage, and the programming languages used, as well as indirect metrics like flow velocity, feature development impact, bug trends, git commits and pull requests. It required several iterations to refine these metrics, but they have been invaluable in assessing the impact of GitHub Copilot on our engineering efficiency. We conduct quarterly Copilot user surveys to get direct reports from our teammates. We also conduct experience sampling and case studies. With the combination of telemetry metrics, developer survey and experience sampling, we get a balanced view of different signals and trends.

Ongoing Support and Consultation

The deployment of GitHub Copilot is not a one-time event but an ongoing process. We continue to conduct 1:1 check-ins and consultations with our teams to address any challenges and gather feedback. This continuous engagement ensures that our teams are supported and can fully utilize GitHub Copilot to enhance their productivity.

In conclusion, the adoption of GitHub Copilot at WellSky has been a meticulously planned and executed process. Through leadership support, thorough evaluation, structured governance, comprehensive training, and continuous support, we have successfully integrated this powerful tool into our engineering workflow. As we move forward, we remain committed to leveraging generative AI to drive innovation and deliver exceptional results.

‍