Software Engineer III, Copilot Models Inference

Software Engineer III, Copilot Models Inference

United States

Engineering

Individual Contributor

Yes

5769

Full Time

Get future jobs matching this search

or

Job Description

About GitHub

GitHub is the world’s leading platform for agentic software development — powered by Copilot to build, scale, and deliver secure software. Over 180 million developers, including more than 90% of the Fortune 100 companies, use GitHub to collaborate, and more than 77,000 organisations have adopted GitHub Copilot.

Locations

In this role you can work from Remote, United States

Overview

As a software engineer at GitHub, you will enhance the collaboration experience at

GitHub by working closely with a community of engineers and designers with a

distributed, diverse and passionate team delivering the services that millions of

developers depend on. In this role you will design, prototype, implement, ship and

support highly performant and inspiring user experiences with your team.

The GitHub Copilot Models Inference team builds and operates the backend services

and APIs that connect Copilot experiences to AI model providers. These systems make

models available safely and reliably at global scale while managing the production

realities of provider integrations, request routing, service health, latency, capacity, and

changing model availability.

As a Software Engineer III, you will deliver production features end to end and help

operate the services you change. You will participate in design and code reviews, build

tests and operational safeguards, prepare changes for rollout, and use production

signals to improve reliability and performance. Clear communication matters on this

distributed team. You will work with engineers and partners across Models Lifecycle,

product, safety, infrastructure, and external model providers through a mix of live

conversations and thoughtful written updates. This is a hands-on individual contributor

role with opportunities to grow ownership through increasingly independent delivery.

Responsibilities

● Design, implement, test, deploy, and maintain backend services and APIs that

connect GitHub Copilot experiences to model providers.

● Own scoped features end to end, from technical design and implementation through operational readiness, safe rollout, measurement, and follow-up.

● Integrate new models and provider capabilities while accounting for availability, compatibility, capacity, failure modes, and safe fallback behavior.

● Improve service latency, throughput, capacity efficiency, and high-availability

behavior using measurable production outcomes.

● Add and maintain automated tests, dashboards, alerts, runbooks, and diagnostics that make service and model health understandable and actionable

● Keep teammates and partners informed through clear discussions, written updates, and documentation, especially when decisions, risks, or incidents affect others.

● Participate in code reviews and design discussions, incorporating feedback and helping teammates ship maintainable changes.

● Participate in on-call and first-responder rotations, investigate incidents, communicate impact, restore service, and complete appropriate repair and automation work.

● Use progressive delivery practices and production evidence to reduce risk when launching models, changing routing behavior, or updating provider integrations.

● Collaborate with Models Lifecycle, product, safety, infrastructure, and provider partners to deliver model availability and operational improvements across team boundaries.

Qualifications

Required Qualifications:

4+ years experience in Software Engineering, Computer Science, or related technical discipline with proven experience maintaining and delivering production software languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python.

OR Associate’s Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 3+ years experience.

OR Bachelor's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field AND 2+ years experience in Computer Science, or related technical discipline with proven experience coding in languages including, but not limited to, C, C++, C#, JavaScript, Go, Ruby, Rust, or Python.

OR Master's Degree in Computer Science, Electrical Engineering, Electronics Engineering, Math, Physics, Computer Engineering, Computer Science, or related field.

OR equivalent experience

Preferred Qualifications:

Experience applying software design principles such as clear interfaces, useful abstractions, and composition to build maintainable and testable systems.

Experience working with both relational databases and NoSQL data stores, including making appropriate data-modeling and access-pattern tradeoffs.

Experience building or operating cloud-native services, including containers, Kubernetes, or managed cloud infrastructure.

Experience with AI or large language model inference systems, model APIs, or model-provider integrations.

Experience with observability and reliability engineering, including improving latency, capacity, resilience, or incident response for production services.

Experience with canary releases, feature flags, progressive delivery, or other safe

rollout practices.

Compensation Range

The base salary range for this job is USD $107,700.00 - USD $285,900.00 /Yr.

• 4+ years software engineering experience or equivalent degree plus relevant experience in languages like C, C++, C#, JavaScript, Go, Ruby, Rust, or Python.
• Proven ability to maintain and deliver production software.
• Degree in Computer Science, Electrical Engineering, Math, Physics, or related technical field, or equivalent experience.