Microsoft is hiring a Software Engineer II for its Bangalore office. This opportunity is ideal for experienced backend software engineers who enjoy solving challenging distributed systems problems at massive scale. The selected candidate will become part of Microsoft's AI infrastructure engineering team responsible for building the backend systems that power large-scale Generative AI inference across Microsoft cloud services.
As a Software Engineer II, you will build highly scalable backend services responsible for processing millions of AI inference requests efficiently. Your work will directly impact how Microsoft's AI services allocate compute resources, schedule workloads, balance traffic, and deliver reliable AI experiences to enterprise customers worldwide.
You will work closely with engineers designing cloud-native distributed systems capable of handling asynchronous workloads while maintaining high availability, low latency, and optimal resource utilization. The role provides excellent exposure to hyperscale cloud infrastructure, intelligent scheduling systems, backend architecture, and production-grade service engineering.
Key Responsibilities
- Design scalable backend services supporting AI inference workloads.
- Develop intelligent scheduling systems for efficient compute utilization.
- Build request routing systems for large-scale distributed infrastructure.
- Optimize latency, throughput, and infrastructure cost.
- Improve service reliability and availability.
- Develop production-quality backend software.
- Write clean, maintainable, and well-tested code.
- Participate in architecture discussions.
- Perform performance profiling and optimization.
- Implement telemetry and monitoring solutions.
- Build alerting systems for production services.
- Participate in on-call engineering rotations.
- Collaborate with AI platform teams.
- Support deployment of new AI models.
- Improve scalability of existing backend services.
- Review pull requests and mentor fellow engineers.
- Follow engineering best practices.
- Ensure system security and compliance.
- Contribute to continuous service improvements.
Required Qualifications
- Bachelor's Degree in Computer Science or related field.
- Minimum 3 years of software engineering experience.
- 2+ years developing backend or distributed systems.
- Experience with C#, Java, Go, or C++.
- Cloud platform experience (Azure, AWS, or GCP).
- Strong understanding of distributed system fundamentals.
Preferred Qualifications
- Experience with distributed computing.
- High throughput system design.
- Queueing systems.
- Load balancing.
- Request routing.
- Container technologies.
- Docker.
- Kubernetes.
- Monitoring tools.
- Production service operations.
- System observability.
Why Join Microsoft?
- Work on cutting-edge Generative AI infrastructure.
- Solve engineering challenges at hyperscale.
- Hybrid work flexibility.
- Competitive salary.
- Learning and career growth.
- Global engineering collaboration.
- Excellent work culture.
- World-class engineering practices.
- Exposure to Azure AI platform.
- Opportunity to work with industry-leading engineers.