ML Software Engineer - Platform LLM Training and Inference
OTHER•Pakistan
About this role
At Rayn, we don’t just work—we innovate, collaborate, and create solutions that leave a lasting impact. As part of our team, you’ll have the opportunity to shape the future of a purpose-driven organization that is redefining how technology can address societal & public health challenges.
We are looking for a software engineer to help design and build the core of the ILM platform. This role sits at the intersection of systems engineering, AI infrastructure, and developer tooling. You will work on runtime orchestration, GPU scheduling, and model lifecycle workflows that power training and inference across heterogeneous environments. This is a hands-on role suited for engineers who enjoy working close to the metal, care about performance, and want to build foundational infrastructure rather than application-layer features.
What you will bring to Rayn as a ML Software Engineer
- Design and implement core services for the ILM orchestration platform
- Build unified pipelines for fine tuning and training methods including LoRA, QLoRA, and RL-based approaches
- Integrate and manage multiple LLM runtimes such as vLLM, TensorRT, and llama.cpp
- Implement GPU-aware scheduling, resource allocation, and workload isolation
- Optimize VRAM usage, KV-cache management, and inference throughput
- Build internal APIs and developer tooling for model lifecycle management
- Collaborate closely with hardware, platform, and research teams
Who We’re Looking For
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Strong experience in JavaScript, Python and C++ with a minimum of 6+ years of experience in software development, architecture, and team leadership roles.
- Solid understanding of Linux systems, containers, and runtime environments
- Experience working with GPU-accelerated workloads
- Familiarity with deep learning inference and training frameworks such as PyTorch
- Familiarity with fine tuning and training techniques for large language models
- Practical experience deploying and optimizing AI infrastructure
- Strong problem-solving skills and attention to performance and reliability
- Experience with CUDA, llama.cpp, VLLM, LlamaFactory, TensorRT, or GPU profiling tools.
- Experience building distributed systems or schedulers
- Experience in AI infrastructure, MLOps, or HPC environments
What you'll do
- Design and implement core services for the ILM orchestration platform and build unified pipelines for fine-tuning and training methods. Collaborate closely with hardware, platform, and research teams to optimize GPU scheduling and resource allocation.
About Rayn Group
At Rayn, we approach everything we do with authenticity and with passion. As entrepreneurs, operators, and investors, we not only empathize with but embrace the unique struggle involved in building something special. We become deeply invested in our partner companies and together we believe we can tackle the hardest problems.
Ready to join Rayn Group?
Take the next step in your career journey
Frequently Asked Questions
What does a ML Software Engineer - Platform LLM Training and Inference do at Rayn Group?
As a ML Software Engineer - Platform LLM Training and Inference at Rayn Group, you will: design and implement core services for the ILM orchestration platform and build unified pipelines for fine-tuning and training methods. Collaborate closely with hardware, platform, and research teams to optimize GPU scheduling and resource allocation..
Is the ML Software Engineer - Platform LLM Training and Inference position at Rayn Group remote?
The ML Software Engineer - Platform LLM Training and Inference position at Rayn Group is based in Pakistan, Pakistan. Contact the company through Clera for specific work arrangement details.
How do I apply for the ML Software Engineer - Platform LLM Training and Inference position at Rayn Group?
You can apply for the ML Software Engineer - Platform LLM Training and Inference position at Rayn Groupdirectly through Clera. Click the "Apply Now" button above to start your application. Clera's AI-powered platform will help match your profile with this opportunity and guide you through the application process.