Karan Singh
Research Software Engineer
I build machine learning and high performance systems, and I like to prove they actually work, from scientific computing at CERN (LHCb) to open benchmarks that test AI reliability.
I focus on making systems verifiable. That can mean profiling how CERN's analysis software behaves on a codebase of over a million lines, or measuring when a retrieval based AI assistant quietly returns an out of date answer. I build from first principles, report honest numbers (including negative results), and ship open, reproducible work.
Now: Currently a member of the HEP Software Foundation, working on AI evaluation and moving community software tools to run on GPUs. Also contributing to sktime and preparing StaleBench for arXiv and JOSS.
Research & Publications
StaleBench: measuring answer freshness in retrieval based AI
Preprint (arXiv cs.LG, in submission) · JOSS in preparation
Across 10 open models, about half of answers stay out of date after a fact changes, even when the new document is added instantly. The cause is the model's position bias, not the search step. Shipped as an open benchmark and tool.
Sustainability studies of big data processing in high energy physics
Poster, ACAT 2025 (co authored)
Improvements on QAOA for particle trajectory reconstruction
Poster, ACAT 2025 (co authored)
JOSS reviewer. Five research artifacts archived on Zenodo (StaleBench, RNArrow, AXON, Arkhe, DisElect).
Selected Projects
StaleBench: answer freshness benchmark for retrieval based AI
An open benchmark and tool that measures how long an AI system keeps giving the old answer after a fact changes, and separates the model's behaviour from the search step.
10 models · about 50% stale rate · Wilson confidence intervals
RNArrow: RNTuple to Apache Arrow gateway
A standalone C++ library and streaming server that exposes CERN's new on disk data format as Apache Arrow, a bridge that did not previously exist outside experiment frameworks. Benchmarks reported honestly, including where it is slower.
under 2× C++ overhead · about 2× faster with bulk reads · about 5% network overhead
Graviton: data local Kubernetes scheduler
A Kubernetes operator in Go that runs each job where its data physically lives, avoiding slow transfers across networks in large physics data systems. A gap the default scheduler ignores.
Go · controller runtime · Helm · Prometheus
Telemetry: time series engine and streaming pipeline
A time series storage engine written from scratch in C++ with custom compression, fronted by a Python API, paired with a Kafka and Spark pipeline that flags anomalies in real time. Memory safety checked in CI.
267K messages per second ingest · about 2× compression · about 12 ms query
AXON: inference as a service (SONIC) reproduction
A faithful proof of concept of the pattern CMS uses to serve machine learning models from shared GPUs, built with real NVIDIA Triton, ONNX, and gRPC, and benchmarked against a strict latency budget. Reports the honest bottleneck.
dynamic batching: 3.7× throughput · 5.6 ms median latency
DisElect: auditing election disinformation in open AI models
An AI safety audit of whether open weight models comply with election disinformation prompts across four elections, and how much a constitution style safeguard reduces it.
220 prompts · 97% to 12% compliance with a safeguard
Experience
Member, HEP Software Foundation (HSF) · CERN community
2026 to present- Part of the community that builds the shared software used across particle physics worldwide.
- Working on evaluating AI tools and on moving key community software to run on GPUs so it is faster and cheaper.
Research Collaborator, LHCb · IFIC Valencia · CERN
Oct 2025 to Mar 2026- Built a tool that measures how much time and energy CERN's analysis software uses, on a codebase of over a million lines.
- Found where the software was slow and showed how moving work to the GPU could speed it up.
Open Source Developer, Google Summer of Code · CERN HSF
Jun to Sep 2025- Built a Python tool comparing classical, AI, and quantum methods for a physics problem across 25 million data points.
- Improved the accuracy of the machine learning model from about 50% to over 90%.
AI Intern · Nihin Media K.K.
Jan to Jul 2025- Built and deployed an AI assistant used by more than 5,000 people every day.
- Made its answers more reliable (about 70% fewer made up answers) and cut its response time roughly in half.
Mentee and Contributor, sktime · Open source machine learning
2026- Selected for a 12 week mentorship contributing to a widely used machine learning library.
Education
B.Tech, Computer Science and Engineering
SRM University, Delhi NCR (Sonepat) · CGPA 8.2 / 10
Supplementary coursework: selected programs from IIT Madras, NVIDIA, Meta, and IBM.