Research Engineering Intern – AI Evaluations

Karya is an emerging AI‑driven technology startup headquartered in Bengaluru, focused on building data‑centric platforms that enable businesses to harness the power of large language models (LLMs) and other advanced machine‑learning solutions. Founded by a team of seasoned researchers and engineers, Karya has quickly positioned itself as a bridge between cutting‑edge AI research and real‑world product deployments, serving clients across finance, healthcare, and e‑commerce sectors. The company culture emphasizes rapid experimentation, open collaboration, and a strong commitment to ethical AI, encouraging interns and fresh graduates to contribute meaningfully from day one.

The Research Engineering Intern – AI Evaluations role is designed for students or recent graduates who are passionate about assessing and improving the performance of AI and language models. Interns will work closely with Karya’s research and engineering teams to design, implement, and refine evaluation pipelines that measure model accuracy, robustness, and fairness across a variety of benchmarks. This position offers a unique blend of hands‑on engineering, data analysis, and research support, providing a solid foundation for a career in AI research or applied machine learning.

Key responsibilities include

• Designing and developing automated tools for evaluating LLMs and other AI models.
• Curating and preprocessing datasets used for benchmark testing.
• Running large‑scale evaluation experiments and logging results.
• Analyzing model outputs to identify systematic errors and quality gaps.
• Implementing statistical analysis and visualisation of evaluation metrics.
• Collaborating with research scientists to propose improvements based on evaluation findings.
• Maintaining documentation of evaluation workflows and best practices.
• Assisting in the creation of new evaluation benchmarks tailored to specific industry use‑cases.
• Contributing to internal knowledge‑sharing sessions and code reviews.
• Supporting the deployment of evaluation pipelines in cloud environments.

The tech stack primarily revolves around Python, with extensive use of libraries such as PyTorch, TensorFlow, Hugging Face Transformers, Pandas, NumPy, and evaluation‑specific tools like EvalAI and MLflow. Familiarity with cloud platforms (AWS/GCP) and containerisation (Docker) is advantageous. Interns will gain exposure to state‑of‑the‑art LLMs, data annotation pipelines, and rigorous scientific evaluation methods.

Growth path at Karya is clearly defined: high‑performing interns may transition to full‑time Research Engineer or Machine Learning Engineer roles, with opportunities to lead independent research projects, publish findings, and influence product direction. Joining Karya offers the chance to work on impactful AI problems, learn from seasoned researchers, and become part of a fast‑growing startup that values innovation, learning, and ethical AI development.