Microsoft is a global technology leader that empowers every person and organization on the planet to achieve more. With a heritage of innovation spanning over four decades, Microsoft builds products and services that touch billions of lives – from Windows and Office to Azure, GitHub, and the rapidly expanding Microsoft Security portfolio. The company’s culture is rooted in a growth mindset, encouraging employees to learn continuously, collaborate across boundaries, and drive impact at scale. In India, Microsoft operates large engineering hubs in Bengaluru, Hyderabad, and other cities, offering a vibrant ecosystem of learning, mentorship, and career progression. The Data Engineer II role sits within the Microsoft Security – Purview Agents Platform team. This team creates next‑generation data agents powered by generative AI, large language models, and advanced analytics to help customers secure their data across clouds, on‑premises, and hybrid environments. As a hands‑on Data & AI Automation Engineer, you will design, build, and operate robust data pipelines, automate operational workflows, and integrate intelligent copilots and agents that surface actionable insights in real time. The position blends deep data engineering expertise with AI‑driven product thinking, making it ideal for engineers who enjoy turning raw telemetry into governed, self‑service data products. Key responsibilities include: 1. Design and operate batch and near‑real‑time pipelines using Azure Synapse, Fabric, Kusto (ADX), SQL, Python, and REST APIs. 2. Build governed ingestion, transformation, and serving layers for cost, utilization, and operational data. 3. Implement data‑quality frameworks covering schema validation, lineage, freshness monitoring, and performance optimization. 4. Create reusable data products, semantic models, APIs, and self‑service portals with clear ownership. 5. Redesign legacy workflows into AI‑native, integrated pipelines. 6. Develop Python services, FastAPI endpoints, CLI tools, scheduled jobs, and event‑driven workflows. 7. Automate monitoring, validation, failure investigation, reporting, and recovery actions. 8. Build copilots and agents that leverage Kusto, Azure OpenAI, and vector search to provide real‑time assistance to platform users. 9. Implement telemetry, health dashboards, alerts, and anomaly detection for operational excellence. 10. Partner with product, security, and compliance teams to translate business needs into scalable technical designs. The tech stack revolves around Azure Data services (Synapse, Data Lake, Fabric), Kusto/ADX, Python, SQL, FastAPI, Azure DevOps, CI/CD pipelines, and emerging GenAI tools such as Azure OpenAI, embeddings, and retrieval‑augmented generation. Successful engineers can expect a clear growth path from individual contributor to senior technical lead, with opportunities to move into architecture, AI product management, or security engineering roles. Joining Microsoft offers exposure to cutting‑edge cloud technologies, a supportive learning environment, competitive compensation, and the chance to impact security solutions used by enterprises worldwide.