Staff Data Engineer

Company: NVIDIA Corporation
Location: Santa Clara
Posted on: September 16, 2023

Job Description:

NVIDIA is hiring senior distributed systems with a data engineering emphasis to develop and scale its AI and deep learning platforms. Our team is building a software 2.0 developer platform with a focus on datasets for AI application development. Together, we will advance NVIDIA's capacity to build and deploy leading solutions for a broad range of AI-based applications such as autonomous vehicles, healthcare, virtual reality, graphics engines, and visual computing. Together, with NVIDIA partners, we will bring autonomous vehicles to life!What you'll be doing:

• Architect and build scalable and distributed commitment to improvement, compute, and data pipelines that will help power the IT Data Lake as a centralized data platform
• Design and build PB sized scalable data lake and structured/unstructured data query interfaces and microservices to ingest, index, mine, transform, and compose large datasets.
• Build Cloud Cost and Usages data patterns to crawl, collect and transform TBs of data on daily basis
• Enabling data models/views across TBs of data which can be consumed by analytical tool as PowerBI. Build PBI analytics for finance reporting
• Build and implement support for versioned, traceable, and immutable datasets in a data lake in a distributed and scalable manner.
• Enable efficient and thoughtful data selection - one of the key ingredients for successful machine learning!
• Hands-on writing code of high quality, good design & architecture, fully tested and peer reviews.
• Collaborate with multiple product/engineering teams to understand their data and compute requirements (SW, HW, Automobile, AI) to integrate amazing innovations and algorithms into our production systems
• Automate everything for measuring, testing, updating, monitoring, and alerting the data platformWhat we need to see:
  • Bachelors (or equivalent experience) or Masters in Computer Architecture, Computer Science, or related data-intensive Engineering Degree.
  • 8+ years of proven experience in Data Engineering, worked on designing and developing software with Big Data, Data Lake/ Lake House ecosystem, Data Analytics, backend microservices architecture, and heterogeneous data types at scale
  • Proven in-depth experience in creating ETL pipelines using Databricks, Spark, Python, SQL, Scala, Kafka, Presto, Parquet, Streaming, events, bots, AWS/cloud ecosystem
  • Proficient in developing Micro Services and using AWS frameworks such as SQS, Stream, Kubernetes, EC2, S3, Lambda etc
  • Experience with data pipelines/analysis/visualization tooling such as Elastic stack, Logstash, Kibana, Kafka, Grafana, Splunk, Pandas, Message brokers, Data modeling.
  • Expertise in Data Lakehouse architecture and end-to-end Databricks techniques including Data Science components
  • Worked on end-to-end data lifecycle from Data Ingestion, Data Transformation, and Data Consumption layer. Versed with API and its usability
  • Knowledge of Cloud solutions like Kendra, SageMaker, Auto-ML, Big Query, RedShift, Glue, Athena.Ways to stand out from the crowd:
    • Understanding and experience around Cost and Usages analytics is plus
    • Expert in Spark, Parquet, steaming, events, Kafka, telemetry, MapReduce, Hadoop, Hive, Presto, Spark, data query approaches, and dashboarding.
    • The one who have implemented Enterprise use cases like CMDB, Governance, time series classification, telemetry anomaly detection, logs, and real-time data ingestion through APIs
    • Experience with structured data such as Avro, Parquet, Protobuf, Thrift, and concepts like schema evolution.
    • Working knowledge of Amazon Web Services, Kubernetes, Docker is a plus.NVIDIA is widely considered to be one of the technology world's most desirable employers. We have some of the most forward-thinking and hardworking people on the planet working for us. If you're creative and autonomous, we want to hear from you!The base salary range is $160,000 - $304,750. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits . NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law. About Us 0:00 -/ - 3:32 NVIDIA is a Learning Machine NVIDIA pioneered accelerated computing to tackle challenges no one else can solve. Our work in AI and the metaverse is transforming the world's largest industries and profoundly impacting society.

Keywords: NVIDIA Corporation, Santa Clara , Staff Data Engineer, Engineering , Santa Clara, California