Senior Data Engineer - Enterprise Data

Company: News-Corp
Location: Santa Clara
Posted on: August 4, 2022

Job Description:

At realtor.com, we process terabytes of data everyday and transform that data into information that powers decisions for millions of home buyers, renters, dreamers, and real estate professionals. We aim to radically simplify home buying/selling and help more people achieve the American dream on our realtor.com website and mobile apps.
What will you do?
We have lots of data, in motion and at rest. Our goal is to use this data to make the home buying experience a breeze for our consumers. We empower them with the most up to date information on properties, and we help match our consumers with the most suitable realtors who are equipped to meet their individual needs.
The Enterprise Data Team works on the coolest data projects and products that reports, predicts and affects the outcome - using data. This impactful role will promote the build-out and usage of the Customer data platform, data-driven decisions and data products for realtor.com , it's customers and partners. Additionally, this role will be part of the core team contributing to the design and development of realtor.com 's Customer Data platform.
Responsibilities:
Design, develop, and operate highly scalable, high-performance and low-cost data pipelines in distributed data processing platforms with AWS/cloud technologies..
Collaborate with Engineers, Analysts and Data Scientists in the organization to construct complex data sources for algorithms and machine learning models
Own and Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
Creation and support of Batch and real-time data pipelines built on AWS technologies including Glue, Kinesis, EMR, Lambda and Athena.
Develop tools, plug-ins and solutions for CI/CD workflow using tools such as Jenkins, Cloud Formation, Docker, Github and other microservices.
Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency.
Drive and participate in architecture discussions, influence product roadmap, and take ownership and responsibility over new projects.
Maintain and support existing platforms and evolve to newer technology stacks and architectures.
Collaborate with the business applications team, product owners, Business stakeholders, Scrum-master to refine and estimate stories/epics.
Be an integral part of the scrum team to deliver on commitments on time and with good quality.
Review and ensure all code documentation is complete and updated periodically. Review work of Junior associates in the team.
Analyze the business Problems and deliver Optimal solutions using AWS Technologies.
Basic Qualifications
BS in Computer Science or equivalent degree; MS in Computer Science preferred.
5+ years of industry experience in software development, data engineering, business intelligence, or related fields with a track record of manipulating, processing, and extracting value from large datasets.
Demonstrated strength in data modeling, data transformation, and data warehousing.
Ability to write complex SQL statements.
Proficiency in programming language - Python.
Experience working with AWS big data technologies (S3, Athena, EMR, Glue, Lambda, Kinesis, API gateway, RDS, DynamoDB, Step Function etc).
E xperience in architecting, implementing, and supporting Snowflake infrastructure and topologies is a plus.
Experience in Snowpipe for handling Streaming data is a plus.
Experience with software configuration management tools(Git) and CI/CD pipelines and their enabling tools such as Jenkins, Nexus, etc.
Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations.
Strong analytical and problem-solving skills. Takes pride in efficient designs and accurate results
Objectively analyzes the pros, cons, and tradeoffs of a design path and helps the team to arrive at the most optimal solution, which may not be their own.
Ability to work with Product and Sales Ops to gather requirements and translate them into data engineering tasks.
Effective team player. Honest and respectful of others.
Additional Preferred Skills:
Experience in Web application development
Experience using business intelligence reporting tools (Tableau, Quicksight etc.).
Experience using big data technologies (Hadoop, Hive, Hbase, Spark etc.)
Familiar with agile development and sprints.
About realtor.com
At realtor.com -, we believe that everyone deserves a home of their own. We're a community of nearly 2,000 employees who work hard to ensure that from the moment someone starts dreaming about a new home, to the moment they walk in the door and beyond, we're there to lend a helping hand. Every month, over 85 million people trust us with their journey home by visiting our site and mobile apps, and we'd love to have you join our team to help.
We've got great offices in the U.S. and Canada with lots of sweet jobs to choose from, so we're hoping you'll join us on our journey to make buying, selling, renting, and living in homes easier and more rewarding for everyone.
Let's make a difference, together. For Real.
Equal Opportunity Employer
All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets

Keywords: News-Corp, Santa Clara , Senior Data Engineer - Enterprise Data, Engineering , Santa Clara, California