LivoniaRecruiter Since 2001
the smart solution for Livonia jobs

Data Engineer

Company: Reliable Software Resources
Location: Livonia
Posted on: February 21, 2021

Job Description:

Job Title: Data engineerLocation: US-MI-Livonia (begin with remote) Duration: 6+Months Job Overview:One of our large clients is expanding their current data footprint on the cloud to provide analytics, BI and data APIs. Majority of data will be batch processed with data validation, data quality and transformation into a multitude of data platforms such as Redshift, Postgres and Hive. A Senior Technical Consultant is expected to be knowledgeable in two or more technologies within (a given Solutions/Practice area). The Senior Technical Consultant is expected to have strong development and programming skills in Spark with a focus on Scala/Java and other ETL development experience in the big data space. You are expected to be experienced and fluent in agile development and agile tools as well as code repositories and agile SLDC/DevOps frameworks. You will work with architects and infrastructure teams to develop, test, deploy and troubleshoot your code as well as provide input into solutions and design of the system. You will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions focused on our clients' business needs. Responsibilities:

  • Work with data engineering team to define and develop data ingestion, validation, transformation and data engineering code.
  • Develop open-source platform components using Spark, Scala, Java, Oozie, Hive and other components
  • Document code artifacts and participate in developing user documentation and run books.
  • Troubleshoot deployment to various environments and provide test support.
  • Participate in design sessions, demos and prototype sessions, testing and training workshops with business users and other IT associates. Qualifications:Data Engineers are required to design and build data products and data pipelines. They will ensure the robust flow of data from acquisition through curation, and governance. Data Engineers will enable data as a service and drive all critical data driven initiatives.
    • Experts of the data and its application by users
    • Understands data landscape and environments: sources, elements, update freq, completeness, stewards/contacts, platforms.
    • Manages ETL: use programming and tools for data ingestion, configure pipelines, apply transformations and decoding, integrate and fuse data, move and securely deliver.
    • Translates business requirements to build repeatable, sustainable, efficient, coded processes that can be productionized by Software Engineers and readily modified by other Data Engineers.
    • Creates POC processes in Dev/QA and works with Software Engineers to productionize and align best coding and process practices that maximize efficiency, speed, stability, system resources and capabilities.
    • Leverages frameworks in place with big data tools: Hadoop, Spark, Python, Kafka, etc.
    • Experience with relational SQL and NoSQL databases
    • Awareness of and compliance with: data privacy, security, legal and contractual guidelines
    • Incorporates data quality and privacy checks/alerts to minimize bad data being consumed by end users, models and dashboards, and to protect customer data; revises checks for new data issues
    • Maintains feedback loop with Data Stewards on data issues, standards, fit for use (Data Stewardship is a subset of data engineering which would include responsibilities like data curation)
    • Validates data products and pipelines are functioning as expected following system or application upgrades, source changes, etc.
    • Responsible for data architecture including sources, table structures, physical models
    • Works closely with Architects to align systems, tools and applications being utilized with business use case and performance requirements
    • Communicates with end users to set expectations and ensure alignment around data accuracy, completeness, timeliness and consistency
    • Provides data product support and maintenance
    • Establishes, tracks and monitors KPIs related to specific data products and deliverables Preferred Skills and Education:
      • Bachelors's degree in Computer Science or related field
      • Certification in Spark, Azure or other cloud platform Educational Qualifications:
        • Required - Bachelor's degree in Computer Science, Information Technology, Computer Engineering or closely related or equivalent
        • Preferred - Master's degree in Management Information Systems (MIS), Computer Science, Big Data or Analytics or equivalent Travel:
          • Open to travel based up on the nature of the engagement Thanks & RegardsSrikanth Donkani Sr. Talent Acquisition Specialist(w): (E):2260 Haggerty Road, Suite 285 Northville, MI 48167Equal Employment OpportunityReliable Software employment does not discriminate on the basis of race, religion, gender, sexual orientation, age or any other basis as covered by federal, state, or local law.Employment decisions are based solely on qualifications, merit and business needs. - provided by Dice

Keywords: Reliable Software Resources, Livonia , Data Engineer, Engineering , Livonia, Michigan

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category
within


Other Engineering Jobs


Application Engineer
Description: Description br br Job ID: 48370 br Application Engineer br Do you have a strong desire to assist in the launch of new automotive prototypes and vehicle launches Are you keen to support the development (more...)
Company: VanderHouwen
Location: Detroit
Posted on: 02/26/2021

Team Leader, Network Engineering
Description: Minimum Qualifications ul li Bachelor s degree in computer science or a related field or equivalent experiencePreferred Qualifications ul li 3 years working in a systems and infrastructure role (more...)
Company: Quicken Loans, LLC
Location: Detroit
Posted on: 02/26/2021

Powertrain Design & Release Engineer
Description: Job Title: Powertrain Design Release EngineerDescription: STG is a SEI CMMi Level 5 company with several Fortune 500 and State Government clients. STG has opening for Powertrain Design Release Engineer. (more...)
Company: Systems Technology Group Inc. (STG)
Location: Auburn Hills
Posted on: 02/26/2021


Controls Engineer
Description: Pursuant to Section 20 C.F.R. 656.10 d ii , this notice is being provided as a result of the filing of an Application for Permanent Employment Certification for the following position: Controls Engineer (more...)
Company: KUKA North America
Location: Sterling Heights
Posted on: 02/26/2021

Firmware Engineer
Description: About the Company:We design, produce and support a complete line of wireless tracking and sensing devices andapplications. We have product deployed in over 160 countries supporting hundreds of globalCustomers. (more...)
Company: ProArch Consulting Services
Location: Detroit
Posted on: 02/26/2021

Controls Design / Robot Programming Engineers
Description: A diversified engineering machine tool builder in Welding Equipment, Special Machines, and Assembly Equipment located in Macomb Oakland County has immediate openings for Controls Design
Company: Utica Enterprises
Location: Sterling Heights
Posted on: 02/26/2021

Project Executive (Structural Engineering)
Description: Putting People at the Center of Innovation br With a history that dates back over 100 years, IMEG Corp. grew from several firms coming together under one uniting vision: people-centered engineering. (more...)
Company: IMEG Corp
Location: Detroit
Posted on: 02/26/2021

Diesel Mechanic
Description: First Vehicle Services, a leading fleet maintenance provider in North America, is now hiring a Fleet Technician II intermediate skill level br br The Fleet Technician II performs routine and some (more...)
Company: First Vehicle Services
Location: Saint Clair
Posted on: 02/26/2021

Emissions Technician
Description: Description:Specific Responsibilities:Stage vehicles to and from test sites Set up of vehicles on dynamometers such as: Chocking wheels in place, Connecting vehicle exhaust to collection systems, connecting (more...)
Company: Aerotek
Location: Wixom
Posted on: 02/26/2021

Mechanic
Description: City of Allen Park Mechanic The City of Allen Park is accepting applications for a full-time position of Mechanic. A qualified candidate will work in active garage servicing nearly 100
Company: city of allen park
Location: Allen Park
Posted on: 02/26/2021

Log In or Create An Account

Get the latest Michigan jobs by following @recnetMI on Twitter!

Livonia RSS job feeds