• What is dbt_artifacts?

    dbt_artifacts is a package for modeling a dbt project and its run metadata. It includes the following models to help you understand the current state of a dbt project and its performance over time. It...

  • Data migration from the Salesforce to snowflake Data warehouse by using Matillion

    Purpose of the Article: This blog explains how to load the data from Salesforce to Snowflake Intended Audience: This article will help our Organizational level kind of developers working on data migra...

  • WiMAX-A Study of Mobility

    1. Introduction The wireless market is growing rapidly; being pushed through wireless connectivity demand various wireless connectivity means are emerging (WLAN/802.11, WMAN/802.16a, WWAN/802.16d) [IE...

  • Weblogic Upgrade

    Patch Name wls1036_upgrade_generic.jar Reference https://www.youtube.com/watch?v=Be6hrYTANQE Just Click Next and Next MAKE SURE ALL OBIEE SERVICES ARE DOWN AND CLOSED IN PROCESS Open CMD prompt C:Mid...

Tags

bics installation OBIEE obiee 11g installation guide Oracle

Categories

Senior Data Engineer

Senior Data Engineer

Senior Data Engineer

Job LocationsUS-MA-Somerville | US-MA-Boston
Job ID 2023-1397
# of Openings 1
Category Analytics & Data

Overview

Senior Data Engineers play a crucial role in helping our client organizations manage and leverage their data effectively. Their responsibilities include: Data Architecture Design; Data Warehousing; Data Quality Assurance; Scalability and Performance Optimization; and Data Security. As senior-level individual contributors, SDEs are also responsible for recommending and implementing data engineering tools and technologies that best suit the client’s needs.

 

Because our clients are mostly US-based organizations, we look for the ability to communicate with professional proficiency in English, verbally and in writing.

Responsibilities

Data Engineering Expertise: You are responsible for building the infrastructure to support the storing and movement of data, so that it can be prepared by analytics engineers to eventually be interpreted by analysts. Your job is informing, developing, and implementing data accessibility solutions, enabling our clients to utilize data for performance evaluation and optimization. As a senior member of the Data Engineering function, you serve as a mentor to other engineers, both individually and in group settings.

  • Consistently seeks out and delivers on engagement level vision, tasks and problems
  • Actively assists in scoping and executing most impactful work for the team
  • Regularly delivers large features and product improvements that have a meaningful impact on clients’ data infrastructure and capabilities
  • Autonomous in approach and may direct or coach other less experienced Engineers
  • Actively mentors other Engineers in the team on individual basis or in group settings
  • Helps others grow through technical guidance, code reviews, documentation, etc.

Cross-Team Collaboration: You are responsible for collaborating with peers and other functional departments to develop and implement data engineering strategies and approaches that support engagement goals and understanding client needs.

  • Promotes a positive culture within and across different teams, collaborating with  analytics engineers and data analysts on end-to-end client requirements
  • Collaborate with clients and functional managers to plan for data engineering needs for a product or feature launch
  • Pair with a teammate or with someone at a client on strategies for solving a data engineering problem
  • Create a process or reporting template that helps cross-functional teams solve for common data engineering problems
  • Regularly engages with other teams to make the organization more effective
  • Take initiative to identify and solve important problems. Coordinates with others on cross-cutting technical issues
  • Drives data solutions improvements that impacts the client experience or empowers internal stakeholders (teams like Operations, Growth / Partnerships, Finance, etc.) to do their job effectively

 

Project Delivery: You are responsible for ensuring that large and/or more complex data engineering projects are delivered on time, within scope, and within budget.

  • Architects and designs services/systems using design patterns that allow for iterative delivery and future scaling
  • Proactively identifies and tackles technical debt becoming too big through planning work and aligning the team. Does this with careful evaluation of additional cost on development
  • Optimizes for the predictability and regular cadence of deliverables
  • Keeps reliability, maintainability and scalability of our clients’ systems top of mind
  • Embraces long-term ownership of projects while training others to reduce the bus factor or becoming a blocker
  • Prioritizes and values undesirable/unowned work that enables the team to move faster

Skills & Qualifications

Tools & Technologies

    • Programming languages (e.g. SQL, Python)
    • Data Processing (e.g. Apache Spark, dbt)
    • Cloud-based data warehouses (e.g., Snowflake, Google BigQuery)
    • Data orchestration (e.g., Apache Airflow, Azure Data Factory, Prefect)

Technical Skills (Hard Skills)

See the latest Data Engineering framework.

    • Data Movement: You can reduce latency of end-to-end pipelines through data orchestration in addition to incrementalization or streaming. You have strong knowledge of common data integration patterns (CDC, ELT, etc.).
    • Data Warehousing: You have a high proficiency in warehousing, including working knowledge of common ingestion SaaS platforms (e.g., Fivetran) and / or frameworks (e.g., Meltano, Airbyte), an ability to configure warehouse ingestion tools (e.g., Snowpipe) and can provision, maintain and optimize at least one cloud data warehouse (e.g., Snowflake).
    • Programming: You are considered a highly proficient programmer, approaching your code holistically, achieving a high standard routinely. You can optimize performance for large workloads and are able to troubleshoot complex queries / functions. Proficiency in Python  required.
    • Domain Expertise: You have a strong foundation of knowledge in domains in which you’re working. You are able to relate how the business works with the goals of the immediate team.
    • Technical Management: Is able to display a clear technical confidence and understanding. For the most part, can use organizational- and team-specific tools independently.

 

Bonus points for:

  • Data Modeling & Transformation: You have high proficiency with data transformation tools such as dbt and expert proficiency in data modeling approaches and philosophies (Kimball, OBT).
  • Data Orchestration. You’re familiar with at least one data orchestration platform (Azure Data Factory, Apache Airflow, Prefect, etc.).
  • Data Infrastructure: You understand more complex infrastructure approaches, including the implications and suitability of different deployment options and how to deploy self-hosted applications for clients with high security requirements.

Essential Skills (Intangible Skills)

    • Curiosity & Versatility: You help your immediate peers to make decisions based on what projects need, not what they feel most comfortable doing. You have taken the initiative to seek out new ways to apply existing skills and knowledge.
    • Collaboration & Partnership: You can facilitate collaborative group activities and/or workshops with colleagues or external stakeholders.  You are considered a role model for collaboration and creating alignment across teams because of your consistency and predictability.
    • Effective Communication: You reliably foster a culture of clear, concise, effective, audience-oriented communication for your team, other departments, and external stakeholders, ensuring those around you are actively listening as well as are understood.
    • Developing Others: You understand your team’s domain, share knowledge frequently with your teammates and contribute to the team’s documentation. You proactively watch for opportunities to share knowledge and encourage others to do the same.
    • Culture & Togetherness: You’ve openly stated your expectations of how your team works and acts, then demonstrated those expectations yourself. You help to coordinate and activate efforts towards a fairer, more diverse and safer workplace, using your position of influence to get things done.

Qualifications (Must Haves)

  • Proven experience as a Data Engineer or related role, with a focus on designing and developing data pipelines.
  • Strong programming skills in Python and SQL. Experience with Scala and Rust is a plus but not required.
  • Deep knowledge of data warehousing and ETL/ELT processes.
  • Intermediate / expert proficiency with common data integration / orchestration platforms (e.g., Fivetran, Azure Data Factory, Apache Airflow)
  • Hands-on experience with data warehouses like Snowflake, BigQuery, Databricks, or similar.
  • Experience with streaming solutions such as Spark Streaming, Kafka, or Flink is desirable but not required.
  • Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
  • Familiarity with machine learning operations (MLOps) techniques and platforms is a plus but not required.
  • Experience mentoring and advising other engineers
  • Strong analytical and problem-solving skills.
  • Excellent communication and collaboration skills.

 

Physical Requirements

  • Frequent sitting at a desk performing work on a computer
  • Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions

Core Company Values

 

Miaz-Tech is an established mid-sized agency with a top-tier portfolio of clients, ranging from the world’s largest non-profits to Fortune 500 brands. We pride ourselves on our people-first culture and a low-ego workplace that embraces experimentation, collaboration and continuous improvement. We have a fun office environment located in Davis Square (Somerville, MA) and offer competitive pay and excellent benefits.

  • Take the Long View – Ensure the company is built to last
  • Be Courageous – Make the right decisions even when they aren’t the easiest decisions
  • Be Genuine – Bring honesty and authenticity to all that you do
  • Work with Focus + Passion – Display purpose and pride in your work and never stop learning

 

As an equal opportunity employer, we are firmly committing to diversity, equity, and inclusion in our hiring efforts. We recognize that we need team members from all backgrounds and experiences to successfully shape a positive employee experience as well as deliver our product and service solutions. To that end, we actively seek candidates who can bring diverse experiences and backgrounds to our team. We know that complex factors and systemic bias can get in the way of us meeting strong candidates, so please don’t hesitate to apply even if you’re not 100% sure.

 

At this time, Miaz-Tech does not sponsor candidates and unfortunately cannot accept those on OPT or CPT.

 

#LI-Remote

Date

October 10, 2023

Category

DA