Example · Data Engineer

70% of your week as a data engineer is still yours.

AI now writes much of the pipeline code. Owning production when it breaks stays with you.

Make my planA typical week at the middle pace. Your plan uses your own tasks.

Is data engineering safe from AI?

In a typical week at the middle pace, 40% of this work needs your judgment and 30% is work you do using AI. AI could do about 30%. Turning a clear spec into pipeline code, configuration and tests is about 30% of your week, and a frontier model with your repo open does that today. If you have had one write the pipeline and dbt models for a new source, you have seen it get most of the way.

Read more

The two parts of the job many data engineers like least, the 3 a.m. page and the regulated file that goes out the door, are the two most protected things you do. The spec-to-code work you may be fastest at is the work a model does best. A model cannot be the named person when the number is wrong at 3 a.m., own the file that goes to a regulator or partner, or know which of a source system's status codes lie.

That can put you in a better spot than most people early in software. Two outside reasons explain it. In regulated industries a named human is answerable for the data you ship, and many security teams will not let a model near production. The second reason may not last. The plan turns protection that comes from policy into protection that comes from your own work.

The week, task by task

Who does each hour of a typical 40-hour week.

See every task
You40% · 16 hours

Needs your judgment.

  • Novel incident diagnosis: "why is yesterday different," CDC gaps, silent duplicates, the 3 a.m. page10%
  • Regulated and partner outputs: furnishing file, partner reconciliation, finance close feeds8%
  • Schema-change negotiation with app teams; impact analysis; saying "not this sprint"6%
  • On-call ownership: named responder, incident comms to finance and partners, postmortems6%
  • + 2 smaller tasks9%
You using AI30% · 12 hours

AI does the slow parts. You still own it.

  • Known-failure triage: reruns, connector resets, backfills after a recognized failure8%
  • Warehouse modeling for a new domain: grain, keys, slowly changing dimensions, business-rule edge cases8%
  • Data-quality monitoring: writing checks, choosing thresholds, tuning noisy alerts6%
  • Code review of teammates' PRs5%
  • + 1 smaller task4%
AI30% · 12 hours

AI could do most of it.

  • New pipeline code from a clear spec: an ingestion DAG, dbt models for a new source or column16%
  • Boilerplate and config: dbt tests and YAML, docs, Terraform for new resources, schema migrations7%
  • Routine analyst support: ad-hoc queries, "where does X live," access requests7%
Why some tasks still need a person
Being the person who is paged, and who says what happened afterwards

A runbook can rerun a task. It cannot be the named responder finance escalates to, or write a postmortem a partner will accept. The rotation exists because someone must answer.

Regulated and partner outputs: a credit-bureau furnishing file, a partner-bank reconciliation, finance close feeds

In consumer lending, Reg V (12 CFR 1022.42), the federal rule on furnishing credit data, requires written procedures for the accuracy and integrity of what is furnished to the bureaus, and a partner bank's regulators hold it responsible for your feeds under the 2023 interagency third-party guidance. A wrong file harms consumers and leaves a paper trail, so someone has to sign for it.

Knowing which of the source database's status codes lie

A model reading the schema sees a clean enum. Knowing that CLOSED sometimes means refinanced, and that refinances double-count on the loan key, is in your head, and a wrong answer looks correct until the reconciliation breaks.

"What breaks if we drop this column?"

Lineage tools show edges. They do not show that the finance close, a regulated file and one analyst's forgotten notebook all depend on that column in different ways, or that the app team's PM (product manager) will accept a two-week delay if you ask on Tuesday.

Deciding who may see which columns

The accountability here comes from whoever owns the security program (in financial services, the Safeguards Rule's Qualified Individual). Access reviews and masking policies are a control someone attests to. A model can propose a policy, but it cannot attest to it.

Good to know

What is hard about this field, and what you can do about it.

The hard truth

The plumbing you build is becoming a product vendors sell.

Moving and modeling data used to be custom work. Vendors keep packaging it, and in October 2025 two of the biggest agreed to merge.

What you can doGet strong at what tools don't cover: messy source systems, data contracts, cost control and governance. Learn the managed tools well enough to advise when to buy.
Sources (1)
  • Fivetran and dbt Labs announced an all-stock merger, with about $600 million in combined annual recurring revenue and 10,000+ customers. Technical.ly, 2025-10

What's next for a data engineer?

Plan A is the best move inside the work you do now.

APlan A

Grow in the job you have

The main focus. Most weekly sessions build toward it.

Your next stepOwn reliability for the whole platform
Each week
4 sessions
Ready in
About 3 to 4 months
Why it fits

Why you. Things break on your watch, so you know what breaks, and usually nobody keeps a page log. If you want to stop being the on-call person forever, this is the way out.

The gap. You need to learn SLO design (written reliability targets), error budgets and postmortem practice. You also need to show finance that "the number by 8" is a service level with a cost. The free Google SRE (site reliability engineering) books cover the mechanics.

The trade-off. Your name goes on the outages, including the ones an agent caused. On a small team this is a half-role; it only works paired with the regulated-output reviewer move. And if the company hires a Head of Platform from outside, "reliability" can read as keeping the lights on.

2 other moves that fit
  • Be the reviewer of record for the regulated outputsReady in about 4 to 6 months
  • Own the platform for your core source system (the hedge)Ready in about 6 to 9 months

What's a plan B for data engineers?

Plan B is meant to be different from this job. It was picked because skills from this role carry over.

BPlan B

Get ready for a new kind of job

A second option. A newer job AI is creating that these skills already fit.

A job to aim forLLM evaluation data scientist

You already build the checks that catch bad data before anyone sees it, and scoring model outputs runs on the same pipelines and SQL.

Each week
1 session
Outlook
Hiring now
Pay
$150.4K to $385K a yearTwo postings: Apple (Oct 2026, base pay) and Perplexity (now closed).
2 other options
  • AI cost and vendor analystYour warehouse and compute bills are already yours to explain, so tracking model and token spend is a short step with some finance practice.
  • Context engineer and knowledge curatorKnowing which source tables are trustworthy and how they join is the core of curating what an AI agent is allowed to know.

In your own plan you can switch to any of the 47 roles.

The 13 weeks

Small steps that add up. One plan for both: most sessions build Plan A, one a week builds Plan B.

By week 13 you will have

AFor Plan A

  • A six-month page log with draft SLOs
  • A written control document for your regulated file
  • A one-page memo asking to own platform reliability

BAnd for Plan B

  • A regression table with a significance check
  • A published eval write-up for your portfolio
Month 1Weeks 1 to 4
  1. 1Measure your week
  2. 2Test an AI tool safely
  3. 3Decide what should page
  4. 4Your on-call week

You finish withA one-page list of controls that already need a human sign-off

Month 2Weeks 5 to 8
  1. 5Map the rules on your file
  2. 6Add checks with reasons
  3. 7Propose a safe AI setup
  4. 8Build a gate and decide

You finish withA written decision on which move to focus on

Month 3Weeks 9 to 13
  1. 9Draft a data contract
  2. 10Your second on-call week
  3. 11Add up your results
  4. 12Make your ask
  5. 13Choose your next step

You finish withA one-paragraph decision with a date

Three courses from the plan
  • Site Reliability Engineering; The Site Reliability WorkbookGoogle, free online at sre.google/books · Free

    Yes, Weeks 2, 3 and 6. It is the highest-value reading in this report, and it is free.

  • FTC Safeguards Rule: What Your Business Needs to KnowFederal Trade Commission · Free

    Yes, Week 4, if you work in financial services.

  • 12 CFR 1022.42 and Appendix E (Reg V furnisher accuracy and integrity)eCFR · Free

    Yes, Week 5, if your company furnishes to the credit bureaus.

Checked 2026-09-22.

The fuller picture

Risks beyond AI: moderatePolicy and cycle risk

Two risks here have nothing to do with what models can do. First, part of your protection is your employer's security policy (no model on production data, no agent with credentials), and policy changes in one meeting. Second, headcount in sectors like lending tracks interest rates and funding. An unfilled junior seat is as likely to come from that cycle as from the AI vision doc.

Where this could be wrong
  • The shares are guesses until Week 1. If your log shows much more spec-to-code work, the share of your week tools can do passes 40% and junior openings shrink faster.
  • A good part of the work that holds up is protected by policy, and policy is a decision. If security approves an in-boundary model next quarter, re-run this report; diagnosis and triage move first.
  • We assumed a regulated employer with an output someone must sign for. If your company has no regulated file, the reviewer-of-record move does not exist and owning the source platform is the plan. If your company is public, SOX IT controls, from the federal financial reporting law, add a second anchor and this report is too pessimistic.
  • Our estimate of how well AI diagnoses new kinds of failures is uncertain in both directions. With log and lineage access, agents diagnose most known-pattern failures today; on a change-data-capture (CDC) gap in a database whose status codes lie, the evidence is thin.

This is an example. Your plan uses your own week.

Make my plan

Also matched: data engineer, analytics engineer, etl developer. Research checked 2026-09-22. Page updated .