Data Engineering on Microsoft Azure (DP-203) Exam Blueprint

Implement a partition strategy for files; Implement a partition strategy for analytical workloads; Implement a partition strategy for streaming workloads; Implement a partition strategy for Azure Synapse Analytics; Identify when partitioning is needed in Azure Data Lake Storage Gen2

Create and execute queries by using a compute solution that leverages SQL serverless and Spark clusters; Recommend and implement Azure Synapse Analytics database templates; Push new or updated data lineage to Microsoft Purview; Browse and search metadata in Microsoft Purview Data Catalog

Design and implement incremental data loads; Transform data by using Apache Spark; Transform data by using Transact-SQL (T-SQL) in Azure Synapse Analytics; Ingest and transform data by using Azure Synapse Pipelines or Azure Data Factory; Transform data by using Azure Stream Analytics

Handle duplicate data; Avoiding duplicate data by using Azure Stream Analytics Exactly Once Delivery; Handle missing data; Handle late-arriving data; Split data; Shred JSON; Encode and decode data; Configure error handling for a transformation; Normalize and denormalize data

Develop batch processing solutions by using Azure Data Lake Storage Gen2, Azure Databricks, Azure Synapse Analytics, and Azure Data Factory; Use PolyBase to load data to a SQL pool; Implement Azure Synapse Link and query the replicated data

Scale resources; Configure the batch size; Create tests for data pipelines; Integrate Jupyter or Python notebooks into a data pipeline; Upsert batch data; Revert data to a previous state; Configure exception handling; Configure batch retention; Read from and write to a delta lake

Create a stream processing solution by using Stream Analytics and Azure Event Hubs; Process data by using Spark structured streaming; Create windowed aggregates; Handle schema drift; Process time series data; Process data across partitions; Process within one partition; Configure checkpoints and watermarking during processing; Scale resources; Create tests for data pipelines; Optimize pipelines for analytical or transactional purposes; Handle interruptions; Configure exception handling; Upsert stream data; Replay archived stream data; Read from and write to a delta lake

Trigger batches; Handle failed batch loads; Validate batch loads; Manage data pipelines in Azure Data Factory or Azure Synapse Pipelines; Schedule data pipelines in Data Factory or Azure Synapse Pipelines; Implement version control for pipeline artifacts; Manage Spark jobs in a pipeline

Implement data masking; Encrypt data at rest and in motion; Implement row-level and column-level security; Implement Azure role-based access control (RBAC); Implement POSIX-like access control lists (ACLs) for Data Lake Storage Gen2; Implement a data retention policy; Implement secure endpoints (private and public); Implement resource tokens in Azure Databricks; Load a DataFrame with sensitive information; Write encrypted data to tables or Parquet files; Manage sensitive information

Implement logging used by Azure Monitor; Configure monitoring services; Monitor stream processing; Measure performance of data movement; Monitor and update statistics about data across a system; Monitor data pipeline performance; Measure query performance; Schedule and monitor pipeline tests; Interpret Azure Monitor metrics and logs; Implement a pipeline alert strategy

Compact small files; Handle skew in data; Handle data spill; Optimize resource management; Tune queries by using indexers; Tune queries by using cache; Troubleshoot a failed Spark job; Troubleshoot a failed pipeline run, including activities executed in external services