Professional-Data-Engineer Exam Dumps | You have a data analyst team member who needs to analyze data by using BigQuery. The data analyst wants

<< Prev Question Next Question >>

Question 158/186

You have a data analyst team member who needs to analyze data by using BigQuery. The data analyst wants to create a data pipeline that would load 200 CSV files with an average size of 15MB from a Cloud Storage bucket into BigQuery daily. The data needs to be ingested and transformed before being accessed in BigQuery for analysis. You need to recommend a fully managed, no-code solution for the data analyst. What should you do?

A. Create a Cloud Run function and schedule it to run daily using Cloud Scheduler to load the data into BigQuery.

B. Use the BigQuery Data Transfer Service to load files from Cloud Storage to BigQuery, create a BigQuery job which transforms the data using BigQuery SQL and schedule it to run daily.

C. Build a custom Apache Beam pipeline and run it on Dataflow to load the file from Cloud Storage to BigQuery and schedule it to run daily using Cloud Composer.

D. Create a pipeline by using BigQuery pipelines and schedule it to load the data into BigQuery daily.

Correct Answer: B

The requirements are for a daily scheduled load, ingest, and transformation, and specifically a fully managed, no-code solution.
* Ingest (Load): The BigQuery Data Transfer Service (DTS) is the fully managed, serverless, and no- code solution for batch loading files (including CSV from Cloud Storage) into BigQuery on a schedule. This is the "ingest" part.
* Transform: After loading the raw data into a staging table using DTS, the transformation can be done using BigQuery SQL. This transformation query can then be automated using a Scheduled Query in BigQuery, which is also a fully managed and no-code feature that runs on a schedule.
* Fully Managed & No-Code: Both DTS for Cloud Storage and Scheduled Queries are native BigQuery features that are fully managed and configured through the console without requiring code, directly meeting the constraints.
* Correcting other options:
* A (Cloud Run + Script): Cloud Run requires writing a custom Python script, which violates the no-code requirement.
* C (Dataflow + Apache Beam + Cloud Composer): This is a powerful, highly scalable ETL solution, but it requires writing custom code (Apache Beam) and requires setting up and managing a workflow orchestrator (Cloud Composer/Airflow), which violates both the fully managed (Dataflow is serverless, but the code/pipeline itself is custom and needs maintenance) and no-code requirements.
* D (BigQuery pipelines): "BigQuery pipelines" is not a distinct, official product name in the Google Cloud documentation that fulfills a no-code scheduled ETL. The closest product is the combination of DTS and Scheduled Queries, as described in option B.
Reference: Google Cloud Documentation on BigQuery Data Transfer Service and Scheduled Queries:
"The BigQuery Data Transfer Service automates data movement into BigQuery on a scheduled, managed basis... The BigQuery Data Transfer Service supports loading data from Cloud Storage in one of the following formats: Comma-separated values (CSV)..." (Source: What is BigQuery Data Transfer Service? and Introduction to Cloud Storage transfers)
"A scheduled query is a query that BigQuery automatically runs at regular intervals. When you configure a scheduled query, you specify the GoogleSQL SELECT statement to run, the destination table for the query results, and the frequency of the query." (Source: Scheduling queries) This combination delivers a fully managed, no-code ELT (Extract-Load-Transform) pipeline.

Question 158/186

LEAVE A REPLY

Download PDF File