Is there a way to create a minute-level partition in BigQuery? I ask because we are getting a high volume of iot data every minute from our nprobe server. We need to process the data at minute level.

Is this correct?

CREATE OR REPLACE TABLE your_dataset.your_table
PARTITION BY TIMESTAMP_TRUNC(event_timestamp, MINUTE) AS
SELECT
    event_id,
    event_timestamp,
    event_data
FROM
    your_source_table;

Is there a way to create a minute-level partition in BigQuery? I ask because we are getting a high volume of iot data every minute from our nprobe server. We need to process the data at minute level.

Is this correct?

CREATE OR REPLACE TABLE your_dataset.your_table
PARTITION BY TIMESTAMP_TRUNC(event_timestamp, MINUTE) AS
SELECT
    event_id,
    event_timestamp,
    event_data
FROM
    your_source_table;

• google-bigquery

Share Improve this question Follow edited Mar 13 at 1:33 President James K. Polk 42.1k2929 gold badges109109 silver badges145145 bronze badges asked Mar 11 at 13:45 Vishal ranaVishal rana 1977 bronze badges

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

BigQuery doesn't support minute partition. Also if you really partition by minutes, per the maximum 10K partitions per table limit, you can only keep 7 days of data in the table.