| Type: | Package |
| Title: | An Interface to Google's 'BigQuery Storage' API |
| Version: | 1.3.0 |
| Maintainer: | Bruno Tremblay <openr@neoxone.com> |
| Description: | Easily talk to Google's 'BigQuery Storage' API from R (https://docs.cloud.google.com/bigquery/docs/reference/storage/rpc). |
| License: | Apache License (≥ 2) |
| Depends: | R (≥ 4.1) |
| Imports: | nanoarrow (≥ 0.6.0), rlang, assertthat, Rcpp, lifecycle, bit64, tibble |
| Suggests: | blob, bigrquery, testthat, hms, wk (≥ 0.3.2), tzdb, base64enc, data.table |
| LinkingTo: | Rcpp, nanoarrow |
| Encoding: | UTF-8 |
| URL: | https://github.com/meztez/bigrquerystorage |
| BugReports: | https://github.com/meztez/bigrquerystorage/issues |
| SystemRequirements: | grpc/protobuf headers and compilers. For example libprotobuf-dev libgrpc++-dev protobuf-compiler-grpc (Debian) or protobuf-devel grpc-devel grpc-plugins (Fedora/RHEL) |
| Biarch: | TRUE |
| Config/roxygen2/version: | 8.1.1 |
| NeedsCompilation: | yes |
| Packaged: | 2026-11-22 01:09:17 UTC; bruno |
| Author: | Bruno Tremblay [aut, cre], Google LLC [cph, fnd] |
| Repository: | CRAN |
| Date/Publication: | 2026-11-22 08:10:02 UTC |
bigrquerystorage: An Interface to Google's 'BigQuery Storage' API
Description
Easily talk to Google's 'BigQuery Storage' API from R (https://docs.cloud.google.com/bigquery/docs/reference/storage/rpc).
Author(s)
Maintainer: Bruno Tremblay openr@neoxone.com
Authors:
Bruno Tremblay openr@neoxone.com
Other contributors:
Google LLC [copyright holder, funder]
See Also
Useful links:
Report bugs at https://github.com/meztez/bigrquerystorage/issues
Initialize bigrquerystorage client
Description
Initialize bigrquerystorage client
Close bigrquerystorage client
Usage
bqs_auth()
bqs_deauth()
Details
Will attempt to reuse bigrquery credentials.
About Credentials
If your application runs inside a Google Cloud environment that has a default service account, your application can retrieve the service account credentials to call Google Cloud APIs. Such environments include Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions. We recommend using this strategy because it is more convenient and secure than manually passing credentials.
Additionally, we recommend you use Google Cloud Client Libraries for your application. Google Cloud Client Libraries use a library called Application Default Credentials (ADC) to automatically find your service account credentials. ADC looks for service account credentials in the following order:
If the environment variable GOOGLE_APPLICATION_CREDENTIALS is set, ADC uses the service account file that the variable points to.
If the environment variable GOOGLE_APPLICATION_CREDENTIALS isn't set, ADC uses the default service account that Compute Engine, Google Kubernetes Engine, App Engine, Cloud Run, and Cloud Functions provide.
If ADC can't use either of the above credentials, an error occurs.
Value
No return value, called for side effects.
Download table data
Description
This retrieves rows block in a stream using a grpc protocol. It is most suitable for results of larger queries (>100 MB, say).
Usage
bqs_table_download(
x,
parent = getOption("bigquerystorage.project", ""),
snapshot_time = NA,
selected_fields = character(),
row_restriction = "",
sample_percentage,
n_max = Inf,
quiet = NA,
as_tibble = lifecycle::deprecated(),
bigint = c("integer", "integer64", "numeric", "character"),
max_results = lifecycle::deprecated(),
type = c("tibble", "data.frame", "data.table", "arrow"),
max_stream_count = 0L,
compression = c("auto", "zstd", "lz4", "none")
)
Arguments
x |
Table reference |
parent |
Used as parent for |
snapshot_time |
Table modifier |
selected_fields |
Table read option |
row_restriction |
Table read option |
sample_percentage |
Table read option |
n_max |
Maximum number of results to retrieve. Use |
quiet |
Should information be printed to console. |
as_tibble |
Should data be returned as tibble. Deprecated; use |
bigint |
The R type that BigQuery's 64-bit integer types should be mapped to.
The default is |
max_results |
Deprecated |
type |
The output container type: |
max_stream_count |
Maximum number of streams to request from BigQuery Storage
( |
compression |
Arrow record batch compression codec: |
Details
More details about table modifiers and table options are available from the API Reference documentation. (See TableModifiers and TableReadOptions)
Multi-stream reads are downloaded in parallel, each worker on its own gRPC
connection, and the Arrow record batches of each stream are then decoded in
parallel. Up to 16 streams are read concurrently by default; set
options(bigrquerystorage.threads = n) to change the number of download
and decoding threads.
Value
By default (type = "tibble"), a tibble::tibble. Alternatively a
data.frame, data.table::data.table, or nanoarrow_array_stream according to type.
Upload table data
Description
This uploads rows to a BigQuery table using the BigQuery Storage Write API (gRPC protocol with Arrow RecordBatch serialization).
Usage
bqs_table_upload(
x,
values,
fields = NULL,
create_disposition = c("CREATE_IF_NEEDED", "CREATE_NEVER"),
write_disposition = c("WRITE_EMPTY", "WRITE_APPEND", "WRITE_TRUNCATE"),
stream_type = c("AUTO", "PENDING", "DEFAULT"),
row_batch_size = 100000L,
quiet = getOption("bigrquery.quiet", NA),
max_stream_count = 0L,
compression = c("auto", "zstd", "none"),
...
)
Arguments
x |
Table reference ( |
values |
Data to upload: a |
fields |
Optional schema specification ( |
create_disposition |
Behavior if the destination table does not exist:
|
write_disposition |
Behavior if the destination table already has data:
|
stream_type |
Write stream mode: |
row_batch_size |
Maximum number of rows per |
quiet |
Should progress information be suppressed? Default |
max_stream_count |
Maximum number of parallel write streams to open
( |
compression |
Arrow record batch buffer compression codec: |
... |
Additional arguments passed to |
Details
More details about the BigQuery Storage Write API are available in the API Reference documentation.
Upstream bigrquery can delegate bq_table_upload(..., api = "arrow") directly to
bqs_table_upload().
Value
Invisibly returns x.