Amazon QuickSight – Data Source Quotas

Amazon QuickSight is a cloud-based business intelligence and data visualization platform that allows users to connect to various data sources to create interactive dashboards and reports. When using QuickSight, it’s important to understand data source quotas, which are limits placed on the amount of data that can be imported and processed from each data source.

• The data source quotas in Amazon QuickSight vary depending on the type of data source. 
• The maximum size for an imported file is 250 MB, while the maximum size for a single dataset is 100 GB. These limits apply to both individual files and the combined size of all files within a dataset.
• There are also limits on the number of data sources that can be used.
• The number of queries that can be run per second.
• The number of concurrent queries that can be executed. 
• These limits are designed to ensure that the QuickSight service remains reliable and available for all users.

To help users manage data source quotas, Amazon QuickSight provides detailed monitoring and tracking tools, which allow users to monitor usage and set up alerts to notify them when they approach or exceed their limits. Users can also purchase additional capacity to increase their data source quotas, if necessary.

Understanding data source quotas is an important part of using Amazon QuickSight effectively, as it can help users avoid unexpected limitations and ensure that their data analysis and visualization projects proceed smoothly. So, the data sources that you use with Amazon QuickSight must conform to the following quotas.

• SPICE quotas for imported data
• Quotas for direct SQL queries

SPICE quotas for imported data

SPICE enforces limits on the number of rows that can be added to a dataset when creating a new dataset in Amazon QuickSight. Data can be ingested into SPICE either from a query or a file, with a maximum of 2,000 columns allowed per file. Additionally, column names are limited to 127 Unicode characters and individual fields can have a maximum of 2,047 Unicode characters.

To manage larger datasets, QuickSight offers features such as column deselection and filters to retrieve a subset of data. When importing data from Amazon S3, each manifest can specify up to 1,000 files.

SPICE quotas are as follows:

• 2,047 Unicode characters for each field
• 127 Unicode characters for each column name
• 2,000 columns for each file
• 1,000 files for each manifest
• For Standard edition, 25 million (25,000,000) rows or 25 GB for each dataset
• For Enterprise edition, 1 billion (1,000,000,000) rows or 1 TB for each dataset

The quotas for SPICE datasets, including row-level security, are applicable to all QuickSight accounts. However, in certain situations where large rows are ingested into SPICE, the gigabyte quota for datasets may be reached before the row quota. The dataset size is calculated based on the SPICE capacity that is utilized after the data is ingested into SPICE.

Quotas for direct SQL queries

Different quotas apply for space and time when not importing data into SPICE. Timeouts can occur during operations like connecting, sampling data or generating visuals, with some resulting from timeout quotas set by the source database engine. Amazon QuickSight generates a timeout after 2 minutes, but not all database drivers react to this limit, leading to long-running queries. To alleviate this, queries can be canceled on the database server to free up resources.

Direct queries have a limit of 2,000 columns per result set, with each column name capped at 127 Unicode characters. To reduce data size, users can deselect columns, apply filters, or use predicates like WHERE or HAVING in SQL queries. If timeouts occur during direct queries, users can optimize execution time by simplifying queries or importing data into SPICE.

Quotas for queries are as follows:

• 127 Unicode characters for each column name.
• 2,000 columns for each dataset.
• 2-minute quota for generating a visual, or an optional dataset sample.
• Data source timeout quotas apply (vary for each database engine).