Before you can query Censys data through Google BigQuery, you'll need to add the Censys Dataset to your BigQuery project. This article explains how to add the dataset once you have created a Google Cloud account.
Note: BigQuery access is only available to users of an enterprise data contract and verified non-commercial researchers. If you have any questions about access, please contact email@example.com.
Currently, the BigQuery Console has a new UI in beta. Through our testing, we've discovered that the ability to display the censys-io project does not exist. That said, we recommend using the Classic UI for the initial setup. Once completed, you can switch to the new UI if you wish and all of your changes will be reflected as expected.
Displaying Censys Project
Log in to the Google BigQuery web interface. If your account is set up correctly, you should see your primary project and the standard query interface:
Expand the options associated with your primary project and select Display Project:
Enter censys-io as the Project ID:
Verify that the project is shown in your navigation panel:
Standard SQL Only
Before writing your first query, note that you'll need to use Standard SQL to query the Censys datasets. Unfortunately, the Classic console UI of Google BigQuery defaults to Legacy SQL and the option to change your query to Standard SQL is hidden. Once you are in the Compose Query screen click Show Options and uncheck Use Legacy SQL.
Another option is to include the following command at the top of your SQL statement in the web interface as shown below:
<SQL GOES HERE>
For more information on constructing SQL statements, check out the BigQuery SQL Reference.
Note: If you choose to utilize the new console UI after adding the censys-io project standard SQL is set by default. Thus you do not need to specify the language or modify any settings.
Execute Test Query
Once the Censys project has been added, you can query the Censys datasets just like you would query any of our own data. Click Compose Query to get started. For example, you can check for the certificates that are valid for censys.io:
SELECT fingerprint_sha256, parsed.names, metadata.updated_at
UNNEST(parsed.names) AS names
WHERE names = 'censys.io'
Note: Censys datasets can only be queried using Standard SQL. We do not support Legacy SQL.
Now that you've added the datasets, to your account, here's some more information on how to write queries.