About the Censys Universal Internet Dataset BigQuery Dataset

The Censys Universal Internet Dataset is the most comprehensive Internet-wide scan data in the industry. This dataset supersedes the legacy IPv4 and IPv4 Banners datasets.

Censys global scanning engine continuously walks the entire IPv4 space, detecting 107 protocols on over 3500 ports to produce visibility in a high-resolution map of the public Internet. This provides a significant tool for threat hunters, attack surface managers, and other security professionals.

A 'snapshot' of every service on the Internet is taken every single day and is added as a partition to a dataset available to Censys Enterprise customers to query this data using Google BigQuery.

Scan data in any snapshot is never more than 10 days old, and the data for a service is either structured, protocol-specific key-value pairs, or, if the service is using an unknown protocol, banner data and/or TLS information.

Want to know the port distribution of SMTP services using the compromised TLSv1_0 encryption method on Jun. 9, 2021? Try this query:

 SELECT
  tls.version_selected,
  service_name,
  port,
  COUNT(*) AS count
FROM
  `censys-io.universal_internet_dataset.20210609`,
  UNNEST(services)
WHERE
  tls.version_selected = 'TLSv1_0'
  AND service_name = 'SMTP'
GROUP BY
  1,
  2,
  3
ORDER BY
  4 DESC

Want to see the software (in CPE format) reported by Telnet services across all ports today? Try running this query:

 SELECT
  software.vendor,
  software.product,
  software.version,
  port,
  COUNT(*) AS count
FROM
  `censys-io.universal_internet_dataset.current`,
  UNNEST(services),
  UNNEST(software) AS software
WHERE
  service_name = 'TELNET'
GROUP BY
  1,2,3,4
ORDER BY
  5 DESC

Or see a count of all the Kubernetes services across the Internet on May 25, 2021 grouped by the port they’re running on:

 SELECT
  port,
  COUNT(*) AS count
FROM
  `censys-io.universal_internet_dataset.20210525`,
  UNNEST(services)
WHERE
  service_name = 'KUBERNETES'
GROUP BY
  1
ORDER BY
  2 DESC