Censys Search 2.0 Release & Timeline FAQ 

What is Censys Search 2.0?

Censys Search 2.0 is a major update to Censys Search released in April 2021, and is available to everyone at search.censys.io. Search 2.0 introduces a new, port-independent data model, a new search language, historical data access, and a 180x increase in port coverage. The changes affect both the Censys Search UI and REST API. Search 2.0 replaces the original search engine that was available at censys.io. 

What is changing?

There are several significant changes in Censys Search 2.0:

• Backend Scan Process. Search 2.0 is powered by a new, continuous scan engine that scans around 2,500 ports, automatically detects the protocols on each port, scans from multiple geographic locations, and prioritizes network ranges that change most often. 

• Simplified Data Model. Search 2.0 uses a new data model for IPv4 hosts that accounts for the fact that any service could appear on any port. The data you receive back when viewing a host will be in a different format from the legacy Search product.

• New Censys Search Language. Search 2.0 introduces a new search DSL that allows more flexibility in querying across ports and protocols—particularly across multiple ports and protocols; by and large, queries written for the legacy Search product will need to be updated.

• Location. Search 2.0 is accessible at search.censys.io rather than censys.io. The REST API has also moved.

When will the change happen?

The data that powers Search 2.0 is already available in our Attack Surface Management platform and was made available to enterprise data customers in January 2021 as part of the Universal Internet Dataset. 

What are the plans for Search 1.0?

We will continue to post new enterprise snapshots in the legacy format until 11/30/2021, at which time we will continue to provide historical downloadable files for the IPv4 and IPv4 Banner datasets, but new files will not be published. Search 1.0 Web UI and API will be available until 11/30/2021 as well, after which they will be deprecated. 

How has the schema changed?

We are introducing a new data model for representing hosts on the Internet. Our new host data model provides cohesive records about individual IPv4 hosts, but decouples ports and protocol data. In Search 1.0, port numbers were statically defined as keys in the schema.

Now, we present top-level information about the host—IP address, location, routing information—and then an array of statically defined services. We have also removed the subprotocol abstraction layer and instead combine all information about a specific protocol in a single record. 

Software and operating system (OS) identification in the new dataset follows the Common Platform Enumeration (CPE) specification, so the same software may appear differently in the new dataset from how it appeared in the legacy datasets.

Read more information about the new data model.

Has the Search Language changed?

Censys Search 2.0 has a new search language with a syntax that reflects the port-independent, software schema. Read more about the 2.0 Search language.

Has the Certificate Dataset changed?

No changes have been made to the data model and search language of the certificate dataset.  This means your certificate queries will continue to work. The only caveat is the URL for the certificate search is changing from censys.io/certificates to search.censys.io/certificates.  We do expect that you will see more host-certificate relationships now because of the increased host coverage. 

Has the Search API changed?

There are minor changes to the Search API under the /api/v2 namespace. API paths are now focused on resources rather than actions, to make v2 API endpoints feel more REST-ful.

Read more about the new API paths, requests, and responses.

Are there any changes for free users? 

We are making our new scan data available to everyone, including free community users. Free users will have access to the same functionality as today with one exception — RegEx queries will only be available to paid users going forward. Regular expressions are seldom used, and poorly constructed regexes can take down our search service, so we’re reserving that functionality for paid customers. Access to historical data is determined by pricing tier. Free users will have access to the past week of history while Pro users will have access to the last 30 days, and Enterprise users, the past year. 

What’s the difference between Search 2.0 and the Universal Internet Dataset?

Censys Search 2.0 is powered by the Universal Internet Dataset. In addition to the new backend scan process and simplified data model that the Universal Internet Dataset offers, Censys Search 2.0 gives users the ability to interact with that dataset using the new search language in the UI and via the API. Censys Enterprise customers who download snapshots of the Universal Internet Dataset have the added benefit of Censys Search 2.0 mirroring the data presented in those snapshots. 

What’s missing from Search 2.0 that was in the old?

We’ve stopped scanning for several TLS vulnerabilities that have since been patched nearly everywhere. We are removing data related to the Heartbleed, FREAK, and Logjam attacks. 

We’re also planning to discontinue our scans of the Alexa Top Million websites. We recommend that users consider Crawler.Ninja as an alternative dataset.

How can I access Censys Search 2.0?

The web UI for the Search 2.0 website is located at search.censys.io. Search 2.0 API documentation is available here.

For Enterprise customers, the Universal Internet Dataset is available in Google BigQuery and can also be downloaded via API. For more information about Enterprise changes, refer to the UIDS FAQ.

Additional Information

• The Search 2.0 data model
• The Search 2.0 search language
• Internet Scanning Overview