1. Censys
  2. Censys Search
  3. Upgrading to Censys Search 2.0

Other Articles In This Section

In this article:

Translate Queries to Search 2.0

  • Updated

Censys Search 2.0 host data better reflects the real state of the Internet, but how do you update your old queries?

update.png

Browse these "before and after" examples that showcase differences in searches on the Censys legacy host data set and the Search 2.0 host data set. Observe the differences in data fields, query syntax, and number of results, and then apply the changes to your favorite queries!

Service Searches

Services on hosts are identified by a service name and are reachable on a port. You will recognize most names as traditional L7 protocols, although not all services are (.e.g., ELASTICSEARCH).

Search for Hosts Running a Service

These queries return hosts running an FTP service.

Search Product

Search Query

No. of Results

1.0

protocols: "22/ssh"

18,058,155

2.0

services.service_name: SSH

23,316,370

Service data is irrespective of port number in Search 2.0: notice how the query makes no reference to port 21 as the legacy search did. The greater number of results also indicates that the additional FTP services returned in 2.0 are running on ports other than the standard (i.e., 21).

Search for a Service With an Attribute

These queries return hosts with a web page title containing the word "Dashboard."

(Words such as "Dash" and "Dashboards" are not returned. Whitespace-delimited phrases such as "Dashboard Panel" are returned.)

Search Product

Search Query

No. of Results

1.0

16992.http.get.title: "Dashboard" OR 16993.https.get.title: "Dashboard" OR 443.https.get.title: "Dashboard" OR 80.http.get.title: "Dashboard" OR 8080.http.get.title: "Dashboard" OR 8888.http.get.title: "Dashboard"

69,995

2.0

services.http.response.html_title: Dashboard

174,298

The legacy search product presented HTTP service results on five ports, whereas Search 2.0 can find HTTP on any port.

Search for Services With an Exact Attribute

These queries return hosts with an HTTP page title that is exactly "Dashboard."

Search Product

Search Query

No. of Results

1.0

8888.http.get.title.raw: "Dashboard" OR 8080.http.get.title.raw: "Dashboard" OR 80.http.get.title.raw: "Dashboard" OR 16992.http.get.title.raw: "Dashboard" OR 16993.https.get.title.raw: "Dashboard" OR 443.https.get.title.raw: "Dashboard"

3,352

2.0

services.http.response.html_title=Dashboard

19,434

In Search 1.0, exact matches could be sought in fields that had an accompanying field that ended in .raw, but not every field in the schema had an equivalent .raw.

In Search 2.0, searching for an exact match for a phrase is as simple as using the exact match operator (=).

Search for Services With Some Matching Attributes

These queries return hosts running any of the Remote Access Protocols that Censys can detect.

Search Product

Search Query

No. of Results

1.0

protocols: "5632/pca" OR "3389/rdp" OR "22/ssh" OR "23/telnet" OR "5900/vnc"

22,630,907

2.0

services.service_name: {OpenVPN, PCA, RDP, RDP_UDP, SSH, TELNET, VNC, X11, XDMCP, ARD, TEAM_VIEWER} and not services.truncated: true

30,235,106

Two things to notice in the 2.0 query:

  1. There’s no OR.

    Search 2.0 uses boolean operators like or, but it also has a set operator {}, which more succinctly denotes what would otherwise be a long (but perfectly valid) or statement.

  2. There’s an additional specification in the 2.0 query to only return hosts that do not have a truncated service.

    This is to weed out a class of hosts that Censys calls "superhosts" such as honeypots and firewalled hosts, which often present hundreds of open ports without any real services. When Censys finds these superhosts, it truncates the data about each of the host’s (pseudo)services.

Software Searches

Search for Services Running a Software Product

These queries return hosts with a service using any version of Microsoft Exchange Server.

Search Product

Search Query

No. of Results

1.0

25.smtp.starttls.metadata.product: "Exchange Server" OR 143.imap.starttls.metadata.product: "Exchange Server" OR 110.pop3.starttls.metadata.product: "Exchange Server" OR 993.imaps.tls.metadata.product: "Exchange Server" OR 995.pop3s.tls.metadata.product: "Exchange Server"

175,521

2.0

services.software.uniform_resource_identifier: "cpe:2.3:a:microsoft:exchange_server:*:*:*:*:*:*:*:*"

189,909

While Search 2.0 has a text field called services.software.product that can be set to "Exchange Server," Censys recommends using the newly supported CPE-formatted URIs to search for software.

CPE URIs yield more results because they are standardized, whereas a field like product may contain variations that would not be returned for a text search.

For example, searching for "Exchange Server" in the product field would not return hosts whose product was "Exchange 2003 Server" or "Exchange 2007 Server." (Show me what gets missed using the product field)

TLS-Related Searches

Many protocols use TLS, and because many users want to query TLS data across protocols, Search 2.0 hosts present TLS information in a separate field at the same layer as other top-level service fields.

Note the resulting simplification of search queries in the following examples.

Search for Hosts Using a TLS Version

These queries return hosts running a service that uses an out-of-date TLS version (1.0).

Search Product

Search Query

No. of Results

1.0

110.pop3.starttls.tls.version: TLSv1.0 OR 143.imap.starttls.tls.version: TLSv1.0 OR 1433.mssql.banner.tls.version: TLSv1.0 OR 1521.oracle.banner.tls.version: TLSv1.0 OR 1883.mqtt.banner.tls.version: TLSv1.0 OR 25.smtp.starttls.tls.version: TLSv1.0 OR 3306.mysql.banner.tls.version: TLSv1.0 OR 3389.rdp.banner.tls.version: TLSv1.0 OR 443.https.tls.version: TLSv1.0 OR 465.smtp.tls.tls.version: TLSv1.0 OR 5432.postgres.banner.tls.version: TLSv1.0 OR 5672.amqp.banner.tls.version: TLSv1.0 OR 587.smtp.starttls.tls.version: TLSv1.0 OR 631.ipp.banner.tls.version: TLSv1.0 OR 8883.mqtt.banner.tls.version: TLSv1.0 OR 9200.elasticsearch.banner.tls.version: TLSv1.0 OR 993.imaps.tls.tls.version: TLSv1.0 OR 995.pop3s.tls.tls.version: TLSv1.0

3,300,020

2.0

services.tls.version_selected: TLSv1_0

4,768,094

Note the difference in TLS version orthography between the two Search products: (TLSv1.0 vs. TLSv1_0)

Tip
 If you’re not getting results in Search 2.0 and you think it might be because of a difference in the orthography of field values, try running a report on the field you’re querying to see how the values look.

Search for Hosts Presenting a Certificate With an Attribute

These queries return hosts presenting a self-signed certificate.

Search Product

Search Query

No. of Results

1.0

110.pop3.starttls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 143.imap.starttls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 1433.mssql.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 1521.oracle.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 1883.mqtt.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 25.smtp.starttls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 3306.mysql.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 3389.rdp.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 443.https.tls.certificate.parsed.signature.self_signed: "TRUE" OR 465.smtp.tls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 5432.postgres.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 5672.amqp.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 587.smtp.starttls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 631.ipp.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 8883.mqtt.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 9200.elasticsearch.banner.tls.certificate.parsed.signature.self_signed: "TRUE" OR 993.imaps.tls.tls.certificate.parsed.signature.self_signed: "TRUE" OR 995.pop3s.tls.tls.certificate.parsed.signature.self_signed: "TRUE"

12,335,116

2.0

services.tls.certificates.leaf_data.signature.self_signed: true

25,003,271

Again, since TLS data is located at the top level of the service object, no protocol specification is necessary in the Search 2.0 query. Any service with any name that has TLS data will be searched and returned if the self-signed attribute is true.

Related articles

Comments

0 comments

Please sign in to leave a comment.