Regular Expressions (Regex) in Censys Search

A regular expression provides great flexibility in defining search criteria to return relevant matches from large data sets.

As a simple example, take a wildcard search such as:

services.http.response.body: *.js*

which asks, "Which (unnamed) hosts with an HTTP service contain a reference to any string containing .js?"

The query above, while simple, is noisy, with link::https://search.censys.io/search?resource=hosts&sort=RELEVANCE&per_page=100&virtual_hosts=EXCLUDE&q=services.http.response.body%3A+*.js*[~26,300,000 results].

A regular expression is a way to provide criteria about what the value must look like without limiting it to single static string.

For example:

services.http.response.headers.location=/.*(\.\.\/)+.*(\.asp|\.php|\.js|\.cgi).*/

asks, "Which hosts have an HTTP location header that includes the sequence ../ ( which is vulnerable to directory traversal attacks), followed by one of the more common executable page types like .js, .php, or .asp?"

What are all the back slashes for?

A back slash is used to escape a character that is otherwise interpreted as an operator. For example, because periods (.) separate pieces of the pattern defined by a regular expression, you must put a backslash before a period to actually look for that character.

Regular expressions are extremely powerful but computationally slow. While understanding when to use a regex comes with experience, the most basic guideline is to use a regex when a simple pattern match won’t do.

Regex against fields whose values are robust strings

While regular expressions are valid for most fields, the best use case is for fields whose values are long strings.

Popular host fields to write regular expressions for:

Popular certificate fields to write regular expressions for:

These are example queries using regular expressions that show the power of regex.

Use this official regex syntax reference to see how to construct regular expressions.