Certificate Transparency and Pre-certificates in Censys
Certificate Transparency is a security standard that aims to make the issuance of digital certificates more transparent and auditable. It works by requiring Certificate Authorities (CAs) to publish information about the certificates they issue to publicly accessible logs. This allows anyone to monitor the issuance of certificates and identify any potential misuse or fraudulent activity.
Modern browsers require an X.509 document for HTTPS connections and a signed certificate timestamp (SCT) from a Certificate Transparency (CT) log to confirm that the certificate is submitted to a public ledger.
This framework and accompanying browser requirements have two notable effects on the Censys certificates data set: the presence of pre-certificates and multiple fingerprints for a single certificate.
Pre-certificates
As a result of CT requirements, many Certificate Authorities (CA) now submit a pre-certificate (sometimes written as “pre-cert”) to CT logs to obtain the log’s signed certificate timestamp (SCT).
A pre-cert contains all of the same information as a final certificate, but it has an X.509 "poison" extension (OID: 1.3.6.1.4.1.11129.2.4.3) that is marked critical, which prohibits the pre-cert from being trusted.
After a CA receives the SCT from the CT log, the timestamp is added to the certificate via the X.509 "SCT" extension (OID: 1.3.6.1.4.1.11129.2.4.2). The poison extension is removed, and the certificate is signed and issued to the requester. Some CAs, such as Let’s Encrypt, also submit the final certificate to the CT log, while many others do not.
Because of these differences, the searchable Censys certificate repository can contain records for:
- A final certificate
- In the case where both documents were submitted to a CT log or in the case where the pre-cert was submitted to a CT log, the corresponding certificate was observed during a Censys scan of the Internet.
- In the case where the certificate was found during a Censys scan of the Internet.
- A pre-certificate
- In the case where only the pre-cert was submitted to a CT log and the corresponding certificate has not been observed during a Censys scan of the Internet.
Important
If a certificate repository contains both a final certificate and its corresponding pre-certificate, only the final certificate will be indexed. However, you can still find the pre-certificate by its SHA-256 fingerprint.
Multiple fingerprints for a single certificate
The information contained in a pre-cert/certificate pair is identical. Only the poison extension on the pre-cert and the SCT extension on the certificate (respectively) distinguish them.
Importantly, however, these extensions result in disparate fingerprints (identifiers) because a fingerprint is a hash algorithm applied to the certificate data to make it smaller.
To further complicate matters, different web browsers employ different hash functions to generate fingerprints. This means that a single certificate can have multiple fingerprints within the Censys dataset:
-
fingerprint_sha256 -
fingerprint_sha1 -
fingerprint_md5
While you can search for certificates using any of these fingerprints, Censys search functionality is optimized for fingerprint_sha256. To view a specific certificate, you need to provide its fingerprint_sha256 value.
Searching across certificates and pre-certs in Censys
To make finding a document easier when you don’t know what kind of record is in the Censys repository, Censys calculates a SHA-256 fingerprint for the intersection of a pre-cert and cert, called tbs_noct_fingerprint.
This field provides a hash value for the content of a pre-cert/certificate pair so that you can search across both types of records.